A-Body Autocross?

It depends a whole lot on what sanctioning organization and class you chose. In the USA, SCCA is the largest and most dominant organization and rule set. Even so, there are other options and the local events sometimes have local classes like vintage.

I won't bore you with SCCA rules 'cause you're in Canada. But keep in mind that the least modified class will be the least expensive and the easiest to drive.

For the later A-body, I would stick with the Duster - especially if it is already a non-restoration project car. This way if the class allows wider tires than fit, you won't feel bad about widening or flaring the openings. Same with installing a roll bar in a way to gain some additional structural stiffening if allowed.

If you're driving courses where a mustang can't maneuver, I don't see any advantage to a Duster or an second gen barracuda. Get yourself a Kart if they allow 'em. That or look for a different group to run with that has a class and course setup for pony cars.

So, I don't think you're out of your mind - depending on your expectations. :)