Bumpers will be the biggest difference when it comes to Duster interchange from 1973 - 1976. They started out with "regular" bumper brackets in '73, went to the shock absorber brackets in the rear in '74 and had them in the front as well in '75 and '76. Fabricating is required to swap the 2 styles of bumpers.
'73/'74 grilles can be swapped with '75/'76 if the corresponding headlight buckets are used.
Was it '75 that they started using a catalytic converter which required a different bucket seat track?
'75 has a 3 piece taillight bezel that stretches all the way across the back of the car. The other years had the same tailight bezels. The '75 will have extra stud holes for the middle panel. Not sure if the '75 end bezels have the same hole location as the other years.