Clunk Noise

Checked the ring and pinion backlash today and it is within specs. If I rotate the driveshaft by hand I can hear the clunk, but much quieter, when I rotate it in the reverse direction. Both u joints are new, but the rear one seems to move just a hair. Possible problem with the pinion or u joint? Thanks.