Something has to be loose - how do I check?

I've got a 71 Duster with manual steering. I rebuilt the suspension about 5 years ago and everything was fine. Lately I've been hearing a fairly loud knocking sound when making right hand turns. I thought it was the wheel bearings, so I changed those and it didn't help anything. I've been under the car wiggling everything and so far nothing seems loose. I tried wiggling the tires up and down and left to right and there is no movement at all. Yet when I make right turns, I can hear clanking around down there.

There isn't much slop in the box. I can turn it back and forth quickly and its fairly tight. One weird problem is that I'll turn it say 2 turns to the left or right and wiggle it back and forth and at that point you can hear a little knocking. Do you guys think that could be my problem? Its just the knocking I hear when driving sounds way too loud to just be the steering box.

What are some ways I can go about testing the suspension for loose components other than just wiggling stuff around? I already had a helper turn the wheel back and forth while I was under there and I didn't see anything loose. Another weird thing is that both of the lower ball joint rubber cups seem to be "blown out". I'll have to replace the rubber, do you think that could be the problem? I'm just taking shots in the dark at this point.