First thing to do is do some SERIOUS looking as to "what's loose." Put your dirty clothes on, and get a friend.
Jack up the car and block it under the lower control arms. This will "unload" the spring pressure on the wheels, tires, and ball joints. Now you can take a small bar and wiggle and pry on various parts of the suspension, look for slop in anything and everything.
Forcefully wiggle the idler arm up/ down. Basically it should not move, at least not very much. Have your friend help Forcefully "horse" one wheel at a time left/ right, while you are under there looking for movement in the joints. and tire rod ends, any joint that moves or swivels is suspect.
Is it possible you have the box "off center?" That may amplify the problem
Harder to check is the lower control arm bushings, because they are under tension from the torsion bars. It would be wise to count the turns, and back off the lower arm adjusting bolts until they are "slack." For this you'll have to jack the car up and support it on the body stub frames. Once you get the torsion bar tension relieved, pry and wiggle the lower arms, looking for slop, cracks and so on
Leave nothing "out of the picture." Upper and especially lower control arms can wear out and or break or crack. Even the K frame can crack and flex in some cars
Tires. Even poor tires--tires that you cannot see any evident problem with-- can cause horrid handling problems. You might borrow a pair to see if it changes the symptoms, AND IF YOU ARE running mis-matched tires from front to rear, this can be a factor.
(We once had a new pair of tires put on my bosses then- nearly- new Dodge pickup and it felt like it was going to "go around" going down the freeway. He finally had me go back and replace all four. Didn't matter as to tire pressure, either)
From your description of the symptoms you have NO BUSINESS driving this car until IT IS FIXED
