I’m hoping you didn’t use those crappy new style UCA cam bolts, with 1/2 of the diameter shaved off of the bolt itself. When I did my 66, I used the new cam bolts that came with the kit, and they wouldn’t tighten down for ****, causing the same problem as you’re having. I started saving all the upper cam bolts I came across after that, and only use the used ones now.
Funny thing on my car was if I pushed it forward to roll it around the garage, it would do what you’re experiencing, although not to that severe of a degree. Pushing it backwards set the wheel back in place, or very close to it. So I tightened the bolts as best I could, headed the 2 miles to town with the used bolts in hand, and had the alignment shop change them and realign the car. No problems since.