Would this be a deal breaker?

Have a very good look at the frame rails and the torsion bar cross member.
If those have holes in them or are just recently undercoated, you probably have a $5,500 parts car. Did you actually determine yourself that the engine and trans are original to that car? Or are you just taking the guy's word for it?
If it's worth it to fix it up is only something you can decide.
Restorations aren't cheap even if you do the majority of the work yourself.
And they are very time and space consuming.
How about more pictures?