Installed new coil, now the tach acts funny

IF we are not missing something, and you have the meter set correctly...........

(what does the switch position say exactly I cannot read it? EDIT looks like AC 200V?)
And what make/ model is the meter?

That is TERRIBLE. AC ripple should be less than a volt by a long ways. You likely have a--more than one--bad diode(s) or other problem in the alternator and it is causing interference in the electrical system.

I have NO idea why the tach would not work with the alternator unhooked. It should work fine.