Installed new coil, now the tach acts funny

IF we are not missing something, and you have the meter set correctly...........

(what does the switch position say exactly I cannot read it? EDIT looks like AC 200V?)
And what make/ model is the meter?

That is TERRIBLE. AC ripple should be less than a volt by a long ways. You likely have a--more than one--bad diode(s) or other problem in the alternator and it is causing interference in the electrical system.

I have NO idea why the tach would not work with the alternator unhooked. It should work fine.

It’s a Gardner Bender GDT-311. The lowest AC setting is 200v, so that’s where it’s set. It didn’t matter whether the ground was on the body of the alternator or grounded at the fender well or engine, I got the same reading.

I should update with the fact that I tried the tach once again with the voltage regulator disconnected, and this time it worked (to the same level it has/hasn’t been working). I must have missed something the first time.