OK guys I'm pretty sure I've narrowed it down to an ignition issue, specifically the distributor. I tried @George Jets idea of spraying starting fluid down the carb when it was chugging; that just made it worse, almost stalled out and I didn't spray much at all.
I then thought about how yesterday I replaced the distributor rotor; "old" one was the unit that came with the Pertronix kit, "new" one was an Accel. I noticed the Accel rotor didn't fit as snugly into the magnetic wheel thingy that triggers the Pertronix module. Today I put the Pertronix rotor back in and made sure it was pressed down tight; lo and behold I started up the engine and put it in gear, it stayed smooth!!! I also have a feeling I wasn't getting the cap seated fully on the distributor body, it felt... different this last time I put it on, like I was able to slide it down a tad farther. Might have been hanging up on one of the wires from the Pertronix module.
Here's a video of it running "fixed", I guess now I need to find a replacement points distributor with good bushings...?
EDIT: The vacuum might seem low but I'm at 5000' elevation and that gauge is not accurate, I dropped it once and the diaphragm inside got bent and I just took it apart and bent it back till it was at zero again. Probably need a new one lol.
I took a video of me moving the distributor shaft around. It has roughly 1/16" play side-to-side (radially) and almost 1/8" play up-and-down (axially).
