You have timing coming from both vacuum advance and mechanical advance... When you plug the hose in to the vacuum port it brings in timing, that timing causes the rpm's to increase, when the rpm's increase that causes the mechanical advance to bring in more timing..... Which causes the rpm's to climb even higher....
The problem is originally the vacuum advance was connected to ported vacuum, that means there is no vacuum at idle, it starts coming in at around 1200 rpm's....
The port on the side of the carb is manifold vacuum, depending on your actual carb there is probably a ported vacuum source on the front edge of the base plate...
FWIW Sorry to hear of your grandfathers passing but he left you a cool car that should remind you of him for years to come..