vacuum advance idle issue?

OK, it would help to answer some simple questions. You're giving too much info in a confused fashion; not trying to be mean at all, it is just not clear enough to make heads or tails of.
1. When you remove the vacumm advance hose from the distributor, do you plug it? You need to do so.
2. Are you using a timing light to set timing? If so, then have you identifed the timing marks on the cover and on the pulley? The mark on the pulley is a smal vee-shaped notch. If you are not using a timing light, then you can't set timing properly.
3. Do you have a factory service manual? Many can be found on-line. That wil lshow you where to connect the vacuum line to the distrobutor and to the choke pull-off. By the way, the early Holley carbs did not have an external choke pull-off but a choke piston in the top of the throat so that can be confusing.
4. Can you take and post pictures? If so can you post a few pix of the carb?
5. So your distributor is not stock with points? Is it a Mopar distributor with a Mopar ignition control unit? Pix?

We really need a more complete baseline of info to work from.

4.