Take the carb off,, and check both of the base gskts to make sure they don't leave a small gap somewhere,, or somewhere that the gskt will suck in with high vacuum,, and close at low vac.. (wot)...
That is the first thing I'd be looking at (loose carb body or internal gaskets)
Just for kicks you might want to disconnect ALL vacuum lines from it and see if it still does it before you tear into it. (Brake booster and all if you have one)