Connecting 'vacuum amplifier' causes hesitation off idle

It sounds like you must have a vacuum leak somewhere in the vacuum controls system, because when everything is working right the engine does not get any extra air to cause this.
It is supposed to only draw vacuum until it has a reserve of it in the amplifier, and not get extra constantly.