Just curious; what idle vacuum and what year booster?
If the pressure switch works properly, that plumbing diagram should work.
Even a 292/108 cam made enough vacuum for my 73 Dart booster. No it didn't work at idle at first start-up. But a couple of blips of the throttle during the warm-up period, and she was ready for action. And the check-valve maintained the vacuum during long coast-down brake applications. It only took about 13 inches of vacuum for it to work on my car. Engine idled at about 11.It was a stick-car tho, with a 10.97 starter gear, so 1000rpm was 7 mph, and by 1000 the vacuum was enough.So I never had an issue.
If your engine makes at least 13 inches by 10 mph,even with 3.23s and an automatic, your booster should be on-line, without the pump,by that 10mph. And the check-valve in the booster should maintain that vacuum, coming down from some higher mph,down to at least that same 10 mph during a stop. So the only problem area should be in that first 10 mph, and only at driveaway on a cold-start. Well, if your legs are sensitive, you might feel it on a long slow-down, in those last 10 mph. But usually, by this time , you are easing off the pedal anyway. If you lose vacuum at a stoplight, enough that it becomes a problem, I would suspect the check-valve first, and the diaphragm second and the control-valve third.