You are getting some suggestions from hard-core racers, and you are a newbie. If you change to a 4 bbl, you would need to install a 4 bbl manifold (~$150), which is a chore, plus different throttle and transmission linkages. Unless you care about screaming the engine at 4000+ rpm, the 2 bbl is fine. Chrysler sold most V-8's that way for a reason.
I agree that your problem is the choke, and that is a simple and inexpensive fix. I would get the parts from a much later car to get the better electronic setup. I pulled them off an 80's Diplomat type car w/ 318, which aren't hard to find at the junkyard and most used the same Carter BBD carburetor as yours (w/ more emissions stuff). You can find some of those parts new, but not everything, so JY is best. Buy a new "choke pull-off", which is the vacuum diaphragm at the top of the carb. Adjust everything per the manual. Basically, you want the choke to close fairly tightly when the engine is cold. That is important to get the initial fire. As soon as the engine fires, the choke pull-off should pop the choke open slightly (manual says measure w/ a drill bit) so it doesn't run rich, flood, and maybe foul the spark plugs with soot. Unless all that works right, you will always have cold-starting problems.
Down the road, install an O2 sensor in the exhaust, so you don't fly blind. I have bought new Holley or MSD lean-rich indicator kits for $25 on ebay ($170 list), and even a new kit for Onan generators for $5 that worked fine (but long time constant). Wideband O2 is even better, and is getting affordable. The Carter BBD is great because you can adjust the metering rods instead of having to install a different main bleed, but don't touch those until you have an O2 sensor or you might screw it all up. BTW, I bought a new BBD for $28 on ebay a few years ago for my 65, but you must spend a lot of time trolling to beat this CMB.