Now, let's instead imagine that you could go to a wrecking yard and pull a 440 from a 2001 ram with closed chamber heads, roller cam, 1.6 rockers and basically bolt all your factory accessories straight on to your 340ci engine.
Again, it'd be 440's all day!
You're right that larger displacenent engines usually have bigger bores, but I'm going to have to disagree that that bore size is the reason they have more potential.
The reason bigger displacement engines have more potential is that they are bigger displacement. And bigger displacement makes more torque.