A (forged) 340 Crank is Not a (forged) 273/318 Crank?

I wonder how many have switched cranks between 340 and 273/318 without taking into consideration those holes? What is the worst that could happen?