Opinions on installing A motors in B bodies

I picked up a 1966 Plymouth Belvedere II 4 door last year and it has a real weak 400 in it and Im trying to decide if I want to stay big block in it or switch back to a small block I already have. This car was an original 318 car and of course in B bodies dont require a K member swap to install either one. Considered building a 440 that I already have but I would have to buy parts for, or putting in this 408 Magnum small block which is already built. When I was a kid nobody put small blocks in B bodies but in reality they are not much heavier than an E body with a small block. Anybody else have one that would care to share their opinion?