worried....... a little
But wouldn't a bent cam also make it hard to turn at any point?
Not always. Nothing is ever perfectly straight, perfectly round, or perfectly aligned. The degree to which they're 'out' increases the more parts are involved.
The bores in the block could be slightly out of alignment, slightly different sizes, and all slightly egg shaped. Same for the cam journals.
There will be a point in rotation where everything aligns, or the opposite and maligns. It could be as simple as journal diameters that are a couple thou larger than the cam it's replacing.
Point being: be careful and take measurements. Guessing does no good, and though chances are that there won't be a catastrophic failure, it's far easier and cheaper to measure parts on hand now than to do it later should an issue arise. Most engine shops would be able to put the cam 'between centers' and check runout on the journals, between that and checking the journal sizes, there's some peace of mind that can be hand.