How much bearing damage is too much

Maybe the camshaft tunnel is not straight the machine shop can tell
You take the cam to the shop installing the bbearings so they can verify free rotation. Then you need the cam out to clean the oil passages and the whole block before reassembly. I like to stand the block on the bell housing end to lower the cam in. It should rotate easily with the index finger and thumb.