I have had experience that the key being tight, i would lay the key on a flat fine file and try to remove a little metal each side plus checking the key slot for burr's.
Yeah sometime you need to lap them somehow to knock down some thickness. You just need to be aware the key also engages an pulley, gear or something and not make that too sloppy.