Go for it.
Wire wheels throw little metal wires EVERY where. So I don’t like to use them in internal machined applications. There are nylon ones but I have the same concern there.
Never had any issues with a bit of elbow grease and a scotch pad. Could use a straight razor if you needed to as well