Cam bearing issue

All of the lathe work I have done and still do never had that problem. The spindle speed does not have to be a high RPM, and taking care and experience comes into play here. But lathe safety is not the issue here, is it? But thanks for your suggestion.
A paint stir stick with the abrasive fastened to it is a much better approach.