Not enough money in the world for me to take a sanding disk to lead filler. Aerosolized lead dust? Yeah, no thanks. Breathe some of that in and you’re screwed. Get it on your skin and you can absorb it too if the particles are small, like they will be if you take a roloc disk to it.
That’s why you only use body files on lead. Bigger pieces, no chance of breathing it in or getting coated in dust. And clean up is easy, just sweep up the shavings. If you make dust it’ll get everywhere.
The surface isn’t as consistent as I’d like to see it, but metal-2-metal over what the OP has isn’t going to cause him any issues at all.