I'm definitely not a body man, but when it comes to my duster I make sure to get all the rust out of the area I'm working on. I cut/sand an area quite a bit larger than I need to just to make sure. I hate body work, I don't want to do it again in a few years.
I'd take my time and sand it all...