I'm not sure what you're doing but I always do this on the last coat of primer. 320 grit dry sand with a soft block and that should find all the needed areas to be fixed. then sand it again with 500 grit dry paper or you can wet sand with 600 if you want.