Maybe on the cam, but I wouldn't start with it. Seems that is not all that uncommon on the magnums. Maybe the factory just let it go means it a roller. Lifters I'll have to clean and inspect. I have other lifters if I cannot use them.
I had the same issue on my 318. Just changed the one that was on that lobe when I did my cam swap. The rest cleaned up and felt fine. 11k miles so far and no noises or issues.