Starter grinding

I have a 72 Scamp with 318 and dual exhausts. It has a Mopar Performance mini starter. Anytime I drive it, if I turn it on, and it doesn't have time to cool down, when I go to start it, the gear on the starter grinds, I assume against the flywheel, and I'm not able to start it. Sometimes this is solved by letting it cool down longer, or by hitting it with a rubber mallet. Occasionally it does it for a split second on a cold start but it never continues to grind when it's starting for the first time of the day. I just hear the split second grind.

I have heard Chevys have starter shims. Has anyone had this similar issue? What are some ideas? I have considered header wrap on the exhaust but I do not think it will make such a great difference that it will solve the issue. I have used a starter heat shield before with little results. Any help would be great.