If you have stock axles and didn’t have a thrust button in there you wouldn’t have been able to tighten the adjuster to get the end play. So if you were able to set the end play without bottoming out the adjuster it’s not likely to be a thrust button issue.
Drums rubbing the backing plates is almost always a mismatch of small bolt pattern and big bolt pattern brakes and axles. The backing plates are specific to the axles, so SBP axles use different backing plates than BBP axles. Redrilled axles have to use the backing plates and brakes to match the original bolt pattern.
Regardless of the cause, grinding the backing plates or drums so they clear is the wrong answer. Brakes are safety equipment, find the problem and fix it.