I think your diagnosis is BS. Lots of rear drums are not swedged. I have seen fronts such as on pickups and 4x4s that slip on just like the rears.
Did you have them turned? Did you check them for out of round? Has anybody been whanging on anything, AKA bent, damaged? How did you get the original swedges off? You can screw up the HUBS if you are not careful
In this day and age of China's version of QC, I would suspect the drums themselves as well
Spoiler alert. I am 76. I spent nearly 15 years at two different parts stores, and I have literally turned THOUSANDS of drums and rotors. All the way from Fiat/ VW bugs to over the road heavy truck/ trailers. You know, like air brakes. I have also relined a few shoes, here and there.