Almost lost a wheel
Wheel spacers are a touchy topic and you will get mixed information everywhere. But the facts are identical to keep them on the car vs keeping any other wheel on the car… proper lug nuts, proper torque, and if the installation seems questionable to you, it probably is. You got extremely lucky considering it sounds like you questioned the install from the day you installed them and to make up for less lug nut contacting the adapter taper seat, you compensated by overtorquing the lug nuts and only checked when you had a problem. If you wouldn’t install a wheel directly to the hub that way, don’t install an adapter to the hub that way.
Where the mixed information comes in is everyone has had an experience and not the same experience. I had a wheel issue on a trailer due to an incorrect lug nut and replaced the studs in the hub only to have a stud issue later that month on that hub. Minimum solution needed was replace that one hub; so I replaced both axles, all wheels, tires, and all the suspension. Needed no, but with two issues I was eliminating any further issues because the initial cause was the trailer was abused before I owned it, so to eliminate that abuse, replace the parts abused.
You converted from 5x4 to 5x4.5 pattern with those adapters, so now you need new studs minimum all the way around due to over torquing all lug nuts. But now you should ask how do your brakes perform, could be a perfect time to swap to 1973+ 5x4.5 front disc brakes and eliminate adapters at least on the front and maybe redrill the rears axles for 5x4.5. And since you like spirited driving, it’s a bonus to have better brakes and no adapters…
On my Duster I wanted to move the rear wheels out 1” to make it look better, I could have done it with 1” wheel spacers easy, but I don’t like adding variables so I bought 2 new wheels with 1” less backspace. Cost me $300 for new wheels, and the wheels I replaced had 200 miles on them since I bought them new for $300 2 years prior. But I like spirited driving and $300 is only $250 more expensive than 1” wheel spacers and 25 grand less than replacing the car… and peace of mind that a wheel adapter on my car isn’t likely to cause an accident that gets anyone hurt, because no adapter. Don’t shortcut safety… these cars feel small enough around these ever growing SUVs, no need to create a way to get tangled up with one.