That's my point- you don't swap them to the opposite side. Only the spindle gets swapped side-to-side, not the ball joint. The recesses stay towards the center of the car and receive the bolts as normal- you just are bolting them to the swapped spindle.
I've done this many times, there is no issue.
I'm swapping spindles not the ball joint it's just occurring on my office floor. The recesses towards the center. Thanks for trying but they would only fit with washers in the recesses.