I am not thinking movement up or down or left and right; I am thinking warpage that will occur in a casting like that when torqued down. A few thousandths movement at the mounting face in or out will move the retainer hole's position relative to the crank centerline around quite a bit. I've found .004"-.005" movement in cast iron blocks back in the BH mount area with as little as 25 ft lbs of torque on BH bolts. So such movement of a thin AL casting is quite possible.
It bears IMHO checking since things look to be so far out. If the bolts were ALL in, and ALL torqued when the last measurements were done, then that would take care of the concern.