restoring ralley mirrors

The non remote mirror has a different ball/socket assembly with resistance like a inside rearview mirror has. Those would loose the tension and change position every time the door is closed.
What you plan to do will probably work, maybe better than a non remote mirror of the same age.
If you decide you no longer need that little piece of steel that is shown on your controller, can I have it ? :)