It's not. The starter motor's load drags line voltage down significantly—on every car, not just yours—and they start up just fine, even in the cold when line voltage during cranking can be in the neighbourhood of 10v.
Taking the loads off the ignition switch contacts and firewall pass-through connectors isn't a bad idea and won't hurt, but the car's lack of such a relay now is pretty much certainly not causing your hard starting.