can i leave starter on,pulling motor

jdsduster said
727 staying in,since its bolted to trans i think it can stay

I have pulled the motor with the stater still bolted to the starter...but had to remove starter to get the engine back in..