Hot start problem

I've been having an intermittent problem that has just gotten more persistant, it's happened about 6 or 7 times now:
My car starts just fine in the morning and runs ok(needs a tune up), but after a while at operating temp, if I shut the engine off, it wont restart.
When I turn the key, I just hear the relay clunk, then nothing. The oil light operates normaly with the key turn, but no start. Once before I shorted the terminals on the starter and got it to kick over. Today, it wouldn't.
Whacked the starter with a small ball pein, put the car in D and let it roll about 10' to check if the bendix was hung up, tried to start again and nothing. Key turn clicks the relay, shorting the starter terminals yields nice big sparks but nothing else.
The last couple of times this happened, if I let enough time pass, the car would start as if nothing was wrong. Sorry for the novel, just trying to supply details. Electrical issues are my weak suit, I dont have access to my tools at the moment, and I dont want to start buying random parts to try to solve the problem. My suspicion is that the starter has just had it. Am I close?
Thanks in advance for your help.
225 bone stock slant with a 727 pushbutton in a 64 dart 270.
Baz