Sometimes just clicks when try to start

I have a 72 Duster 340 4 speed. The problem I am having is sometimes when I turn the key I just get a click. When I try again it starts. Does not happen all the time, but I am afraid one day it will not start and leave me stranded. I thought it might be a loose connection so I checked all the connections to make sure they are tight. I am thinking it might be the starter relay but not sure so thought I would get some input before I start changing parts. I not very good at electrical issues. Any help would be appreciated.