Starter Will Not Disengage

As the title says starter will not disengage unless the battery is disconnected while running. When the ignition is turned off without manually disengaging the starter it will keep turning over. I have only replaced starter relay and did not solve the issue. This is on a mild 318, 80,000 miles, cheap summit headers, edelbrock performer intake 1405 carb, msd wires and blaster 2 coil with new orange box and ballast. A/C system has also been deleted.
Any help would be much appretiated!