Car starts without key!

Just something to check is there still 2 wires running to the starter. You said the previous owner put a push button start in it. On circle track cars we run a copper strip from the bug lug to the small lug on the starter, then run the big cable to one side of a Ford starter solenoid, then a wire from a push button activates the solenoid sending power to the starter. Don't think this mod would work with a Chrysler solenoid, but it is worth a check.