Eliminating the column shifter

I have a 66 Valiant with an auto shifter on the column and I am switching to a 4 speed. I want to get rid of the shifter on the column, do I have to switch the whole column out or can I just get the collar and replace that? Or grind off the shifter and patch up the hole?
My 66 Valiant has manual steering, a V-8 with the 904 automatic and a column shifter right now. I am gathering up as many of the parts I need before I make the 4 speed swap. I passed up a chance to get a floor shift column because it was for P/S and I didn't know if I could have just used my manual steering shaft and swapped it into the P/S floor shift column. I know the P/S column has a shorter steering shaft and thought of just swapping out the shafts. That column is no longer available.