Replacing column shift lever

I've got the bearing and it's housing off. I need to remove the lock plate now and was looking for other methods than a hammer and a punch. I guess I can support the end of the shaft to the floor with a 2x4 cut to length but would rather not pound on it.

I have the pin retaining collar off (red arrow) but need to remove the roll pin (black arrow).

View attachment lock.jpg