Distributor thrust bushing roll pin

I managed to knock my distributor off the shelf and when it fell it shattered the nylon-lookin' thrust bushing where the shaft enters the main body. I have a computercontrolly magnum dist I can steal the part from but I'm having issues getting the roll pins out without damaging the plastic. Anybody have any tips or tricks to get this guy off whole?