How do I remove this piece?

Do not tap on the shaft with a steel hammer; you'll be soooooooorry.