Should piston gudgeon pin retainers be replaced at rebuild?

Always. Unless you like ruined blocks.