To felt or not to felt?

That is the question.
IDK. Daily driver.
Replacing chain and pump and seal.
Didn't have one on it for years.
Seems like it will rub the outer rubber on the seal.
Does it matter?