I have to do a patch panel huh?

One thing to consider is that the panel got rock chips on it and it rusted from the outside in.......
The body shop just ''bondoed'' over the pin holes and caused the problem to get worse. The plastic filler actually acts as a sponge and absorbs moisture through the already present rust holes from behind. The time is now to fix it right.
It's really an easy fix to cut out and correct the problem area with new metal.
I really wouldn't be too worried about it, there are lots of videos on you tube about this if you do some research.