Get that thing off of there and clean the hell out of it.
Get under the rivet with a small chisel but it can’t have a sharp end. If it does you’ll knock the head off of it.
Gently tap in the chisel with a small hammer. Not some 32 ounce framing hammer but an 8 ounce ball peen hammer works nicely.
Once you get under the head and get it away from the manifold, you can grab it with vise grips and wiggle it back and forth to get it out.
If you are careful you can use the chisel and get it all the way out.
If you bend the rivet a bit it won’t matter. Just don’t knock the head off of it and leave the stem in there.
Once that plate is off get it clean. Any crap left in there will come out and jack your new engine right up. So get it clean.
Sometimes you get lucky and you can tap the rivets right back in and they will still have some press fit. I use either stud and bearing mount locktite or the red stuff.
If you don’t have enough press fit on the rivet, take a small, dull pointed punch and about an 1/8 inch from the hole put a nice punch mark.
Then do it three more times at about 90 degrees. That will put some press back on the rivet so with that and the locktite it will never come out.
But you damn sure need to get it off and clean it up.