Where the blue circles are is where your ifr’s are now. If you can’t get them out, drill them as big as you can so they aren’t a restriction any more.
Then drill the holes circled in red (if you have to, they may be to size already…I just don’t remember if they are or aren’t) and tap them 6-32 and then drill your brass set screws to .026 and screw them in those holes.
That way the ifr’s always have a head over them.
View attachment 1716009927