I pulled #1 exh lifter with a magnet but the drain back hole is not big enough to pull it out. I was able to rotate it around. It actually did not look bad. It certainly was not chewed up.
On the SB heads I have here to look at, the cavity by cyls 3/6 looks to be the largest.
Just work the lifter rearward to that hole and it should fit thru.
Nonetheless, it seems as though you’ll be doing a cam swap.