Removing Tappets

I am planning on replacing the lifters but keep the cam so could I just knock them out?