steering box removal question

i am removing my box to have firm feel rebuild it. my question is,do i HAVE to knock loose the steering column to get the box out? i got the coupler pin out,just need to remove the box bolts and get it out.