Repalcement Door Handles
sounds like the rod that runs from the handle linkage down to the latch itself has a slight bend in it. which would cause it not to "throw down" far enough when you engage the door handle button. (there are two seperate rods that run to the latch ....one for interior handle, one for exterior handle) which would explain why it opens from the inside and not the outside. being you had the same problem with the original exterior handle as the new.....and that you can open from the inside fine....I am lead to believe that neither the latch or handle are to blame....but that the rod is!
take it out and look for any bends.....straighten it out a bit to add more length to it and then i bet it works fine.