Something doesn't sound right!

Doesn't sound like lower end to me. But it's not "just" and exhaust leak there. I'd pull the valve covers and intake and take a look at the cam and lifter preload.
My best guess via laptop speakers is a lifter. A cylinder sounds weak. The sound occurring after the engine and oil is warmed up supports the idea of a lifter and a bad/stuck check valve. Did you actually check the lifter preload? That's hard to impossible to do with the intake still on.

Have you done a compression test on all cylinders?

What oil weight and brand are you running?