Oil dipstick rattling, hitting crank?

Older post but I have a similar problem.
1964 225.
The oil light came on last October while descending a long hill while putting miles on after the complete engine rebuild. I pulled over and there was no oil on the dipstick. The dipstick was actually out of the block a couple inches and oil had been escaping until it turned the oil light on. I figured the angle of the hill had been enough to cause the remaining oil to move away from the pickup and because I was less than a mile from home on level ground, decided to carefully drive home. No damage done.

The next time I had an incident was when I started hearing a steady ticking sound after getting on the throttle a bit. This time the dipstick was out of the block a couple inches and the crank was obviously hitting it. It had a slight banana curve but was not broken. I bent a piece of wire through the finger hole of the stick and hooked it to a hole in the motor mount. This effectively keeps it in place and have not had any further problems.

My question: Is there a positive way to retain the stick in the tube? Right now it just drops into place without any resistance. I checked the dipstick on a 340 I have here and it's the same way. I never remembet seeing any grommet or o-ring on any of my Mopars.

I should mention that I welded a couple small plates at the front and rear of the sump just because the pan was sitting on a welding table, I had the material and welder handy, and it seemed like a good idea at the time. What was i thinking? After driving it for 600 miles, its obvious this 64 Valiant is not a roadracer. In retrospect, maybe that's causing windage and pushing the dipstick out. The crankcase venting is 100% stock 1964 vintage.

Any ideas? I can remove the pan and cut out the baffles if needed.