Long story short: I took off the intake AGAIN... Here's why:
I finished cleaning up the valve tips, put it all back together and fired her up. Same, if not worse clacking. Oil pressure took longer than before to build up.
At some point I had not been super confident on the oil level, so I when I drained the oil and changed the filter, I ignored the dipstick and filled the car with 5 quarts of oil, which is the supposed capacity of a 318 with a stock pan. I started the engine, let it sit over night, then marked the stick where that level was, which was about another quart.
When I started the engine now with the new rockers, I had drained about a quart out of it again thinking I was wrong and possibly there was too much oil in the engine. Since the oil pressure at startup was worse again, it's safe to say that the dipstick is the wrong one. So I really need to find a correct dipstick as the method described above is likely not accurate.
It's also safe to say then that I drove the car without enough oil in it, which could have caused the bad lifters.
Anyways, I also discovered that the was a coolant leak at the intake, so I didn't hesitate to take off the intake.
I inspected each of the lifters for wear on the bottom, but it seems like they're all spinning and haven't eaten into the cam. I'm no expert, though.
However, I found one collapsed lifter and it was where the loudest clacking was coming from.
So, as devastated and frustrated I am, this is what happened with this engine: Incorrect dipstick, so engine was run with too little oil. Lifters went bad. I replaced the lifters, but at least one of them was bad, collapsed or never pumped up.
So if I replace that lifter and get a correct dipstick & tube, I might have a chance at an engine without problems.
Here is a video of the cam. It doesn't appear to be hurt?!