Slant Six Oil light coming on — need a new pump?

I would say what you found is the anti-drain back valve, most oil filters have them. Discard the pieces and install a new filter, over and done with. You did not mention what weight oil you went with, I would go with a straight weight oil maybe a 40 OR 50 weight. Rotella makes a pretty good oil in this weight class.
Usually when you find a car like this...is probably why it was parked and forgotten. Best of luck, it is a cool old car.