Any Pontiac guys on here? Need insight.

Another possibility.....worn thrust brg.
Engine thrust brg is on #4 main. Lever back & forth on crank. Normal endplay is 0.008" or less.
More likely to be damaged [ worn ] if used with a T400 trans. Oil pump drive shaft runs veeeery close to a counterweight; could come in contact & make noise if brg is worn.