It was the threads in the crank that stripped out. I didn't notice it until I already had the engine rebuilt and went out to the garage to periodically turn it over while it sat on the stand. It felt weird, like the bolt was turning a little bit while the crank wasn't. Pulled the bolt out and threads from the crank came out with it. It wasn't 100% stripped but it was enough for me to start using the flex plate to turn it instead.
Previous owner must have hit it with an impact or something because I wasn't doing anything out of the ordinary.