Ok folks, thanks for all the brainpower, just got off the phone with Ray and we have the issue figured out. Weak small spring in the advance mechanism.
What was happening was the spring weakened with time, 7 months of storage, and that it was adding timing at idle and under 1k. around 750 it was adding 2 ish degrees, 850 around 4, 950, 6 and so on so that at a high idle of 1k rpm it was adding almost 10 degrees. Timing was almost all in by 2k (which was causing the detonation). When I mentioned "swapping distributor" earlier I misspoke.. what I did was cannibalize another distributor using the same advance weights and springs... so of course problem followed. Visually, there wasn't really anything to indicate the spring had weakened other then light corrosion over everything.
As the motor heated up, it wanted to idle higher, which would pull in advance, which would make it idle higher, and of course making it impossible to accurately set base timing. It wasn't until I sat with the timing light and observed the advance through the entire rev range that I was able to see what was going on.