Too big of cam?

Haynes says to set timing at 5 degrees btdc but I don't think I can use that spec due to the larger cam.

That would be correct. The stock spec was arrived at including a number of factors which likely don't apply here..

Since you don't have any smog issues, set it where the engine wants using a vacuum gauge. You'll probably have to limit mechanical advance when you are done. The engine will be a bunch more responsive down low, run cleaner, etc., when you get the initial timing where the engine wants it.