How much does temperature affect motor?

Did you block the heat riser when installing the manifold?
Have you re-torqued the manifold (and carb) after a few heat cycles ?

I did not block the heat exchange section of the intake when installing it. When I put it all together, I didn't even know that it might cause any problems.
I have not gone back to re-torque the intake but I have removed the carb multiple times trying different spacers to try and eliminate these issues.