Drill the stat, put it back in and be done with it for good.
You will never have that "burp" problem again.
You can fill it with coolant, and then as soon as it's running you can top it off because the 1/8 hole in the stat will let the bubbles from the block out right away instead of getting hot spots till it opens.
Now if you want to try and keep it cooler than actual running temp during initial break in, that's a different story.
Leaving the stat out could very well cause it to overheat and have to be shut down before your break in time is done. (like already mentioned)