The factory head gasket is a known problem on the Neons. The Felpro (revised design) gasket takes care of the problem.
The only other issue I ever saw with the Neons, was the stupid Aluminum/Plastic radiator, they like to blow the transmission cooler ports clean out of the lower plastic tank. If that doesn't happen, the tanks crack and leak. Either way it's a horrible design (but I see this frequently on all makes/models with those dumb alum/plastic radiators)
My dad bought a '96 Neon about 6-7 years ago for $50 that needed the head gasket replaced. We replaced the head gasket, and never had a problem with the car after that.
I worked on a lot of Neons at the shop, the only time they ever came in was for Basic Maintenance (Tune up, oil change, etc), The Head Gasket (never had one come back for a head gasket after going with the Felpro gasket), or the Radiator (but like I said, I see that all the time, not just Neons)