What's the best way to find a dead cylinder?

I'm assuming that you mean to spray a Windex bottle stream on the exhaust manifold adjacent to the engine and see if it boils off. Could this harm the exhaust manifold or crack it?
A little stream won't hurt it