When to worry about engine noises?

Probably a stupid question but you wouldn't by chance have an OEM exhaust manifold with the thermostatic damper in it? Those will rattle like crazy.