New Starter Issues

Any noise from the starter while the engine is running is bad. Double check the installation to be sure its centered and bolts are tight. Sounds like its just not disengaging. May have to replace it again.
You can also try to bump the key a few time to see if frees up.