Drawbacks?

Before you get aftermarket bell housing measure EVERYTHING. I did what you are saying. Switching starter from pass to driver. Originally my car had a 904 with custom headers made. I wanted to keep that orientation. ( I have QuickTime bell) offers a gen3 hemi to tkx ( I know you have t56) check your input shaft length! The gen 3 specific driver side starter quicktime bell is too tall.. I had to make a custom input shaft bearing. Reference the driver and passenger side starter bell depths cross reference it with input shaft length. Whatever option you do measure everything.
My opinion cast bell housings are.. eh. I know they work and have better tolerances than older ones but I like my feet..

The TKX is an aftermarket trans and comes in a Ford or GM flavor. Sounds like you had a bell for the Ford version but had a GM style TKX. Yikes.

He is talking about an OEM TR6060 which is going to be different than any of the TKX's that are available and there is only one bellhousing available as far as I have seen.

Unfortunately, being that it's the only bellhousing option for a Mopar TR6060 that can do a DS starter, it is cast aluminum or nothing. There is a scatter proof QT one for the TR6060, but it is PS starter only and basically just replaces the OEM cast bell.