If you want authenticity then keep it and find out why you are having the fade issues after some drive time! Sounds like you really have a lining issue.
I have a bit of an issue anytime, cold or warm but know the cam hurts it. Authenticity of this piece matters to me so it stays and I live with what I believe is to hard to press pedal effort short of decent deceleration during braking. No fade issue though!
All my other early Mopars were non power assisted and seemed just fine on pedal pressure, drum or disc!