My Power Disk Brakes are frustrating me...

Ok, so with the engine off I pushed the pedal and it soft at first until it gets firm. I let it roll backwards in the driveway just to see where the brakes catch. Seems like it catches near where it gets firm. How would I know if the master cylinder or booster is bad? How I can I check?