I agree, knock him on his *** and tell him you're done with his bullshit. Just because he's your brother doesn't mean he gets a free pass to be an asshole.
This. Because he's your brother is more reason to knock his a__ out. He only does it because he knows your not going to. Does he berate other people? Probably not because he knows he might take a beating.
Or ignore him and block him from your life. Don't just take his crap.