You should. The breather is the INLET for the PCV system. It's not complete without it. Just because you do it, doesn't mean it's right. Here's a generic diagram for a correct operating PCV system. There is no "maybe" involved. Without the inlet, the PCV cannot do its job.