In the perfect world everything fits and works. Unfortunately someone added tolerances to things using statistical process control measures so they could actually ship product at a profit. And it became good enough. Consider two items assembled together, each at the opposite of ends of their control limits. It may bolt and work but may not be ideal for the assembly.
IMO, It's likely close enough to make the spec. Many have just bolted things together and have been just fine. And BTW, some of those pins can be a PITA to remove too.
But, you don't know if you don't check it. If you have the opportunity to check it, why not check it. At the very least you would know one way or the other. You may just head off a potential issue. I still believe if you use the bearing the closer you can get to zero the better.