The cam fitting tight isn due to and undersize cam journal
You have to open up "4 can journal a few though on most blocks I have messed with or the can turms hard or will not slide into #4 at all. The correct method is adjusting to journal size. If a person doesn't have the necessary means to do a job such as that they break out a bearing knife which is more for tractors and heavy equipment in the field but it works.
I can't imagine doing any type of close tolerance work with a Dremel tool especially when you can easily chuck the camshaft in a small lathe so tolerance can be maintained. If you already spent the money to get a roller cam and all of the necessary hardware; not grooving number four when the cam is made is like stepping over a dollar to pick up a dime.