Rocker Shafts- Washers?

I just put the Angelucci blocks on. As mentioned, they need a shave. All of the rockers pivot, but I'd call it a tight slide fit. The alignment on all but 3 and 6 is good. I'll improve their alignment when I have them shaved.

Two rockers and the spacer in between them measure 3.341". How much clearance do I need? I have to consider that the blocks aren't doweled, and they'll move a little.

I guess my concern is how much the heat when it's running will decrease the clearance.