It doesn't slow your car down appreciably to run a bigger tire, what you do get though is the ability to hook your car when the track starts going away and that will make you faster than spinning your small tires, and loads more consistent.
You should run the biggest tire you are allowed or can physically fit. It's like having insurance that you won't lose the race because you spun your tires.
Even though those 11.5 tires and CalTracs did the job on my Duster 90% of the time, there was always that 10% waiting to bite you in the ***.
Oh and all those really quick 10.5 tire cars run 4 links and 33 inch tall tires.