I noticed it's up and down. Some days it loads up pretty quick and others it's kinda slow. Back when I had dial up it was plain frustrating at times. Ramcharger's right about a virus scanner eating up ram no matter what one you have.
I also use another browser on certain sites. Google Chrome. It seems to load stuff up allot faster. Some pages IE works better on and other pages Chrome works better. Go figure.