That is usually the argument for not being an early adopter of a new operating system or application release.
A service pack isn't an "upgrade" it is an "update". Usually they consist of a consolidation of patches and fixes to date (which you have most likely applied already), with a few new ones added for good measure.
This presents a dilemma, as if you don't apply the new patches, you will leave yourself vulnerable. I am sure that most of us are aware of the increasing trend to have exploits out in the wild hours after a patch release.
Incidentally, I am sure it is not my imagination, but I am sure that SP3 has produced a performance improvement in this box.
@ MsM:
Forgot to mention it, but SP3 loads RDP 6.1 and you need to uninstall RDP 6.0 or you will get an error message :)
