This is actually a misconception.

IE is not a true part of the Windows.

IE resides entirely within user space and uses only standard MSDN documented APIs.
IE gives the impression of being embedded into Windows because it is seamlessly integrated with many other components like Explorer and the help systems.

Media Player has been integrated in the same manner. both IE and MP can be removed but give the impression of being non-removable because there is no functional equivalent to replace them so alternative solutions fall short and cause errors.

IE is just another application its integration does not make it more secure but easier to secure using Group Policy and its integration does not make it less accountable.

cheers,

catch

edit: Still retired, just asked about this by MS Sec.