If you want to just delete the folder itself just go to c:\Documents And Settings\<your username>\Start Menu\ and delete the folder. Nothing bad will come out of deleting shortcuts.

I checked...