Heres something that I heard one time from a sys-admin

Port scanning is the same as walking past a house and looking into the open the windows. As long as you don't step inside your ok, but once you step inside your breaking in.