Results 1 to 7 of 7

Thread: NMAP 3.70 Core Re-write is Out!

  1. #1
    Master-Jedi-Pimps0r & Moderator thehorse13's Avatar
    Join Date
    Dec 2002
    Location
    Washington D.C. area
    Posts
    2,885

    NMAP 3.70 Core Re-write is Out!

    Yea man, just got the news....


    Nmap hackers,

    Nmap's 7th birthday is tomorrow (Sept. 1). Since the proper way to commemorate it is with a major new release, I am pleased to announce the immediate availability of Nmap 3.70. This release brings dramatic changes. The main port scanning engine has been rewritten from scratch to be faster, scan many hosts in parallel, and be gentler against target hosts and networks. Service/version detection also now functions against many hosts in parallel. The UDP system has been overhauled to work in conjunction with version detection and a new "open|filtered" state to avoid false open reports against filtered systems. Nmap now estimates completion time for port/service scans in verbose mode (-v) when they are expected to take at least a couple minutes. A "port scan ping" system can dramatically improve scan times against heavily filtered hosts. There is also a new --exclude option that allow you to skip given hosts or networks in a broad scan. For example, a host may be too sensitive or critical to scan at a given time, or a security admin may not be responsible for certain subnetworks. Or maybe you want to cease scanning CW.Com IPs because they keep sending abuse reports to your ISP . There is a workaround for the Windows SP2 problems (improved from the workaround in 3.55-SP2), and MAC address reporting now works on Windows. There are dozens of other changes, which you can read about in the Changelog entries below.

    Making Nmap faster was one of your top 5 priorities in the last Nmap user survey, and I hope 3.70 will not disappoint. Timing varies dramatically based on network/firewall characteristics, but almost all of the pre-release feedback has been positive. For example, Bill Peterson, an Information Security Analyst at Alcatel, regularly scans a million IP addresses over the Internet to keep the company secure. He reported that with 3.55, "my scans were running for more than two weeks". He switched to a 3.70 pre-release (incidentally on a somewhat beefier machine), optimized the option flags a bit, and was soon finishing the scans in under a day. I've been doing my own testing against thousands of machines as well. The time taken for the command "nmap localhost" on my primary development machine improved from 3 seconds to less than three TENTHS of a second. The command "nmap -T4 scanme.insecure.org" (a filtered-by-default machine) over my home aDSL line improved from 31.8 seconds to 19.7. Results involving multiple machines or UDP scanning are often far more dramatic than these. The official motto for this release is: So fast it deserves a CERT Advisory .

    If you do find a case where the UNIX version 3.70 is slower than 3.55, let me know. I'm afraid that speeds of the Windows version of Nmap may not have improved as dramatically as on Linux/BSD/Mac OS X. I had to spend time working around MS SP2 nonsense rather than focusing on optimizing for that platform.

    You might expect that such dramatic changes to the core of Nmap would load the new release with bugs. There are always some, but I hope you will be pleasantly surprised by 3.70's stability. The nmap-dev list has admirably tested many pre-releases over the last few weeks. I would particularly like to thank Gisle Vanem, Eric of Catastrophe.net, Andy Lutomirski, Dana Epp, Mark-David McLaughlin, William McVey, Arturo "Buanzo" Busleiman, Bill Petersen, and Tom Duffy for major contributions to 3.70.

    Here is the full list of significant changes:

    o Rewrote core port scanning engine, which is now named ultra_scan().
    Improved algorithms make this faster (often dramatically so) in
    almost all cases. Not only is it superior against single hosts, but
    ultra_scan() can scan many hosts (sometimes hundreds) in parallel.
    This offers many efficiency/speed advantages. For example, hosts
    often limit the ICMP port unreachable packets used by UDP scans to
    1/second. That made those scans extraordinarily slow in previous
    versions of Nmap. But if you are scanning 100 hosts at once,
    suddenly you can receive 100 responses per second. Spreading the
    scan amongst hosts is also gentler toward the target hosts. Nmap
    can still scan many ports at the same time, as well. If you find
    cases where ultra_scan is slower or less accurate, please send a
    report (including exact command-lines, versions used, and output, if
    possible) to Fyodor.

    o Added --max_hostgroup option which specifies the maximum number of
    hosts that Nmap is allowed to scan in parallel.

    o Added --min_hostgroup option which specifies the minimum number of
    hosts that Nmap should scan in parallel (there are some exceptions
    where Nmap will still scan smaller groups -- see man page). Of
    course, Nmap will try to choose efficient values even if you don't
    specify hostgroup restrictions explicitly.

    o Rewrote TCP SYN, ACK, Window, and Connect() scans to use
    ultra_scan() framework, rather than the old pos_scan().

    o Rewrote FIN, Xmas, NULL, Maimon, UDP, and IP Protocol scans to use
    ultra_scan(), rather than the old super_scan().

    o Overhauled UDP scan. Ports that don't respond are now classified as
    "open|filtered" (open or filtered) rather than "open". The (somewhat
    rare) ports that actually respond with a UDP packet to the empty
    probe are considered open. If version detection is requested, it
    will be performed on open|filtered ports. Any that respond to any of
    the UDP probes will have their status changed to open. This avoids a
    the false-positive problem where filtered UDP ports appear to be
    open, leading to terrified newbies thinking their machine is
    infected by back orifice.

    o Nmap now estimates completion times for almost all port scan types
    (any that use ultra_scan()) as well as service scan (version
    detection). These are only shown in verbose mode (-v). On scans
    that take more than a minute or two, you will see occasional updates
    like:
    SYN Stealth Scan Timing: About 30.01% done; ETC: 16:04 (0:01:09 remaining)
    New updates are given if the estimates change significantly.

    o Added --exclude option, which lets you specify a comma-separated
    list of targets (hosts, ranges, netblocks) that should be excluded
    from the scan. This is useful to keep from scannig yourself, your
    ISP, particularly sensitive hosts, etc. The new --excludefile reads
    the list (newline-delimited) from a given file. All the work was
    done by Mark-David McLaughlin (mdmcl(a)cisco.com> and William McVey
    ( wam(a)cisco.com ), who sent me a well-designed and well-tested
    patch.

    o Nmap now has a "port scan ping" system. If it has received at least
    one response from any port on the host, but has not received
    responses lately (usually due to filtering), Nmap will "ping" that
    known-good port occasionally to detect latency, packet drop rate,
    etc.

    o Service/version detection now handles multiple hosts at once for
    more efficient and less-intrusive operation.

    o Nmap now wishes itself a happy birthday when run on September 1 in
    verbose mode! The first public release was on that date in 1997.

    o The port randomizer now has a bias toward putting
    commonly-accessible ports (80, 22, etc.) near the beginning of the
    list. Getting a response early helps Nmap calculate response times and
    detect packet loss, so the scan goes faster.

    o Host timeout system (--host_timeout) overhauled to support host
    parallelization. Hosts times are tracked separately, so a host that
    finishes a SYN scan quickly is not penalized for an exceptionally
    slow host being scanned at the same time.

    o When Nmap has not received any responses from a host, it can now
    use certain timing values from other hosts from the same scan
    group. This way Nmap doesn't have to use absolute-worst-case
    (300bps SLIP link to Uzbekistan) round trip timeouts and such.

    o Enabled MAC address reporting when using the Windows version
    of Nmap. Thanks to Andy Lutomirski (luto(a)stanford.edu) for
    writing and sending the patch.

    o Workaround crippled raw sockets on Microsoft Windows XP SP2 scans.
    I applied a patch by Andy Lutomirski (luto(a)stanford.edu) which
    causes Nmap to default to winpcap sends instead. The winpcap send
    functionality was already there for versions of Windows such as NT and
    Win98 that never supported Raw Sockets in the first place.

    o Changed how Nmap sends Arp requests on Windows to use the iphlpapi
    SendARP() function rather than creating it raw and reading the
    response from the Windows ARP cache. This works around a
    (reasonable) feature of Windows Firewall which ignored such
    unsolicited responses. The firewall is turned on by default as of
    Windows XP SP2. This change was implemented by Dana Epp
    (dana(a)vulscan.com).

    o Fixed some Windows portability issues discovered by Gisle Vanem
    (giva(a)bgnett.no).

    o Upgraded libpcap from version 0.7.2 to 0.8.3. This was an attempt
    to fix an annoying bug, which I then found was actually in my code
    rather than libpcap .

    o Removed Ident scan (-I). It was rarely useful, and the
    implementation would have to be rewritten for the new ultra_scan()
    system. If there is significant demand, perhaps I'll put it back in
    sometime.

    o Documented the --osscan_limit option, which saves time by skipping
    OS detection if at least one open and one closed port are not found on
    the remote hosts. OS detection is much less reliable against such
    hosts anyway, and skipping it can save some time.

    o Updated nmapfe.desktop file to provide better NmapFE desktop support
    under Fedora Core and other systems. Thanks to Mephisto
    (mephisto(a)mephisto.ma.cx) for sending the patch.

    o Further nmapfe.desktop changes to better fit the freedesktop
    standard. The patch came from Murphy (m3rf(a)swimmingnoodle.com).

    o Fixed capitalization (with a perl script) of many over-capitalized
    vendor names in nmap-mac-prefixes.

    o Ensured that MAC address vendor names are always escaped in XML
    output if they contain illegal characters (particularly '&'). Thanks
    to Matthieu Verbert (mve(a)zurich.ibm.com) for the report and a patch.

    o Changed xmloutputversion in XML output from 1.0 to 1.01 to note that
    there was a slight change (which was actually the MAC stuff in 3.55).
    Thanks to Lionel CONS (lionel.cons(a)cern.ch) for the suggestion.

    o Many Windows portability fix and bug fixes, thanks to patch from
    Gisle Vanem (giva(a)bgnett.no). With these changes, he was able to
    compile Nmap on Windows using MingW + gcc 3.4 C++ rather than MS
    Visual Studio.

    o Removed (addport) tags from XML output. They used to provide open
    ports as they were discovered, but don't work now that the port
    scanners scan many hosts at once. They did not specify an IP
    address. Of course the appropriate (port) tags are still printed
    once scanning of a target is complete.

    o Configure script now detects GNU/k*BSD systems (whatever those are),
    thanks to patch from Robert Millan (rmh@debian.org)

    o Fixed various crashes and assertion failures related to the new
    ultra_scan() system, that were found by Arturo "Buanzo" Busleiman
    (buanzo(a)buanzo.com.ar), Eric (catastrophe.net), and Bill Petersen
    (bill.petersen(a)alcatel.com).

    o Fixed some minor memory leaks relating to ping and list scanning as
    well as the Nmap output table. These were found with valgrind (
    http://valgrind.kde.org/ ).

    o Provide limited --packet_trace support for TCP connect() (-sT)
    scans.

    o Fixed compilation on certain Solaris machines thanks to a patch by
    Tom Duffy (tduffy(a)sun.com)

    o Fixed some warnings that crop up when compiling nbase C files with a
    C++ compiler. Thanks to Gisle Vanem (giva(a)bgnett.no) for sending
    the patch.

    o Tweaked the License blurb on source files and in the man page. It
    clarifies some issues and includes a new GPL exception that
    explicitly allows linking with the OpenSSL library. Some people
    believe that the GPL and OpenSSL licenses are incompatable without
    this special exception.

    o Fixed some serious runtime portability issues on *BSD systems.
    Thanks to Eric (catastrophe.net) for reporting the problem.

    o Changed the argument parser to better detect bogus arguments to the
    -iR option.

    o Removed a spurious warning message relating to the Windows ARP cache
    being empty. Patch by Gisle Vanem (giva(a)bgnett.no).

    o Removed some C++-style line comments (//) from nbase, because some C
    compilers (particularly on Solaris) barf on those. Problem reported
    by Raju Alluri <Raju.Alluri(a)Sun.COM>

    As usual, 3.70 is available from http://www.insecure.org/nmap/nmap_download.html, including Windows (.zip format) binaries.
    Our scars have the power to remind us that our past was real. -- Hannibal Lecter.
    Talent is God given. Be humble. Fame is man-given. Be grateful. Conceit is self-given. Be careful. -- John Wooden

  2. #2
    AO Veteran NeuTron's Avatar
    Join Date
    Apr 2003
    Posts
    550
    I just downloaded 3.70 and got an error message when I ran it for the first time.
    Error: rawrecv_open: SIO_RCVALL failed (10022) on device eth0

    This was over my wireless connection, not ethernet, which Im sure has something to do with it. I dont remember seeing this with any previous version. Ever heard of it?

  3. #3
    Blast From the Past
    Join Date
    Jan 2003
    Posts
    729
    thehorse13 i didnt no you were on the nmap team.....or am i reading you post wrong

    eitherway this sounds great so im off to download now
    work it harder, make it better, do it faster, makes us stronger

  4. #4
    Senior Member
    Join Date
    Jun 2003
    Posts
    723
    Originally posted here by hexadecimal
    thehorse13 i didnt no you were on the nmap team.....or am i reading you post wrong

    eitherway this sounds great so im off to download now
    This (the first post) is from the nmap hackers mail list if you go to the nmap site you can sign up and will receive an email when a new nmap is released or something special happens.
    Do unto others as you would have them do unto you.
    The international ban against torturing prisoners of war does not necessarily apply to suspects detained in America\'s war on terror, Attorney General John Ashcroft told a Senate oversight committee
    -- true colors revealed, a brown shirt and jackboots

  5. #5
    Senior Member IKnowNot's Avatar
    Join Date
    Jan 2003
    Posts
    792
    First response to reading this: sh**, what the hell is this going to look like in my logs now? Especially with the "port scan ping" system”

    After a second review of the changes with another glass wine I was a little more relaxed, actually relieved and looking forwad to trying it, but curious as one caught my eye:
    o Removed Ident scan (-I) ...

    At one time Ident was almost a requirement, then, as security concerns and paranoia took hold many servers started abandoning it, almost to extinction. Now a resurgence as many IRC channels require it. For someone using these IRC channels this may curtail many of the automated scans based on nmap. Was this the underlying idea?
    " And maddest of all, to see life as it is and not as it should be" --Miguel Cervantes

  6. #6
    Master-Jedi-Pimps0r & Moderator thehorse13's Avatar
    Join Date
    Dec 2002
    Location
    Washington D.C. area
    Posts
    2,885
    thehorse13 i didnt no you were on the nmap team.....or am i reading you post wrong
    You are reading the post wrong. I subscribe to the NMAP mailing list and I do beta testing when asked.



    A little bit of fun stuff is in the code. Sept 1st marks NMAP's 7th birthday. When you run NMAP on Sept 1st, a happy birthday message appears.


    One last thing. For all of you XP SP2 users, you may experience quite a nasty surprise if you try to scan across your SOHO NAT router with NMAP. So far, I have seen those with Linksys BEFSR41 routers experience a spontanious reboot when executing NMAP with the -A -T5 switches. I tried this at my home and sure enough, reboot city. If anyone else sees this with other brands of SOHO routers, please post the info in this thread.

    --TH13
    Our scars have the power to remind us that our past was real. -- Hannibal Lecter.
    Talent is God given. Be humble. Fame is man-given. Be grateful. Conceit is self-given. Be careful. -- John Wooden

  7. #7
    Leftie Linux Lover the_JinX's Avatar
    Join Date
    Nov 2001
    Location
    Beverwijk Netherlands
    Posts
    2,534
    thehorse13, you only get the happy birthday message with the -v option . . .

    thx for the notice..

    and cheers to Nmap, may it live to be 107!
    ASCII stupid question, get a stupid ANSI.
    When in Russia, pet a PETSCII.

    Get your ass over to SLAYRadio the best station for C64 Remixes !

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •