Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Port Scanner Challenge

  1. #1
    Senior Member
    Join Date
    Jan 2003
    Posts
    3,915

    Port Scanner Challenge

    This has been interesting...

    It started out as a comparison between nmap and UnicornScan and the recently released PortBunny. The results were interesting... and it spanned a couple of blog posts.

    Port Scanner Challenge
    And the winner is...
    Port Scanner Challenge Update

    What made this really interesting was that someone associated with UnicornScan jumped in and jumped all over the results... and then went so far to run his own "unbiased" tests... even though I'm independent and he's not.

    Essentially, UnicornScan relies on the user to set the transmission rate... it apparently finds throttling to be too difficult of a task... In order to "properly" use UnicornScan you need plenty of experience and you have to be willing to run multiple scans to tweak the speed. Which apparently every network admin and security admin wanting to quickly scan a box, should have time to do... I'm assuming that some people have never worked in a busy environment. I mean 10 scans at 10 - 60 seconds per scan to get down to that 10 second scan, or 1 scan at 30 seconds (picking arbitrary numbers for comparison here)... the 1 scan is the clear winner. So yeah... it's proved interesting... I've also asked that each software's author provide me with the best string for their product (to optimize it to the max for a single, generic scan) and again I've been told you can't scan with UnicornScan unless you are willing to perform multiple scans to properly tune the speed.

    Anyways... enjoy.

  2. #2
    Right turn Clyde Nokia's Avatar
    Join Date
    Aug 2003
    Location
    Button Moon
    Posts
    1,696
    I strongly disagree with the methods used in your tests. (I'm Harry on your blog by the way, commented in the 'and the winner is' part)

    Scanning with a default scan, with T5 set and no retries is ludicrious and is not indicative of a real world scan - in fact if you were looking foir settings that would guarentee unreliability these would be them.

    I've alo read Fydoors, Robert's and your posts on the Sec Focus lists and would be interested in the test you proposed in your last email.

    Let me know if you need targets to scan - I can put up a PIX 515E, 7 hosts, oracle, MSSQL,, SOlaris, all flavours of windows and a few Linux and have them on different subnets to put the scaners through their paces - could even through and IDS/IPS in - althouhg couldn't guarentee this in time.

  3. #3
    Senior Member
    Join Date
    Jan 2003
    Posts
    3,915
    Quote Originally Posted by Nokia
    I strongly disagree with the methods used in your tests. (I'm Harry on your blog by the way, commented in the 'and the winner is' part)
    Good to know

    Scanning with a default scan, with T5 set and no retries is ludicrious and is not indicative of a real world scan - in fact if you were looking foir settings that would guarentee unreliability these would be them.
    You're right... the nmap settings (for example) were definitely not the best scan methods used... a simple -sS -P0 would have greatly improved the results...

    I've alo read Fydoors, Robert's and your posts on the Sec Focus lists and would be interested in the test you proposed in your last email.
    Did you see the nmap-dev (I believe it was nmap-dev and not nmap-hackers) list as well? The -T5 and no retries settings came from Fyodor on that list, as he was pointing out how nmap could be improved to operate at the same speed as PortBunny... that was one of the driving factors in using those settings.

    Let me know if you need targets to scan - I can put up a PIX 515E, 7 hosts, oracle, MSSQL,, SOlaris, all flavours of windows and a few Linux and have them on different subnets to put the scaners through their paces - could even through and IDS/IPS in - althouhg couldn't guarentee this in time.
    Excellent, thanks... I'm still waiting to hear from them... Robert is still fairly insistent that to be fair I run multiple scans and tune UnicornScan's send rate... I may give into this, but if that's the case I will be adding all of the scan times together. If I do get a response from them, I'm actually hoping to use our lab at work... I'll have between 200 and 300 live IPs to scan.

  4. #4
    Right turn Clyde Nokia's Avatar
    Join Date
    Aug 2003
    Location
    Button Moon
    Posts
    1,696
    It would actually be good to describe the layout of your network to the relevant authors and ask them what switches they propose will be best for their software....then run them.

    Each person would not know what they other programmer is going to suggest, however as the programmers they are suggesting what would work best with thier tools...

    This would aim off for any critique from the authors afterwards about not using the right options for the scan..

    I am actively testing unicornscan and Nmap myself as I have come across quite a few situations lately during PCI auditing where each tool has produced different results and I have had to go out on a limb when producing a final Pen test report about what services are contactable.

    Unicornscan feels like it can do more that Nmap but I think this is because it is so bloody complicated to use it gives you the impression of being able to do everything you want from it.....where as Nmap is so simple to use you would be forgiven to thinking it lacks functionality when compared to unicornscan.......which it most definitely does not...

    I've not really used Port Bunny, TBH I've not even heard of it that much.

    //edit - IMHO I would really not take scan times into consideration, unless they are absurdly long - personally I don't really care if Nmap completes in 40 seconds and unicornscan completes in 60 seconds; it is reliability that would determine what port scanner I use - if one is more complicated and I have to learn the switches and wait an extra 2 minutes for the scan to complete, then so be it; in my line of work I can't afford to have inaccurate results...
    Last edited by Nokia; February 8th, 2008 at 02:53 AM.

  5. #5
    Senior Member
    Join Date
    Jan 2003
    Posts
    3,915
    Quote Originally Posted by Nokia
    I am actively testing unicornscan and Nmap myself as I have come across quite a few situations lately during PCI auditing where each tool has produced different results and I have had to go out on a limb when producing a final Pen test report about what services are contactable.

    Unicornscan feels like it can do more that Nmap but I think this is because it is so bloody complicated to use it gives you the impression of being able to do everything you want from it.....where as Nmap is so simple to use you would be forgiven to thinking it lacks functionality when compared to unicornscan.......which it most definitely does not...
    I find that very interesting... most people, that i hear from, think nmap has too much functionality and is overly complicated... and from what I've seen of unicornscan, I find it's functionality to be limited... especially considering how complicated it is.

    I've not really used Port Bunny, TBH I've not even heard of it that much.
    It's a kernel-based port scanner that was released recently.

    //edit - IMHO I would really not take scan times into consideration, unless they are absurdly long - personally I don't really care if Nmap completes in 40 seconds and unicornscan completes in 60 seconds; it is reliability that would determine what port scanner I use - if one is more complicated and I have to learn the switches and wait an extra 2 minutes for the scan to complete, then so be it; in my line of work I can't afford to have inaccurate results...
    You're right... in the long run.. a few seconds doesn't matter... neither does learning the switches... For me, with unicornscan, the biggest issue is still that the authors are telling me I have to run multiple scans to fine tune it... That to me reeks of bad design... One scan should be sufficient.. maybe I missed ports that I didn't expect to be open.. How do I know I didn't have it tuned properly and that I need to scan again.. I'd rather unprecedented accuracy... and beyond speed..nmap also had the accuracy in my testing.

  6. #6
    Junior Member
    Join Date
    Feb 2008
    Posts
    2
    @HTRegz:

    Of the three portscanners mentioned (nmap, portbunny, unicornscan), only portbunny is designed without tuning options available. Nmap has default settings which would make for very slow scans in filtered environments. Unicornscan stays at a steady 300 packets per second by default in the unicorn.conf configuration file. It is expected that the end user will know what line speed they can test at.

    Point is, with nmap or unicornscan, you will likely run it several times with different options before finding that optimal setting for quick scans and high accuracy.

    Also, please study up on the design decisions made by the various authors. By understanding the problems the authors were trying to solve, you will find test situations that should favor each particular tool. Running one round of tests against a flat unfiltered network does not represent what professional testers see today.

    I meant for the testing I've done at http://loquens-caesu.blogspot.com/ to encourage others to begin testing for themselves. End users are the ultimate judge of what works for their environment. Don't take my word for it. Set up a similar environment. Use the options I did. You will find similar results.

    If you're going to go through the effort to run a test, at least try to understand the tools you are evaluating. Otherwise you come off favoring a car with automatic transmission because you don't understand the efficiency gains of stick shift.

  7. #7
    Senior Member
    Join Date
    Jan 2003
    Posts
    3,915
    Welcome A simple post brings in new membership... I should ask for a cut of the advertising fees.

    Quote Originally Posted by jeneral
    @HTRegz:

    Of the three portscanners mentioned (nmap, portbunny, unicornscan), only portbunny is designed without tuning options available. Nmap has default settings which would make for very slow scans in filtered environments. Unicornscan stays at a steady 300 packets per second by default in the unicorn.conf configuration file. It is expected that the end user will know what line speed they can test at.
    I agree that nmap has to be tuned... that being said... nmap has some "fairly standard" argument sets. With unicornscan, you were telling me that I had to run several scans to ensure that I had the best possible speed... That becomes a fairly different tuning that nmap generally requires.


    Point is, with nmap or unicornscan, you will likely run it several times with different options before finding that optimal setting for quick scans and high accuracy.
    See above

    Also, please study up on the design decisions made by the various authors. By understanding the problems the authors were trying to solve, you will find test situations that should favor each particular tool. Running one round of tests against a flat unfiltered network does not represent what professional testers see today.

    I meant for the testing I've done at http://loquens-caesu.blogspot.com/ to encourage others to begin testing for themselves. End users are the ultimate judge of what works for their environment. Don't take my word for it. Set up a similar environment. Use the options I did. You will find similar results.

    If you're going to go through the effort to run a test, at least try to understand the tools you are evaluating. Otherwise you come off favoring a car with automatic transmission because you don't understand the efficiency gains of stick shift.
    Here's the problem... I can test drive a car with an automatic transmission and a car with a manual... and I can see the difference for myself... I can see pick-up, I can see mileage... That's without racing, without being an engineer... just by driving.

    I took the port scanners out on a drive... I don't see what you are talking about with a simple drive... You are telling me that I need to be a stock car driver, or an automotive engineer to see your benefits because a simple drive isn't a proper test...

  8. #8
    Right turn Clyde Nokia's Avatar
    Join Date
    Aug 2003
    Location
    Button Moon
    Posts
    1,696
    I think the analogy he was using is that if you only know how to drive an automatic car and then jump in to a manual you may not have a very good experience of using it.

    Similar to if you don't understand how to use unicorscan properly then you will not have a good experience of this either ( and could be tempted to give it an unfavorable write up due to lack of knowledge?)
    Last edited by Nokia; February 8th, 2008 at 05:10 PM.

  9. #9
    Senior Member
    Join Date
    Jan 2003
    Posts
    3,915
    Quote Originally Posted by Nokia
    I think the analogy he was using is that if you only know how to drive an automatic car and then jump in to a manual you may not have a very good experience of using it.

    Similar to if you don't understand how to use unicorscan properly then you will not have a good experience of this either ( and could be tempted to give it an unfavorable write up due to lack of knowledge?)
    I don't think that lack of knowledge has anything to do with it... Could I do a scan with several options... sure no problem (it's not like there are that many to consider)... is that what I was demoing? Nope... I've never once called my testing the be-all-end-all or even said it was definitive.

    Unicornscan on default settings was sub-par to the other products... end of story... That doesn't apply to everyone's use case, that's fine... I know plenty of people for whom that is a very standard use case.

    Quite honestly Robert's original emails to me were what was detrimental to my view of UnicornScan... prior to that I was interested in investigating it further... However multiple emails telling me I need to scan, tune and scan again until I find the ultimate settings for the network / device... that tells me that the tool isn't conducive to my general requirements of a quick scan. It might be great for people doing IDS/IPS evasion.. that wasn't what I was doing, nor is it something I have to do regularly, so it wasn't in my scope.

  10. #10
    Junior Member
    Join Date
    Feb 2008
    Posts
    1
    DISCLAIMER i am the author of unicornscan.

    Quote Originally Posted by HTRegz

    What made this really interesting was that someone associated with UnicornScan jumped in and jumped all over the results... and then went so far to run his own "unbiased" tests... even though I'm independent and he's not.
    speaking for myself, i was confused about what your tests were actually
    trying to measure. If you were trying to measure accuracy, I think your
    measurements were too coarse. I also think your test scenario was
    unrealistic compared to what a pen tester actually sees.

    Quote Originally Posted by HTRegz
    Essentially, UnicornScan relies on the user to set the transmission rate... it apparently finds throttling to be too difficult of a task...
    Ignoring your mildly offensive tone here, i will point out that unicornscan
    had a different design goal of being asynchronous and stateless, so i will
    chalk up your misunderstanding here as ignorance of why stateless systems
    do not handle retransmissions in a statefull way.

    Quote Originally Posted by HTRegz
    In order to "properly" use UnicornScan you need plenty of experience and you have to be willing to run multiple scans to tweak the speed.
    this is true, it is designed for pen testers or other professionals who have
    extensive understanding about networks, and experience measuring them.
    But unicornscan is opensource, so if you have great ideas or specific things
    that you dont like, i am more than happy to try and help. I have noticed
    that some people really like unicornscan and others hate it, and i am OK
    with that too.

    Quote Originally Posted by HTRegz
    I've also asked that each software's author provide me with the best string for their product (to optimize it to the max for a single, generic scan)
    Minor correction, you did not ask me anything, but I may have told you the
    same exact thing anyhow.

Similar Threads

  1. Port List
    By ThePreacher in forum Miscellaneous Security Discussions
    Replies: 17
    Last Post: December 14th, 2006, 09:37 PM
  2. Help with Trojans!!!
    By Jubei_Yagyu_14 in forum Newbie Security Questions
    Replies: 19
    Last Post: February 19th, 2004, 08:42 PM
  3. ports
    By hatebreed2000 in forum AntiOnline's General Chit Chat
    Replies: 1
    Last Post: March 14th, 2003, 06:36 AM
  4. My firewall block this attempt.. but need info
    By LordChaos in forum Firewall & Honeypot Discussions
    Replies: 19
    Last Post: October 4th, 2002, 11:58 AM
  5. Port list
    By Badassatchu in forum Other Tutorials Forum
    Replies: 13
    Last Post: March 23rd, 2002, 03:18 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •