An argument against OpenBSD, qmail, et al.
Page 1 of 4 123 ... LastLast
Results 1 to 10 of 36

Thread: An argument against OpenBSD, qmail, et al.

  1. #1
    Banned
    Join Date
    May 2003
    Posts
    1,004

    An argument against OpenBSD, qmail, et al.

    Much has been made about the security of various security minded, open source products like OpenBSD and qmail, but are these accolades warranted? The data seems to speak for itself:

    “Only one remote hole in the default install, in more than 8 years!”
    -http://www.openbsd.org

    “My offer ($500) still stands. Nobody has found any security holes in qmail.”
    -http://cr.yp.to/qmail/guarantee.html

    These claims of nearly a decade of security do make a strong argument, the claims may have qualifications like how the insecurity must exist only in the current version or a minimal amount of functionality… however these don’t change the fact that the security record is excellent. Does a strong record mean they are secure?

    You could argue that most systems have such appallingly bad audit trails, that perhaps they have been exploited by real attackers and just not by bugtraq junkies, but this would be just plain pessimistic and argumentative.

    You could argue that OpenBSD is a downright rare operating system and even then mostly run by enthusiast and not high risk installations, consequently there has been minimal effort to break it. While this argument makes sense for OpenBSD it does not ring true for qmail, which is frequently listed as the third most popular SMTP server. Clearly it is more than just scarcity to account for a low rate of vulnerability.

    The simple fact of the matter is that guys like Theo de Raadt and Dan Bernstein are vulnerability experts and very good programmers (or at least they know good code when they see it, I have no idea if either of them lift a finger or just “lead” like myself). They have done a remarkable job in removing more or less all code level errors that allow common exploits to occur.

    If this is the case, why then do I feel that OpenBSD, qmail, et al are in fact poor examples of “secure products”? To understand that, we must first understand exactly what vulnerabilities are why they occur and more importantly, and why they are unavoidable at this time.

    For the context of this document, it is important to understand both the classes of flaws that lead to vulnerabilities and their introduction within a product.

    The seven classes (as defined by the Information Sciences Institute) are:

    1. Incomplete Parameter Validation (Buffer Overflow)
    2. Inconsistent Parameter Validation (Different routines w/ incongruent data formats)
    3. Implicit Sharing of Privileged/Confidential Data (Covert channels)
    4. Asynchronous Validation / Inadequate Serialization (Race Conditions)
    5. Inadequate Identification / Authentication / Authorization (Trojans, Weak Passwords)
    6. Violable Prohibition / Limit (Poor Bounds Handling)
    7. Exploitable Logic Error (Poor Error Handling, Bad Resource Allocation)

    These flaws may be introduced within the three primary elements of development (Requirements / Specification Design, Source Code, or Object Code), during maintenance, or even operation. 44% of these flaws occurring in the Requirements / Specification Design phase and 18% in the Operation phase compared to 30% in the source code itself according to Naval Research Laboratory studies.

    Perhaps part of the reason such a large percentage of errors are introduced at the design level is because so few commercial products (an even fewer open source ones) even have a Formal Top-Level Specification (FTLS), much less a good one. This FTLS should state more or less exactly what conditions the product is to meet, and then in the case of a secure product the FTLS must be consistent its security policy. Much of the point of an FTLS is focused on validation, “Does this product do what we need it to?”

    Products with an FTLS are then further verified with a formal (documented, precise, comprehensive) security policy, which has undergone formal (mathematical proofs, pre/post-conditions mapped) verification. Products lacking an FTLS may have either a formal security policy that has been formally verified or informal (un/incompletely documented, ambiguous) with no verification at this phase.

    Where verification becomes more difficult is ensuring that implicit rules are not violated. Implicit policies differ from explicit rules in that they are not actually spelled out anywhere in the security policy. For example: “User data cannot exceed the allocated buffer” Is implicit while “Operators can delete logs older than seven days.” is explicit. While products with an FTLS can mathematically determine the implicit rules correctness, informal products merely address explicit rules (at best) implicit rules can only be verified with testing.

    While testing is useful, (primarily for usability issues) it is not comprehensive enough to offer any real level of assurance as testing cannot prove the absence of vulnerabilities. Now review the flaw list above and think to yourself for a moment, “How many of these have you ever tested for?” Now keep in mind, I’ve only listed a few of the attacks under each classification and worse is that new ones arise all the time. How many of you knew what a Heap Overflow Attack was five years ago? Would you have though to test if such an attack could violate your security policy having never heard of such an attack? No amount of informal verification can overcome this lack of perfect knowledge and the potential infiniteness of the implicit rules.

    Does secure software exist? Yes. Can secure software exist without formal validation and verification? No.

    In time, as products harden, attackers will need to discover more varied attacks as the more trodden avenues become effectively shored up. New ways to exploit the flaws classes will be discovered and abused as there is currently no way to test such flaws, informally validated and verified systems will constantly find themselves in a race against attackers. A long history of success in this race is no guarantee of success in the future.

    Cheers,

    catch

  2. #2
    Senior Member gore's Avatar
    Join Date
    Oct 2002
    Location
    Michigan
    Posts
    7,177

    Re: An argument against OpenBSD, qmail, et al.

    Originally posted here by catch
    Much has been made about the security of various security minded, open source products like OpenBSD and qmail, but are these accolades warranted? The data seems to speak for itself:

    “Only one remote hole in the default install, in more than 8 years!”
    -http://www.openbsd.org

    “My offer ($500) still stands. Nobody has found any security holes in qmail.”
    -http://cr.yp.to/qmail/guarantee.html


    You could argue that OpenBSD is a downright rare operating system and even then mostly run by enthusiast and not high risk installations, consequently there has been minimal effort to break it. While this argument makes sense for OpenBSD it does not ring true for qmail, which is frequently listed as the third most popular SMTP server. Clearly it is more than just scarcity to account for a low rate of vulnerability.

    The simple fact of the matter is that guys like Theo de Raadt and Dan Bernstein


    catch
    Dan has more than Qpopper though, he has DNS stuff as well. His students were the ones that found those flaws in Linux. Now, ummm, Catch I do have to go with Qpopper being fairly secure here, if you offer cash and no one stands up and says "Look at me" that's at least got something going.

    Open BSD on the other hand, well, SUSE does code audits, if they release the next SUSE after a very big code audit with all things turned off (Un-useable) they could say it was the most secure OS in the world too. SUSE was just given the OK for Govt use as it got EAL 4+.

    I have to agree though that I've rarely ever seen someone actually using OBSD.

  3. #3
    Senior Member
    Join Date
    Mar 2003
    Posts
    245
    There has been a definite decline in the 'production' usage of OpenBSD. I think this is for two reasons. A) as you mentioned catch, the typical installation is on an enthusiasts machines; and many of those have abandoned it in favor of the more up to date and easier to use FreeBSD. B) Where OpenBSD used to be very popular in small ISP's and InfoAssurance/InfoSec applications, there are now dedicated, 'supported' commercial products on the market from folks like RSA, and EdgeBlue (BlueCoat) that have largely replaced them.

    Also in my expreience anyway, IS budget managers seem to be very uncomfortable with products that don't come with a large degree of accountability on the part of the vendor. That's one reason why we continue to purchase licenses for TrustedSolaris. Besides being a great OS for our financial data/SAP, knowing that a Sun security engineer is only a phone call away gives folks a warm fuzzy.

    Last, but most important these days for 'real' system managers working at companies in the U.S. is the dreaded and mandatory SOX Audit . If you haven't heard of this yet, and are an IS/IT manager or administrator, look out it can be painful. Systems like OpenBSD, and for that matter many Linux distro's don't have the necessary support to run products like Symantec's ESM; an absolute must if you need to provide 'proof' that several hundred UNIX servers have proper access controls, and that full system/group/user auditing is taking place on each one. Personally I love OpenBSD and am a big supporter, but realize that without some big comercial 'must-have' security product that runs on it, it's days are most certainly numbered in 'real' environments.

    Great post.

    -- spurious
    Get OpenSolaris http://www.opensolaris.org/

  4. #4
    Senior Member IKnowNot's Avatar
    Join Date
    Jan 2003
    Posts
    792
    Nice, enjoyable ( as usual ).

    But ( nit-picking again ):
    Can secure software exist without formal validation and verification? No.
    I disagree. That is like saying if a tree falls in the forest and no one hears it, it did not make a sound.

    ( Don’t mind me, I still think E=MC2 is incomplete )

    Now if you said that the chances are extremely slim that secure software could be created without formal validation and verification, I would probably agree. But that does not mean it can’t. What you seem to be looking for here is what you termed real level of assurance .

    Remember, software ( and hardware ) are made by humans. Humans are fallible. The system you describe merely creates checks and balances in an attempt to overcome these fallacies, but is not in-of-itself a guarantee. The motivation behind the creation of software varies, but the bottom line is usually money, fame, or a quick fix. These motivations propel innovation in the industry which is why we are at where we are today. Too much haste, too many people who don’t know and/or care about quality or pride in their work.

    IMHO software flaws are discovered mostly by those motivated as described above. Flaws in “ secure software” are found and/or prevented by those described below.

    What is needed are idealists, such as yourself. ( Just guessing here ) Those who believe the job is not finished until every avenue has been explored, examined, and accounted for ( oops, does spell checking count? ) People who think not just in the here-and-now, but in the future, and how their actions and creations will be utilized and judged.

    Well, while writing this, spurious_inode provided an example of the current state of affairs:
    ... IS budget managers seem to be very uncomfortable with products that don't come with a large degree of accountability on the part of the vendor.
    Accountability? That is just someone to point a finger at when what you did, did not work. But how much finger pointing can one do if one actually reads the warranties, disclaimers, liability limitations, etc. of the software they are relying on? Until the courts actually hold commercial software vendors responsible for what they peddle ( and the pendulum is beginning to swing ) there is only the illusion of accountability ( or, as they say, assurance ).

    Just my thoughts.
    " And maddest of all, to see life as it is and not as it should be" --Miguel Cervantes

  5. #5
    AO Senior Cow-beller
    Moderator
    zencoder's Avatar
    Join Date
    Dec 2004
    Location
    Mountain standard tribe.
    Posts
    1,177

    Slightly off topic

    Originally posted here by spurious_inode
    ...<snip> IS budget managers seem to be very uncomfortable with products that don't come with a large degree of accountability on the part of the vendor. That's one reason why we continue to purchase licenses for TrustedSolaris. <snip>...
    You've just summed up the pinnacle point in the argument against Open Source software in the business place that many of my 'long haired zealot' compatriots can't get through their thick pony-tailed heads.

    I'm not against Open Source at all, but I can acknowledge why a lot of companies won't embrace it; your statement is the reason. If you're network security solution tanks and my company takes a major hit, if you can be found liable, I've just removed myself from the hotseat and put you on it...at least partially.

    Ok, back on topic.
    "Data is not necessarily information. Information does not necessarily lead to knowledge. And knowledge is not always sufficient to discover truth and breed wisdom." --Spaf
    Anyone who is capable of getting themselves made president should on no account be allowed to do the job. --Douglas Adams (1952-2001)
    "...people find it far easier to forgive others for being wrong than being right." - Albus Percival Wulfric Brian Dumbledore

  6. #6
    Senior Member
    Join Date
    Sep 2004
    Posts
    117
    Can secure software exist without formal validation and verification? No

    well i disagree about this
    and i think 100 % secure software exist... if it is a software that does not contain code... or does nothing :P and even donno if ppl will find something in it :P

    If this is the case, why then do I feel that OpenBSD, qmail, et al are in fact poor examples of “secure products”?
    i wouldn't say poor...1 in 8 years
    how would you describe other product...1 critical every 10 minutes
    i would love to hear your description of some * products like windows, iexplorer... i think they will not be in your list of software

    and about not lot of ppl using and testing qmail and freebsd and some open source... well i think that the guys who make the exploits in other products are the one protecting and running those ones cuz they use them, they live on them :P

  7. #7
    Senior Member
    Join Date
    Jan 2002
    Posts
    1,207
    I have personally never heard of any business software (including operating systems) being written using formal methods.

    These might of course be true for ultra-critical applications (nuclear power stations, aircraft guidance systems etc), but I have never worked on any of those (thankfully).

    I don't know how hard it would be to formally specify, even a very simple piece of business software. Probably impossible, due to the large number of informally-specified pieces of other software it would have to interoperate with.

    You'd need a formal specification for any interface it used to interoperate with anything, including the OS. Business OSs don't have formal specifications (some very tiny subsystems of them might have very inaccurate ones).

    Personally I don't believe that formal specification is useful in business software. In software to do mathematical proofs, maybe. In software to guide aircraft, possibly. In business software? No.

    My rationale is:
    - Formal methods experts are very few, and almost certainly very expensive
    - Formal methods are obscenely difficult to do anything useful with
    - Business software needs to be developed to a reasonable timescale and budget.

    How many jobs do you see "Wanted - formal methods expert - must hold Compsci. PhD with specialisation in formal methods"??

    Slarty

  8. #8
    Banned
    Join Date
    May 2003
    Posts
    1,004
    disagree. That is like saying if a tree falls in the forest and no one hears it, it did not make a sound.
    Without formal V&V a "secure" software product is little more than assumptions:

    I assume this product the type of security I need.
    I assume that the implicit rules have been sufficiently tested.
    I assume that there exists no ways to take advantage of flaws that I don't know about.

    As an example, your security policy states: "A cannot write to C. A can write to B. B can write to C." (this is really simplified of course) Can A write to C?

    If you happened to develop a secure product without the use of formal V&V, how would you know?

    i think 100 % secure software exist
    By the context it is clear what you mean. I agree, 100% secure software can exist, but such software cannot be developed informally. Otherwise you are left with a product that is "Tested Secure" (which can mean near anything) vs. "Proven Secure."

    cheers,

    catch

  9. #9
    AO Ancient: Team Leader
    Join Date
    Oct 2002
    Posts
    5,197
    'long haired zealot' compatriots can't get through their thick pony-tailed heads.
    *COUGH*

    You might want to view my profile before you start claiming that my "pony-tailed head" is "thick"..... The damned thing is _solid_...... and I'm not for open source in a business environment for many of the reasons already laid down here.....
    Don\'t SYN us.... We\'ll SYN you.....
    \"A nation that draws too broad a difference between its scholars and its warriors will have its thinking done by cowards, and its fighting done by fools.\" - Thucydides

  10. #10
    Banned
    Join Date
    May 2003
    Posts
    1,004
    Slarty, there have been a number of COTS level formally developed products.

    I don't know how hard it would be to formally specify, even a very simple piece of business software. Probably impossible, due to the large number of informally-specified pieces of other software it would have to interoperate with.
    A big component of formal structure and development is black box theory, it doesn't matter that you send to it or what it sends to, so long as the data falls within the range of some pre-approved format.

    You'd need a formal specification for any interface it used to interoperate with anything, including the OS. Business OSs don't have formal specifications (some very tiny subsystems of them might have very inaccurate ones).
    Nah, read above about black boxing. Several buisness OSes have differeing degrees of formal design specifications (though few have comprehensive formal verification) and include AIX, HP-VV, NT, Pitbull, Solaris (trusted and un), IRIX, HYDRA, and SecureOS (the last two offer formal verification as well)

    Formal methods experts are very few, and almost certainly very expensive
    Such experts are only needed by developers, not the end users and with the advent of automated boyer-moore theorem proves formal verification has gotten much simpler/cheaper.

    Formal methods are obscenely difficult to do anything useful with
    Most new systems mimic the security architectures of previous systems, giving the developers at least a road map... and again these should be of little to concern for the end user.

    Business software needs to be developed to a reasonable timescale and budget.
    Of course, the idea here wasn't so much "Is a tested secure product better than a proven secure product for installation X?" More to the point of what constitutes "secure" anyhow? The beautiful thing about formal models is that they can be reused and once you have one development is a much faster process with a better end product, however as a rule such products do tend to be slightly more expensive (though perhaps the size of their market is more accountable for this than) not prohibitively so, especially for financial or R&D systems.

    cheers,

    catch

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •