The history of the Mac line of Operating systems

Ok, before I begin:

This is NOT my work. I found this on Slashdot, and I thought it was very interesting. Being an OS person, I enjoy reading this type of stuff. Also, most of the Mac users here may get a kick out of this stuff.

I may also seem like a Mac hater. Well, to an extent, I am, and to another, I'm not. I can't stand most of the user base, because they walk around the same way Gentoo Linux users do, believing that their loved OS/Platform of choice, were some pioneers on the computing landscape. Both of these groups rarely understand that BSD was a huge basis onn these OSs, and also, Mac was NOT the first GUI OS, Palo owns that bragging right. And Gentoo did NOT come up with that portage thing, BSD has used that for as long as I can remember it.

I'm uploading sonme pics that are relevent to this discussion, so have a ball looking at them too.


Without further ranting:




The History Of The MacOS:



A History of Apple's Operating Systems:



Introduction


Mac OS X is a unique operating system in that it represents a rather successful coming together of paradigms, ideologies, and technologies that have usually resisted each other in the past.


It is a result of the trials and tribulations of Apple and NeXT, as well as their user and developer communities. Mac OS X is perhaps one of the best examples of how a capable system can result through the direct or indirect efforts of corporations, academic and research communities, the Open Source and Free Software movements, and even individuals.


Apple has been around since 1976, and many accounts of its history have been told. If the story of Apple as a company is fascinating, so is the "technical" story of Apple's operating systems.


This document discusses operating systems that Apple has created in the past, and many that it tried to create. Through this discussion, we will come across several technologies the confluence of which eventually led to Mac OS X. An important goal of the discussion is to better understand the reasons, and if possible, the rationale behind Mac OS X and its important components. This, in turn, will be helpful in understanding and appreciating the system as it is today.



My approach in this discussion is to have somewhat high magnitudes of the depth and breadth of historical coverage, as long as I believe the discussion is interesting and relevant from a system design perspective, within reason. For example, while I will briefly describe "the influence of Xerox PARC" and the background of Mach, I will not go to farcical limits and talk about the invention of the transistor, or the discovery of Silicon.





First Bytes into an Apple



As 1975 came to an end, Steve Wozniak finished his prototype of what would become the first Apple computer. Wozniak's employer at that time, Hewlett-Packard, was not interested in his creation, so he requested, and was soon granted, a release of the technology. Apple was founded on April 1, 1976 by Steve Jobs, Steve Wozniak, and an Atari engineer named Ronald Wayne. The company's first product was Wozniak's computer, the Apple I.



Apple I



The Apple I was based on an 8-bit microprocessor, MOS Technology's 6502, running effectively at just below 1 MHz. The 6502 was similar to the more expensive 6800 from Motorola.
The Intel 8080 was also around, but the 6502 was chosen primarily because it was cheap. The computer had a built-in video terminal, sockets for 8K bytes of onboard RAM, a keyboard interface, and a cassette board meant to work with regular cassette recorders.



The "computer" was simply a motherboard: the user had to provide a case, an AC power source, an ASCII keyboard, and a display device. The Apple I could be directly connected to a television with an RF modulator that resulted in a scrolling display with 24 lines of 40 characters each. It was introduced at a price of $666.66 that included 4K bytes of RAM and a tape of Apple BASIC.


The Apple I came with a firmware resident System Monitor, a program that can be thought of as its operating system. The monitor program was 256 bytes in size, and made use of the keyboard and display to present the user with a command line for viewing memory contents, typing and running programs, etc.


Compared to the UNIX general-purpose time-sharing system, which was in its Sixth Edition at that time, the Apple I's "operating system" doesn't sound like much. However, a computer running UNIX would have costed many thousands, perhaps even tens of thousands, of dollars then. The Apple I was an attempt to make computing affordable for hobbyists, and hopefully, for the masses.


Apple II



The Apple I would have a life span of less than a year, but its successor would live much longer. Wozniak had begun work on the Apple II that, although based on the same 6502 microprocessor, was introduced as an integrated computer: it came in a beige plastic case, with a built-in keyboard. Upon its release it was the first personal computer to display color graphics.



Various Apple II machines followed: the Apple II+, IIe, IIc, IIc+, IIe Enhanced, IIe Platinum, and finally the 16-bit IIgs, introduced in 1986. Many of these models had several revisions themselves.


A number of operating systems were created for the Apple II family.


Apple DOS



Shortly after the release of the Apple II in 1977, it was realized that a disk drive was imperative for the computer. Wozniak created a brilliant design for a floppy disk drive, the Disk II, and thus there was need for a disk operating system (DOS). Apple's first version of a DOS was released as Apple DOS 3.1 in July 1978.


Note that this was unrelated to Microsoft's popular MS-DOS. During a time when it was a luxury to have disk drives, and for an operating system to support them, many such "disk operating systems" had the term DOS in their names.


The first release was called 3.1 and not something like 1.0 because one of the implementers, Paul Laughton, incremented a revision counter x.y every time he recompiled the source code: it started with x = 0, y = 1, and every time y reached 9, x was incremented by 1. Apple DOS was beta tested as version 3.0.



Apple Pascal


The p-System from University of California at San Diego (UCSD) was very popular in the 1970's and the early 80's. It was a portable operating system, essentially a virtual machine, running p-code (akin to bytecode), with UCSD Pascal being the most popular programming language for it.



Apple had a Pascal system for the Apple II derived from UCSD Pascal II.1, an implementation of the p-code architecture. Two UCSD students, Mark Allen and Richard Gleaves developed a 6502 interpreter in the summer of 1978.



This later became the basis for Apple II Pascal released in 1979.
Apple Pascal lived as a product for five years.


Apple CP/M


Microsoft introduced a co-processor circuit board named Softcard in 1980. It was originally called the Microsoft Z-80 SoftCard, but Microsoft had to rename it to avoid a lawsuit from Zilog, the makers of the Z-80. The Softcard enabled the Apple II to run Z-80 programs based on the popular CP/M operating system that had a rich software library of programs, such as dBase and WordStar.


There were coprocessor cards other than Microsoft's, both for the Z-80 and other processors such as the Motorola 6809. The "Stellation Mill" 6809 card allowed OS-9, a real-time operating system to run on compatible Apple machines.



Apple SOS



The Apple III was introduced in 1980 as a computer for business users. It had a new operating system called SOS ("sophisticated" operating system, although it apparently was an acronym for "Sara's Operating System", after an engineer's daughter).


Every SOS program loaded the operating system into memory as well. A SOS application disk consisted of a kernel (SOS.kernel), an interpreter (SOS.Interp), which could be the application itself, or something that the application used, and a set of drivers (SOS.Driver). SOS evolved into Apple ProDOS.



Apple ProDOS



ProDOS, first released as version 1.0 in October 1983, was Apple's replacement for DOS 3.3, and was based on SOS. ProDOS provided better facilities for programming in BASIC, assembly language, and machine language, better interrupt handling, faster disk I/O with direct block access, and so on. It also had a relatively sophisticated hierarchical file system with features such as:

Multiple logical volumes on one physical volume

Support for up to 20 different file types (10 could be user defined)

Up to 8 open files at a given time

An arbitrary number of files in a subdirectory (the volume directory was limited to a maximum of 51 files, however)

When the 16-bit Apple II came out, ProDOS (then at version 1.1.1) forked into ProDOS 8 and ProDOS 16 (for 8- and 16-bit processors, respectively).




Inspirations



1984 is well known in the Apple world as the year the Macintosh was introduced. In 1983, Apple had released the Lisa. At this point, it would only be appropriate to go back to 1968, before the advent of UNIX, many years before Apple or Microsoft were even founded, and in fact, almost seventeen years before the first version of Microsoft Windows was released.
NLS: The oNLine System


On December 9, 1968, an astounding technology demonstration happened at the Convention Center in San Francisco during the Fall Joint Computer Conference (FJCC). Douglas Engelbart and his team of 17 colleagues working in the Augmentation Research Center at the Stanford Research Institute (SRI) in Menlo Park, California, presented NLS (oNLine System), an "online" system they had been creating since 1962. The "astounding" adjective is justified by the amount of innovation demonstrated on that one day.


Engelbart said at the beginning of his presentation that: "The research program that I am going to describe to you today is quickly characterizable by saying: if in your office, you as an intellectual worker were supplied with a computer display backed up by a computer that was alive for you all day and was instantly responsible, err, responsive ... how much value could you derive from that? Well that basically characterizes what we've been pursuing for many years ..."



The Computer Mouse



Engelbart demonstrated the first computer mouse, a three button "pointing device" with a tracking spot, or "bug" on the screen. The underside of the mouse had two wheels that could roll or slide on a flat surface. Each wheel controlled a potentiometer.

As the user moved the mouse, the respective rolling and sliding motions of the two wheels resulted in voltage changes that were translated to relative coordinate changes on the screen.
A 5-Chord Key Set



Another input device Engelbart used in his demonstration was a chord key set, a five finger equivalent of a full-sized keyboard. The key set could be used to input up to 31 (25 minus the state when no keys are pressed) characters.


Document Processing

Engelbart showed that text could be entered, dragged, copied and pasted, formatted, scrolled, grouped hierarchically in multiple nested levels (for example, multiple line of text could be collapsed into a single line), and so on. The text so created could be saved in files, with provision for storing meta-data (the file's owner, time of creation, etc.) The use of a mouse made these operations much easier. Engelbart referred to the overall mechanism as view control.


The system was useful while editing code as well: blocks of code could be expanded and collapsed, with support for auto-completion.
Furthermore, documents could contain embedded statements for markup, which allowed formatting them for a specific purpose, like printing.


Hypertext and Imagemaps

Using hypertext, that is, text with hyperlinks, Engelbart could jump from one location to another. This could happen either as a result of searching (by name, say), or by explicit use of live hyperlinks, that could be visible or invisible.
The system also had picture drawing capabilities, and even pictures could have live hyperlinks (similar to later day imagemaps).


Searching

The system provided powerful search facilities where keywords could be weighted, and search results were ordered accordingly. The results could also be presented as hyperlinks.



Windows

The computer screen could be split into a frozen display and a scanning window. Thus, while you were reading a manual, say, and you needed to look up a term, you could split the screen and view the term's definition in the dynamically changing scanning window, rather like modern day frames in web pages.



Collaboration


The system also kept track of who you were and what you were doing. People could work collaboratively on files, annotate each other's text, and leave notes for each other, similar in some respect to modern-day document versioning systems.
It was also possible to leave message for one or more specific people. A special language, essentially a programmable filter, would allow a "test" to be associated with pieces of text. Thereafter, a reader could only view what he was allowed to, as determined by the result of the context-sensitive test.




Interactive Collaboration


The SRI team demonstrated live audio-video conferencing. The communicating parties could even have collaborative screen sharing with independent capabilities. For example, two people could look at the same display, but one of them would only have read-only privileges.



The Result


Engelbart stated that their system was a vehicle to allow humans to "operate" (compose, study, and modify) within the domain of complex information structures - where content represents concepts. NLS was meant to be a tool to navigate complex structures, something linear text could not do well.


Engelbart was also involved in the creation of DARPA's ARPANet, the precursor to the Internet. His team planned to create a special service for ARPANet that would provide relevant network information, one that would answer questions like: Who is providing what services? What protocol do I use to get there? Who is down today and who is up?
The inherent philosophy, explicitly stated by Engelbart as well, was bootstrapping, that is, to build tools that let you build better tools, recursively. A very successful example of this philosophy is Unix.


Even with such impressive innovations, NLS ran into misfortune. Many of the team members went to the nascent Xerox PARC, where they hoped to create a distributed network version of NLS (as opposed to time-sharing). Worse still, SRI dropped the program and there was no funding. Engelbart went to a phone networking company called Tymshare, where he used to sit in a cubicle in an office building in Cupertino: very near to the birthplace of the Macintosh.



Smalltalk


The work done at Xerox PARC would greatly influence the face (and surely the interface) of computing. The first version of Smalltalk was deployed at PARC in 1972. Inspired by Simula, the first language to use object concepts, Smalltalk was a truly object-oriented programming language. It was also an operating environment with an integrated user interface, overlapping windows, cut-and-paste, integrated documents, etc. and would facilitate development of useful and interesting tools such as a WYSIWYG editor, a music capture and editing system, an animation system, and so on.


Later on, Smalltalk would also be one of the inspirations behind the Objective-C programming language.

Another effort began at Xerox PARC in 1972: one to create a personal computer.


Xerox Alto

The "personal" in Xerox's personal computing effort implied a non-shared system containing sufficient processing power, storage, and I/O capability to support the computing needs of a single user. The result was the Alto: a system consisting of a 16-bit medium scale integrated, or MSI (referring to the number of electronic components on a chip) processor, a bit-mapped 606 x 808 point graphical display, a keyboard, a five-finger keyset, a 3-button mouse (one with a ball, rather than SRI's wheels), and a computer box that housed the processor, disk(s), and their power supplies.


It had interfaces for connection to printers and plotters, and a 2.94 Mbps ETHERNET interface via which it could be connected to other Altos and laser printers. The Alto's processor did not support virtual memory.


By 1979, over 1500 Altos were in use, within and outside of Xerox.


Alto OS



The Alto ran a lowest-priority task (task 0) called the Emulator, the microcode for which was stored in the standard Alto microcode ROM. It emulated a standard instruction set, thus aiding in portability.


Other standard tasks included tasks for the disks (Disk sector task, Disk word task), networking (ETHERNET task), display (Display word task, Cursor task, Display horizontal task, Display vertical task), and memory (Memory refresh task, Parity task).


The Alto's operating system was implemented in BCPL. It provided drivers for disks, keyboard, and display, management of memory, the clock, interrupts, and other events, a file system, a BCPL environment, and miscellaneous functions. The system's debugger, called "Swat", saved machine state in a file named "Swatee". The Alto Executive was equivalent to a Unix shell. Its networked version, the NetExec, loaded programs from a boot server available via the ETHERNET rather than from the local disk.




Applications



The Alto began with and went on to have many more productivity applications, such as Bravo (for powerful, multi-windowed text processing), Laurel (a display-oriented message system providing facilities to display, forward, classify, file, print messages and their replies), Markup (a document illustration application), Draw (an interactive illustrator program for creating pictures composed of lines, curves, and text captions), Neptune (a file utility program for managing file directories on Alto disks), etc. Many programming languages became available on the Alto, such as Mesa (similar to Pascal), Smalltalk, and Lisp.



The applications mentioned above were usually very sophisticated. Draw, for example, divided the screen into multiple areas: brush menu, command menu, font menu, picture area, caption area, and a message area for displaying informational, error, or prompting messages. Officetalk, an experimental forms-processing system on the Alto, was an inspiration for the STAR System developed later.


As inter-network communication facilities were developed at PARC (leading to an internet packet format called PARC Universal Packet, or Pup), Alto also included FTP and TELNET abilities.


Xerox used the Alto's technology to come up with a system designed for office professionals, the STAR System.




Xerox STAR System


Xerox introduced the 8010 STAR Information System at a Chicago trade show in April 1981. The STAR's hardware was based on the Alto, with "better" components (more memory, bigger disks, higher resolution display, faster ETHERNET, etc.)


An important difference was that the STAR user interface was designed before actually building the hardware or software.

The STAR user interface provided the user with an electronic metaphor for the physical office. There were electronic analogs of common office objects: paper, folders, file cabinets, mail boxes, calculators, and so on. It would be an understatement to say that the STAR interface influenced many systems that came after it.

Consider:
The user's first view of the working environment was the Desktop, which displayed small pictures (icons) of familiar objects such as documents, folders, file drawers, in- and out-baskets, etc.


The user could click on an icon, and push the OPEN key to "open" an icon, which resulted in a window displaying the icon's contents. Icons could represent either "Data" (Document, Folder, Record File), or "Function". The user could copy, delete, file, mail, move, open, close, and print data icons. The function icons operated on data icons, and many of them are analogous to today's "application" icons. Examples of function icons included File Drawer, In- and Out-Baskets, Printer, Floppy Disk Drive, User, User Group, Calculator, Terminal Emulator, and Network Resource Directory.


Windows had title bars displaying the icon name and a context-sensitive command menu, with context-sensitive help accessible via the "?" button. They also had horizontal and vertical scroll bars (with page-up, page-down, and jump-to). However, the STAR windows were explicitly designed not to overlap (considered a nuisance in the Alto by STAR's designers): they could only be tiled.


An abstraction called Property Sheets was analogous to today's "control" or "property" panels. A related abstraction called Option Sheets implemented a visual interface for providing "options" (arguments) to commands. For example, the "Find" option sheet was a powerful tool for searching text (including textual properties such as case, font, size, face, position, etc.) in a part or whole of a document or a selection. It also allowed changing of the "found" text (and its properties) to something else, with optional confirmation.


The Graphical Age at Apple



Apple released the $9995 Lisa in January 1983, a year before the Macintosh was introduced. Lisa had a 5 MHz 32-bit Motorola 68k processor. It was inspired by the work at Xerox PARC described in the previous section, many details of which Apple was made privy to, thanks to a deal involving Xerox getting Apple stock in return for sharing their technology with Apple.
Note that the Lisa project had begun before Apple visited Xerox PARC to look at their technology. It has always been somewhat of a controversy as to what exactly was "inspired" by PARC's work, and what exactly was an Apple in-house innovation. Apple's engineering has usually been exemplary since its founding days, and it is not our goal to resolve this controversy here.




Lisa OS


Lisa's operating system, the Lisa Office System (OS), had a fully graphical user interface. There was a file browser with clickable, active icons. A folder icon would display its contents in a window, and a document icon would launch the appropriate application. It came with a spreadsheet (LisaCalc), a chart tool (LisaGraph), an outline builder (LisaList), a project scheduler with integrated PERT/Gantt (LisaProject), a drawing program (LisaDraw), a DEC VT/ANSI terminal emulator (LisaTerminal), and some other software.


Lisa introduced several aspects that would become part of Apple's systems to come. It had a menu bar at the top of the screen, although without an Apple menu. Menu commands had an Apple symbol however (instead of the cloverleaf symbol used later). Double clicking on an icon caused the resulting window to come up animated. Items were deleted by dragging them to a trash can icon.


Like the STAR system, Lisa strived to present a physical office metaphor. Like real paper, Lisa's screen displayed a white background. Since a white screen flickers more, Lisa required a higher refresh rate for its display, which added to its price. The addition of more memory and a disk drive pushed Lisa's price well over $10,000. It also took a long time to start up. In real life, Lisa was not quite the perfect computer it was designed to be.
SCO Xenix was also available for the Lisa.



The Macintosh


Steve Jobs unveiled the Macintosh on January 24, 1984, at the Flint Center in De Anza College, Cupertino. Later known as the Mac 128K (due to the 128 KB of built-in RAM), it had an 8 MHz Motorola MC68000 processor (with no memory management unit, floating point unit, or L1/L2 caches), and a built-in 9 inch 512 x 342 black-and-white monitor. There was a single 3.5" floppy drive that accepted 400 KB disks.

The Macintosh ran a single-user, single-tasking operating system, initially known simply as Mac System Software. It came on one 400 KB floppy disk.


The Macintosh ROM contained both low-level code (such as for hardware initialization, diagnostics, drivers, etc.) and the higher level "Toolbox". The Toolbox was a collection of software routines, somewhat like a shared library, that applications could use.


This way, a programmer did not have to "re-invent the wheel", and could create a consistent and standard user interface. Toolbox functionality included management of dialog boxes, fonts, icons, menus, and windows, handling of events, routines for text entry and editing, arithmetic and logical operations, and so on.


The QuickDraw portion of the Toolbox contained highly optimized primitives for drawing shapes (almost the same as the Lisa's graphics routines), dialogs, pull-down menus, scroll-bars, windows, and so on. With time, the Toolbox would have an incredible amount of functionality (API's) packed into it, and it would eventually be in the way of Apple's attempts to create a modern operating system while maintaining backwards compatibility.


The default application that ran as the system came up was called the "Finder" It was an interface for browsing the file system and launching applications. The single-tasking nature of the system required the user to quit a running application in order to work in the Finder.
The Macintosh File System (MFS) was a flat file system: all files were stored in a single directory. However, the system software presented a hierarchical view that showed nested folders.


Each disk contained a folder called "Empty Folder" at its root level. New folders were created by renaming this folder, which caused a replacement Empty Folder to appear.
The Macintosh contained many of the Lisa's characteristics, such as a menu bar at the top (but with an Apple menu) and an iconic trash can (that was automatically emptied every time the system booted). It also heralded Apple's Human Interface Guidelines.


The Lisa 2 was introduced simultaneously with the Macintosh. A version of the Lisa 2 with a disk was later sold as the Macintosh XL and came with MacWorks, an emulator to run the Macintosh operating system.


Many Systems for Many Apples



Apple spent the next few years improving the Macintosh operating system, and creating some other noteworthy systems.
System Software Releases 2 - 6
For a long time, there were multiple, independent versioning schemes in effect for system components: a System Software Release, a System Version, a Finder Version, a MultiFinder Version, a LaserWriter Version, etc. Eventually there were attempts to unify these versions.


Some improvements made during this time included:

Continued speed improvements for the Finder, including a disk cache and a "minifinder" to make application launching faster
Commands for common tasks such as shutting down, creating new folders, ejecting disks, etc.
A hierarchical file system (HFS) that supported true hierarchy, that is, folders could be nested without illusory aid
Support for multiple monitors
Support for large disk drives
AppleShare client features


An important improvement came when Apple incorporated cooperative multitasking via the MultiFinder. Initially included as a separate piece of software (along with the original Finder), MultiFinder soon became non-optional. It allowed the user to have several programs open simultaneously, as well as assign RAM to these programs.


Apple also made usability improvements like providing a progress bar with cancel button for "copy file" and "erase disk" operations. So far, the Finder did not use color even on color capable systems. This was "fixed" with the introduction of Color QuickDraw.
What color is your system?



After the Macintosh had been around for four years, some Apple engineers and managers met at an off-site in March 1988. As they brainstormed to come up with future operating system strategies, they noted down their ideas on three sets of index cards: blue, pink, and red:
Blue would be the project for improving the existing Macintosh operating system. It would go on to form the core of System 7.


Pink would soon become a revolutionary operating system project at Apple. It was supposed to be object-oriented, and would have full memory protection, lightweight threads, large number of protected address spaces, multi-tasking, and many more modern features. After languishing for many years at Apple, Pink would move out to Taligent, a company jointly run by Apple and IBM, as discussed later.


Since red is "pinker than pink", ideas considered too advanced even for Pink were made part of the Red project.
At this point, the System Software was at version 6.x. The 1980's were drawing to an end. System 7, a result of the Blue project, would be Apple's most significant system, both relatively and absolutely. However, that would not be until 1991. Apple would come out with two interesting operating systems before that: GS/OS and A/UX.




GS/OS



As stated earlier, the Apple II had a rather long life span. After the release of the Macintosh in 1984, the Apple II was still around. The Apple IIGS was introduced in 1986, almost as a bridge between the old and the new.


It was the first and only 16-bit Apple II, and had impressive multimedia abilities (the "GS" stands for graphics and sound). It had a 6502-compatible processor, two super high resolution graphics modes (200 x 320, 16-color palette, or 200 x 640, 4-color palette), a 32-voice Ensoniq Digital Oscillator chip that could be driven by firmware to produce up to 15 musical instruments, and many other additions and improvements over previous Apple II machines.



Apple ProDOS was forked into 8- and 16-bit versions to accommodate the Apple IIGS.
After using ProDOS 16 as the computer's operating system for a short time, Apple replaced it with GS/OS, a new 16-bit, native-mode system that significantly improved performance (boot time, disk access time, program launch time) and added several modern features.



GS/OS had the concept of file system translators (FST's), a generic file interface that allowed many different file systems to be read from and written to. The concept was along similar lines as AT&T's file system switch, Sun Microsystems'vnode/vfs, and DEC's gnode, that were being introduced in the mid-1980's to allow multiple file systems to co-exist. GS/OS eventually went on to have FST's for various file systems: Apple Pascal, Apple DOS 3.3, ISO/High Sierra, Macintosh HFS disks, MS-DOS, ProDOS, and AppleShare (that allowed GS/OS to access an AppleShare file server over AppleTalk).



The Finder could browse over the network, and GS/OS could even be network-booted.
The graphical control panel in GS/OS was a facility for controlling numerous system settings. Control panel "devices" (CDEV's) could be added by third-party developers.
The last version of GS/OS (4.02) shipped with Apple IIGS System 6.0.1.




A/UX



Apple came out their own version of POSIX compliant Unix, A/UX, in late 1988. The earliest A/UX was based on 4.2BSD and AT&T UNIX System V Release 2, but it would go on to derive from 4.3BSD and various subsequent System V releases.


A/UX included features such as job control, signals, networking (AppleTalk, STREAMS, TCP/IP, sockets, NFS with YP, etc.), Berkeley File System (ffs), SCCS, printing, the X Window System, compatibility with the BSD and System V API's (in addition to POSIX), compilers such as 'cc' and 'f77', and so on.


More interestingly, A/UX combined various features of the Macintosh operating system with Unix. A/UX 2.x used System 6, while A/UX 3.x was a combination of the above mentioned Unix environment with System 7.


The A/UX file system appeared as a disk drive icon in the System 7 Finder. It was possible to run a Macintosh application, Unix applications (command line and X Window), and even DOS applications (if you had SoftPC installed) side-by-side. You could also create "hybrid" applications that made use of the Unix system call interface as well as the Macintosh Toolbox.


Note that while the Unix processes ran with preemptive multitasking, the Macintosh MultiFinder still did cooperative multitasking. The startmac (or startmac24, depending on whether 32-bit or 24-bit addressing was in effect) application was responsible for creating a Macintosh environment under A/UX. Many aspects of this environment were customizable, including what application to run as the Finder.


A/UX was regarded as the holy grail of Unix systems by its proponents. Its installation procedure, provided you had compatible hardware, was probably one of the simplest ever (essentially one-click) for a Unix system.


From trivialities such as similar commands (appleping, for example) to the more elaborate marriage of Unix and the Macintosh environment, vestiges of insights gained through A/UX can be seen in Mac OS X today. A/UX used "/.mac/<host>/Desktop Folder" and "/.mac/<host>/Trash" as the Unix path names for directories containing items visible on the Macintosh Desktop and in the trash can, respectively. Moreover, Unix uses the '/' character to separate path components, while Macintosh file systems use ':'. Invisible translation was done while accessing or moving files from one environment to the other. Home directories for user accounts lived in /users, and so on.


The last version of A/UX, 3.1.1, was released in 1995. A/UX only ran on 68k Apple machines with an floating point unit (FPU) and a paged memory management unit (PMMU).




Seeking Power



By the beginning of the 1990's, Apple was making great efforts to overhaul its operating system. Of the three colored projects mentioned earlier, Blue would emerge as System 7.
System 7
When released in 1991, System 7 represented a gigantic leap relative to earlier Macintosh systems. Some of its features were:
MultiFinder built-in
Built-in networking (via AppleTalk) and file-sharing (via AppleShare)
Support for 32-bit memory addressing
A virtual memory (VM) implementation


The first Macintosh with an MMU was introduced in 1987: the Macintosh II. System 7 was the first to make use of it. However, the virtual memory support was only preliminary. VM features such as protected address spaces, memory mapped files, page locking, shared memory, etc. were not present.


Technologies such as AppleScript (system-level macro language for automating tasks), ColorSync (color management system), PowerTalk (collaboration and email software), QuickTime (multimedia software for viewing, copying, and pasting video, animations, images, and audio), TrueType (font technology), and WorldScript (system-level support for several languages) introduced, although there were not all bundled with the first release of System 7


System 7 had several usability improvements. Users could view and switch between running applications via a menu. The trash can had the same status as any other folder, and now had to be proactively emptied. Text could now be dragged between applications (in addition to cut-and-paste).


Aliases could be created for access to documents and applications from two or more locations. Control Panels and Extensions were organized hierarchically on disk.
Even with all these improvements, System 7 only did cooperative multi-tasking, and lacked memory protection.


Around this time, Apple's formed an alliance with IBM and Motorola that put the PowerPC on Apple's hardware roadmap. This required fundamental changes in the design of the Macintosh operating system.




AIM for POWER



The emphasis on making the semantics of a computer architecture close to those of higher level programming environments had led to very complex processors. However, the benefits of simplicity in computer architecture design was understood by people like Seymour Cray even in the early 1960's.


His CDC 6600 supercomputer, as well as the CRAY-1, were RISC machines, although the term had not been coined yet. RISC (Reduced Instruction Set Computer) does not only imply fewer instructions. RISC architectures are predominantly load-store, are register centric, usually employ fixed-format instructions, have efficient pipelining and a low clock cycles per instruction number, and so on.


Note that the line between RISC and CISC, as defined initially, has been growing fuzzier over the years.
IBM started research on the 801, a RISC architecture, in 1975, and in the early 1980's, Berkeley and Stanford began work on the RISC and MIPS projects, respectively. By 1990, there were several competing RISC architectures in the market, such as MIPS, HP Precision Architecture (PA-RISC), SPARC V8, Motorola 88k, IBM RS/6000, etc. The RS/6000 was an implementation of a second generation RISC architecture called POWER.


As part of a project code-named Jaguar, Apple had briefly tried, unsuccessfully, to use the Motorola 88k as their future RISC-based hardware platform. They turned to the PowerPC next.


In 1991 Apple, IBM, and Motorola joined forces to form the AIM Alliance with the goal of creating a Common Hardware Reference Platform (CHRP), which resulted in the PowerPC Architecture. The first PowerPC processor was the 601, which included most of the POWER instructions. PowerPC was designed as a 64-bit architecture, and can be dynamically switched between the 64- and 32-bit modes. An implementation can implement only the 32-bit subset, as was the case with the PowerPC processors that Apple used prior to the G5. PowerPC has continually been the mainstay of Apple hardware.



Mac OS for PowerPC



System 7.1.2 was the first to support the PowerPC. Porting all components of the operating system to a new architecture would have taken prohibitively long. Moreover, it was extremely important for Apple to provide a way so that "older" (68k based) applications continue to run. The system architecture devised to take care of these issues included a hardware abstraction layer (HAL) and a 68k emulator.


A nanokernel (a term sometimes used for a kernel even smaller than a microkernel) was used to "drive" the PowerPC. It executed in supervisor mode as the HAL and exported low-level interfaces for interrupt management, exception handling, memory management, etc. to the rest of the system. Only the system software and possibly debuggers could use the nanokernel API.



The 68k emulator was initialized by the nanokernel during boot-time. It only emulated a 68LC040 user-mode instruction set, without emulating the PMMU or the FPU. Exception stack frames were created as per a slightly older processor, the 68020, for better compatibility. There were a few other caveats and limitations (as compared to a real 68LC040).
A/UX did not run on this emulator, since it used the PMMU directly.


Since two instruction set architectures were in effect simultaneously, a system level component called the Mixed Mode Manager was used to handle context switches between the two types of code. Code pieces belonging to the two architectures could also call each other. While the manager was transparent to 68k code, PowerPC code was aware of it.


Note that the initial PowerPC ports of Mac OS had little native PowerPC code. Most existing applications, device drivers, system extensions, and large parts of the Toolbox and the operating system itself were not PowerPC native initially. Apple would port many of these to the PowerPC, and it was hoped that the nanokernel would form the basis for future Apple systems, possibly as a robust microkernel.




MAE



A Mixed Mode Manager was also used in a product called Macintosh Application Environment (MAE). MAE was an X Window application available for SPARC (SunOS) and HP 9000/700 (HP-UX) that provided an emulated Macintosh environment on these systems. MAE also contained a 68k emulator. The Mixed Mode Manager was used to optimize execution by translating 68k code to native instructions if possible. MAE provided an emulated System 7.x, and supported (depending on MAE version) features such as Aliases, AppleEvents, AppleTalk, NFS, QuickDraw, QuickTime, TrueType, and many more.


A similar solution was Apple's Mac-on-Unix (MAS), meant for running Macintosh applications under Unix on the PowerPC.


System 7.5 and later included OpenTransport (an implementation of STREAMS based on Mentat Portable STREAMS), QuickDraw 3D, a Java runtime, and OpenDOC (a software component architecture analogous to Microsoft's OLE). "Fancy" features such as a startup screen with a Mac OS logo and a progress bar were added.




AIX for PowerPC


A few years later in 1996, Apple would have a short-lived product called Apple Network Server. The PowerPC based server had hardware features and options like hot-swappable drive bays (using RAID), hot-swappable power supplies and fans, provision for adding several SCSI devices and PCI cards, processor on a daughterboard (for easy upgradability), an easy to replace logic board, and an LCD status display on the outside.


The Network Server came with the AIX for Apple Network Servers operating system, based on IBM's AIX, and did not support Mac OS. With AIX, Apple had an advanced operating system with features like memory protection, preemptive multiprocessing, multithreading, support for various networking protocols (including AppleTalk and AppleTalk services), RAID, Journal File System (JFS), Logical Volume Manager (LVM), and so on. The user could choose to work with the command-line interface, or could have one of the AIXwindows or Common Desktop Environment (CDE) graphical interfaces.


The Network Server line was discontinued in 1997, and Apple would not have a serious server offering until 2002, when they introduced the Xserve.





Quest for the Operating System



Microsoft's Windows 3.x had been extremely successful since its release in 1990. Their new generation system code-names "Chicago", initially intended for a 1993 release, kept slipping and would be eventually released as Windows 95. Microsoft did, however, release Windows NT in 1993. It was an advanced operating system meant for high-end client/server applications.

NT included the Win32 API, and had various important features such as a preemptive scheduler, integrated networking, subsystems for OS/2 and POSIX, virtual machines for DOS and 16-bit Windows, SMP support and a new file system called NTFS.
Apple needed an answer to Microsoft's onslaught, particularly in the face of the upcoming Windows 95.


The Pink and the Red projects would turn out to be rather unsuccessful. A number of attempts would be made to solve the "OS problem" one way or the other.


Star Trek


This was a bold effort, run jointly with Novell, to port Mac OS to run on x86 processors. A team consisting of engineers from both Apple and Novell actually succeeded in creating a very reasonable prototype in an incredibly short amount of time. The project was cancelled, however, for various reasons: Apple had already committed to the PowerPC, some people thought it would disrupt Apple's existing business model, vendor feedback was not encouraging, etc.


Many years later, Darwin, the core of Apple's much more successful Mac OS X, would run on both the PowerPC and the x86. While the Star Trek prototype showed the "Happy Mac" logo while booting up, the SystemStarter application in Darwin/x86 would say "Welcome to Macintosh".



Raptor



Raptor was in many respects the Red project. It was meant to run on any architecture, and was supposed to provide Apple with a next-generation microkernel. As the Star Trek project was being cancelled, it was considered for absorption by Raptor, which itself would die due to budgetary limitations, employee attrition, and various other reasons.




NuKernel



This was the name of a kernel project at Apple that was meant to result in a modern kernel on more than one occasion.



TalOS



Taligent was a company formed by Apple and IBM in early 1992 to continue work on the Pink project. Pink originally aimed to be an object-oriented operating system, but later morphed into an object-oriented environment called CommonPoint that ran on many modern operating systems, such as AIX, HP-UX, OS/2, and Windows 95/NT.


It was also supposed to run on Apple's NuKernel. Taligent Object Services (TalOS) was the name given to a set of lower-level technologies that would be built around Mach 3.0. TalOS was meant to be an extensible, portable operating system, with a small footprint and good performance.


TalOS was object-oriented from the kernel up, with even device drivers, network protocols etc. implemented in this fashion. Taligent's object-oriented building blocks were known as frameworks. There were frameworks for user interface, text, documents, graphics, multimedia, fonts, printing, and low-level services (drivers, network protocols).


These, along with the provided development tools explicitly strived to shift the burden of programming from application developers to application system engineers.


Note that even though other systems at that time (like NEXTSTEP) had object-oriented application frameworks, Taligent aimed to be better by building its entire programming model around objects. In NEXTSTEP, the developers who created frameworks had to map object behavior to the underlying libraries, Unix system calls, or Display PostScript, all of which were procedural. In fact, CommonPoint applications were not meant to use the host OS api at all.
In 1995, Taligent became a wholly owned subsidiary of IBM. The Pink project did not give Apple the next-generation OS it sought.



Copland


Apple made an announcement in early 1994 that they would channel more than a decade of experience into the next major release of their operating system, Mac OS 8. It was code-named Copland.


Apple had already started work on Copland a few months before, and it was expected that Copland would be Apple's real response to Windows. Apple hoped to achieve several goals, many of them long elusive, with this project:
Adopt RISC as a key foundation technology by making the system fully PowerPC native
Integrate, improve, and leverage their existing technologies such as OpenDOC QuickDraw GX (a graphics architecture for type, graphics, color, and printing), ColorSync, QuickDraw 3D, and collaboration software such as PowerTalk and PowerShare
Retain and improve the ease-of-use of the Mac OS interface, while making it multi-user and fully customizable
Extend interoperability with DOS and Windows
Make Mac OS systems the best network clients
Incorporate active assistance that works across applications and networks (in other words, make it really easy to automate a wide variety of tasks)
Release Copland as a system that can be openly licensed, and foster development of Mac OS compatible clones
In order to achieve these goals, Copland was supposed to have system features such as:
A hardware abstraction layer (HAL) that would also help 3rd party vendors in creating compatible systems
A microkernel (the NuKernel) at its core
Symmetric multiprocessing with preemptive multitasking
Improved virtual memory with protection
A flexible and powerful system extension mechanism
Critical subsystems such as I/O, networking, and file systems running as services on top of the kernel
Low-level networking facilities such as X/Open Transport Interface (OTI), System V STREAMS, and Data Link Provider Interface (DLPI)



Work on Copland picked up speed at the beginning of the 1990s, and by mid-1990s, Copland was heavily counted on by everybody to do wonders for Apple. However, the project kept slipping. A few Driver Development Kit (DDK) releases, essentially prototypes, went out, but a 1996 release (as planned and hoped) did not seem feasible. Due to various pressures, full memory protection had not been included after all.


Apple's then CEO Gil Amelio described the state of Copland as "... just a collection of separate pieces, each being worked on by a different team ... that were expected to magically come together somehow ..."



Apple finally decided to cancel Copland in May 1996. Amelio announced that Copland's best pieces would be shipped with future releases of their existing system, beginning with the upcoming System 7.6, whose name was formally changed to Mac OS 7.6.




Gershwin



After the Copland debacle, Apple's need for a new operating system was more dire than ever. Focus shifted briefly to the "effort" named Gershwin, which was to include the painfully elusive memory protection, among other things. However, it was apparently nothing more than a code-name, and it is believed that nobody ever worked on Gershwin.


BeOS


Apple briefly considered partnering with Microsoft to create an Apple OS based on Windows NT. They also considered Solaris from Sun Microsystems, and BeOS from Be.
The acquisition of Be actually came close to materializing.


Be had been founded by Jean-Louis Gassée, Apple's ex-head of Product Development, and its capable engineering team had created an impressive operating system.


It had memory protection (!), preemptive multi-tasking, symmetric multi-processing, and it even ran on the PowerPC (later ported to the x86), thus fitting with Apple's hardware agenda. BeOS was designed to be very good at handling multimedia. It even had a meta-data rich file system, allowing for files to be accessed via numerous attributes. However, BeOS was an unfinished, unproven product. For example, it did not have support for file sharing or printing. Not many applications had been written for it.


Gassée was rather confident that Apple would buy Be, and he wanted over $500 million. The total investment in Be at that time was estimated to be only $20 million, and Apple valued Be at about $50 million. Apple negotiated up to $125 million, and Be negotiated down to $300 million. When things still did not work out, Apple offered $200 million, and even though it is rumored that Gassée was actually willing to accept this offer, it is also believed that he came back with a "final price" of $275 million, hoping Apple will bite the bullet. The deal did not happen.


There was another contender alongside Be: NeXT, whose operating systems had at least been proven in the market, unlike BeOS. Despite NeXT not having any resounding successes, OPENSTEP was well received in the enterprise market. Steve Jobs pitched NeXT's technology very strongly to Apple, and asserted that OPENSTEP was many years ahead of the market.



Plan A


The deal with NeXT did work out. Apple acquired them in February 1997 for over $400 million. Amelio later quipped that "We choose plan A instead of Plan Be".
The next section discusses the background of NeXT's systems.



The NeXT Chapter


All of Steve Jobs' operational responsibilities at Apple were "taken away" on May 31, 1985. Around this time, Jobs had come up with an idea for a startup for which he pulled in five other Apple employees. The idea was to create the perfect research computer for universities, colleges, and research labs.

Jobs had earlier met up with Nobel laureate biochemist Paul Berg, whose reaction to Jobs' idea of using a computer for various simulations was apparently positive. Although Apple was interested in investing in Jobs' startup, they were outraged, and actually sued Jobs later in 1985 when they learnt about the five Apple employees joining him. Apple dropped the suit early next year after some mutual agreements. The startup was NeXT Computer, Inc.


NeXT's beginnings were promising. Jobs put in $7 million of his own money. A number of larger investments would be made in NeXT, such as $20 million from Ross Perot, and $100 million from Canon a few years later.

NeXT strived to create a computer that would be perfect in form and function. The motherboard had a clever, visually appealing design. The magnesium case of the cube was painted black with a matte finish.

The monitor stand required a great deal of engineering as well. An onboard digital signal processing chip allowed the cube to play stereo quality music. These machines were manufactured in NeXT's own state-of-the-art factory.




NEXTSTEP and OPENSTEP


Jobs unveiled the NeXT cube on October 12, 1988, at the Davies Symphony Hall in San Francisco. The operating system was called NEXTSTEP, and used a port of CMU Mach 2.0 (with a 4.3BSD environment) as its kernel.


Its window server was based on Display Postscript, a marriage of page-description language and window system technologies (Sun Microsystems had announced NeWS, their own Display Postscript Window System, earlier in 1986).


The Mach port used in NEXTSTEP also included NeXT specific features, as well as features from later versions of CMU Mach.


NEXTSTEP used Objective-C as its native programming language, and included Interface Builder, a tool for designing application user interfaces graphically. A number of "software kits" (collections of reusable classes, or object templates) were provided to aid in application development, such as: Application Kit, Music Kit, and Sound Kit.



Objective-C


Objective-C is an object-oriented, compiled programming language invented by Brad Cox in the early 1980's. It is an object-oriented superset of C, with dynamic binding and a messaging syntax inspired by Smalltalk.

It is meant to be a simpler language than C++, and does not have many features of C++, such as multiple inheritance and operator overloading.


Brad Cox founded StepStone Corporation, from which NeXT licensed the language and created their own compiler. In 1995, NeXT acquired all rights to StepStone's Objective-C related intellectual property.



Apple's Objective-C compiler used in Mac OS X is a modified version of the GNU compiler.
At the time of the cube's announcement, NEXTSTEP was at version 0.8, and it would be another year before a 1.0 mature release would be made.


NEXTSTEP 2.0 was released exactly a year later after 1.0, with improvements such as support for CD-ROM's, color monitors, NFS, on-the-fly spell checking, dynamically loadable device drivers, and so on.


In the fall of 1990, the first web browser (offering WYSIWYG browsing and authoring) was created at CERN by Tim Berners-Lee on a NeXT computer. Tim's collaborator, Robert Cailliau, later went on to say that "...

Tim's prototype implementation on NEXTSTEP is made in the space of a few months, thanks to the qualities of the NEXTSTEP software development system ..."

In the 1992 NeXTWORLD Expo, NEXTSTEP 486, a $995 version for the x86 was announced.



The last version of NEXTSTEP, 3.3, would be released in February, 1995, by which time NEXTSTEP had very powerful application development facilities, thanks to tools like Project Builder, Interface Builder, and others.


There was an extensive collection of libraries, such as for user interfaces, databases, distributed objects, multimedia, networking, etc. It also had an object-oriented tool kit (Driver Kit) for writing device drivers.

NEXTSTEP ran on the 68k, x86, PA-RISC, and SPARC platforms, and you could create a single version of your application containing binaries for each supported architecture. Such multiple architecture binaries are known as "fat" binaries.


Despite the virtues of NEXTSTEP and the elegance of its hardware, NeXT had proven to be economically unviable over the years. NeXT announced in early 1993 that it was getting out of the hardware business, and would continue development of NEXTSTEP for x86.


NeXT partnered with Sun Microsystems to jointly release specifications for OpenStep, an open platform comprised of several API's and frameworks that anybody could use to create their own implementation of an object-oriented operating system, running on any underlying core operating system.

The OpenStep API was implemented on SunOS, HP-UX, and Windows NT. NeXT's own implementation, essentially an OpenStep compliant version of NEXTSTEP, was released as OPENSTEP 4.0 in July, 1996, with 4.1 and 4.2 to follow shortly afterwards.


The OpenStep API and the OPENSTEP operating system did not seem to turn things around for NeXT, even though they caused some excitement in the business, enterprise, and government markets.

NeXT started to focus more on their WebObjects product, a multi-platform environment for rapidly building and deploying web based applications.


As described earlier, NeXT was purchased by Apple in early 1997. Mac OS X would be based on NeXT's technology, while WebObjects would keep up with advancements in its domain (such as its support Web Services and Enterprise Java). Apple's web sites, for example its online store and the .Mac offering, are built using WebObjects.


What NeXT? The Mach Factor



Along with NeXT's operating system came the kernel: a version of Mach, which became the kernel foundation of Apple's future systems. It would be instructive to briefly discuss the origins and evolution of Mach.



Rochester's Intelligent Gateway



A group of researchers at the University of Rochester, New York, began development of an "intelligent" gateway system named RIG (Rochester's Intelligent Gateway) in 1975. RIG was meant to provide uniform access (via terminals, for example) to a variety of computing facilities, both local and remote.


Local facilities could be locally connected disks, magnetic tapes, printers, plotters, batch processing or time-sharing computers, and so on. Remote facilities could be available through a network like the ARPANET. RIG's operating system, called "Aleph", ran on a Data General Eclipse minicomputer.



The Aleph kernel was structured around an interprocess communication (IPC) facility. RIG processes could send messages to each other, where the destination was specified by a process number and a port number. A port was an in-kernel message queue identified by the same { process #, port # } tuple.



A process could have several ports defined within itself, each of which could be used to wait for a message to arrive on. A process X could "shadow" another process Y (X receives a copy of every message sent to Y), or X could "interpose" Y (X intercepts all messages sent to, or originating from Y).



RIG was killed a few years later due to several fundamental shortcomings in its design: a 2KB limit on the size of a message (because of the limited address space provided by the underlying hardware) and the resulting IPC inefficiency, no protection for the ports (port numbers were global, and any process could create and use them - thus, any process could send a message to any other process), no way to notify the failing of a process to another process that depends on it (without explicit registration of such dependencies), etc.



Accent


One of the people who worked on RIG was Richard Rashid. In 1979, he moved to Carnegie Mellon University, where he worked on Accent, a network operating system kernel. Like RIG, Accent also used IPC as the basic system structuring tool.


However, Accent addressed RIG's shortcomings: ports now had "capabilities", and copy-on-write memory mapping was used to facilitate large message transfers.


Messages could be sent to processes on another machine through an intermediary process. Accent had flexible and powerful virtual memory management, which was integrated with IPC and file storage.



Accent was developed to support two distributed computing projects, Spice (distributed personal computing) and DSN (a fault-tolerant distributed sensor network). Accent was the name of a product (a spice) sold by Accent Inc. The only ingredient of this product was monosodium glutamate (MSG).


In computing, one often abbreviates "message" as "msg".
Within a few years, the future of Accent did not look promising as well. It needed a new hardware base, including support for multiprocessors and portability to various hardware. Accent also had difficulty supporting Unix software.




Mach


The sequel to Accent was called Mach. It was designed with goals such as:
To support multiprocessing
To exploit other features of modern hardware architectures that were emerging then
To reduce the number of features in the kernel, and therefore make it less complex, and give the programmer a very small number of abstractions to work with
To have full compatibility with UNIX
To address the shortcomings of previous systems such as Accent
When Mach was developed, UNIX had been around for over fifteen years.


Mach's implementation used 4.3BSD as the starting code base. Mach's designers had RIG and Accent to refer to when it came to creating message-passing kernels. DEC's Tenex Operating System provided some ideas for Mach's virtual memory subsystem.



When it was published in 1986, the original Mach paper hailed it as "A New Kernel Foundation For UNIX Development". While not everybody saw or sees it that way, Mach went on to become a rather successful system. As far as Apple is concerned, the paper's title might as well have been "A NuKernel Foundation ..."



Richard Rashid recounts that after a series of unsuccessful naming attempts, he came up with the name MUCK (Multiprocessor Universal Communication Kernel). One of his colleagues, Darlo Giuse, who was Italian, inadvertently pronounced MUCK as "Mach", and the name stuck.


Initially the Mach designers presented four basic abstractions in the kernel:
A Task is a container for the resources (virtual memory, ports, processors, ...) of one or more thread.


A Thread is a basic unit of execution. One task may have several threads. The task provides an execution environment, while the threads actually run. The various threads of a task share its resources, although each thread has its own execution state (program counter and other registers, etc.)


A Port is similar to ports in Accent. It is an in-kernel message queue with capabilities, and forms the basis for interprocess communication in Mach.


A Message is a collection of data that threads in two different tasks, or the same task, can send to each other using ports.


Another basic Mach abstraction is a memory object. A memory object can be thought of as a container for data (including file data) that is mapped into the address space of a task.
Mach required a paged memory management unit (PMMU), and provided an excellent interface to the machine dependent MMU facilities via its pmap interface.


The VM subsystem was designed to support large, sparse virtual address spaces, and was integrated with IPC. Unlike traditional UNIX, which implied a contiguous virtual memory space (with the heap and the stack growing towards each other), Mach allowed for sparse address spaces, where regions of memory could be allocated from anywhere in the address space.


Memory could be shared read/write, in a structured manner, and copy-on-write (COW) was used both for sharing memory between tasks as well as to optimize copy operations. The generalized memory object abstraction allowed for "external" (out-of-kernel, user-space) memory pagers (to handle page faults and page-out data requests), that could reside even on another machine.



An important decision of CMU was to provide all Mach software free of distribution fees or royalties, with unrestrictive licensing.


Mach did not provide (or mean to provide) any I/O, networking, or file system capabilities by itself. It was meant as an operating system to create operating systems from. It was hoped that this would keep things simple, and promote portability of operating systems.


The real operating system was supposed to run as a user-level Mach task. Release 2.0 of Mach, as well as the rather successful Release 2.5, had monolithic implementations - BSD and Mach resided in the same address space.


Release 2.5 was used by the Open Software Foundation (OSF) as a kernel basis for their OSF/1 operating system. 2.x was also the basis of Mt. Xinu, NEXTSTEP, OPENSTEP, Omron LUNA/88k, Multimax (Encore) and so on.


The Mach 3 effort was started at CMU and continued by OSF. This was the first "true microkernel" version in the sense that BSD ran as a user space Mach task. Other changes/improvements included:
Real-time support (kernel preemption and a real-time scheduling framework)
Low-level device support (devices presented as port to which data or control messages could be sent, with support for both synchronous and asynchronous I/O)
A completely rewritten IPC implementation
System call redirection (a set of system calls could be handled by code in user-space, running within the calling task)
Use of continuations, a kernel facility that gives a thread the option to block by specifying a function (the continuation function) that would be called when the thread runs
Richard Rashid went on to become the head of Research at Microsoft, while another co-author, Avie Tevanian, went on to become the Chief Software Technology Officer at Apple.



MkLinux


Apple and OSF began a project to port Linux to a variety of Power Macintosh platforms running on top of OSF's implementation of Mach. This led to "osfmk", and the overall system was MkLinux.

The first release of this "Linux hosted on Mach" system, based on Mach and Linux 1.3.x, came out as MkLinux DR1 in early 1996. Later releases moved to Linux 2.0.x and beyond. One of the releases was incorporated into Apple's Reference Release.

Note that MkLinux uses a single-server approach: the monolithic Linux kernel runs as a single Mach task.



Mac OS X uses osfmk as base (with BSD running as part of the kernel itself), and includes many MkLinux enhancements.

OSF first became Open Group, and then Silicomp.



Musical Names


Apple's OS strategy after acquiring NeXT was two-pronged: they would keep improving Mac OS for the consumer desktop market, and would create a high-end operating system called Rhapsody based on NeXT technology. Rhapsody would mainly be targeted towards the server and enterprise markets.

In contrast to the chromatic aberrations such as Pink and Red, Apple also had a string of musically inspired code-names for their operating system projects. Copland and Gerswhin were named after Aaron Copland and George Gerswhin, both American composers. Rhapsody in Blue is a famous work of Gershwin.




Strategies



The first release of Mac OS after NeXT's purchase was 7.6. The version originally meant to be released as 7.7 became Mac OS 8.0.
Mac OS 8 and 9


Over the years, some important features that were either invented or improved for Copland were added to Mac OS 8.x/9.x as originally intended, such as:
Mac OS Extended (HFS+) file system
A search engine that could search on local drives, network servers, and the Internet (released as Sherlock)


Copland API (evolved into Carbon gradually)


The Platinum-look user interface
Multiple users with their own preferences
Mac OS 8 had a multi-threaded Finder that allowed several file-oriented operations simultaneously, contextual menus activated by a control-click, personal web hosting, and important enhancements to power-management, USB, and FireWire. Microsoft Internet Explorer and Netscape Navigator were also bundled.


Apple's implementation of the Java runtime environment, the the Macintosh Runtime for Java (MRJ), was part of the system. Version 8.5 was PowerPC only. The nanokernel was overhauled in 8.6 to integrate multi-tasking and multi-processing. It included a preemption-safe memory allocator. The multiprocessor API library could now run with virtual memory enabled.


Mac OS 9 came out in 1999, and was hailed by Apple as the "best Internet operating system ever". It was the first Mac OS version that could be updated over the Internet. It included useful security features such as file encryption and the "Keychain" mechanism for storing passwords. It could also use the AppleTalk protocol over TCP/IP.
An important part of Mac OS 9 was a mature installation of the Carbon API's, which at the time represented about 70 percent of the legacy Mac OS API's, and provided compatibility with Mac OS 8.1 and later.

The last release of this "old" Mac OS (later referred to as "Classic") was 9.2.2, released in late 2001.








Rhapsody




Rhapsody was first demonstrated at the 1997 World Wide Developers Conference (WWDC). It was based on OPENSTEP, and consisted of the following primary components:


A kernel and related subsystems based on Mach and BSD
An extended OpenStep API implementation, called the Yellow Box
A Java virtual machine
A Mac OS compatibility subsystem called the Blue Box
A Mac OS like user-interface, with some features from OPENSTEP
There were plans to port most of the Mac OS frameworks to Rhapsody: QuickTime, QuickDraw 3D, QuickDraw GX, ColorSync, etc. Rhapsody was also to support a number of file systems such as Apple Filing Protocol (AFP), FAT, HFS, HFS+, ISO9660, and UFS.
There were two developer releases of Rhapsody, dubbed DR1 and DR2. These were released both for the PowerPC and x86 platforms.




Blue Box



Apple extended the PowerPC DR1 with a Mac OS compatibility environment called Blue Box shortly afterwards. Implemented by a Rhapsody application (MacOS.app), Blue Box was a virtual environment that appeared as a new Macintosh hardware model.


The application loaded a Macintosh ROM file from disk, and Mac OS was run mostly unchanged within this environment. Blue Box initially ran Mac OS 8.x, full-screen, with the ability to switch between Rhapsody and Mac OS. It could only support applications that did not access the hardware directly, did not use internal undocumented Mac OS API's, etc. Support for newer versions of Mac OS, as well as for running the Blue Box windowed, was added later.



Yellow Box



Rhapsody's development platform was called the Yellow Box. It was hosted on Rhapsody for Power Macintosh, Rhapsody for x86, and was also available independently for Microsoft Windows.



Yellow Box included most of OPENSTEP's integrated frameworks (shared object libraries), augmented by a run-time and development environment. There were three core object frameworks, with API's available in Objective-C and Java:
Foundation: a collection of "base" classes providing API's for allocating, deallocating, examining, storing, notifying, and distributing objects.


Application Kit: a set of API's for creating user-interfaces, managing and processing events, and using services such as color and font management, printing, cut-and-paste, text-manipulation, etc.


Display PostScript: a set of API's for drawing in PostScript, compositing images, etc. It could also be considered as a part of Application Kit.

Yellow Box included Project Builder, an integrated development environment (IDE) and Interface Builder, a visual graphical interface creation tool.

The Windows NT implementation of Yellow Box provided the same environment, through a combination of Apple provided Windows system services (machd, the Mach emulation daemon, and nmserver, the Netname Server) and application programs (WindowServer and pbs, the pasteboard server).

Earlier implementations of the OpenStep API for other platforms (such as Solaris) used a similar architecture.


Towards Mac OS X


After Rhapsody's DR2 release, Apple would still alter its operating system strategy, but would finally be on its way towards achieving its goal of having a "new" system. It would further take three years to reach that goal however. We shall refer to a pictorial approximation of the progression towards Mac OS X in the discussion that follows.



Core Services



The Core Services layer can be visualized as sitting atop the kernel. This layer's most important sub-components are CoreFoundation.framework and CoreServices.framework.


It contains various critical non-GUI system services (including APIs for managing threads and processes, resource, virtual memory and filesystem interaction):
CarbonCore: Core parts of Carbon, such as various Carbon managers. Carbon has traditionally been a very critical Mac OS API family, and is so in Mac OS X as well.
CFNetwork: An API for user-level networking that includes several protocols such as FTP, HTTP, LDAP, SMTP, ...


OSServices: A framework that includes various system APIs (accessing disk partitions, the system keychain, Open Transport, sound, power, etc.)
SearchKit: A framework for indexing and searching text in multiple languages.
WebServicesCore: APIs for using Web Services via SOAP and XML-RPC.


CoreFoundation also includes a large number of other services. For example, it provides ways so that applications can access URLs, parse XML, maintain property lists, etc. In the directory /System/Library/Frameworks/, refer to the directory CoreFoundation.framework/Headers/ for headers belonging to CoreFoundation.




Application Services


This layer can be visualized as being on top of Core Services. It includes services that make up the graphics and windowing environment of Mac OS X.


The core of the windowing environment is called Quartz. Quartz consists of broadly two entities:
Quartz Compositor: consists of the window server (the WindowServer program) and some private libraries. Quartz implements a layered compositing engine, in which every pixel on a screen can be shared between different windows in real time.
Quartz 2D: a 2D graphics rendering library.



While NeXT used "Display PostScript" (DPS) for the imaging model in NEXTSTEP, Quartz uses PDF for its drawing model (or as its native format). This makes possible some useful features, such as automatic generation of PDF files (you can save a screenshot "directly" to PDF), import of PDF data into native applications, rasterization of PDF data (including PostScript and EPS conversion), etc.



There are Python bindings to the Quartz PDF engine. Note however, that Quartz's PDF support is not a replacement for, say, Adobe's professional level PDF tools.
Quartz also has an integrated hardware acceleration layer called Quartz Extreme that automatically becomes active on supported hardware.



The graphics environment also has other rendering libraries, for example, OpenGL (2D and 3D), QuickDraw (2D) and QuickTime.


The Application Services layer also includes various other component frameworks:
Apple Events: a mechanism for inter-application communication.


Apple Type Services: a framework for managing and accessing fonts on Mac OS X.
ColorSync: Apple's color management system that's integrated with Mac OS X.
CoreGraphics: a graphics framework that's used for implementing Quartz.
FindByContent: an API that allows searching specific volumes or folders for files containing the specified content.



HIServices: framework providing human-interface related services, such as for icon management, translation, "pasteboard" (clipboard) etc.
LangAnalysis: an API to the Language Analysis Manager, allows for analyzing morphemes (a morpheme is a meaningful linguistic unit, that is, a distinctive collocation of phonemes, having no smaller meaningful parts) in text.



LaunchServices: a medium-level interface to starting applications, including opening documents with either a specified or the default application, open URLs, etc.
PrintCore: an API to the printing subsystem.
QD: a 2D graphics engine.



SpeechSynthesis: an API for generating synthesized speech.
QuickTime is both a graphics environment and an application environment. It has excellent features for interactive multimedia that allow for manipulating, streaming, storing and enhancing video, sound, animation, graphics, text, music, and VR.




Application Environments




There are multiple execution environments on Mac OS X within which respective applications execute:
BSD: This application environment is similar to a traditional *BSD system and provides a BSD-based POSIX API. It consists of a BSD runtime and execution environment. Mac OS X uses FreeBSD as a reference code base for its BSD derivations (Panther derives from FreeBSD 5.0). The libraries and headers for this environment reside in their traditional location (/usr/lib and /usr/include, respectively).



Carbon: This is a set of procedural C-based APIs for Mac OS X that are based on the "old" Mac OS 9 APIs. Note that Carbon does not include all the old APIs - a subset of the old APIs has been modified to work with OS X. Some APIs have been dropped as they are not applicable any more because of the radical differences between Mac OS X and Mac OS 9.



The fact that Mac OS X includes APIs and abstractions from so many different systems (Mach, *BSD, Mac OS 9, etc.) makes things rather confusing and messy sometimes. Consider that Mach uses tasks (that contain one or more threads), FreeBSD uses processes (with a proc structure, pid, etc.) while Carbon uses its own notion of processes in the Carbon Process Manager, with process serial numbers (PSNs) which are not the same as a BSD pid!



If a process is running under the Classic emulator, then multiple Carbon Process Manager processes inside Classic are using one BSD process. Consider the following excerpt from the output of the ps command:
% ps -axw | grep Safari 345 ?? S 4:18.19 /Applications/Safari.app/Contents/\ MacOS/Safari -psn_0_917505

Safari is linked against both the Carbon and Cocoa frameworks, among others. The above output means that Unix process id 345 maps to Carbon Process Manager PSN 917505.
You can use the Carbon function GetProcessForPID(pid_t, ProcessSerialNumber *) to obtain the PSN for a process given its Unix pid (note that not all processes will have both).


Classic: This is a compatibility environment so that Mac OS 9 applications can be run on Mac OS X. The Classic emulator runs in a protected memory environment, with multiple processes in Mac OS 9 layered on top of one BSD process.



Cocoa: This is an object-oriented API for developing applications written in Objective-C and Java. Cocoa is an important inheritance from NEXTSTEP (a fact testified by the various NS* names in its API).


It is very well supported by Apple's rapid development tools, and is the preferred way of doing things on Mac OS X if what you want to do can be done through Cocoa. There are many parts of Mac OS X that have not "converted" to Cocoa completely, or at all.


A Cocoa application can call the Carbon API. Cocoa is largely based on the OpenStep frameworks, and consists of primarily two parts: the Foundation (fundamental classes) and the Application Kit (classes for GUI elements).



Java: This environment consists of a JDK, both command-line and integrated with Apple's IDE, a runtime (Hotspot VM, JIT), and various Java classes (AWT, Swing, ...).
Cocoa includes Java packages that let you create a Cocoa application using Java as the programming language. Moreover, Java programs can call Carbon and other frameworks via JNI.

Finally, although Java is considered an Application Environment, the Java subsystem can itself be represented as different layers above the kernel. The Java Virtual Machine along with core JDK packages is analogous to the Core Services layer, and so on (see picture).



Filesystem Hierarchy



Although Mac OS X has many directories similar to a traditional *nix system, such as /etc (a symbolic link to /private/etc, /usr, /tmp (a symbolic link to /private/tmp, etc., it has many others that are unique to it, for example:

/ root, the Local domain
/Applications GUI applications
/Applications/Utilities GUI utilities
/Developer Xcode developer tools
/Library User installed frameworks
/Network the Network domain
/System the System domain
/System/Library System frameworks
/Users Home directories, the User domain
~/Desktop Your desktop
~/Documents Your documents
~/Movies Your movies directory (iMovie uses it by default)
~/Music Your music directory (iTunes uses it by default)
~/Pictures Your pictures directory (iPhoto uses it by default)
~/Public For sharing (contents visible to others)
~/Sites Your www (apache) directory



What is Mac OS X?




Mac OS X System Startup


This page briefly describes the sequence of events that happen when Mac OS X boots.
Some details of the boot process from power-on until the kernel is up and running are covered in Booting Mac OS X and XNU: The Kernel. To recapitulate:
Power is turned on.


Open Firmware code is executed.


Hardware information is collected and hardware is initialized.
Something (usually the OS, but also things like the Apple Hardware Test, etc.) is selected to boot. The user may be prompted to select what to boot.


Control passes to /System/Library/CoreServices/BootX, the boot loader. BootX loads the kernel and also draws the OS badges, if any.



BootX tries to load a previously cached list of device drivers (created/updated by /usr/sbin/kextcache). Such a cache is of the type mkext and contains the info dictionaries and binary files for multiple kernel extensions. Note that if the mkext cache is corrupt or missing, BootX would look in /System/Library/Extensions for extensions that are needed in the current scenario (as determined by the value of the OSBundleRequired property in the Info.plist file of the extension's bundle.


The init routine of the kernel is executed. The root device of the booting system is determined. At this point, Open Firmware is not accessible any more.
Various Mach/BSD data structures are initialized by the kernel.
The I/O Kit is initialized.



The kernel starts /sbin/mach_init, the Mach service naming (bootstrap) daemon. mach_init maintains mappings between service names and the Mach ports that provide access to those services.


From here on, the startup becomes user-level:
mach_init starts /sbin/init, the traditional BSD init process. init determines the runlevel, and runs /etc/rc.boot, which sets up the machine enough to run single-user.


During its execution, rc.boot and the other rc scripts source /etc/rc.common, a shell script containing utility functions, such as CheckForNetwork() (checks if the network is up), GetPID(), purgedir() (deletes directory contents only, not the structure), etc.


rc.boot figures out the type of boot (Multi-User, Safe, CD-ROM, Network etc.). In case of a network boot (the sysctl variable kern.netboot will be set to 1 in which case), it runs /etc/rc.netboot with a start argument.


/etc/rc.netboot handles various aspects of network booting. For example, it performs network and (if any) local mounts. It also calls /usr/bin/nbst to associate a shadow file with the disk image being used as the root device. The idea is to redirect writes to the shadow file, which hopefully is on local storage.


rc.boot figures out if a file system consistency check is required. Single-user and CD-ROM boots do not run fsck. SafeBoot always runs fsck. rc.boot handles the return status of fsck as well.


If rc.boot exits successfully, /etc/rc, the multi-user startup script is then run. If booting from a CD-ROM, the script switches over to /etc/rc.cdrom (installation).
/etc/rc mounts local file systems (HFS+, HFS, UFS, /dev/fd, /.vol), ensures that the directory /private/var/tmp exists, and runs /etc/rc.installer_cleanup, if one exists (left by an installer before reboot).


/etc/rc.cleanup is run. It "cleans" a number of Unix and Mac specific directories/files.
BootCache is started.

Various sysctl variables are set (such as for maximum number of vnodes, System V IPC, etc.). If /etc/sysctl.conf exists (plus /etc/sysctl-macosxserver.conf on Mac OS X Server), it is read and sysctl variables contained therein are set.
syslogd is started.


The Mach symbol file is created.
/etc/rc starts kextd, the daemon process that loads kernel extension on demand from kernel or client processes.


/usr/libexec/register_mach_bootstrap_servers is run to load various Mach bootstrap based services contained in /etc/mach_init.d
portmap and netinfo are started.

If /System/Library/Extensions.mkext is older than /System/Library/Extensions, /etc/rc deletes the existing mkext and creates a new one. It also creates one if one doesn't exist.
/etc/rc starts /usr/sbin/update, the daemon that flushes internal file system caches to disk frequently.


/etc/rc starts the virtual memory system. /private/var/vm is set up as the swap directory. /sbin/dynamic_pager is started with the appropriate arguments (swap filename path template, size of swap files created, high and low water alert triggers specifying when to create additional swap files or delete existing ones).


/etc/rc starts /usr/libexec/fix_prebinding to fix incorrectly prebound binaries.
/etc/rc executes /etc/rc.cleanup to clean up and reset files and devices.


/etc/rc finally launches /sbin/SystemStarter to handle startup items from locations such as /System/Library/StartupItems and /Library/StartupItems.

A StartupItem is a program, usually a shell script, whose name matches the folder name.


The folder contains a property list file containing key-value pairs such as Description, Provides, Requires, OrderPreference, start/stop messages etc.


You can run SystemStarter -n -D as root to have the program print debugging and dependency information (without actually running anything).
The CoreGraphics startup item starts the Apple Type Services daemon (ATSServer) as well as the Window Server (WindowServer).


By default, the loginwindow application (loginwindow.app under /System/Library/CoreServices) is executed for the console device. You can change this line in /etc/ttys if you do not want a graphical login.




What is Mac OS X?



Mac OS X Filesystems




Like most modern day operating system implementations, Mac OS X uses an object-oriented vnode layer. xnu's VFS layer is based on FreeBSD's, although there are numerous minor differences (for example, while FreeBSD uses mutexes, xnu uses simple locks; XNU's unified buffer cache is integrated with Mach's virtual memory layer, and so on).





Local Filesystems



HFS
HFS (Hierarchical File System) was the primary filesystem format used on the Macintosh Plus and later models, until Mac OS 8.1, when HFS was replaced by HFS Plus.
This section briefly describes the various filesystems supported by "stock" Mac OS X.
HFS+
HFS+ is the preferred filesystem on Mac OS X. It supports journaling, quotas, byte-range locking, Finder information in metadata, multiple encodings, hard and symbolic links, aliases, support for hiding file extensions on a per-file basis, etc. HFS+ uses B-Trees heavily for many of its internals.



Like most current journaling filesystems, HFS+ only journals meta-data. Journaling support was retrofitted into HFS+ via a simple VFS journaling layer in XNU that's actually filesystem independent.


The journal files on an HFS+ volume are called .journal and .journal_info_block (type jrnl and creator code hfs+). HFS+, although not a cutting-edge filesystem, supports some unique features and has worked well for Apple.


Similar to HFS
HFS+ is architecturally similar to HFS, with several important improvements such as:
32 bits used for allocation blocks (instead of 16). HFS divides the disk space on a partition into equal-sized allocation-blocks.


Since 16 bits are used to refer to an allocation-block, there can be at most 216 allocation blocks on an HFS filesystem. Thus, using 32 bits for identifying allocation blocks results in much less wasted space (and more files).



Long file names up to 255 characters
Unicode based file name encoding
File/Directory attributes can be extended in future (as opposed to being fixed size)
In addition to a System Folder ID (for starting Apple operating systems), a dedicated startup file that can easily be found (its location and size are stored in the volume header in a fixed location) during startup, is also supported so that non-Apple systems can boot from a HFS+ filesystem
Largest file size is 263 bytes
Aliases

Aliases are similar to symbolic links in the sense that it allows multiple references to a file or directory. However, if you move the target (without replacing it), a symlink would break, while an alias would not.

This is possible because under HFS+, each file/directory has a unique, persistent identity, which is stored along with the pathname.

If one of the two (pathname or unique identity) is wrong (the file cannot be found using it), the alias updates it with the "right" one (using which the file could be found).

This feature is the reason why you can keep moving applications to different places on your disk without having to worry about breaking their Dock "shortcuts".

In order to make use of aliases, an application must use either Carbon or Cocoa APIs, since this feature is not available through the POSIX API.




Optimizations



HFS+ also has a few specific optimizations. When a file is opened on an HFS+ volume, the following conditions are tested:
The file is less than 20 MB in size
The file is not already busy
The file is not read only
The file is fragmented (the eighth extent descriptor in its extend record has a non-zero block count)


The system uptime is at least 3 minutes
If all the above are satisfied, the file is relocated (de-fragmented) - on-the-fly.
Another optimization is "Hot File Clustering".

This is a multi-staged (the stages being DISABLED, IDLE, BUSY, RECORDING, EVALUATION, EVICTION and ADOPTION) clustering scheme that records "hot" files (except journal files, and ideally quota files) on a volume, and moves "hot" files to the "hot" space on the disk (0.5% of the total filesystem size located at the end of the default metadata zone - at the start of the volume).


The scheme uses an on-disk B-Tree file for tracking (/.hotfiles.btree on a volume):
# ls -l /.hotfiles.btree -rw------- 1 root admin 196608 17 Dec 10:09 /.hotfiles.btree
At most 5000 files, and only files less than 10 MB in size are "adopted" under this scheme.








Multiple Forks



HFS+/HFS files have two "forks" - traditionally called the data and resource forks, either of which may be empty.

Historically, a resource fork has been used for various things such as custom icons, preferences, license information, etc. As expected, this is incompatible with traditional Unix filesystems, and care must be taken while moving files across filesystems.


The article Command Line Archival in Mac OS X takes a brief look at the issue and some solutions. From a BSD command line, a file's resource fork can be accessed thus:
% ls -l Icon 128 -rwxrwx--- 1 amit amit 0 11 Jun 2003 Icon # this means the data fork is empty % ls -l Icon/rsrc 128 -rwxrwx--- 1 amit amit 65535 11 Jun 2003 Icon/rsrc # the resource fork has is 65535 bytes

Thus, unlike aliases, multiple forks can be accesses via the POSIX API.





Device Driver Partitions




Although this is not related to HFS+, Mac OS X can load block device drivers from various places: the ROM, a USB or FireWire device and special partitions on a fixed disk. In order to support multiple operating systems or other features, a disk can have more than one device driver installed, each in its own partition.



For example, viewing your disk drive's information in /Applications/Utilities/Disk Utility.app might tell you that Mac OS 9 drivers are installed on this disk. A "standard" partition scheme on the PowerBook might look like the following:
# pdisk /dev/rdisk0 -dump /dev/rdisk0 map block size=512 #: type name length base ( size ) 1: Apple_partition_map Apple 63 @ 1 2: Apple_Driver43*Macintosh 56 @ 64 3: Apple_Driver43*Macintosh 56 @ 120 4: Apple_Driver_ATA*Macintosh 56 @ 176 5: Apple_Driver_ATA*Macintosh 56 @ 232 6: Apple_FWDriver Macintosh 512 @ 288 7: Apple_Driver_IOKit Macintosh 512 @ 800 8: Apple_Patches Patch Partition 512 @ 1312 9: Apple_HFS Untitled 156299656 @ 1824 ( 74.5G) 0: Apple_Free 0+@ 156301480
In the above dump, the Apple_partition_map is a meta-data partition describing the partitions on the disk.


The patch partition is a meta-data partition containing patches that must be applied to the system before it can boot. The various "Driver" partitions contain drivers (Apple_Driver43 contains SCSI Manager 4.3, for example).


Insensitive!


Note that HFS+ is a case preserving, case insensitive filesystem, which can be rather jarring in situations such as the following:
# tar -tf freebsd.tar FreeBSD.txt freebsd.txt # The tar file contains two files # tar -xvf freebsd.tar FreeBSD.txt freebsd.txt # ls *.txt freebsd.txt
The Apple Technical Note titled HFS Plus Volume Format describes HFS+ internals in great detail.



ISO9660
ISO9660 is a system-independent file system for read-only data CDs. Apple has its own set of ISO9660 extensions. Moreover, you would likely run into Mac HFS/ISO9660 hybrid discs that contain both a valid HFS and a valid ISO9660 filesystem. Both filesystems can be read on a Mac, while on "other" systems, you would typically read the ISO9660 data. Note that this doesn't mean there has to be redundant data on the disc: usually the data that needs to be accessed from both Macs and PCs is kept on the ISO9660 volume, and is aliased on the HFS volume.


MSDOS
Mac OS X includes support for MSDOS filesystem (FAT12, FAT16 and FAT32).


NTFS
Mac OS X includes read-only support for NTFS.


UDF
UDF (Universal Disk Format) is the filesystem used by DVD-ROM (including DVD-video and DVD-audio) discs, and by many CD-R/RW packet-writing programs. Note that at the time of this writing, Mac OS X "Panther" only supports UDF 1.5, and not UDF 2.0.


UFS
Darwin's implementation of UFS is similar to that on *BSD, as was NEXTSTEP's, but they are not really compatible. Currently, only NetBSD supports it. Apple's UFS is big endian (as was NeXT's) - even on x86 hardware. It includes the new Directory Allocation Algorithm for FFS (DirPref). The author of the algorithm offers more details, including some test results, on his site.





Network Filesystems



AFP



The Apple Filing Protocol (AFP) is an Apple proprietary protocol for file sharing over the network. A comparison of AFP and NFS is outside of the scope of this document, but there exists software that enables the two to co-exist (AFP shares can be made to look like NFS shares and vice-versa).
/usr/sbin/AppleFileServer is the AFP daemon, which is launched when you select the "Personal File Sharing" checkbox under System Preferences/Sharing.



FTP
The mount_ftp command mounts locally a directory on an FTP server. Note that this functionality is read-only currently. It works transparently through the Finder and the Web Browser.


mount_ftp ftp://user:password@hostname/directory/path node
NFS
Mac OS X includes NFS client and server support (version 3) from BSD, including the "NQ" (NFS with leases) extensions. The usual supporting daemons (rpc.lockd, rpc.statd, nfsiod, etc.) are present as well.
SMB/CIFS
Mac OS X "Panther" includes Samba 3.0 to support SMB/CIFS.



WebDAV


A WebDAV enabled directory located at a server specified by an appropriate URL can be mounted as a filesystem via the mount_webdav command. Since a .Mac account's iDisk is available through WebDAV, it can be mounted this way.


Note that Mac OS X has a (preliminary) system level event notification framework built around FreeBSD's kqueue/kevent. This can allow graceful mounts and unmounts of network volumes based on changes in network connectivity.



Other/Pseudo Filesystems



cddafs


The cdda filesystem is used to make the tracks of an audio CD appear as aiff files. Moreover, if the track names can be looked up successfully, the track "files" have corresponding names.
When you insert an audio CD in the drive, Mac OS X "mounts" it using cddafs by default, or you can manually do so as follows:
# mount_cddafs /dev/disk<N> /tmp/audiocd



deadfs


When the underlying filesystem is disassociated from a vnode (in the vclean() operation), its vnode operations vector is set to that of the dead filesystem. All operations in the dead filesystem fail, except for close().



deadfs essentially facilitates revocation (of access to the controlling terminal, to a forcibly unmounted filesystem, etc.) Consider a situation where you want a backgrounded job of a logged out user to finish what it's doing, and yet have no access to its (erstwhile) controlling terminal. This is achieved by detaching the terminal from the vnode and replacing it with deadfs.



devfs


devfs, the device filesystem, provides access to the kernel's device namespace in the global filesystem namespace. devfs is typically mounted on /dev and allows entries in there to be built automatically.


devfs, if enabled, is mounted from within the Mac OS X kernel during BSD initialization, although instances of it can be mounted later on using mount_devfs.
# mount -t devfs devfs /tmp/dev




fdesc


The fdesc filesystem is typically mounted on /dev/fd. It's functionality is similar to /proc/<pid>/fd (or simply /proc/self/fd) on Linux, that is, it provides a list of all active file descriptors for the currently running process. Note that a typical Linux system has /dev/fd symbolically linked to /proc/self/fd.



/etc/rc mounts the fdesc filesystem during system startup:
# mount -t fdesc -o union stdin /dev
fifofs
The purpose of fifofs is similar to specfs
loop*

Functionality similar to Linux's "loop" mounts (or "lofi" on Solaris) is available via the Finder (or simply on the Desktop) - simply double-clicking on a disk image file mounts its filesystem (if supported). The command line utility hdid can be used for a finer grained control of this functionality:
# hdid floppy.img /dev/disk3 # hdid http://127.0.0.1/disk.img /dev/disk4
The "disks" disk3 and disk4 can be accessed as regular disks. Note that if the disk image to be mounted using HTTP is a dual-fork file, then it is trickier to use it.
Moreover, hdid can be directed to use only a subset (a range of sectors) of a disk image.


There is also support for encryption, and more importantly, shadowing, wherein a "shadow" file can be used to which all writes can be redirected. When a read occurs in such a case, blocks present in the shadow file have precedence over the ones in the image.



nullfs


The null mount filesystem is a stackable filesystem in 4.4BSD. It allows mounting of one part of the filesystem in a different location. This can be used to join together multiple directories into a new directory tree. Thus, filesystem hierarchies on various disks can be presented as one directory tree, subtrees of a writable filesystem can be made read-only, and so on.


Note that this is slightly different (less seamless) from a union mount (see below). While the latter essentially combines seamlessly the filesystems of the mount point and the mounted, nullfs simply intercepts VFS/vnode operations and passes them through (with the exception of vop_getattr(), vop_lock(), vop_unlock(), vop_inactive(), vop_reclaim(), and vop_print() to the original filesystem (of the mount point).


Note that the null filesystem layer also serves as a prototype filesystem, and new layers can be implemented by using the null layer as a template.
Finally, it should be noted that although nullfs is present in the bsd subtree of Darwin's kernel source, null mounts are not really used by Mac OS X.



ramfs


A ram filesystem can be created under Mac OS X as follows:
# hdid -nomount ram://1024 /dev/disk3

The above command creates a ram disk with 1024 sectors (sector size being 512), and prints the name of the resultant device on the standard output. Thereafter, a filesystem can be created on this device (the corresponding raw device, technically) as follows:
# newfs_msdos /dev/rdisk3 /dev/rdisk3: 985 sectors in 985 FAT12 clusters \ (512 bytes/cluster) \ bps=512 spc=1 res=1 nft=2 rde=512 sec=1024 mid=0xf0 \ spf=3 spt=32 hds=16 hid=0


The disk can be mounted as usual:
# mount -t msdos /dev/disk3 /tmp/msdos # mount /dev/disk3 on /private/tmp/msdos (local) # df /tmp/msdos Filesystem 512-blocks Used Avail Capacity Mounted on /dev/disk3 987 2 985 0% /private/tmp/msdos



Finally, you can get rid of the ram disk as follows:
# hdiutil detach /dev/disk3 "disk3" unmounted. "disk3" ejected.



specfs

Devices (the so called "special" files) and FIFOs can reside on any arbitrary filesystem (that can house such files). This means their names and attributes are maintained by this "host" filesystem. However, their operations cannot be handled by this filesystem - accesses to such device files need to be mapped to their underlying devices (more specifically, the respective device drivers). Moreover, device aliases (for example, same major/minor numbers, but different pathnames on a filesystem, or maybe even different filesystems) need to be detected and handled appropriately.
The specfs layer facilitates the above. Note that specfs is not a user visible filesystem, and it's not "mounted" anywhere.





synthfs


synthfs is a pseudo (in-memory) filesystem used to create arbitrary directory trees (if you wanted to "synthesize" mount points for random things, for example). synthfs is not derived from FreeBSD.


A synthfs mount is similar to a typical pseudo filesystem mount:
# mount -t synthfs synthfs /tmp/synthfs



union


A detailed description of 4.4BSD's "union" mounts, including a short history of similar filesystems, can be found in the USENIX paper titled Union Mounts in 4.4BSD-Lite. In the simplest terms, the union mount filesystem extends the null filesystem by not hiding the files in the "mounted on" directory. It merges the two directories (and their trees) into a single view. Note that duplicate names are suppressed and a lookup locates the logically topmost entity with that name.


Consider the following sequence of commands that illustrates the basic concepts of union mounts:
# hdiutil create /tmp/msdos1 -volname one \ -megabytes 1 -fs MS-DOS ... created: /tmp/msdos1.dmg # hdiutil create /tmp/msdos2 -volname two \ -megabytes 1 -fs MS-DOS ... created: /tmp/msdos2.dmg # hdid -nomount /tmp/msdos1.dmg /dev/disk3 # hdid -nomount /tmp/msdos2.dmg /dev/disk4 # mount -t msdos /dev/disk3 /tmp/union # echo "msdos1: a" > /tmp/union/a.txt # umount /dev/disk3 # mount -t msdos /dev/disk4 /tmp/union # echo "msdos2: a" > /tmp/union/a.txt # echo "msdos2: b" > /tmp/union/b.txt # umount /dev/disk4 # mount -t msdos -o union /dev/disk3 /tmp/union # mount -t msdos -o union /dev/disk4 /tmp/union # ls /tmp/union a.txt b.txt # cat /tmp/union/a.txt msdos2: a # umount /dev/disk4 # ls /tmp/union a.txt # cat /tmp/union/a.txt msdos1: a # umount /dev/disk3 # mount -t msdos -o union /dev/disk4 /tmp/union # mount -t msdos -o union /dev/disk3 /tmp/union # cat /tmp/union/a.txt msdos1: a
As a real-life example, /etc/rc mounts the "descriptor" filesystem as a union mount:
# mount -t fdesc -o union stdin /dev




volfs



volfs, the "volume" filesystem, is a virtual filesystem that exists over the HFS+ (or a filesystem that supports volfs) VFS and serves the needs of two differing APIs (POSIX/Unix pathnames and Mac OS <Volume ID><Directory><File Name>). It is there to support the Carbon File Manager APIs on top of the BSD filesystem.
The filesystems that support volfs are HFS+, HFS, ISO9660 and UDF.
Consider the following example:
# mount /dev/disk0s9 on / (local, journaled) devfs on /dev/ (local) fdesc on /dev (union) <volfs> on /.vol ... # ls -l /.vol total 0 dr--r--r-- 2 root wheel 64 25 Dec 12:45 234881033
The entry in /.vol is nothing but a representation of /. Each mounted volume (a partition, if you will), such as those on external storage devices, would have a representation under /.vol. Consider a file, say /mach_kernel:
# ls -li /mach_kernel 1045670 -rw-r--r-- 1 root wheel 3824080 11 Dec 16:20 /mach_kernel
This file would be accessed under /.volfs as follows (note that 1045670 is the file's inode number):
# ls -li /.vol/234881033/1045670 1045670 -rw-r--r-- 1 root wheel 3824080 11 Dec 16:20 /.vol/234881033/1045670
volfs is mounted during system startup (in /etc/rc):
# mkdir -p -m 0555 /.vol && chmod 0555 /.vol && mount_volfs /.vol



What about XYZ?



While Mac OS X supports many filesystems, you might run into some that are not supported. Linux's ext2/3 and Reiser, for example, are not supported, although you can find an open source implementation of ext2 for Mac OS X.
Interestingly, BootX, the Mac OS X bootloader, does understand the ext2 filesystem, and can load kernels from it.



An important (though not necessarily critical) omission is that of the proc filesystem. This issue, including description of a minimal port of /proc from FreeBSD to Mac OS X, is discussed in /proc on Mac OS X.




What is Mac OS X?



Programming on Mac OS X




Mac OS X is a fertile field for all kinds of programming endeavors, especially if you have a *nix frame of reference. Life is still much better for a developer on Windows than on Mac OS X - no matter what one might think of the usability, etc. of Windows. Apple has been trying to improve things for developers lately, which is a good sign.



This page discusses some programming facilities, frameworks and tools available on Mac OS X.
Application Environments
Since Mac OS X is derived from various sources, it has a multitude of Application Environments.


We discussed these in Above the Kernel. To recapitulate:
BSD
Mac OS X uses FreeBSD as a reference code base for its BSD derivations (Panther derives from FreeBSD 5.0). It includes a BSD-based POSIX API (BSD style system call vector, uap based argument passing, etc.). An example of intercepting system calls on Mac OS X is covered in the article Re-routing System Calls. OS X also supports System V IPC, asynchronous I/O, poll() (emulated over select()), etc. Arbitrary C programming is not any different than on any generic Unix platform. Here is an example of re-routing function calls by overwriting and injecting machine instructions.



Carbon
This is a set of procedural C-based APIs for Mac OS X that are based on the old Mac OS 9 API (actually dating back as far back as Mac OS 8.1). Carbon was originally designed to provide an easy development migration path from Mac OS 9 to Mac OS X. The Carbon APIs are all-encompassing (they include legacy interfaces), covering most things that you are likely to do programmatically on Mac OS X. Carbon specific code is not portable to other platforms.



Classic
Mac OS X includes a Classic (Mac OS 9) emulator that executes in a protected memory environment so as to let users run programs written for Mac OS 9. Apple does not encourage you to actually develop for this environment.



Cocoa
This is an object-oriented Objective-C based API that's the preferred way of doing things in Mac OS X (if what you want to do can be done through Cocoa), particularly because of how well it's supported by Apple's Rapid Development tools.


However, there are many parts of Mac OS X, and applications from 3rd party vendors, that have not converted to Cocoa completely, or at all. A Cocoa application can call the Carbon API when necessary. Cocoa is largely based on the OpenStep frameworks, and consists of primarily two parts: the Foundation (fundamental classes) and the Application Kit (classes for GUI elements).



Although Cocoa is not really portable across platforms, you might be able to get your Cocoa programs to work on a number of platforms if you take GNUstep into account. Here's what the GNUstep FAQ has to say about portability between Cocoa and GNUstep (quoted verbatim):
It's easier from GNUstep to Cocoa than Cocoa to GNUstep.


Cocoa is constantly changing, much faster than GNUstep could hope to keep up. They have added extensions and new classes that aren't available in GNUstep yet. Plus there are some other issues.


If you start with Cocoa:
Be careful with Cocoa.h includes (Use #ifndef GNUSTEP, for instance)
Do not use CoreFoundation
Do not use Objective-C++
Do not use Quicktime or other proprietary extension
You need to convert .nib files (see section 1.1.3 Tools for porting)
Some unfinished classes include NSToolBar and Drawers.


My definition of "portability" (in the current context) is not about feasibility, but the practicality of doing so. Given enough resources, one could port anything to anything - often by emulating/implementing the "source" API on the "target".


The WINE project is a wonderful effort. Microsoft once used help from Mainsoft to get Internet Explorer and Outlook Express to run on Solaris. Still, Win32 code is not portable from a practical viewpoint.



Java
Mac OS X includes a complete J2SE implementation. The Swing implementation generates native OS X GUI elements for a uniform look and feel. JAR files are treated as shared libraries. Note that Cocoa includes Java packages that let you create a Cocoa application using Java as the programming language.



X11


Mac OS X includes (optionally) an X Window System implementation based on XFree86 4.3+. The X server has been optimized for OS X via integration with Quartz and supports OpenGL, rootless and full screen modes, an Aqua-compatible window manager (quartz-wm) and a menu in the Dock.


The presence of a good quality X server and the X11 SDK is a big win because it makes possible to port (in most cases with no or minor changes) a large number of existing X11 applications to Mac OS X, including use of toolkits such as GTK, KDE, various others.


As mentioned in Architecture of Mac OS X, Mac OS X has a number of very different APIs due to the many environments constituting it. The example of BSD and Carbon Process Manager processes was given before.


Similarly, what thread API you use on Mac OS X is determined by what environment you are programming in. Mach provides low-level kernel threads.



The pthread library, /usr/lib/libpthread.dylib (actually a symlink to libSystem.dylib) provides POSIX threads. Carbon includes a threads package for cooperatively scheduled threads (Thread Manager) and another for preemptively scheduled threads (Multiprocessing Services).


Cocoa uses the NSThread class, while Java uses java.lang.Thread. All these are built using pthreads.



Bundles and Frameworks


Mac OS X uses a few concepts not found on many traditional BSD, Linux, Solaris etc. systems.



Bundle


A Bundle is a directory that stores executable code and the software resources (icons, splash images, sounds, localized character strings, interface builder "nib" files, .rsrc resource files, etc.) related to that code.

Although a bundle is a directory containing potentially numerous subdirectories and files, it is treated as a single entity for various purposes.


Mac OS X can have different kinds of bundles:

An "Application" bundle (such as Foo.app) contains everything (except frameworks/libraries coming from elsewhere) needed to run the Foo application.

It is possible to simply drag Foo.app to any location and it will work as expected (you do not even have to do anything to its Dock icon, if any - courtesy the fact that "aliases" on HFS+ do not break if you move a file without replacing it).

The /Applications directory contains many such bundles.


A "Framework" (such as Bar.framework) is a versioned bundle containing resources such as headers, documentation, etc. The /System/Library/Frameworks directory contains numerous frameworks (such as for Kerberos, Python, QuickTime, ScreenSaver, and so on).


A "Loadable" bundle can be a kernel extension (a .kext, similar to a loadable kernel module on Linux, say), many of which exist in /System/Library/Extensions, a Plug-in or a Palette.
One of the Finder flags is kHasBundle, which, if set, makes the bundle appear as a file package (a single opaque entity), with exceptions and specific behavior for different bundle types.


The various bundle extensions referred to above are only conventions - a bundle can have any extension. For example, instead of a .app, you can have a .debug or .profile to imply debug or profile code, respectively.




Framework


A Framework, as stated above, is a type of a bundle that contains shared resources such as dynamic shared libraries, header files, icons and images, documentation, etc. Moreover, frameworks are versioned. Major versions are incompatible while minor versions are compatible. One framework can have multiple major versions.


Consider an example:
# ls -lF /System/Library/Frameworks/OpenGL.framework total 32 lrwxr-xr-x ... Headers@ -> Versions/Current/Headers lrwxr-xr-x ... Libraries@ -> Versions/Current/Libraries lrwxr-xr-x ... OpenGL@ -> Versions/Current/OpenGL lrwxr-xr-x ... Resources@ -> Versions/Current/Resources drwxr-xr-x ... Versions/
Except Versions/, everything else is a symbolic link (to entities from the current major version). The file OpenGL is the dynamic shared library:
# file -L OpenGL OpenGL: Mach-O dynamically linked shared library ppc
The default path for searching frameworks (as used by the dynamic link editor) is:
$(HOME)/Library/Frameworks /Library/Frameworks /Network/Library/Frameworks /System/Library/Frameworks


As we have seen, Mac OS X has complex entities (like a .app directory tree) exposed as a single, click-able entity through the Finder. The same effect as double-clicking on an entity's icon can be achieved on the command line through the open utility. It opens a file, folder, or a URL, in an appropriate manner.

For example, opening a .app folder would launch that application, opening a URL would launch an instance of the default web browser with that URL, opening an MP3 file would open it in the default MP3 player, etc.



Runtime Environments


Mac OS X has two primary runtime environments: one based on the dynamic link editor, dyld, and the other based on Code Fragment Manager (CFM). OS X does not support ELF, and there's no dlopen, although the dlcompat library provides a limited compatibility layer (using native OS X functions) so that common Unix source can be compiled easily.


CFM determines addresses for referenced symbols in executables at build time (a static approach). The executable format used is called PEF (Preferred Executable Format). dyld resolves undefined symbols at execution time. The executable format is Mach-O (Mach object-file-format).


Mac OS X is natively a dyld/Mach-O platform - all system frameworks are built using dyld. In fact, the CFM/PEF environment is itself built on top of dyld/Mach-O. However, there exist provisions to call dyld code from CFM code.


Moreover, if you wish to debug or trace a CFM application using GDB, you would need to use a Mach-O program called LaunchCFMApp:
/System/Library/Frameworks/Carbon.framework/Versions/\ A/Support/LaunchCFMApp
dyld/Mach-O is similar in many respects to ld.so/ELF, although they differ both conventionally and fundamentally.


Some of these are:
Dynamic shared libraries on Mac OS X have the .dylib extension. The functionality of several traditional libraries (such as libc, libdl, libinfo, libkvm, libm, libpthread, librpcsvc, etc.) is provided by a single dynamically loadable framework, libSystem. libc.dylib etc. are simply symbolic links to libSystem.dylib.


Mac OS X builds libraries and applications with a two-level namespace (as compared to a flat namespace in traditional Unix systems). This topic is described in an Apple Developer Note called Two-Level Namespace Executables.


This also means that DYLD_INSERT_LIBRARIES, the dyld analog of ld.so's LD_PRELOAD will not work with two-level namespace libraries. You can force a flat namespace, but that often messes up things enough to stop the application from running at all. Additionally, you cannot have "weak" symbols (symbols that can be overridden) in libraries.


Functionality similar to ldd (on Linux, say) is provided by /usr/bin/otool.

Mac OS X uses Mach-O and not ELF simply because NEXTSTEP used Mach-O. Apple had a large enough TODO list that moving to ELF for the sake of mainstream conformity was not justified.



Like NEXTSTEP, Mac OS X supports "fat" binaries where an executable image contains binaries for more than one platform (such as PowerPC and x86). Apple's port of GNU CC allow for fat binaries to be produced (provided assemblers and libraries are available for each specified architecture).


Prebinding


Mac OS X uses a concept called "prebinding" to optimize Mach-O applications to launch faster. Prebinding is the reason you see the "Optimizing ..." message when you update the system, or install certain software.


The dynamic link editor resolves undefined symbols in an executable (and dynamic libraries) at run time. This activity involves mapping the dynamic code to free address ranges and computing the resultant symbol addresses.


If a dynamic library is compiled with prebinding support, it can be predefined at a given address range. This way, dyld can use predefined addresses to reference symbols in such a library.


Of course, for this to work, libraries cannot have preferred addresses that overlap. Apple specifies address ranges for 3rd party (including your own) libraries to use to support prebinding.



update_prebinding is run to (attempt to) synchronize prebinding information when new files are added to a system. This can be a time consuming process even if you add or change a single file, say, because all libraries and executables that might dynamically load the new file must be found (package information is used to help in this, and the process is further optimized by building a dependency graph), and eventually redo_prebinding is run to prebind files appropriately.



/usr/bin/otool can be used to determine if a binary is prebound:
# otool -hv /usr/lib/libc.dylib /usr/lib/libc.dylib: Mach header magic cputype cpusubtype filetype ncmds sizeofcmds flags MH_MAGIC PPC ALL DYLIB 10 1940 \ NOUNDEFS DYLDLINK PREBOUND SPLIT_SEGS TWOLEVEL
Xcode
Xcode is Mac OS X "Panther"'s developer tools package. It includes components typical of a comprehensive IDE:
A source code editor with code completion
A file browser/organizer
Support for version control (CVS and Perforce)
A documentation viewer that can link symbols in the code to documentation
A class browser
Various compilers (the GNU suite, including gcc 3.3 and integration with distcc for distributed builds, javac, jikes)
GDB based graphical and command-line debugging
An Interface Builder application that provides a GUI for laying out interface objects (various GUI elements), customize them (resize, set and modify attributes), connect different objects together, and so on


Support for packaging


A new Xcode project can be instantiated from a large number of templates. As can be seen, it supports development of various kinds of programs in C, C++, Objective-C, Objective-C++, Java and Assembly. For example, it is almost trivial to create things such as Screen Savers, Preference Panes (the kind you see under System Preferences), etc.



Xcode has some neat and useful features: Predictive compilation runs the compiler in the background as you edit the source. Once you are ready to build, the hope is that most of the building would have been done already.


"Zero Link" links at runtime instead of compile time, whereby only code needed to run the application is linked in and loaded. A related feature is "Fix and Continue", courtesy which you can make a change to your code and have the code compiled and inserted into a running program. Distributed builds are also supported via integration with distcc.



Programming Languages


Compilers and Libraries

Apple provides a customized/optimized GNU CC, with backends for C, C++, Objective-C and Objective-C++. For Java, two compilers are included: javac and IBM's jikes.

Compilers for many other languages are available either precompiled (such as the XL Fortran Advanced Compiler from IBM), or can be compiled from source, which is not any harder in general than compiling the same source on, say, Linux or FreeBSD.


The same goes for development libraries - it should be easy to compile many open source platform-independent / multi-platform libraries from source on OS X.

Many important libraries and APIs are either included with Mac OS X, or are readily available (Gtk/Gtk++, Java, OpenGL, Qt, QuickTime, Tcl/Tk, X11R6). The system comes with several special purpose (and/or optimized) libraries as well, such as for Numerical Computing and Image processing (BLAS, vBigNum, vDSP, vImage, LAPACK, vMathLib, etc.)



Interpreters



A number of scripting languages are included in Mac OS X, such as AppleScript (including the AppleScript Studio IDE), Perl (Mac OS X "Panther" has perl 5.8.1), PHP, Python (Panther has python 2.3, with bindings to CoreGraphics), Tcl, and Ruby.


Mac OS X also supports the Open Scripting Architecture (OSA), using which it is possible to get JavaScript (a port of Mozilla JavaScript) in the form of an OSA component. You can get support for more languages (Lisp, Scheme, ...) via Fink, and it should be straightforward to compile most of them from source, if desired. bash, tcsh, and zsh are the *nix shells included.



Scriptability


AppleScript is the preferred scripting system on Mac OS X, providing direct control of many parts of the system as well as applications. In other words, AppleScript lets you write scripts to automate operations, exchange data with and send commands to applications, etc.

While I personally find AppleScript syntax to be exasperating (I never really enjoyed COBOL for that matter), it is useful.

Consider a contrived example for a taste of AppleScript:
tell application "Finder" set system_version to (get the version) as number do shell script "say 'This is Mac OSX'" & system_version return system_version end tell
Running the above (either in ScriptEditor or through the osascript command line utility) should cause the say utility (see below) to speak the version of Mac OS X (well, assuming you are running Mac OS X) to you.

By the way, you do not need to run a shell command and use the say utility: AppleScript has a say command of its own.


Now, whether you are impressed by (or happy with) Mac OS X's scriptability depends on which system you are contrasting with. As mentioned at the beginning of this section, if you are coming from a traditional Unix/Unix'ish world (FreeBSD, Linux, Solaris, *nix, ...), you should find it impressive, a lot perhaps (barring the miniscule possibility that you are an anti-Mac/pro-XYZ zealot). However, you might not think much of this if you have been using Microsoft's COM/DCOM/.NET.


Along the lines of making a *nix developer/user happy though, Apple has been exposing various aspects of Mac OS X functionality to be driven via a traditional command line.


Consider some examples:
drutil is a command line tool that interacts with the DiscRecording framework. The command:
# drutil getconfig supported
will probably tell you more than you want to know about your CD/DVD burner.


hdiutil is a powerful utility for manipulating disk images.
say is a command line utility to convert input text to speech using the Speech Synthesis manager.


sips is a command line interface to the Scriptable Image Processing Server. The graphical abilities of Mac OS X are exposed through this image processing service. The SIPS architecture contains tools for performing basic image alterations and support various image formats. The goal is to provide quick, convenient, desktop automation of common image processing operations.


Tools

The Developer Framework contains a wealth of tools for various programming, debugging, profiling and other developmental tasks. The CHUD (Computer Hardware Understanding Development) Tools are particularly useful for various kinds of benchmarking and optimization.

Mac OS X Developer Tools Overview contains detailed information on almost all tools included in Xcode. Mac OS X Hacking Tools is a compendium of some useful programs on Mac OS X (mostly outside of Xcode).


Lastly, the OS X Terminal.app, is one of the better terminal applications around, which does matter to me while programming, since I do not use an IDE for everything. The terminal advertises itself as xterm-color, has Unicode support (though broken), drag-and-drop support (dragging an icon to the terminal would drop its pathname or URL string), transparency support (has a minor problem in Panther - there's ghosting of text under certain conditions), etc.


What is Mac OS X?



A Sampling of Mac OS X Features



Mac OS X has many "cool", "interesting", and useful features, a number of which directly contribute to the overall usability of the system. This page briefly describes a few features representative of why Mac OS X is a good (Desktop) operating system.




Aqua


The graphical user interface of Mac OS X is called Aqua. This includes the look and feel, behavior, and integration of GUI elements. The GUI application environments of Mac OS X, Carbon, Cocoa, and Java, all support Aqua.

Classic does not, and Mac OS 9 applications running under Classic look like they did on Mac OS 9. Finally, Mac OS X includes an optimized X Window server, including a native Aqua window manager (quartz-wm) that lets you run X11 applications alongside native Aqua programs.


quartz-wm provides Aqua window controls, drop shadows, etc. However, the X11 application's own look and feel will be the one provided by the particular X11 toolkit being used.


Aqua has numerous distinctive features: Mac OS X uses high-quality photorealistic icons that are rendered at various sizes up to 128x128, allowing for features such as in-place document preview and in-icon status indication.


Mac OS X has a number of functional and unique user interface elements, such as sheets, which are document-modal dialogs that attached to and appear to come out of a document's title bar.


The Desktop, Dock, and the Finder are also different (in my opinion, mostly better, from a productivity/usability point of view) from their counterparts on Windows and *nix.
Aqua Human Interface Guidelines is a thorough description (almost 300 pages) of what guidelines to adhere to while creating applications for Mac OS X as well as an overview of various Aqua GUI elements.


While the lucidity and attractiveness of Aqua is visible immediately as you look at a Mac OS X desktop (with the disclaimer that this is a subjective area, so many people may not like how it looks), it may take a little while to get used to before you appreciate Aqua's usability.



Data and Information Management


While not really Utopian, Mac OS X makes a very good attempt at keeping various data and information organized by context, rather than having files strewn all over the place. System and Application "preferences" can be global (system-wide) or per-user, and are kept organized as such. The various APIs make sure that (if used properly), all of a user's data is stored deterministically.



One of the most useful features of Mac OS X is its support for synchronization of your computer's configuration, or personality, if you will. Currently this data set includes the address book, calendar, and Safari bookmarks, although Apple should add more entities.


If you are doing a new installation or clean upgrade of your system, it is a boon to have the following: on the "old" installation, click a button to synchronize the above information to a device, which can be your iPod, or your .Mac account. On the "new" machine, you can reverse synchronize and have this information injected.



Devices


Apple has excellent support for FireWire devices, hardly surprising since they invented FireWire. You can readily boot from external drives, treat a Mac to behave as if it were an external FireWire drive (boot it with the T key pressed, which puts the computer in Target Disk Mode), and even connect two computers together using TCP/IP over a FireWire cable. The iSight and the iPod use FireWire connections too.



Apple has also been pushing Bluetooth with their newer computers, although you can get adapters for older models. In addition to using Bluetooth for communicating with phones and PDAs, Apple also uses it for their wireless keyboard and mouse, which are very well designed and work well with Mac OS X (well, the mouse still has one button).


Even though Apple computers have custom "chips" (the KeyLargo IC, for example, is an I/O controller that provides USB, UDMA, EIDE, sound, communication support, etc. all on a single IC),


Apple uses a number of "standard" components (RAM, IDE/SATA disk drives, optical drives, ...) in their machines, things are not always black and white. For example, an arbitrary DVD burner may not work with iDVD, though usually a workaround can be found.



I18N


Mac OS X is localized to a number of regions. It supports Unicode 4.0, various input methods, and multiscript support (a single document can contain multiple scripts). Apple provides tools, including support in Xcode, so that developer can internationalize their applications.
Some specific components included are ICU, libiconv and support for wchar_t.


I was most impressed by how easy it is to input various Indian languages, including Hindi, on Mac OS X - out of the box. I can write an email containing English, Hindi, and other languages, using the QWERTY keyboard to input Hindi phonetically - it is very intuitive (assuming you do know Hindi, of course), and far better than my experience on other systems.



iLife


Apple's iLife suite (iDVD, iMovie, iPhoto, iTunes, and GarageBand) are quite possibly the best applications you can get bundled with any operating system. While I do not use iDVD and iMovie myself (I have been experimenting with their professional counterparts, DVD Studio Pro and Final Cut Pro), I find iTunes and iPhoto to be excellent at what they set out to do, although nitpickers can find "issues" no matter what. These two applications manage and organize your assets (music and digital photos), with the option of leaving the "originals" untouched. Asset metadata is stored in a relational database, into which you can have multiple views.



iPhoto


iPhoto can import images from a digital camera. It lets you do some basic operations on images, such as constraining as per various standard sizes, crop, resize, enhance, do red-eye reduction, retouch, convert to black and white, adjust brightness and contrast, etc. You can have a number of output channels for your photos: you can print them on a printer, order prints using Kodak's online print service, email pictures, create a slideshow, burn them onto optical media, create .Mac slides, export them to a thumbnailed web gallery, and even create a book in WYSIWYG fashion that Apple can print and bind for a fee.




iTunes




Unless you have some uniquely specific needs, have a gargantuan gripe against Apple, or are a masochist, you would probably find iTunes on Mac OS X to be the final word in music management.

iTunes is a powerful and sophisticated jukebox, an interface to Apple's online music store, and a companion program to the iPod.

I believe it represents the best that software can possibly do in making your music experience on a computer pleasant.




Power Management


Mac OS X in conjunction with Macintosh hardware make up for some impressive power and thermal management. The four independently controlled thermal zones and the nine fans of the Power Mac G5 have been discussed aplenty. Mac OS X includes drivers and other logic for the 21 temperature sensors in that machine.


The PowerBooks have sophisticated thermal management as well. You can 'grep -i' for "thermal" and "temperature" in the output of 'ioreg -l' on Mac OS X for related miscellaneous information.


Based on power/thermal feedback, processor and bus speeds can be reduced to conserve power and control heat. All current Apple notebook batteries have remaining charge indicators. On the PowerBooks, you can change the battery without powering-off courtesy an internal backup battery that holds charge for a few minutes. Note that you do need to put the machine to sleep to do this.


Note that by default the system tries to keep network connections alive even if the machine sleeps. For example, if you login (via SSH, say) from one PowerBook to another, and both of them go to sleep, your login will stay alive.



Security


Mac OS X includes various security features, services, and APIs (including what's available on typical *nix systems), such as those for controlling/managing passwords, certificates, public/private keys, application-level privileged operations (capabilities), trust policies, etc. Mac OS X supports Kerberos, OpenSSL, and PAM as well.


Note that many of the above services are exposed through the Keychain Services API, which any application can use, for example, to "remember" your passwords.

It is possible to have a single keychain password instead of multiple passwords across different applications.


root login is disabled by default, and sudo is used for administrative access. You can use /usr/bin/security from the command line to control the security framework.


Relatively recent security related enhancements include FileVault (encryption of a user's home directory) and Secure File Deletion (see above).

As mentioned earlier, Mac OS X is not a trusted system, or as focussed on security as say, OpenBSD, but it should at least be as secure as any modern day *nix system.

It does make use of a large amount of open source software, so it would share many of the weaknesses and strengths of those components.

Potentially, Apple's integration of such components might create new weaknesses, but I have found their software engineering to be extremely admirable in general.



Speech Interface


Mac OS X includes both speech recognition (part of Carbon) and synthesis frameworks, that are fairly well integrated with the system. Applications can make use of APIs to these frameworks. "Speakable Items", a user customizable interface to the speech recognition engine, is available to arbitrary applications, wherein you can add your own items.


You can do the following, for example:
Open System Preferences, and click on "Speech" under the "System" category. Click on "Speech Recognition".



Under the "On/Off" tab, select "On".
Under the "Listening" tab, make sure the appropriate microphone (such as "Internal microphone" is selected. You should check its volume settings so that when you speak, the level indicator remains in the green area.


Press the esc (Escape) key, keep it pressed, and after about a second, say "Get my mail" naturally.


The mail application should launch (if not already running), or you should be switched to it (if already running).


Similarly, with esc pressed, say "Switch to Safari". You should either be switched to Safari, or it should launch if not already running.


If you want to add a command called, say, "New window" to Safari (that opens a new browser window, the equivalent of cmd-N, you can do so by saying "Define new keyboard command".


In the "Define keyboard command" sheet that launches, type in cmd and N (the text field should say Command+N). Click on the OK button.


In the "Define keyboard command" window, type in the phrase "New window" in the phrase text field. You can choose to keep the command limited to Safari only, or be allowed in any application. Click on "Save".


With esc pressed, say "Switch to Safari", followed by "New window". It should open a new Safari window.


As another example, you can play Chess by speaking your moves.
The speech interfaces also add to the accessibility features of Mac OS X. In addition to speech recognition and synthesis, Mac OS X offers visual assistance (zoom features, enhanced contrast, grayscale display), aural assistance (screen flashing), typing assistance (sticky keys, slow keys), and mouse assistance (mouse keys, full keyboard access).




Miscellaneous


Mac OS X 10.3 (Panther) introduced a number of new features. While some of these might be immature and even unstable, they should prove to be useful over time. Some of these are: iChat AV (video, audio, and text chat), FileVault (encryption of a user's home directory), Exposé, Secure Trash Deletion, etc.



While Panther does securely delete files via a multi-pass overwriting algorithm courtesy the srm utility, Mac OS X is not a trusted system (it's not intended to be either). If a "secure" file's contents are in the buffer cache, or an application's address space, and if some of it has been swapped out, the information is vulnerable.



You should be able to shut the machine down, connect it to another computer, say using target disk mode, and see "interesting" information through a simple strings on the device. OpenBSD, for instance, supports encryption of virtual memory (swap).


Exposé is a feature of the user interface that, among other things, lets you shrink each window on the screen in-place to a point where no windows overlap on the screen. This is different from automatic window tiling, because the original positions of the windows are not disturbed.


You activate the in-place shrinking via a key press: F9 arranges all windows on the screen so that they don't overlap. You can "find" the window you are looking for visually - although the windows are shrunk, they are the same as the original windows - movies keep playing, etc.


Once you identify the window you are seeking, you can choose it, either through the keyboard or the mouse, and the windows grow back to their original size, in their original positions, with the selected window on top. Exposé has a few other similar features, and Apple might offer a desktop pager based on it soon.


Mac OS X also has an implementation of Zero Configuration Networking, which Apple calls Rendezvous.


What is Mac OS X?




Available Software on Mac OS X



One relatively common notion about Mac OS X seems to be that there's not a lot of software for it. While it is true that the quantity of software available for Mac OS X is not as large as, say, that on Windows or Linux, I believe the software that does exist for Mac OS X provides you as much functionality, if not more, as software on any other platform.


Mac OS X has constantly been improving its support of "alien" APIs and frameworks. The introduction of a native X Window System is an example (although there were non-Apple ports of XFree86). Projects like Fink (see below) have helped a great deal in facilitating easy availability of open source software on Mac OS X.


In my opinion, at this point, the concern that Mac OS X doesn't have 191,700 packages/libraries available is largely unwarranted, as long as existing software's functionality matches that of other platforms.


A glaring exception is games (see below), where Mac OS X is lacking in comparison to Windows.



Bundled Software


Apple includes a phenomenal amount (and quality) of software with Mac OS X. You can refer to the Mac OS X Technical Specs for details of included software.
It's not just the quantity, or even quality, of software that makes Mac OS X a highly productive and usable platform: the entire system has a uniform look and feel, applications are very nicely integrated with each other as necessary, and things mostly happen as you would expect.



Moreover, Apple has tried to extensively leverage a large amount of existing open source software, as is evidenced by the numerous open source packages that constitute the Darwin technology umbrella.


Open Source Software


Bundled


As stated in Architecture of Mac OS X, Darwin critically relies on a lot of open source software for functionality and features. Apache, bind, binutils, cvs, gcc, gdb, pam, perl, postfix, python, rsync, samba, etc. are only some of such packages. Moreover, Apple has integrated many of these software pieces with their platform, and has provided user interfaces that try to make configuration and use of such software easy and seamless.
Fink


The Fink project makes available a large set of Unix open source software on Mac OS X. The fink tool (written in Perl) uses Debian package management tools (dpkg, dselect, and apt-get).


If you need, for example, the text-based web browser lynx, you can always download the source and compile it, which may work "out-of-the-box" for many packages, but not for many others.


Fink takes care of such porting issues, dependencies, version updates, etc. For example:
% sudo apt-get install lynx Password: Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: lynx 0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.


Need to get 1329kB of archives. After unpacking 0B will be used. Get:1 http://us.dl.sourceforge.net 10.3/release/main lynx 2.8.4-12 [1329kB] Fetched 1329kB in 5s (221kB/s) Selecting previously deselected package lynx. (Reading database ... 7563 files and directories currently installed.) Unpacking lynx (from .../lynx_2.8.4-12_darwin-powerpc.deb) ... Setting up lynx (2.8.4-12) ...



You can use the fink list command to see the list of packages currently supported by fink. Version 0.6.2 lists over 1100 packages (including various versions of certain packages)!


Moreover, you can choose to download and install pre-compiled (binary) packages, or download the source (including Mac OS X specific patches, if any) and compile it.

Heck, it's easier to get certain Open Source packages up-and-running via fink than, say, on Red Hat Linux (xv, for example).



Finally, a very nice approach taken by fink is that it does not pollute the rest of the system - it installs everything under the /sw directory.




Other Port Collections



Darwin Ports and GNU-Darwin are also open source port collections for Darwin/Mac OS X.
Random Open Source Software


There is plenty of open source software (possibly arcane or esoteric, but not always) that's not provided by fink. It should be easy to compile it from source, provided it's reasonably standard *nix source.


There are some provisions to make portability easier: the dlcompat library (a non-Apple library providing a subset of the dlopen/dl* API), System V IPC support, <poll.h>, <sys/aio.h>, etc.



OpenOffice


OpenOffice (X11 based, not native) is available for Mac OS X. KOffice is also an option.





Shareware



The Macintosh platform has historically had an active "shareware" community. You can browse a large number of shareware software packages available for Mac OS X on web sites such as VersionTracker, MacUpdate, osx.hyperjeff.net/Apps etc. In addition to shareware, these sites also "track" other kinds of "free" software, as well as commercial software.
VersionTracker also tracks software for other platforms (Mac OS 9, Windows, and Palm OS).





Commercial Software


A lot of commercial software is available for Mac OS X. If you need it, and if you can afford it, you will find plenty of applications to suit a variety of needs. In fact, the kind of commercial software that is available is one key factor that makes Mac OS X the most unique operating system currently.


A Sampling of Available Software



This section lists some examples of important, interesting, or commonly used software available for Mac OS X. This is not a complete list by any means, and there are many more "important" applications that I have not listed. I have categorized the software according to its domain of application, rather than as commercial, shareware, freeware, open source, etc.



For a much more comprehensive list of available software for the Mac, please refer to Apple's Software Guide.



Development




Apple has tried hard to make Mac OS X an excellent platform for various kinds of software development. Apple's developer tools are very good, and you also have other options, such as CodeWarrior). What's more, a large number (if not most) of open source compilers, interpreters, tools, and libraries should be usable on Mac OS X.


More details of programming options on Mac OS X were discussed in a previous section (Programming on Mac OS X).
Graphics, Media, and Publishing Software
Adobe After Effects
Adobe Acrobat Professional
Adobe GoLive
Adobe Illustrator
Adobe InDesign
Adobe Photoshop
Adobe Premiere (discontinued)
Alias Products (ImageStudio, Maya, PortfolioWall, SketchBook Pro, Studio Tools, ...)
Apple DVD Studio Pro
Apple Final Cut Express
Apple Final Cut Pro
Apple Logic Platinum
Apple QuickTime Pro
Apple Shake
Apple Soundtrack
NewTek Products (LightWave 3D, VT[3], ...)
Macromedia Studio MX
Pixar RenderMan
QuarkXPress
Roxio Toast 6 Titanium
Instant Messaging Clients
Apple iChat AV
Gaim
Microsoft MSN Messenger
Yahoo! Messenger
A number of other open source and/or shareware messenger clients (including multiple-protocol clients and IRC clients) are available. You can visit the VersionTracker site for more details.
Mail Clients
Apple Mail.app
IBM Lotus Notes
Microsoft Entourage X
Microsoft Outlook Express (OS 9/Classic)
Mozilla Thunderbird
Qualcomm Eudora
Media Players
Apple QuickTime Player
Microsoft Windows Media Player
Real Player
VLC Player
Networking/Connectivity Software
Apple Remote Desktop
Microsoft MSN
Microsoft Remote Desktop Connection
Nortel Contivity VPN Client
Office, Productivity, and Information Management Software
Apple iSync
Apple Keynote
AppleWorks
IBM Lotus Notes
Microsoft Office v.X
OpenOffice (X11 based, not native)
Sherlock and Watson
Sherlock (or its 3rd party counterpart, Watson) is a remarkably useful tool that you have to use to appreciate. The idea is to present useful information (movies, phone book, TV listings, recipes, dictionary, exchange rates, stock quotes, weather, translation, maps, travel, and many more) without using a web browser, thereby avoiding unnecessary navigation and clutter, while rendering and formatting the information in an appropriate way.
Sherlock is included with Mac OS X, while Watson costs $29.
Tax/Accounting Software
Intuit QuickBooks Pro
Intuit Quicken
Intuit TurboTax
Scientific Software
Wolfram Mathematica
x86 Emulators
Bochs
Microsoft Virtual PC
Bochs is a free, cross-platform, open source x86 emulator. It is extremely flexible and neat, but it is not integrated with or optimized for the Mac, and consequently is rather slow.
Virtual PC is a very useful application that emulates a PC on the Mac. Currently it is the fastest and the most comprehensive x86 emulator for Mac OS X. You can refer to Many Systems on a PowerBook for examples of Virtual PC use.
Web Browsers
Mac OS X includes two web browsers by default: Safari and Internet Explorer (optional in Panther). A number of other web browsers are available for it though (open source and otherwise, free, shareware or commercial, etc.). Here is one list (not necessarily exhaustive):
Amaya
Camino
Firebird
iCab
Internet Explorer
Konqueror
links (text based)
lynx (text based)
Mozilla
Netscape
OmniWeb
Opera
Safari



If you are feeling nostalgic, you can even get a copy of NCSA Mosaic, both as a native Mac OS 9 application (runs under Classic on OS X), or as an X11-based source archive that you can compile from.





What About Games?



I am very ill qualified to comment in this area, having played computer games only on game consoles all my life. From what I hear though, the Mac is (relatively) a far worse candidate than a PC as a gaming platform - not so much because of technical reasons (anymore), but simply because there are not as many games for the Mac.
This is not to say that there are no games at all! I think there are a few, like Breakout, Super Breakout, ...




Apple's web site has a section on available games for Mac OS X.


What is Mac OS X?



Conclusion: Why Mac OS X?



The previous sections have been a commentary on Mac OS X, mostly in a "statement of fact" fashion. The closing discussion on this page assumes familiarity with the contents of the previous pages.




Bottom line


I have positive feelings about Mac OS X. If I were to choose a computing platform today to run my "digital life", it would be Mac OS X, unless my line of work or specific needs require/dictate otherwise.



This doesn't mean what works for me would work for others. This also doesn't mean Mac OS X has no "issues". It has many: it carries a lot of "old" technology and baggage, it is not bleeding edge in many respects, several things are ugly below Apple's pretty interface, it costs money, it needs Apple hardware (although I personally like Apple's machines very much), and it even has kernel panics on its own once in a while.



Still, Mac is currently the most worthy client platform in my opinion: the only operating system currently in production that, within reason, lets you have your cake and eat it too.


In Search of an Ideal Client Computer
Of the operating systems I have used on my primary workstations (for extended periods of time), I believe currently three are "in the running" for client computing: the Linux family, Mac OS X, and the Windows family (in alphabetical order).


Note that we use the phrase "client computing" or "client platform" to mean the same as, or similar to, "desktop computing", which is somewhat of a misnomer since many people use notebook computers in place of desktops. In any case, the phrase is meant to represent a multi-purpose, multi-faceted, everyday, personal machine, that you can use for work and play.



*BSD and Solaris



The various BSDs are well suited for servers and embedded applications. They are great if you want to follow a pedantic approach to learning about operating systems, and might even be ideal primary machines for people with either specific or limited needs, but I don't think any of the BSDs are going to be the client computers of tomorrow by themselves (of course, they might very well be the critical underlying technology of tomorrow's clients, as evidenced by the presence of FreeBSD in Mac OS X).


I have great affinity for Solaris, and I think it still looks very promising (as a server) despite Sun's recent fortunes.


Let us reflect on the three client computing "candidates" briefly.


Note that these operating systems have server versions too, but we are only talking about the client versions.



Windows


The existing "numbers" are ridiculously lopsided in favor of Windows. I have had a more or less neutral attitude towards Windows: I sometimes feel claustrophobic while using Windows, but I do not despise the platform (I think some people do have rather strong emotions against Windows).


I have usually ascribed my "discomfort" with Windows to my background: my first operating system was SVR4, followed by Linux and *BSD. Windows alienates me from my familiar environments, and has several nuances of its own. I have tried various *nix-like environments for Windows over the years: the MKS toolkit for DOS, Cygwin, MKS for Win32, Microsoft Services for Unix, etc.


They all try, admirably, but it's not the same. I admit that the lack of a "true Unix environment" isn't much justification against Windows. In fact, it's a fine platform in many respects - if you have seriously developed on multiple platforms, including Windows, you might agree with this.



I think there is too much entropy on Windows. The operating system is too "busy" (in the colloquial sense). Legacy baggage and backward-compatibility contribute to software malice and malfunction, whether it be doctored or accidental.



Gigantic user base amplifies flaws (particularly security related) to epidemic proportions. Many experts disagree with numerous aspects of the Windows UI. A number of small "everyday" annoyances cumulatively make life difficult.


Consider some examples: the entire system gets unresponsive when you empty a very full recycle bin, or copy large files, IE gets unresponsive when you empty its cache, even though XP boots very quickly to the point of showing the desktop, it takes a long while before the system is usable - and this is without having anything in "Startup". Do you know how many keys there typically are in the Registry - do you know how many Office alone has, and what this does to maintainability?



Have you ever installed Windows and spent hours, if not days, downloading and installing drivers specific to your machine (such as for Sony VAIOs, IBM ThinkPads, ...)? Have you ever had stability issues with Windows?



There are even books dedicated to Windows annoyances.


The default XP theme's color-scheme seems to be Disneyland inspired - not that there's anything wrong with Disneyland. Many people like it, actually.



That said, Windows can be a fun platform, even if you are a hacker (in the good connotation of the word). Consider: you can get all the nice developer tools, debuggers like SoftICE, figure out lots of undocumented APIs, overtake some system calls, extend the system, find and fix security flaws, and so on.


The fact that it has a huge market share might just make your efforts worthwhile. However, since Windows is largely disjoint with the *nix Universe, many (traditional) hackers steer clear of it for the fear of being "left out".


Source code unavailability doesn't help either.
Nevertheless, an overwhelmingly large majority of computer users are on Windows. A lot of software is available for Windows as a result - it's a feedback loop perhaps, although for the typical end-user, the amount of important Windows-only software is probably decreasing.


Still, many people don't have a choice, often because they don't know that they have a choice. There is a lot of inertia in moving away from Windows, both real (and justifiable) and imaginary.


Often, there are existing investments in Windows: monetary, intellectual, legal, political, etc. Most importantly, a lot many people simply don't care - they have no motivation, or interest, in "experimentation".


They have better things to do in life than evaluate computing platforms and switch. Many people don't even like computers - they just have to use them.



Microsoft could address some or most of the existing problems with Windows in future versions. They do seem to have the opportunity and the resources. I opine that they did several nice things with Windows 2000.



Windows has a lot of things in place (including the user base). Microsoft has a lot of technology (theirs, licensed from 3rd parties, and even open source) to draw from. Thus, even though they probably will be the de facto client platform of the future, they might actually make Windows worthy of being so.



Currently Mac OS X is, for lack of a more appropriate word, better in my opinion: for those who have a choice, and can make it.




Linux



I have great fondness for Linux. In fact, I feel much indebted to the GNU, Linux, and BSD communities for providing me (and others) with excellent sources of usable, no strings attached knowledge all these years.


I have made heavy use (sometimes exclusive use) of Linux in academics and my jobs, including my current job. My extreme interest in Linux is one reason why I am somewhat disappointed in the way Linux as a client platform has evolved.



I think Linux is an excellent, if not the best, operating system for a variety of uses: in server applications, all kinds of embedded applications, in research and academia, for technology companies looking for an operating system as a base to develop their offerings; almost every domain except mainstream client computing.



Linux (as a client operating system), like Windows, has a lot of entropy, although of a different kind.


Choice, Choice Everywhere



Linux provides an incredible amount of choices - for everything. You can choose your distribution (and/or flavor) from an insane number of them, although the mainstream ones are not that many (I'm lying: even they are too many, and they all differ, sometimes subtly, sometimes vastly).


You can run Linux on an even more incredible array of platforms - some of which one would never imagine could run anything, let alone an operating system. You can put Linux on watches and PDAs, or make clusters using it cheaply and efficiently.


There are all kinds of devices that run Linux. You can install Linux in seemingly infinite number of ways. You can even compile the entire system from source (Gentoo Linux) - optimized for your particular machine to extract the last ounce of speed.


You can have more number of different filesystems on Linux than you had total files on some older operating systems. You can have a similar number of packet classifiers in the kernel, and far too many other entities to be mentioned here.


You can have support for devices you never knew existed, for protocols nobody even uses, and of course, for those that everybody uses. You can have almost all the window managers ever invented for X11.


You can program in a zillion programming languages on Linux. You can choose from among more than one desktop environments, based on your personal preferences, taste, working style, religion, or whatever else.


You can have APIs from myriad systems and environments on Linux, either natively, or retrofitted/emulated. There's even STREAMS for Linux. Countless organizations, people, software, ... have benefited from Linux (including Mac OS X).



None of the above is exaggeration - I only used some minor figures of speech. Rather than go on and on haphazardly with this, I think that Linux essentially represents the union of a lot of, if not most, operating system (and related) technology ever created.


I think it's a marvel that such a thing could be created, has been created, and is available for free!



Note that we have used the term "Linux" in the above excerpt in its misnomered sense: to mean a Linux kernel based operating system (in other words, a Linux kernel and a lot of accompanying software). There is so much software (applications, libraries, documentation) available for Linux (mostly for free, no strings attached) that it's not even funny.



Even with all these great things, Linux is lacking, perhaps seriously so, when it comes to the specific case of a client platform. It is not so much because of technology deficiencies as it is because of the fact that despite many (some of them very well-meaning and noble) attempts by several entities (people, organizations, businesses, ...) to create operating systems around Linux, a cohesive enough system hasn't resulted.




If you intend to develop a client computing application for Linux, life is not easy for you. This sounds counter-intuitive to some. Life is supposed to be great if you are a developer on Linux. You have the source code for everything.


You have countless eyes looking at all this source, finding problems, improving performance, and so on. There are libraries to do everything: the most glorious "parts-bin" that there can be. You can choose from numerous toolkits and even multiple desktop environments, each of which provides good, sometimes excellent, APIs.



Do you see a problem?

All this is great if you are a student, a researcher, a hobbyist, a company trying to use Linux to come up with something they can sell, or anybody trying to morph Linux into a very custom operating system for arbitrary use, etc.



It is not great if your domain of development is client computing. Say, you are a developer (individual, or part of a team, etc.) working on, for example, a professional quality movie editing software for Linux (yes, I have looked at Kino - it's a great effort). How long do you think it would take (compared to other platforms)?


Which toolkit(s) would you use? Which desktop environment would you integrate it with? Which distribution would you target it for? How do you know what you "like" is "right"? Which all libraries would you leverage so that you don't reinvent the wheel? How would you ensure it works seamlessly on a random installation?


Do you think there are system wide, distribution agnostic, desktop environment independent "frameworks" that expose various components of the system's functionality? Is it even clear what "the OS" is? Surely, there are numerous ways to do most things, so which would you choose? In this situation, too much choice isn't such a wonderful thing after all. Think about the feasibility and effort requirements of creating something professional level, or even something non-trivial consumer level.



In the context of client computing, Linux's greatest strengths are somewhat offset by this fragmentation and lack of consistent, overall architecture.
There is no common "glue" tying the system together.

Having a bleeding edge kernel with large number of pieces of wide-ranging technology clubbed together, even decently, in an operating system might make a hacker's dream come true, but you need a stronger fabric that runs through the system if you want to contend with Windows and Mac OS X.



Linux indeed gives you an incredible number of choices, except the most important choice: suppose I do not want to think of, worry about, or handle any of this. Suppose I want everything that I need to be there, and to "just work".

Suppose I wish to be oblivious of everything that's "under the hood" (maybe I am technically ill-equipped, maybe I do not have the time or inclination, maybe I want to use the computer as a tool so I can create other things, rather than be my own system administrator or software harvester, ...) - would Linux as a client platform "handle it" for me? No. Not today. Not yet.



I think sometimes the "choice" argument is misused. Consider this: how many people write their own disk drivers on Linux because they have the choice to do so? Very few, if any, because it needs uncommon expertise, it's too low-level for most programmers, it's not necessary because the default ones are good enough, etc.

Well, for client computing, the same logic applies to a lot more things than disk drivers. A lot of paying customers probably find everything to be as low-level as a disk driver. If one wants to sell them Linux, one can't tell them to read a recipe of black-magic look alike instructions to do something, go find software on Freshmeat, or read Word documents with strings if OpenOffice failed to read them ...



There have been several steps in the right direction, though the efforts are often not collaborative, thereby not synergistic. People have tried to standardize things, from rather lower-level standards such as the LSB, to entire human interface guidelines (GNOME, KDE, ...) A historically popular remedy has been to create a new Linux distribution.


It is extremely encouraging to see the amount of attention Linux based operating systems have gotten, even from a client computing perspective. Governments and businesses seem to be very interested in Linux, for cost reasons, as a reaction against Microsoft's monopoly, and other political reasons.


Linux is very popular in emerging countries. While this does give Linux headway, it's no guarantee for success. I believe the "concerns" mentioned above are still valid (and they are not the only obstacles Linux is facing - there are numerous other issues that are beyond the scope of this discussion).


Somebody (a company, perhaps) needs to stand behind Linux in a different way (rather than only coming up with yet another "distribution"). I would like to see somebody take a different approach with Linux - maybe even something like what Apple did while coming up with Mac OS X.


I have stopped using Linux on my primary computer (in favor of Mac OS X). I work on Linux at my work, and run it under Virtual PC at home. In general, it is possible to do most things that one does on Linux on Mac OS X, usually with less effort.



As a digression, I can't help but point out that I believe there exists a psychological trap, wherein one spends a lot of time, over and over, tweaking, configuring, making things work, on an installation, while being under the impression that it's a worthwhile activity ... [more] >>>


Mac OS X


It hasn't been a very long time since I got my first Apple computer (April fool's day in the year 2003, complete with drama), but I agree with Apple's general direction with Mac OS X. I have tried to present a view of the current state of Mac OS X in the previous pages, so that you can judge for yourself if it suits your needs.


Mac OS X has managed to become rather Utopian with time. If you care, it doesn't alienate you from *nix. In fact, it gives you a very colorful environment to play with: you have Mach, FreeBSD, a nice driver development environment, and a lot of the system's source code to go with it.


If you are used to *nix (including Linux), you can get a lot of the same, or similar software on Mac OS X. You can compile it yourself, or hope to find it in one of the port collections.
As discussed in Available Software on Mac OS X, there is a lot of professional quality (mostly commercial) software available for the Mac.


In its current form, Mac OS X is uniquely positioned, unlike any other system, to offer the benefits of the *nix Universe as well as counterparts (either the same or with similar functionality) of important software from the Windows Universe. Apple have been busy adding to Mac OS X a lot of valuable software they created in-house.


Apple's traditional emphasis on ease-of-use seems to work most of the time, although there are exceptions. In my opinion, Mac OS X is representative of a "best-effort" approach - Apple took technology they had collected over the years, along with technology that had flourished in the open source world, and put together a reasonable system.
Currently there is nothing that I want to be able to run (as per my needs and wants) that does not run on Mac OS X.


Mandatory Apple Hardware


The biggest attached string with Mac OS X is the mandatory use of Apple hardware required. Some people get turned off by this, and "Apple should release OS X for x86" is sickeningly clichéd.

People even offer "innovative" solutions to Apple's dilemma by saying that Apple could choose to support a fixed set of x86 hardware (so as to maintain the predictability and robustness that OS X currently has).


Well, Apple should have no problem running OS X on x86 hardware, if that's what they want. NEXTSTEP and OpenStep ran on x86 hardware, and so does Darwin. It is rumored that Apple even has internal x86 builds of Mac OS X.


Apple apparently doesn't want to do that. This issue has been discussed to death time and again by all kinds of experts and analysts. I'm neither, so I'd just leave it at that.


As an aside, why wouldn't you use Apple hardware? It could be that you cannot pay for Apple hardware because you don't have the means, or you don't think hardware should cost as much.

It could be that you hate Apple for making you necessarily buy their machines in order to use Mac OS X. Whatever the reason might be, it's your decision, and your beliefs.

If you look at current Apple computers carefully, you'll see that a lot of engineering goes into making them. They are not just "pretty" and stylish (though that might be subjective) - they are functional and well designed. Some people even call them lust worthy.


If you've never, take a look at the inside of a G5 PowerMac. Do an ioreg -l -w 0 on that machine. There are nine fans and twenty one temperature sensors on it. It's amazingly quiet.
The bottom line is that Apple wants a "Mac with Mac OS X" to be a package deal - in my opinion it's not such a bad deal. As a consumer, you can always pass it up!



The 1-Button Mouse

This is the Traveling Salesman Problem of the Mac Universe. Apparently (as I have learnt over time), all existing "problems" with Apple and Mac OS X are reducible to this. I have had people tell me in all seriousness the following: "Apple cannot even make a mouse with more than one button. They are losers. Mac OS X sucks."

I don't have a good answer, or any answer for that matter, to this "issue". There are plenty of 3rd party mice available. Since Apple is adopting technologies such as X11 (which make heavy use of multiple buttons), they might as well just offer one themselves.


At the same time, I suspect some of the revulsion against a one-button mouse might just be anticipatory.


What is Mac OS X?




Mac OS X Hacking Tools



Hacking? Tool?


The Jargon File is a popular lexicographic resource amongst hackers (and non-hackers too). Although it might have some subjective definitions I may not agree with, I have conveniently quoted verbatim the definitions of the terms "hacker" and "tool" as a preface to the contents of this page.



hacker

[originally, someone who makes furniture with an axe]
1. A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary. RFC1392, the Internet Users' Glossary, usefully amplifies this as: A person who delights in having an intimate understanding of the internal workings of a system, computers and computer networks in particular.
more>>>


tool

1. n.A program used primarily to create, manipulate, modify, or analyze other programs, such as a compiler or an editor or a cross-referencing program. Oppose app, operating system; see also toolchain.
more>>>


So?
It is eminently debatable whether one (that means me, you or whosoever else) is a "hacker", but such a debate would probably be fruitless anyway, even meaningless. I do enjoy exploring the details of all sorts of things, including operating systems.

This page is a compendium of some programs you might come across while tinkering with Mac OS X.


Documentation for most of these tools exists, therefore my aim is not to reproduce documentation, but simply to maintain a cache of relevant information. I believe this would be useful to those who are new to Mac OS X, but are interested in exploring the system at a low(er) level.


Note that many of the tools listed here are ones that are either new to Mac OS X (as compared to Unix style systems), or are different from their Unix counterparts. In other words, I have avoided listing "standard" Unix/BSD tools.

Moreover, do realize that some (like dynamic_pager and various daemons) are not really tools.
The following list has not been fully updated for Panther (10.3.x).



Tools


KernelEventAgent
/usr/sbin/KernelEventAgent handles one of the core system services (events such as file systems being mounted and unmounted, low disk space, network connections going down, etc.)


SystemStarter
/sbin/SystemStarter is run during system initialization to handle "startup items". See "Mac OS X System Startup" for details.
aexml
/usr/sbin/aexml forwards XMLRPC and SOAP requests to the AppleEvent manager for further dispatching. More documentation is available on the Apple Developer Web Site.
appleping
/usr/bin/appleping exercises the AppleTalk network by sending packets to a named host.
ardbgd
/usr/sbin/ardbgd is the daemon for the Apple Remote Debugging Service.
asr
/usr/sbin/asr (Apple Software Restore) efficiently copies disk images and volumes, and can also accurately clone volumes.
bless
/usr/sbin/bless is used to set volume bootability characteristics for Macintoshes. The command can be used to select a folder on a mounted volume to act as the blessed system folder, and optionally update Open Firmware to boot from that volume. It can also be used to format and setup a volume for the first time. Finally, it can be used to query the folder(s) that are blessed on a volume. Try the following (non-destructive) commands:
% sudo bless -verbose -info / ... % sudo bless -verbose -plist -info /
blued
/usr/sbin/blued is the Bluetooth daemon.
cac_*
/usr/sbin/cac_* are scripts related to CAC (Common Access Card) support. A CAC can be thought of as a SmartCard that combines multiple cards (functions) into one. A CAC can enable physical access to buildings and controlled places, enable computer network and system access and serve as the primary platform for the PKI token.
cmpdylib
/usr/bin/cmpdylib compares two dynamic shared libraries for compatibility.
createhomedir
/usr/sbin/createhomedir creates and populates local home directories.
ddb
ddb is a debugging mechanism that can be compiled into Mac OS X, similar to BSD's kdb. While gdb can be used over Ethernet (through a kernel stub), ddb is compiled into the kernel and is used over a serial line. Most importantly, ddb requires an actual built-in hardware serial line on the debug target. Fortunately, gdb should suffice for almost all debugging needs unless one is trying to debug an Ethernet driver itself, say.
ddb is not present by default on Mac OS X. It must be compiled from source (xnu/osfmk/ddb in the CVS tree).
defaults
/usr/bin/defaults is used to access (read, write and delete) Mac OS X user defaults from the command line. For example, the following will print out Desktop background settings (including the pathname for the desktop background image, if any):
% defaults read com.apple.desktop Background
dev_mkdb
/usr/sbin/dev_mkdb creates a hash access method database (based on Berkeley DB) in /var/run/dev.db. This database contains the name of all devices under /dev.
diskarbitrationd
/usr/sbin/diskarbitrationd is a daemon that listens for connections from clients, notifies clients of the appearance of disks and filesystems, and governs the mounting of filesystems and claiming of disks amongst clients.
disktool
/usr/sbin/disktool is a command line utility for disk arbitration. It can be used to rename, eject, mount or unmount disks and volumes.
diskutil
/usr/sbin/diskutil is a utility for managing disks and volumes. It can be used to perform operations such as enabling/disabling HFS+ journaling, verifying and repairing permissions, erasing disks (including optical media), partitioning, creating and managing RAID sets etc. You typically need root access to use this utility.
ditto
/usr/bin/ditto copies files and directories to a destination directory. ditto can be used to "thin" "fat" (multiple-architecture) exectuables. It can also copy files selectively based on the contents of a BOM ("Bill of Materials"). One of the most useful features of ditto is that it can preserve resource fork and HFS meta-data information when copying files.
drutil
/usr/bin/drutil is a command line tool that uses the DiscRecording framework to interact with attached CD/DVD burning devices.
dscl
/usr/bin/dscl is the Directory Service command line utility.
dsperfmonitor
/usr/bin/dsperfmonitor is a directory tool for testing plugin performance in Directory Services.
dynamic_pager
/sbin/dynamic_pager is started during system initialization to manage swap files. See Mac OS X System Startup for details.
fdisk
/usr/sbin/fdisk displays or changes the DOS partition table found in the bootsector of x86 bootable disks.
fixPrecomp
/usr/bin/fixPrecomp is a tool for "fixing" precompiled header warnings that occur when headers get out-of-sync with their precompiled versions - after a system update, say.
fixproc
/usr/bin/fixproc is a Perl script that "fixes" a named process by performing the specified action (which can be check, kill, restart, exist or fix).
fs_usage
/usr/bin/fs_usage presents an ongoing display of system call usage information pertaining to file system activity. By default this includes all system processes except the running fs_usage process, Terminal, telnetd, sshd, rlogind, tcsh, csh and sh.
fstat
/usr/bin/fstat identifies open files (including sockets).
heap
/usr/bin/heap lists all the malloc-allocated buffers in the specified process's heap.
hdiutil
/usr/bin/hdiutil uses the DiskImages framework to manipulate disk image files.
hlfsd
/usr/sbin/hlfsd is the home-link file system daemon. It implements a file system containing a symbolic link to a subdirectory within a user's home directory, depending on the user which accessed that link.
installer
/usr/sbin/installer is the Mac OS X system software and package installer tool.
install_name_tool
/usr/bin/install_name_tool changes the dynamic shared library install names recorded in a Mach-O binary.
ioalloccount
/usr/sbin/ioalloccount displays some accounting of memory allocated by IOKit allocators, including object instances, in the kernel. This is useful for tracking memory leaks.
ioclasscount
/usr/sbin/ioclasscount displays the instance count, offset by the number of direct subclasses that have at least one instance allocated, for the classes specified. This is useful for tracking leaks.
ioreg
/usr/sbin/ioreg displays the IOKit registry. Try ioreg -l, for example, and you can see detailed registry information (including object properties) - such as details of various temperature sensors in the system (on the I2C bus).
iostat
/usr/sbin/iostat displays kernel I/O statistics on terminal, disk and cpu operations.
ipconfig
/usr/sbin/ipconfig can be used to get the number of network interfaces active (the ifcount argument), and also to retrieve various options associated with these interfaces. For example, "ipconfig getoption en1 lease_time" prints the DHCP lease time of en1 if applicable. Finally, ipconfig can also be used to set an interface for BOOTP, DHCP etc.
kdump
/usr/bin/kdump displays the kernel trace files produced with ktrace in human readable format.
kextcache
/usr/sbin/kextcache creates or updates kext caches, which are used to speed up kernel extension loading operations and to prepare kexts for inclusion in such media as device ROM.
kextload
/sbin/kextload can be used to explicitly load kernel extensions, validate them to see that they can be loaded by other mechanisms, such as kextd, and to generate symbol files for debugging the kext in a running kernel.
kextstat
/usr/sbin/kextstat displays the status of any kernel extensions currently loaded in the kernel.
kextunload
/sbin/kextunload is used to terminate and unregister IOKit objects associated with a kernel extension and to unload the code and personalities for that kext.
kgmon
/usr/sbin/kgmon generates a dump of the operating system's profile buffers for later analysis by gprof.
ktrace
/usr/bin/ktrace enables kernel trace logging for the specified processes, causing trace data to be logged to a file. Traced kernel operations include system calls, namei translations, signal processing and I/O.
latency
/usr/bin/latency is used for monitoring scheduling and interrupt latency. The tool can also be used to set real time or timeshare scheduling policies.
ld
/usr/bin/ld is the (Mach) object file link editor.
leaks
/usr/bin/leaks examines a specified process for malloc-allocated buffers which are not referenced by the program.
lipo
/usr/bin/lipo creates or operates on multi-architecture ("fat") files. It can list the architecture types in a fat file, create a single fat file from one or more input files, thin out a single fat file to a specified architecture type, and extract, replace and/or remove architecture types from the input file.
lockfile
/usr/bin/lockfile can be used to create one or more (conditional) semaphore files, with the provision of waiting for a specified number of seconds and a specified number of retries.
lsbom
/usr/bin/lsbom interprets the contents of binary bom (bill-of-materials) files. bom is a file system used by the Mac OS X installer to determine which files to install, remove, or upgrade.
lsof
/usr/sbin/lsof lists information about files opened by processes.
lsvfs
/usr/bin/lsvfs lists known (currently loaded) virtual file systems.
mDNSResponder
/usr/sbin/mDNSResponder (Multicast DNS Responder) listens for and responds to DNS-format query packets sent via Multicast to UDP port 5353.
mach_init
/sbin/mach_init is a daemon that maintains various mappings between service names and the Mach ports that provide access to those services.
malloc_history
/usr/bin/malloc_history inspects a given process and lists the malloc allocations performed by it. It relies on information provided by the standard malloc library when debugging options have been turned on.
mig
/usr/bin/mig (Mach Interface Generator) is used to compile procedural interfaces to Mach's message-based APIs, based on descriptions of those APIs.
mkbom
/usr/bin/mkbom creates a bom (bill-of-materials) given a directory.
mkextunpack
/usr/sbin/mkextunpack extracts the contents of a multikext (mkext) archive.
netstat
/usr/sbin/netstat symbolically displays the contents of various network-related data structures.
nibindd
/usr/sbin/nibindd is a daemon that is responsible for finding, creating and destroying NetInfo servers.
nibtool
/usr/bin/nibtool is used for printing, verifying and updating nib files.
nicl
/usr/bin/nicl is a general-purpose utility for operating on NetInfo databases. Its commands allow one to create, read and manage NetInfo data.
nidomain
/usr/sbin/nidomain is an interface to nibindd to which it sends all of its requests about the domains served on a given machine. It can also be used to create and destroy NetInfo databases.
nifind
/usr/bin/nifind finds a directory in the NetInfo hierarchy.
nigrep
/usr/bin/nigrep searches for a regular expression in the NetInfo hierarchy.
niload
/usr/bin/niload loads information from standard input into the given NetInfo domain.
nireport
/usr/bin/nireport prints tables from the NetInfo hierarchy.
niutil
/usr/bin/niutil is used to do arbitrary reads and writes on the given NetInfo domain.
nmedit
/usr/bin/nmedit is used to change global symbols to local symbols. It differs from strip in that it also changes the symbolic debugging information for the global symbols it changes to static symbols so that the resulting object can still be used with a debugger.
notifyd
/usr/sbin/notifyd is a daemon that facilitates processes to exchange stateless notification events.
nvram
/usr/sbin/nvram allows manipulation of Open Firmware non-volatile RAM variables.
objcopy
objcopy is part of binutils that you can download, compile and install. This utility copies the contents of an object file to another, using the GNU BFD (Binary File Descriptor) library to access the object files.
objdump
objdump is part of binutils. It displays information (including disassembly, if required) about one or more object files.
open
/usr/bin/open is a command line utility to open a file (or a directory or URL), just as if you had double-clicked the file's icon.
open-x11
/usr/bin/open-x11 is a wrapper shell script that provides open functionality for X11 applications.
orbd
/usr/bin/orbd is the Object Request Broker Daemon. It is a tool to enable clients to transparently locate and invoke persistent objects on servers in the CORBA environment.
osacompile
/usr/bin/osacompile compiles the given files, or standard input if non are listed, into a single output script.
osalang
/usr/bin/osalang prints information about installed OSA (Open Script Architecture) languages.
osascript
/usr/bin/osascript executes the given script file, or standard input if none is given. Scripts may be plain text or compiled scripts.
otool
/usr/bin/otool displays specified parts of object files or libraries (similar to ldd on Linux).
pagestuff
/usr/bin/pagestuff displays information about the specified logical pages of a file conforming to the Mach-O executable format.
pax
/bin/pax is a tool for reading, writing, and listing members of an archive file. It is also used to copy directory hierarchies. pax supports various archive formats such as cpio, bcpio, sv4cpio, sv4crc, tar, and ustar.
pbcopy
/usr/bin/pbcopy is used to copy standard input to the pasteboard buffer.
pbpaste
/usr/bin/pbpaste prints the contents of the pasteboard buffer.
pcscd
/usr/sbin/pcscd is a daemon used to dynamically allocate/deallocate Smart Card reader drivers at runtime and manage connections to the readers. Related utilities include /usr/bin/pcsctest and /usr/bin/pcsctool. These tools are taken from the MUSCLE (Movement for the Use of Smart Cards in a Linux Environment) project, a project to coordinate the development of smart cards and applications under Linux.
pdisk
/usr/sbin/pdisk is a menu driven program which partitions disks using the standard Apple disk partitioning scheme.
plutil
/usr/bin/plutil can be used to check the syntax of property list files, or convert a plist file from one format to another.
pmset
/usr/bin/pmset changes and reads power management settings such as idle sleep timing, wake on administrative access, automatic restart on power loss, etc.
pstat
/usr/sbin/pstat displays open file entry, swap space utilization, terminal state, and vnode data structures.
redo_prebinding
/usr/bin/redo_prebinding is used to redo the prebinding of an executable or dynamic library when one of the dependent dynamic library changes. The input file, executable or dynamic library, must have initially been prebound for this program to redo the prebinding.
say
/usr/bin/say uses the Speech Synthesis manager to convert input text to audible speech and either play it through the sound output device chosen in System Preferences or save it to an AIFF file.
screencapture
/usr/sbin/screencapture captures the screen (a window selection or a mouse selection) to the clipboard or a file (as PDF).
scselect
/usr/sbin/scselect is used to change current network location, or to list defined locations.
sc_usage
/usr/bin/sc_usage displays an ongoing sample of system call and page fault usage statistics for a given process.
scutil
/usr/sbin/scutil is a tool to communicate with configd, read and write from/to the configuration data store etc.
security
/usr/bin/security provides a command line interface to administer Keychains, manipulate keys and certificates, and do most things the Security framework is capable of.
segedit
/usr/bin/segedit extracts and/or replaces the named sections from the specified input file and creates an output.
setregion
/usr/bin/setregion is the command line utility for setting the DVD drive's "region".
sips
/usr/bin/sips is a command line interface to the Scriptable Image Processing Server. The graphical abilities of Mac OS X are exposed through this image processing service. The SIPS architecture contains tools for performing basic image alterations and support various image formats. The goal is to provide quick, convenient, desktop automation of common image processing operations.
slpd
/usr/sbin/slpd is the Service Location Protocol daemon that advertises local services to the network.
slp_reg
/usr/sbin/slp_reg is a tool to register URLs via the Service Location Protocol in order for remote machines to discover locally registered services.
softwareupdate
/usr/sbin/softwareupdate is a command line utility to perform software updates under Mac OS X.
srm
/usr/bin/srm securely (by overwriting, renaming, and truncating before unlinking) removes files or directories.
sw_vers
/usr/bin/sw_vers prints the product name (such as Mac OS X), version and build number.
sysctl
/usr/sbin/sysctl retrieves kernel state and allows processes with appropriate privilege to set kernel state.
system_profiler
/usr/sbin/system_profiler is the command line system profiling utility.
tcpdump
/usr/sbin/tcpdump dumps traffic on a network.
top
/usr/bin/top displays an ongoing sample of system usage statistics (such as cpu utilization, memory usage etc. for each process).
trpt
/usr/sbin/trpt interrogates the buffer of TCP trace records created when a socket is marked for debugging (via setsockopt()) and prints a readable description of these records.
update_prebinding
/usr/bin/update_prebinding tries to synchronize prebinding information for libraries and executables when new files are added to a system. Prebinding information is pre-calculated address information for libraries used by a given executable or library. By pre-determining where a function in another library is destined to be placed, the dynamic linker does not have to resolve symbols at application startup time.
vm_stat
/usr/bin/vm_stat displays Mach virtual memory statistics.
vmmap
/usr/bin/vmmap displays the virtual memory regions allocated in a specified process, indicating how memory is being used, and what the purposes of memory at a given address might be.
vpnd
/usr/sbin/vpnd is the Mac OS X VPN service daemon.
xcode*
/usr/bin/xcode* are Xcode related commands.
xxd
/usr/bin/xxd creates a hex dump of a given file or standard input. It can also convert a hex dump back to its original binary form.

A pic of The Apple I:

Zipped file of :

Just about every OS that has had something to do with Mac OS, including a Windows NT 3.51 desktop shot.

OMG that is apple I....heh it kinda sucked just like the ones now (i am a apple hatter :D)