Eric S. Raymond Goodbye, "free software"; hello, "open source" by Eric Raymond http://www.catb.org/~esr/open-source.html original call to the community to start using the term ‘open source‘ that I issued on 8 February 1998. The event referred to in the first paragraph is the 23 January announcement of the Mozilla source code release. Goodbye, "free software"; hello, "open source" After the Netscape announcement broke in January I did a lot of thinking about the next phase -- the serious push to get "free software" accepted in the mainstream corporate world. And I realized we have a serious problem with "free software" itself. problem with the term "free software", itself, not the concept. the term "free" is very ambiguous Does "free" mean "no money charged?" or does it mean "free to be modified by anyone", or something else? Second, the term makes a lot of corporate types nervous. ... There's now a chance we can make serious gains in the mainstream business world without compromising our ideals and commitment to technical excellence -- so it's time to reposition. We need a new and better label. I brainstormed this with some Silicon Valley fans of Linux (including Larry Augustin of the Linux International board of directors) the day after my meeting with Netscape (Feb 5th). We kicked around and discarded several alternatives, and we came up with a replacement label we all liked: "open source". >>> a new label We suggest that everywhere we as a culture have previously talked about "free software", the label should be changed to "open source". Open-source software. The open-source model. The open source culture. The Debian Open Source Guidelines. (In pitching this to the corporate world I'm also going to be invoking the idea of "peer review" a lot.) Linus Torvalds has been saying in "World Domination 101" that the open-source culture needs to make a serious effort to take the desktop and engage the corporate mainstream.Of course he's right -- and this re-labeling, as Linus agrees, is part of the process. It says we're willing to work with and co-opt the market for our own purposes, rather than remaining stuck in a marginal, adversarial position. This re-labeling has since attracted a lot of support (and some opposition) in the hacker culture. It's crunch time, people. The Netscape announcement changes everything. We've broken out of the little corner we've been in for twenty years. We're in a whole new game now, a bigger and more exciting one -- and one I think we can win. ************************************* The Cathedral and the Bazaar Eric Steven Raymond http://www.catb.org/esr/writings/cathedral-bazaar/cathedral-bazaar/ Copyright � 2000 Eric S. Raymond This is version 3.0 Linux is subversive. Who would have thought even five years ago (1991) that a world-class operating system could coalesce as if by magic out of part-time hacking by several thousand developers scattered all over the planet, connected only by the tenuous strands of the Internet? I believed that the most important software (operating systems and really large tools like the Emacs programming editor) needed to be built like cathedrals, carefully crafted by individual wizards or small bands of mages working in splendid isolation, with no beta to be released before its time. Linus Torvalds's style of development—release early and often, delegate everything you can, be open to the point of promiscuity—came as a surprise. No quiet, reverent cathedral-building here—rather, the Linux community seemed to resemble a great babbling bazaar of differing agendas and approaches (aptly symbolized by the Linux archive sites, who'd take submissions from anyone) out of which a coherent and stable system could seemingly emerge only by a succession of miracles. By mid-1996 I thought I was beginning to understand. Chance handed me a perfect way to test my theory, in the form of an open-source project that I could consciously try to run in the bazaar style. So I did—and it was a significant success. This is the story of that project. I'll use it to propose some aphorisms about effective open-source development. Not all of these are things I first learned in the Linux world, but we'll see how the Linux world gives them particular point. If I'm correct, they'll help you understand exactly what it is that makes the Linux community such a fountain of good software—and, perhaps, they will help you become more productive yourself. FETCHMAIL development preconditions for successful bazaar-style development, including both the qualifications of the project leader and the state of code at the time one goes public and starts to try to build a co-developer community. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode. Linus didn't try it. I didn't either. Your nascent developer community needs to have something runnable and testable to play with. When you start community-building, what you need to be able to present is a plausible promise. Your program doesn't have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is (a) run, and (b) convince potential co-developers that it can be evolved into something really neat in the foreseeable future. In his discussion of ``egoless programming'', Weinberg observed that in shops where developers are not territorial about their code, and encourage other people to look for bugs and potential improvements in it, improvement happens dramatically faster than elsewhere Before cheap Internet, there were some geographically compact communities where the culture encouraged Weinberg's ``egoless'' programming, and a developer could easily attract a lot of skilled kibitzers and co-developers. Bell Labs, the MIT AI and LCS labs, UC Berkeley—these became the home of innovations that are legendary and still potent. Linux was the first project for which a conscious and successful effort to use the entire world as its talent pool was made. I don't think it's a coincidence that the gestation period of Linux coincided with the birth of the World Wide Web, and that Linux left its infancy during the same period in 1993–1994 that saw the takeoff of the ISP industry and the explosion of mainstream interest in the Internet. Linus was the first person who learned how to play by the new rules that pervasive Internet access made possible. While cheap Internet was a necessary condition for the Linux model to evolve, I think it was not by itself a sufficient condition. Another vital factor was the development of a leadership style and set of cooperative customs that could allow developers to attract co-developers and get maximum leverage out of the medium. cooperative customs But what is this leadership style and what are these customs? They cannot be based on power relationships—and even if they could be, leadership by coercion would not produce the results we see. Weinberg quotes the autobiography of the 19th-century Russian anarchist Pyotr Alexeyvich Kropotkin's Memoirs of a Revolutionist to good effect on this subject: "I began to appreciate the difference between acting on the principle of command and discipline and acting on the principle of common understanding. The former works admirably in a military parade, but it is worth nothing where real life is concerned, and the aim can be achieved only through the severe effort of many converging wills." Linus, by successfully positioning himself as the gatekeeper of a project in which the development is mostly done by others, and nurturing interest in the project until it became self-sustaining, has shown an acute grasp of Kropotkin's ``principle of shared understanding''. Both the fetchmail and Linux kernel projects show that by properly rewarding the egos of many other hackers, a strong developer/coordinator can use the Internet to capture the benefits of having lots of co-developers without having a project collapse into a chaotic mess. Perhaps in the end the open-source culture will triumph not because cooperation is morally right or software ``hoarding'' is morally wrong (assuming you believe the latter, which neither Linus nor I do), but simply because the closed-source world cannot win an evolutionary arms race with open-source communities that can put orders of magnitude more skilled time into a problem. lessons from Linux: I needed a POP3 client. So I went out on the Internet and found one. Actually, I found three or four. I used one of them for a while, but it was missing what seemed an obvious feature, the ability to hack the addresses on fetched mail so replies would work properly. The problem was this: suppose someone named `joe' on locke sent me mail. If I fetched the mail to snark and then tried to reply to it, my mailer would cheerfully try to ship it to a nonexistent `joe' on snark. Hand-editing reply addresses to tack on <@ccil.org> quickly got to be a serious pain. This was clearly something the computer ought to be doing for me. But none of the existing POP clients knew how! And this brings us to the first lesson: 1. Every good work of software starts by scratching a developer's personal itch. 2. Good programmers know what to write. Great ones know what to rewrite (and reuse). 3. ``Plan to throw one away; you will, anyhow.'' (Fred Brooks, The Mythical Man-Month, Chapter 11) In a software culture that encourages code-sharing, this is a natural way for a project to evolve. I was acting out this principle: 4. If you have the right attitude, interesting problems will find you. 5. When you lose interest in a program, your last duty to it is to hand it off to a competent successor. 6. Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging. The power of this effect is easy to underestimate. In fact, pretty well all of us in the open-source world drastically underestimated how well it would scale up with number of users and against system complexity, until Linus Torvalds showed us differently. In fact, I think Linus's cleverest and most consequential hack was not the construction of the Linux kernel itself, but rather his invention of the Linux development model. When I expressed this opinion in his presence once, he smiled and quietly repeated something he has often said: ``I'm basically a very lazy person who likes to get credit for things other people actually do.'' Lazy like a fox. Or, as Robert Heinlein famously wrote of one of his characters, too lazy to fail. Linus was treating his users as co-developers in the most effective possible way: 7. Release early. Release often. And listen to your customers. 8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone. Or, less formally, ``Given enough eyeballs, all bugs are shallow.'' I dub this: ``Linus's Law''. ``will be transparent to somebody''. Linus demurred that the person who understands and fixes the problem is not necessarily or even usually the person who first characterizes it. ``Somebody finds the problem,'' he says, ``and somebody else understands it. And I'll go on record as saying that finding it is the bigger challenge.'' Linus's Law can be rephrased as ``Debugging is parallelizable'' 9. Smart data structures and dumb code works a lot better than the other way around. 10. If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource. 11. The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better. 12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong. 13. ``Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.'' 14. Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected. 16. When your language is nowhere near Turing-complete, syntactic sugar can be your friend. 17. A security system is only as secure as its secret. Beware of pseudo-secrets. 18. To solve an interesting problem, start by finding a problem that is interesting to you. 19: Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one. I think the future of open-source software will increasingly belong to people who know how to play Linus's game, people who leave behind the cathedral and embrace the bazaar. This is not to say that individual vision and brilliance will no longer matter; rather, I think that the cutting edge of open-source software will belong to people who start from individual vision and brilliance, then amplify it through the effective construction of voluntary communities of interest. Perhaps in the end the open-source culture will triumph not because cooperation is morally right or software ``hoarding'' is morally wrong (assuming you believe the latter, which neither Linus nor I do), but simply because the closed-source world cannot win an evolutionary arms race with open-source communities that can put orders of magnitude more skilled time into a problem. ----- reverent cathedral-building speed barely imaginable to cathedral-builders ************************* rms bio http://www.catb.org/esr/writings/rms-bio.html I have known RMS since we were teenage SF fans back in the mid 1970s. I've met his family. I've collaborated with RMS and argued with him and hung out with him for more than twenty-five years. For all that events since 1998 have forced us into a sort of rivalry, RMS and I remain among each others' oldest friends. the mythology RMS created around himself. RMS's account of his own development tends to be streamlined, narratized. Once the story had been absorbed into Richard's personal narrative of prophecy, struggle, and mission, the actual time and circumstances of the story became unimportant to him and forgettable. Richard's own tendency to emphasize the printer-driver incident over the consequences of the Symbolics War tends to downplay the cultural context in which he operated — one consisting not merely of other MIT hackers such as Steele, Gosper, Abelson, Moon, et. al. but of hackers in other traditions — the Unix guys, the Xerox PARC crowd, the SAIL group. The real untold story here isn't Richard's individual development of a sense of mission — that prophet-out-of-the-desert tale has been rehearsed way too many times. It's the way in which RMS's choices grew organically from a hacker culture that already existed, and which informed RMS's values and the range of his choices on many levels. I knew there was something monstrously wrong about a software-production system that alienated me from the product of my creativity, buried my work under secrecy and bad management decisions. But I thought Richard's anti-propertarian dream was crazy, a nightmare that would lead to starving programmers and no good result. the central problem RMS was grappling with — how to fix the proprietary-software system that was crushing the creative life out of us? We couldn't sign on to Richard's eccentric messianism. Richard's genius was that he exteriorized his personal journey as a plan for a movement, then mythologized it and took it on the road — providing a model for me when I felt it necessary to do the same thing ten years later. It could have been any of us, though. Could have been Seth, or Alan, or Don, or Oz, or Mark, or Karl, or possibly any of a half-dozen other geekboy geniuses I knew in SF fandom whose names have escaped me. While it took individual vision and moral courage to act out this feeling as thoroughly as Richard did, the sense of obligation, of being elected by the fact of one's own capability wouldn't have surprised any hacker of that day or today. Hackers often react like this. See something that needs doing, do it — don't count the cost. In this respect as in many others, RMS was as much a product of the hacker culture as a shaper of it. He did act. He did change the world, if not perhaps in the exact direction and degree he intended. RMS put his stamp on the hacker culture so pervasively that it is sometimes difficult to tell where Richard's reinvention of himself ends and the cultural matrix around him begins. I admired him greatly for that even as I disagreed with him, having no clue that within a few years I would be doing the same sort of thing myself. Richard would never have described himself the way Linus Torvalds and I later did, as accidental revolutionaries. Linus and I had to be dragged into that role and we still aren't entirely comfortable with it, but for the man who became RMS, it was purposeful revolution from the beginning. Because when you're a genius in the secularized-Jewish tradition that produced Spinoza and Marx, that's what you do — you seize on a big moral idea and redeem the world with it. RMS's drama, the eremite hero/prophet whose individual moral vision and self-martyrdom changes the world. That story-line requires that, despite intense suffering and rejection, the prophet's vision ultimately triumph — the greater the suffering, the more complete the eventual victory.So FAIF requires the canonical ending — that RMS's "star has grown", that his moral crusade is stronger today than it has ever been. The failure of the GNU project at its central goal (production of the HURD kernel) became undeniable in 1995-1996, and as a result RMS has been steadily losing influence to others since then. In 1995 nobody could even name anyone but Stallman with a credible claim to lead the hackers. Now, all but the hardest-core FSF supporters would rank Linus Torvalds above him in influence, a good many would add Larry Wall to that list, and not a few would add me. As an index of the change, do a Google hit count on the terms "open source" and "free software" (with "GNU or GPL or FSF or Stallman") as qualifiers In other words, the number of webpages that endorse RMS's "free software" label and his refusal to countenance "open source" dropped from 38% to 16% of the whole set during that year, and not just in market share but in absolute numbers as well. This was a crude test, but it fits what I observe on my travels. It's not a picture of burgeoning success, or even of a long march to victory. The community has spoken. RMS's propaganda tactics have been found wanting and largely abandoned for more effective ones. the "free software" message seems in the process of being completely eclipsed by the "open source" message there is indeed a sense in which RMS is winning, but not at the exact game he wanted to play. RMS's artifacts — GCC, Emacs, the GNU General Public License — really have changed the world. The process of open, collaborative development he did so much to help invent is triumphing. His code and his license have succeeded; it is only his rhetoric and moralizing that have failed. The tragedy is that RMS himself values his moralizing more than his code. Had it not been so, had RMS been a bit less consumed by his big moral idea and a bit better salesman, the hacker culture would not have landed on me the job of saying what he could not. I might have been a fully paid-up member of his crusade, rather than winding up as an unwilling rival. I often think I would have preferred that outcome. Perhaps then RMS's story would have had the happy, redemptive outcome FAIF projects. But that's not the history we got. Instead, I think RMS will never be satisfied with the victory he gets, even as it transforms the world.