Open source development as a software engineering concept


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Pbm: in English, there is an ambiguity between free=zero dollars and free=freedom.
  • From
  • Public domain: no copyright XFree86 style: can be locked in a commercial product Copylefted: prevent restrictions when redistributing or modifying the software.
  • Rationale: Encouraging lots of improvement is a good thing, but users have a right to know who is responsible for the software they are using. Authors and maintainers have reciprocal right to know what they're being asked to support and protect their reputations. Accordingly, an open-source license must guarantee that source be readily available, but may require that it be distributed as pristine base sources plus patches. In this way, "unofficial" changes can be made available but readily distinguished from the base source.
  • Sun cannot prevent Microsoft to distribute OpenOffice for instance. Rationale: In order to get the maximum benefit from the process, the maximum diversity of persons and groups should be equally eligible to contribute to open sources. Therefore we forbid any open-source license from locking anybody out of the process. Some countries, including the United States, have export restrictions for certain types of software. An OSD-conformant license may warn licensees of applicable restrictions and remind them that they are obliged to obey the law; however, it may not incorporate such restrictions itself.
  • Big problem for researchers. Most of the time, they allow the usage of their software (and modification of their source code) for research purpose only.
  • Rationale: This clause is intended to forbid closing up software by indirect means such as requiring a non-disclosure agreement.
  • Rationale: Distributors of open-source software have the right to make their own choices about their own software. Yes, the GPL is conformant with this requirement. GPLed libraries "contaminate" only software to which they will actively be linked at runtime, not software with which they are merely distributed.
  • CaB, rule 1, page 4. Can be reformulated “necessity is the mother of invention”. Example: People complain about the development of Mozilla: skins have been added whereas the bulk code was not stable. CaB, rule 18, page 18.
  • CaB, rule 2, page 4. Personnal example: adapting an opensource program TableGen to generate Java Beans from tables stored in databases. It did not work properly with mSQL. Getting the source, it was possible to adapt the code for mSQL. Pbm: it was not a beautiful hack, so I didn’t distribute it. CaB, rule 11, page 13
  • CaB, rule 3, page 5. From Fred Brooks, “The mythical man month”, Chapter 11. Example: Windows suffers from code inherited from Dos for backward compatibility. CaB, Rule 12, page 14. CaB, Rule 13, page 15. Quote from Antoine de Saint Exupéry.
  • CaB, rule 6, page 6. CaB, rule 10, page 12
  • CaB, rule 7, page 8 “ Linus was keeping his hacker/users constantly simulated and rewarded – stimulated by the prospect of having an ego-satisfying piece of the action, rewarded by the sight of constant (even daily) improvement in their work.” CaB, page 9 Stimulating is important since only interested people will provide good work (cf rule 1).
  • CaB, rule 8, page 9 Linux Law: “Given enough eyeballs, all bugs are shallow.” “ Somebody finds the problem, and somebody else understands it. And I’ll go on record as saying that finding it is the bigger challenge.” Delphi effect: “ Sociologists years ago discovered that the averaged opinion of a mass of equally expert (or equally ignorant) observers is quite a bit more reliable a predictor than that of a single randomly-chosen one of the observers. They called this the ``Delphi effect''. It appears that what Linus has shown is that this applies even to debugging an operating system -- that the Delphi effect can tame development complexity even at the complexity level of an OS kernel. ” CaB page 10. CaB, page 21: “ In ``The Mythical Man-Month'', Fred Brooks observed that programmer time is not fungible; adding developers to a late software project makes it later. He argued that the complexity and communication costs of a project rise with the square of the number of developers, while work done only rises linearly. This claim has since become known as ``Brooks's Law'' and is widely regarded as a truism. But if Brooks's Law were the whole picture, Linux would be impossible. Gerald Weinberg's classic ``The Psychology Of Computer Programming'' supplied what, in hindsight, we can see as a vital correction to Brooks. In his discussion of ``egoless programming'', Weinberg observed that in shops where developers are not territorial about their code, and encourage other people to look for bugs and potential improvements in it, improvement happens dramatically faster than elsewhere. ”
  • Lawrence reply to Eric Raymond about CaB.
  • Quake is working as client/server game. After the release of Quake 1 sources under GPL, new clients with ability to cheat appeared. (some information were provided to the client before the player can see them, an the client got the responsibility to use them adequately).
  • Artistic license: notion of “Standard Version”.
  • Homesteading the Noosphere, page 12.
  • Open source development as a software engineering concept

    1. 1. Open source development as a software engineering concept Daniel Le Berre The University Of Newcastle
    2. 2. Overview of the lecture <ul><li>History of open Source </li></ul><ul><li>Definition </li></ul><ul><li>Major features of Open Source development </li></ul><ul><li>Why to open source (summary) </li></ul><ul><li>Making money with open source </li></ul><ul><li>Case study: Netbeans/Forte4J (Sun) </li></ul>
    3. 3. Brief History of Open Source Software <ul><li>1984: The GNU (GNU’s Not Unix) project is launched by Richard Mathew Stallman (RMS) Notion of “Free Software” </li></ul><ul><li>1991: Linus Torvalds announces the availability of his Unix kernel, Linux . </li></ul><ul><li>1997: Eric Raymond writes “The cathedral and the Bazaar”. (using “free software”). </li></ul><ul><li>1998: The term “ open source ” is used by ER after the announce of Netscape to release the sources of its browser. </li></ul>
    4. 4. Open-Source: Two religions Richard Stallman (FSF) Eric Raymond (OSI)
    5. 5. Free Software Foundation <ul><li>Stallman is a saint in the Church of Emacs---Saint IGNUcius. </li></ul><ul><li>Sainthood in the Church of Emacs requires living a life of purity--but in the Church of Emacs, this does not require celibacy (a sigh of relief is heard). Being holy in our church means installing a wholly free operating system --GNU/Linux is a good choice--and not putting any non-free software on your computer . Join the Church of Emacs, and you too can be a saint! </li></ul>
    6. 6. Open Source Initiative <ul><li>[…] the real reason for the re-labeling is a marketing one . We're trying to pitch our concept to the corporate world now. We have a winning product , but our positioning, in the past, has been awful. The term &quot;free software&quot; has a load of fatal baggage ; to a businessperson, it's too redolent of fanaticism and flakiness and strident anti-commercialism. Mainstream corporate CEOs and CTOs will never buy &quot;free software,&quot; manifestos and clenched fists and all. But if we take the very same tradition, the same people, and the same free-software licenses and change the label to &quot;open source&quot; – that, they'll buy. </li></ul>
    7. 7. The meaning of “free” software
    8. 8. Quick Open Source definition (1) <ul><li>Open source promotes software reliability and quality by supporting independent peer review and rapid evolution of source code. </li></ul>
    9. 9. Quick Open Source Definition (2) <ul><li>Open source promotes software reliability and quality by supporting independent peer review and rapid evolution of source code. To be OSI certified, the software must be distributed under a license that guarantees the right to read, redistribute, modify, and use the software freely. </li></ul>
    10. 10. The GNU Project <ul><li>A free Unix-like software system </li></ul><ul><li>It is not just an OS, but also </li></ul><ul><ul><li>Compiler </li></ul></ul><ul><ul><li>Debugger </li></ul></ul><ul><ul><li>Linker </li></ul></ul><ul><ul><li>Shell </li></ul></ul><ul><ul><li>Editor </li></ul></ul><ul><ul><li>… </li></ul></ul>
    11. 11. Linux <ul><li>Initially a new kernel to extend Minix </li></ul><ul><li>Then it develops by himself </li></ul><ul><li>Linux is just a kernel , not a complete OS! </li></ul><ul><li>GNU Tools + Linux = GNU/Linux </li></ul><ul><li>(alternative to GNU/Linux exists: GNU/Hurd for instance) </li></ul>
    12. 12. The cathedral and The bazaar <ul><li>First worldwide claim of the importance of free software development </li></ul><ul><li>Consequences </li></ul><ul><ul><li>Netscape decides to release Mozilla </li></ul></ul><ul><ul><li>The Halloween documents written at Microsoft </li></ul></ul><ul><ul><li>Huge media cover! </li></ul></ul>
    13. 13. Famous Open Source Projects <ul><li>Emacs /GCC/GNU Tools (FSF) </li></ul><ul><li>GNU/Linux, FreeBSD, OpenBSD (Free Unix) </li></ul><ul><li>Apache (daemon http, 60% of the web servers) </li></ul><ul><li>KDE /GNOME (desktops for Linux) </li></ul><ul><li>MySQL (database) </li></ul><ul><li>… </li></ul>
    14. 14. Some [Commercial] Open Source projects <ul><li>Mozilla / Netscape: web browser www. mozilla .org </li></ul><ul><li>OpenOffice / StarOffice: MS Office clone www. openoffice .org </li></ul><ul><li>NetBeans / Forte for Java: Java IDE www. netbeans .org </li></ul><ul><li>Darwin / MacOs X : operating system www. opensource </li></ul>
    15. 15. Open Source definition (from www. opensource .org ) <ul><li>Open source doesn't just mean access to the source code ! </li></ul><ul><ul><li>Free Redistribution </li></ul></ul><ul><ul><li>Source Code </li></ul></ul><ul><ul><li>Derived Works </li></ul></ul><ul><ul><li>Integrity of The Author's Source Code </li></ul></ul><ul><ul><li>No Discrimination Against Persons or Groups </li></ul></ul><ul><ul><li>No Discrimination Against Fields of Endeavor </li></ul></ul><ul><ul><li>Distribution of License </li></ul></ul><ul><ul><li>License Must Not Be Specific to a Product </li></ul></ul><ul><ul><li>License Must Not Contaminate Other Software </li></ul></ul>
    16. 16. Free Redistribution <ul><li>The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale. </li></ul>
    17. 17. Source Code <ul><li>The program must include source code, and must allow distribution in source code as well as compiled form . Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost–preferably, downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program . Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed. </li></ul>
    18. 18. Derived Works <ul><li>The license must allow modifications and derived works , and must allow them to be distributed under the same terms as the license of the original software. </li></ul>
    19. 19. Integrity of The Author's Source Code <ul><li>The license may restrict source-code from being distributed in modified form only if the license allows the distribution of &quot;patch files&quot; with the source code for the purpose of modifying the program at build time. The license must explicitly permit distribution of software built from modified source code. The license may require derived works to carry a different name or version number from the original software. </li></ul>
    20. 20. No Discrimination Against Persons or Groups <ul><li>The license must not discriminate against any person or group of persons . </li></ul>
    21. 21. No Discrimination Against Fields of Endeavor <ul><li>The license must not restrict anyone from making use of the program in a specific field of endeavor . For example, it may not restrict the program from being used in a business, or from being used for genetic research. </li></ul>
    22. 22. Distribution of License <ul><li>The rights attached to the program must apply to all to whom the program is redistributed without the need for execution of an additional license by those parties. </li></ul>
    23. 23. License Must Not Be Specific to a Product <ul><li>The rights attached to the program must not depend on the program's being part of a particular software distribution . If the program is extracted from that distribution and used or distributed within the terms of the program's license, all parties to whom the program is redistributed should have the same rights as those that are granted in conjunction with the original software distribution. </li></ul>
    24. 24. License Must Not Contaminate Other Software <ul><li>The license must not place restrictions on other software that is distributed along with the licensed software. For example, the license must not insist that all other programs distributed on the same medium must be open-source software. </li></ul>
    25. 25. Key of Success <ul><li>Launching a project/participating is cheap : </li></ul><ul><ul><li>Need only a computer and an internet access </li></ul></ul><ul><li>Internet = worldwide! </li></ul><ul><li>Evolutionary concept (genetic algorithm, Darwinian process) </li></ul><ul><ul><li>Several variants of a program can be created, the best will survive. </li></ul></ul><ul><li>Reward: </li></ul><ul><ul><li>Making something </li></ul></ul><ul><ul><li>Peer recognition </li></ul></ul>
    26. 26. “ Every good work of software starts by scratching a developer’s personal itch ” <ul><li>“ To solve an interesting problem, start by finding a problem that is interesting to you .” </li></ul><ul><li>Usually, open source software is coming from someone specific need (writing a driver for its own graphic card, an emacs mode for its software, etc.) </li></ul><ul><li>People needs are different: </li></ul><ul><li>Efficiency </li></ul><ul><li>Features </li></ul><ul><li>GUI </li></ul><ul><li> The software evolves in different ways in parallel ! </li></ul>
    27. 27. “ Good programmers know what to write. Great ones know what to rewrite (and reuse)” <ul><li>“ The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better. ” </li></ul><ul><li>Adapting software for its own need save time and money </li></ul><ul><li>Classical concept of OOP/software engineering </li></ul>
    28. 28. “ Plan to throw one away; you will, anyhow.” <ul><li>“ Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.” </li></ul><ul><li>“ Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.” </li></ul>
    29. 29. “ Treating your users as co-developers is your least-hassle route to rapid improvement and effective debugging.” <ul><li>“ If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource. ” </li></ul><ul><li>Beware: works well only if your users are also developers. </li></ul>
    30. 30. “ Release early. Release Often. And Listen to your customers.” <ul><li>Releases can happen every day! </li></ul><ul><li>Using a stable/development policy can help. </li></ul><ul><ul><li>(Linux kernel: 2.0.2 stable, 2.1.4 developer) </li></ul></ul><ul><li>Helps to keep users/developers interested </li></ul>
    31. 31. “ Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly” <ul><li>Internet is worldwide, so many (an unexpected) potential beta-tester available. </li></ul><ul><li>You do not pay for the software, so you understand some bugs can occur. </li></ul><ul><li>“ Delphi” effect </li></ul><ul><li>Against the law: “adding developers to a late project makes it later.” </li></ul>
    32. 32. Necessary Preconditions to Bazaar design <ul><li>Needs something runnable and testable to play with. </li></ul><ul><li>Certain level of design and coding skills. </li></ul><ul><li>A bazaar project coordinator or leader must have good people and communication skills. </li></ul>
    33. 33. Against the Bazaar <ul><li>Running unstable software (defect installation) </li></ul><ul><li>“ People are quick to volunteer to help, but slow to help.” </li></ul>
    34. 34. Open Source and Security <ul><li>Doesn't closed source help protect against crack attacks? </li></ul><ul><li>Security holes can be fixed quickly with open source software. </li></ul><ul><li>The Quake cheats: </li></ul><ul><ul><li>Problem of conception (tradeoff security/performances), not of Open Source </li></ul></ul>
    35. 35. Why Open Source ? <ul><li>Reliability/stability/scalability are critical </li></ul><ul><li>Correctness of design and implementation is not readily verified by means other than independent peer review </li></ul><ul><li>the software is critical to the user’s control of his/her business </li></ul><ul><li>the software establishes or enables a common computing and communication infrastructure. </li></ul><ul><li>Key methods are part of common engineering knowledge </li></ul>
    36. 36. Some licenses <ul><li>GNU General Public License (GNU GPL) </li></ul><ul><ul><li>All distributed software including GPL code must be GPL. </li></ul></ul><ul><li>GNU Lesser GPL (GNU LGPL) </li></ul><ul><ul><li>Non-contaminating GPL (for libraries) </li></ul></ul><ul><li>The Artistic license </li></ul><ul><ul><li>To keep an “artistic”control over the development of a package (“standard version”). </li></ul></ul>
    37. 37. Others Open Source Licenses <ul><li>The BSD license </li></ul><ul><li>The MIT license </li></ul><ul><li>The Mozilla Public License v. 1.0 (MPL) </li></ul><ul><li>The Qt Public License (QPL) </li></ul><ul><li>The IBM Public License </li></ul><ul><li>The MITRE Collaborative Virtual Workspace License (CVW License) </li></ul><ul><li>The Ricoh Source Code Public License </li></ul><ul><li>The Python license </li></ul><ul><li>The zlib/libpng license </li></ul><ul><li>The Apache Software License </li></ul><ul><li>The Vovida Software License v. 1.0 </li></ul><ul><li>The Sun Internet Standards Source License (SISSL) </li></ul><ul><li>The Intel Open Source License </li></ul><ul><li>The Mozilla Public License 1.1 (MPL 1.1) </li></ul><ul><li>The Jabber Open Source License </li></ul>
    38. 38. Making money from Open Source? <ul><li>There are several strategies </li></ul><ul><ul><li>Loss-Leader/Market-Positioner (Netscape) </li></ul></ul><ul><ul><li>Widget Frosting (Apple) </li></ul></ul><ul><ul><li>Give Away the recipe, Open a restaurant (Redhat) </li></ul></ul><ul><ul><li>Accessorizing (O’Reilly) </li></ul></ul><ul><ul><li>Free the future, sell the present (Alladin) </li></ul></ul><ul><ul><li>Free the software, sell the brand </li></ul></ul><ul><ul><li>Free the software, sell the content </li></ul></ul><ul><li>(see The Magic Cauldron, Eric Raymond, for more details) </li></ul>
    39. 39. Loss-leader/Market Positionner <ul><li>To create or maintain a market position for proprietary software that generates a direct revenue stream. In the most common variant, open-source client software enables sales of server software, or subscription/advertising revenue associated with a portal site. </li></ul><ul><li>Examples: Mozilla, OpenOffice </li></ul>
    40. 40. Widget Frosting <ul><li>For hardware manufacturers (including anything from peripheral boards all the way up to entire computer system) </li></ul><ul><li>Advantage: nothing to loose </li></ul><ul><li>Example: Apple with Darwin </li></ul>
    41. 41. Give away the recipe, open a restaurant <ul><li>Create a market position for services (instead of proprietary software in the Loss-Leader/Market-Positioner case) </li></ul><ul><li>Examples: Cygnus Solution (translation of GNU tools) RedHat, Mandrake and all other Linux commercial distributions. </li></ul><ul><li>Local example (Newcastle): Cybersite www. cybersite </li></ul>
    42. 42. Accessorizing <ul><li>To sell accessories for open source software. (from mugs, T-shirt up to books) </li></ul><ul><li>Example: O’Reilly </li></ul>
    43. 43. From closed to open source <ul><li>A software can be initially released as closed software and becoming open source </li></ul><ul><li>Example: the game DOOM (Id Software) </li></ul><ul><ul><li>Initially (late 93): the first one in his category! (better to stay closed source) </li></ul></ul><ul><ul><li>After a while, competitors appeared (Duke Nukem) </li></ul></ul><ul><ul><li>Late 97: releasing source opened a second market for the game! </li></ul></ul>
    44. 44. Case study: NetBeans/Forte4J <ul><li> </li></ul><ul><li>NetBeans started as a student project in the Czech Republic (originally called Xelfi), in 1996. The goal was to write a Delphi-like Java IDE in Java. A company was formed around this project, called NetBeans. There were two commercial versions of NetBeans, called Developer 2.0 and 2.1. Around May of 1999, NetBeans released a beta of what was to be Developer 3.0 - some months later, in October '99, NetBeans was acquired by Sun Microsystems . After some additional development time, Sun released the Forte for Java Community Edition IDE - the same IDE that had been in beta as NetBeans Developer 3.0. </li></ul><ul><li>There had always been interest in going Open Source at NetBeans. In June 2000, Sun open-sourced the NetBeans IDE … </li></ul>
    45. 45. Forte4J conclusions (1) <ul><li>Feature Set </li></ul><ul><li>The developer community gets more of the features they want, with none of the commercial clutter. </li></ul><ul><li>The community really has gotten more of the features they want . However, a few points should probably be deducted for some &quot;commercial clutter&quot; features that nevertheless help the product. This probably rates as an &quot;A–&quot;. </li></ul><ul><li>Evolution </li></ul><ul><li>The technology evolves faster as more developer effort can be applied to the features. </li></ul><ul><li>The technology has evolved very quickly, and the collaborative effects are notable. Several companies are helping push the technology forward, but there are few miracles and the laws of software development physics still apply . This probably rates a &quot;B&quot;. </li></ul>
    46. 46. Forte4J conclusions (2) <ul><li>Stability </li></ul><ul><li>Quality happens sooner as every beta tester can be a bug fixer. </li></ul><ul><li>This has proven true, particularly in the early days of the NetBeans project . If the population of users gets biased increasingly toward </li></ul><ul><li>commercial developers less inclined towards fixing tool bugs, this effect will diminish. On balance, however, the effort involved in the testing of bug fixes can be expected to increase as the code base grows, so the bug fixes themselves become a less significant benefit. This probably gets a &quot;B&quot; as well. </li></ul>
    47. 47. Forte4J conclusions (3) <ul><li>Cost </li></ul><ul><li>Development costs are lower as a result of harnessing talent pools from around the world. </li></ul><ul><li>Development costs are not measurably lower, and there is a lot of conflicting data. Since it can't really be measured, this aspect will have to be given an &quot;incomplete&quot;. </li></ul><ul><li>Users </li></ul><ul><li>User populations are higher as the software is free in binary and source form. </li></ul><ul><li>User populations are indeed dramatically higher . In fact, the biggest 'competitor' to the Forte for Java IDE is Emacs– another free software product. This aspect deserves an &quot;A&quot;. </li></ul>
    48. 48. Forte4J conclusions (4) <ul><li>ISVs (independent software vendors) </li></ul><ul><li>ISVs are more likely to base products on open source, rather than proprietary code basis. </li></ul><ul><li>This also seems to have been borne out in reality. It is interesting to note that most ISVs have not yet been willing to put their own intellectual property into Open Source, but they are very attracted by Sun's doing so . This also earns an &quot;A&quot;. </li></ul>
    49. 49. References <ul><li>www. opensource .org Eric Raymond parish “Open Source Initiative” </li></ul><ul><li>www. fsf .org Richard Stallman parish “Free Software Foundation” </li></ul><ul><li> esr / Eric Raymond writings </li></ul>
    50. 50. Open Source = gift culture? <ul><li>“ Gift cultures are adaptations to abundance. Appears in populations that do not have significant material-scarcity problems with survival goods.” </li></ul><ul><li>“ The only available measure of competitive success is reputation among one’s peer” </li></ul><ul><li> Same thing for research culture/academia </li></ul>