The Open Source Secret Sauce




 15 Sep 2011
 Rochester NY
 Lugor User Group
Abstract
How do volunteer open-source projects
create and maintain so many compelling,
competitive products?
What is the Open Source Secret Sauce?
Join open-source insider, Ted Husted, as
he takes us deep inside the Apache
Software Foundation, to show how the
sausages are made.
About the Speaker
Ted Husted was a senior member of the Apache Struts
development group from 2000 to 2008.
His books include Google Wave Explained, JUnit in Action,
Struts in Action, and Professional JSP Site Design.
Ted has consulted with teams throughout the United States,
including CitiGroup, Nationwide Insurance, and Pepsi
Bottling Group.
He has been a regular speaker at ApacheCon US and the
Ajax Experience, as well as to various user groups.
Ted is currently working with NimbleUser of Pittsford NY, as
a business analyst.
For more, visit http://ted.husted.com/
Open Source Secret Sauce
               ● What’s the opportunity?
               ● Does open source matter?
               ● How does it work?
               ● Can you do it too?




    http://www.slideshare.net/ted.husted/
What’s the problem?
● Overall, about 15% of all projects are
  delivered on-time and on-budget.




            Source: The Standish Group
What’s the problem?
● Overall, about 15% of all projects are
  delivered on-time and on-budget.
● About 30% of software projects are
  cancelled before completion.




            Source: The Standish Group
What’s the problem?
● Overall, about 15% of all projects are
  delivered on-time and on-budget.
● About 30% of software projects are
  cancelled before completion.
● Half of all software projects cost almost
  twice as much as anticipated.



            Source: The Standish Group
What’s the problem?
● Overall, about 15% of all projects are
  delivered on-time and on-budget.
● About 30% of software projects are
  cancelled before completion.
● Half of all software projects cost almost
  twice as much as anticipated.
● More than 50% of software projects
  fail.
            Source: The Standish Group
Why do projects fail?




http://www.stylusinc.com/Common/Concerns/SoftwareProjectsFailure.php
Why do projects succeed?




http://www.stylusinc.com/Common/Concerns/SoftwareProjectsFailure.php
Best job in America?
   ● Software Engineer (2006)
         ○ $80,500 average pay
         ○ 46% 10-year growth
         ○ 44,800 annual job openings
         ○ Stress: B
         ○ Flexibility: B
         ○ Creativity: A
         ○ Ease of Entry: C




http://money.cnn.com/magazines/moneymag/moneymag_archive/2006/05/01/8375749/index.htm
Best job in America?
● Software Architect (2010)
   ○ $119,000 median pay
   ○ 34% 10-year growth
   ○ 110,000 total jobs
   ○ Low Stress: C
   ○ Flexibility:A
   ○ Personal Satisfaction: B
   ○ Future Growth: A




  http://money.cnn.com/magazines/moneymag/bestjobs/2010/snapshots/1.html
What's the opportunity?
             ● Software projects often fail
             ● More successful projects
               pay for less successful
               projects
             ● Software engineering is a
               growth industry


   http://www.slideshare.net/ted.husted/
Open Source Secret Sauce
              ● What’s the opportunity?
              ● Does open source matter?
              ● How does it work?
              ● Can you do it too?




    http://www.slideshare.net/ted.husted/
Does open source matter?
     “Covalent has earned a great reputation in the
     marketplace with its support for Apache
     projects, including Tomcat and Apache HTTP.
     “Its hundreds of support customers include
     more than half of the Fortune 500, and
     household names like Pfizer, Johnson &
     Johnson, British Telecom (BT), NASA, Intel,
     Royal Bank of Scotland and Bear Stearns.”


http://blog.springsource.com/main/2008/01/29/some-decisions-are-easy-%e2%80%93-like-springsource-acquiring-covalent/
Does open source matter?
     “Research by BZ Media and others shows that
     Apache Tomcat is the leading open source
     application server with a 64% market
     penetration.”




http://blog.springsource.com/main/2008/01/29/some-decisions-are-easy-%e2%80%93-like-springsource-acquiring-covalent/
http://ztrek.blogspot.com/2007/10/bea-oracle-market-share.html
Tomcat’s long tail …
  "The Tomcat numbers aren't driven by
  vendor marketing, but by thousands of
  companies making similar decisions."




http://blog.springsource.com/main/2007/12/24/is-it-a-tomcat-or-the-elephant-in-the-room/
What about frameworks?
"Hibernate and
Struts topped the
list with more
than 71 % of
[OpenLogic]
customers using
each."


http://press-releases.techwhack.com/15764-openlogic-5
Does open source work?




http://www.indeed.com/jobtrends?q=struts+java%2C+jsf+java&0
Are there many Apache projects?
 ●   HTTP Server     ●   Incubator    ●   Santuario
 ●   ActiveMQ        ●   Jackrabbit   ●   ServiceMix
 ●   Ant             ●   Jakarta      ●   Shale
 ●   APR             ●   James        ●   SpamAssassin
 ●   Beehive         ●   Labs         ●   STDCXX
 ●   Cayenne         ●   Lenya        ●   Struts
 ●   Cocoon          ●   Logging      ●   Synapse
 ●   Commons         ●   Lucene       ●   Tapestry
 ●   Continuum       ●   Maven        ●   TCL
 ●   DB              ●   Mina         ●   Tiles
 ●   Directory       ●   MyFaces      ●   Tomcat
 ●   Excalibur       ●   ODE          ●   Turbine
 ●   Felix           ●   OFBiz        ●   Velocity
 ●   Forrest         ●   OpenEJB      ●   Wicket
 ●   Geronimo        ●   OpenJPA      ●   Web Services
 ●   Gump            ●   Perl         ●   Xalan
 ●   Hadoop          ●   POI          ●   Xerces
 ●   Harmony         ●   Portals      ●   XML
 ●   HiveMind        ●   Roller       ●   XMLBeans
 ●   HttpComponent                    ●   XML Graphics
     s
Can HTTPD compete?




    http://news.netcraft.com
Does open source matter?
              ● Open source servers and
                frameworks succeed in the
                marketplace
              ● Many proprietary
                applications use open
                source products as
                dependencies


    http://www.slideshare.net/ted.husted/
Open Source Secret Sauce
              ● What’s the opportunity?
              ● Does open source matter?
              ● How does it work?
              ● Can you do it too?




   http://www.slideshare.net/ted.husted/
Once upon a time in 1994
Once upon a time in 1994
Once upon a time in 1994
http://nothingtosay.firstround.com/2007/01/the_internet_ye.html
Once upon a time
● 1994 – Rob McCool leaves
  NCSA for Netscape. Web
  server development stalls.
● 1995 - Brian Behlendorf started
  collecting patches to be applied
  to the last version of the public
  domain web server.
Once upon a time
● 1994 – Rob McCool leaves
  NCSA for Netscape. Web
  server development stalls.
● 1995 - Brian Behlendorf started
  collecting patches to be applied
  to the last version of the public
  domain web server.
Once upon a time
● Coined in homage to the
  Apache (Inde) people and some
  people liked the pun on "a
  patchy server".
● Apache 1.0 released in
  December 1995.
● Apache Software Foundation
  (US 501c) formed in June 1999.
Who makes the decisions
● User: Anyone who tries the product
● Contribute: Anyone who helps
● Committer: Anyone with write access
● PMC Members: Folks with binding
  votes
● Foundation Members: Elect the Board
Who makes the decisions
● Action Items
   ○ Product changes
   ○ Showstoppers – must fix issues
   ○ Release plans
   ○ Release grades
● Voting flavors
   ○ +1
   ○ -1
   ○ +/- 0
Who makes the decisions
● Why not count all the votes?
  ○ The ASF is a meritocracy
     ■ Them that do the work, make the decisions.
  ○ The ASF is a corporation
     ■ Chain of accountability
How does it work
          ● At the ASF, a corporate
            infrastructure creates a
            safe environment where
            developers can do the
            work and make the
            decisions.



http://www.slideshare.net/ted.husted/
Portal
Repository
Repository
Repository
Issue Tracker
Mailing List
Author: niallp
Date: Thu Oct 7 12:43:39 2004
New Revision: 54010

Modified:
 struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java
Log:

Remove change made in Revision 54006 for
Bug 31585

As pointed out by Kris Schneider this could
cause Tag Pooling problems
Modified: struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java
==============================================================================
--- struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java
(original)
+++ struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java
Thu Oct 7 12:43:39 2004

@@ -382,7 +382,6 @@
    // Clean up our started state
    started = false;
    iterator = null;
 ●     collection = null;
    // Continue processing this page
    return (EVAL_PAGE);
Automated Build
PRIMA: The Secret Sauce
● Portal
● Repository
● Issue Tracker
● Mailing List
● Automated Build
PRIMA: The Secret Sauce
● Portal
● Repository
● Issue Tracker
● Mailing List
● Automated Build
How to be an insider
● Follow the mailing list
● Inspect the code
● No hidden forums or closed meetings
● Everything that happens, happens on
  the list.
Open Source Secret Sauce
               ● What’s the opportunity?
               ● Does open source matter?
               ● How does it work?
               ● Can you do it too?




   http://www.slideshare.net/ted.husted/
Can you do it too?
http://code.google.com/
:::::::::::::::::
:::                 http://unfuddle.
:::::::::::::::::
:::
                    http://www.collab.
http://www.jira.
http://trac.edgewall.
Team Best of Breed
● Confluence
● FishEye
● JIRA
● Bamboo




          http://www.atlassian.
Open Source Secret Sauce
              ● What’s the opportunity?
              ● Does open source matter?
              ● How does it work?
              ● Can you do it too?




   http://www.slideshare.net/ted.husted/
Open Source Secret Sauce
         ● What’s the opportunity?
           ○ High ROI
         ● Does open source matter?
         ● How does it work?
         ● Can you do it too?
Open Source Secret Sauce
         ● What’s the opportunity?
           ○ High ROI
         ● Does open source matter?
           ○ Success breeds success
         ● How does it work?
         ● Can you do it too?
Open Source Secret Sauce
         ● What’s the opportunity?
           ○ High ROI
         ● Does open source matter?
           ○ Success breeds success
         ● How does it work?
           ○ Transparency + Infrastructure
         ● Can you do it too?
Open Source Secret Sauce
         ● What’s the opportunity?
           ○ High ROI
         ● Does open source matter?
           ○ Success breeds success
         ● How does it work?
           ○ Transparency + Infrastructure
         ● Can you do it too?
           ○ Hosted or standalone
Open Advocacy University Series

Open Source Secret Sauce - Lugor Sep 2011

  • 1.
    The Open SourceSecret Sauce 15 Sep 2011 Rochester NY Lugor User Group
  • 2.
    Abstract How do volunteeropen-source projects create and maintain so many compelling, competitive products? What is the Open Source Secret Sauce? Join open-source insider, Ted Husted, as he takes us deep inside the Apache Software Foundation, to show how the sausages are made.
  • 3.
    About the Speaker TedHusted was a senior member of the Apache Struts development group from 2000 to 2008. His books include Google Wave Explained, JUnit in Action, Struts in Action, and Professional JSP Site Design. Ted has consulted with teams throughout the United States, including CitiGroup, Nationwide Insurance, and Pepsi Bottling Group. He has been a regular speaker at ApacheCon US and the Ajax Experience, as well as to various user groups. Ted is currently working with NimbleUser of Pittsford NY, as a business analyst. For more, visit http://ted.husted.com/
  • 4.
    Open Source SecretSauce ● What’s the opportunity? ● Does open source matter? ● How does it work? ● Can you do it too? http://www.slideshare.net/ted.husted/
  • 5.
    What’s the problem? ●Overall, about 15% of all projects are delivered on-time and on-budget. Source: The Standish Group
  • 6.
    What’s the problem? ●Overall, about 15% of all projects are delivered on-time and on-budget. ● About 30% of software projects are cancelled before completion. Source: The Standish Group
  • 7.
    What’s the problem? ●Overall, about 15% of all projects are delivered on-time and on-budget. ● About 30% of software projects are cancelled before completion. ● Half of all software projects cost almost twice as much as anticipated. Source: The Standish Group
  • 8.
    What’s the problem? ●Overall, about 15% of all projects are delivered on-time and on-budget. ● About 30% of software projects are cancelled before completion. ● Half of all software projects cost almost twice as much as anticipated. ● More than 50% of software projects fail. Source: The Standish Group
  • 9.
    Why do projectsfail? http://www.stylusinc.com/Common/Concerns/SoftwareProjectsFailure.php
  • 10.
    Why do projectssucceed? http://www.stylusinc.com/Common/Concerns/SoftwareProjectsFailure.php
  • 11.
    Best job inAmerica? ● Software Engineer (2006) ○ $80,500 average pay ○ 46% 10-year growth ○ 44,800 annual job openings ○ Stress: B ○ Flexibility: B ○ Creativity: A ○ Ease of Entry: C http://money.cnn.com/magazines/moneymag/moneymag_archive/2006/05/01/8375749/index.htm
  • 12.
    Best job inAmerica? ● Software Architect (2010) ○ $119,000 median pay ○ 34% 10-year growth ○ 110,000 total jobs ○ Low Stress: C ○ Flexibility:A ○ Personal Satisfaction: B ○ Future Growth: A http://money.cnn.com/magazines/moneymag/bestjobs/2010/snapshots/1.html
  • 13.
    What's the opportunity? ● Software projects often fail ● More successful projects pay for less successful projects ● Software engineering is a growth industry http://www.slideshare.net/ted.husted/
  • 14.
    Open Source SecretSauce ● What’s the opportunity? ● Does open source matter? ● How does it work? ● Can you do it too? http://www.slideshare.net/ted.husted/
  • 15.
    Does open sourcematter? “Covalent has earned a great reputation in the marketplace with its support for Apache projects, including Tomcat and Apache HTTP. “Its hundreds of support customers include more than half of the Fortune 500, and household names like Pfizer, Johnson & Johnson, British Telecom (BT), NASA, Intel, Royal Bank of Scotland and Bear Stearns.” http://blog.springsource.com/main/2008/01/29/some-decisions-are-easy-%e2%80%93-like-springsource-acquiring-covalent/
  • 16.
    Does open sourcematter? “Research by BZ Media and others shows that Apache Tomcat is the leading open source application server with a 64% market penetration.” http://blog.springsource.com/main/2008/01/29/some-decisions-are-easy-%e2%80%93-like-springsource-acquiring-covalent/
  • 17.
  • 18.
    Tomcat’s long tail… "The Tomcat numbers aren't driven by vendor marketing, but by thousands of companies making similar decisions." http://blog.springsource.com/main/2007/12/24/is-it-a-tomcat-or-the-elephant-in-the-room/
  • 26.
    What about frameworks? "Hibernateand Struts topped the list with more than 71 % of [OpenLogic] customers using each." http://press-releases.techwhack.com/15764-openlogic-5
  • 29.
    Does open sourcework? http://www.indeed.com/jobtrends?q=struts+java%2C+jsf+java&0
  • 30.
    Are there manyApache projects? ● HTTP Server ● Incubator ● Santuario ● ActiveMQ ● Jackrabbit ● ServiceMix ● Ant ● Jakarta ● Shale ● APR ● James ● SpamAssassin ● Beehive ● Labs ● STDCXX ● Cayenne ● Lenya ● Struts ● Cocoon ● Logging ● Synapse ● Commons ● Lucene ● Tapestry ● Continuum ● Maven ● TCL ● DB ● Mina ● Tiles ● Directory ● MyFaces ● Tomcat ● Excalibur ● ODE ● Turbine ● Felix ● OFBiz ● Velocity ● Forrest ● OpenEJB ● Wicket ● Geronimo ● OpenJPA ● Web Services ● Gump ● Perl ● Xalan ● Hadoop ● POI ● Xerces ● Harmony ● Portals ● XML ● HiveMind ● Roller ● XMLBeans ● HttpComponent ● XML Graphics s
  • 33.
    Can HTTPD compete? http://news.netcraft.com
  • 34.
    Does open sourcematter? ● Open source servers and frameworks succeed in the marketplace ● Many proprietary applications use open source products as dependencies http://www.slideshare.net/ted.husted/
  • 35.
    Open Source SecretSauce ● What’s the opportunity? ● Does open source matter? ● How does it work? ● Can you do it too? http://www.slideshare.net/ted.husted/
  • 37.
    Once upon atime in 1994
  • 38.
    Once upon atime in 1994
  • 39.
    Once upon atime in 1994
  • 40.
  • 41.
    Once upon atime ● 1994 – Rob McCool leaves NCSA for Netscape. Web server development stalls. ● 1995 - Brian Behlendorf started collecting patches to be applied to the last version of the public domain web server.
  • 42.
    Once upon atime ● 1994 – Rob McCool leaves NCSA for Netscape. Web server development stalls. ● 1995 - Brian Behlendorf started collecting patches to be applied to the last version of the public domain web server.
  • 43.
    Once upon atime ● Coined in homage to the Apache (Inde) people and some people liked the pun on "a patchy server". ● Apache 1.0 released in December 1995. ● Apache Software Foundation (US 501c) formed in June 1999.
  • 50.
    Who makes thedecisions ● User: Anyone who tries the product ● Contribute: Anyone who helps ● Committer: Anyone with write access ● PMC Members: Folks with binding votes ● Foundation Members: Elect the Board
  • 51.
    Who makes thedecisions ● Action Items ○ Product changes ○ Showstoppers – must fix issues ○ Release plans ○ Release grades ● Voting flavors ○ +1 ○ -1 ○ +/- 0
  • 52.
    Who makes thedecisions ● Why not count all the votes? ○ The ASF is a meritocracy ■ Them that do the work, make the decisions. ○ The ASF is a corporation ■ Chain of accountability
  • 53.
    How does itwork ● At the ASF, a corporate infrastructure creates a safe environment where developers can do the work and make the decisions. http://www.slideshare.net/ted.husted/
  • 57.
  • 58.
  • 59.
  • 60.
  • 61.
  • 62.
  • 63.
    Author: niallp Date: ThuOct 7 12:43:39 2004 New Revision: 54010 Modified: struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java Log: Remove change made in Revision 54006 for Bug 31585 As pointed out by Kris Schneider this could cause Tag Pooling problems
  • 64.
    Modified: struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java ============================================================================== --- struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java (original) +++struts/trunk/src/share/org/apache/struts/taglib/logic/IterateTag.java Thu Oct 7 12:43:39 2004 @@ -382,7 +382,6 @@ // Clean up our started state started = false; iterator = null; ● collection = null; // Continue processing this page return (EVAL_PAGE);
  • 65.
  • 66.
    PRIMA: The SecretSauce ● Portal ● Repository ● Issue Tracker ● Mailing List ● Automated Build
  • 67.
    PRIMA: The SecretSauce ● Portal ● Repository ● Issue Tracker ● Mailing List ● Automated Build
  • 68.
    How to bean insider ● Follow the mailing list ● Inspect the code ● No hidden forums or closed meetings ● Everything that happens, happens on the list.
  • 69.
    Open Source SecretSauce ● What’s the opportunity? ● Does open source matter? ● How does it work? ● Can you do it too? http://www.slideshare.net/ted.husted/
  • 70.
    Can you doit too?
  • 71.
  • 72.
    ::::::::::::::::: ::: http://unfuddle.
  • 73.
    ::::::::::::::::: ::: http://www.collab.
  • 74.
  • 75.
  • 76.
    Team Best ofBreed ● Confluence ● FishEye ● JIRA ● Bamboo http://www.atlassian.
  • 77.
    Open Source SecretSauce ● What’s the opportunity? ● Does open source matter? ● How does it work? ● Can you do it too? http://www.slideshare.net/ted.husted/
  • 78.
    Open Source SecretSauce ● What’s the opportunity? ○ High ROI ● Does open source matter? ● How does it work? ● Can you do it too?
  • 79.
    Open Source SecretSauce ● What’s the opportunity? ○ High ROI ● Does open source matter? ○ Success breeds success ● How does it work? ● Can you do it too?
  • 80.
    Open Source SecretSauce ● What’s the opportunity? ○ High ROI ● Does open source matter? ○ Success breeds success ● How does it work? ○ Transparency + Infrastructure ● Can you do it too?
  • 81.
    Open Source SecretSauce ● What’s the opportunity? ○ High ROI ● Does open source matter? ○ Success breeds success ● How does it work? ○ Transparency + Infrastructure ● Can you do it too? ○ Hosted or standalone
  • 83.