Software Engineering and Open Source           (A few things I have learned doing Mozilla since 1998)           Brendan Ei...
Happy to be here!                               mozilla           2Friday, April 29, 2011
Agenda           • Problem statement           • Mozilla’s history and contributions           • The big picture          ...
Problem statement                               mozilla           4Friday, April 29, 2011
Problem statement           • Software, especially “in the large”, remains quite challenging                              ...
Problem statement           • Software, especially “in the large”, remains quite challenging           • “What makes code ...
Problem statement           • Software, especially “in the large”, remains quite challenging           • “What makes code ...
Problem statement           • Software, especially “in the large”, remains quite challenging           • “What makes code ...
Problem statement           • Software, especially “in the large”, remains quite challenging           • “What makes code ...
The scale problem is worse over time                                                  mozilla           5Friday, April 29,...
The scale problem is worse over time           • Path dependence in any large codebase creates “lock-in” (see sunk cost fa...
The scale problem is worse over time           • Path dependence in any large codebase creates “lock-in” (see sunk cost fa...
The scale problem is worse over time           • Path dependence in any large codebase creates “lock-in” (see sunk cost fa...
Mozilla’s history: 1998                                     mozilla           6Friday, April 29, 2011
Mozilla’s history: 1998           • An “escape pod” jettisoned by Netscape, the first big commercial open source           ...
Mozilla’s history: 1998           • An “escape pod” jettisoned by Netscape, the first big commercial open source           ...
Mozilla’s history: 1998           • An “escape pod” jettisoned by Netscape, the first big commercial open source           ...
Mozilla’s history: 1998           • An “escape pod” jettisoned by Netscape, the first big commercial open source           ...
Mozilla’s history: 1998           • An “escape pod” jettisoned by Netscape, the first big commercial open source           ...
Mozilla’s history: 1998           • An “escape pod” jettisoned by Netscape, the first big commercial open source           ...
The beginner’s error: rewrite!                                            mozilla           7Friday, April 29, 2011
The beginner’s error: rewrite!           • October 1998: I declared it time to rebuild almost all of the code using:      ...
The beginner’s error: rewrite!           • October 1998: I declared it time to rebuild almost all of the code using:      ...
2000           • Netscape 6 - “Black and Blue”, circular buttons, slow, crashy, AOL IM built in...           • Based on “M...
2001-2002                         mozilla           9Friday, April 29, 2011
2001-2002           • After Netscape 6 flopped, Mitchell and I leveled the community playing field:                         ...
2001-2002           • After Netscape 6 flopped, Mitchell and I leveled the community playing field:                 • New Ne...
2001-2002           • After Netscape 6 flopped, Mitchell and I leveled the community playing field:                 • New Ne...
2001-2002           • After Netscape 6 flopped, Mitchell and I leveled the community playing field:                 • New Ne...
2001-2002           • After Netscape 6 flopped, Mitchell and I leveled the community playing field:                 • New Ne...
2002-2003                         mozilla           10Friday, April 29, 2011
2002-2003           • Mozilla 1.0 released in June 2002                                                 mozilla           ...
2002-2003           • Mozilla 1.0 released in June 2002           • mozilla/browser, to be renamed Phoenix, then Firebird,...
2002-2003           • Mozilla 1.0 released in June 2002           • mozilla/browser, to be renamed Phoenix, then Firebird,...
2002-2003           • Mozilla 1.0 released in June 2002           • mozilla/browser, to be renamed Phoenix, then Firebird,...
2002-2003           • Mozilla 1.0 released in June 2002           • mozilla/browser, to be renamed Phoenix, then Firebird,...
2002-2003           • Mozilla 1.0 released in June 2002           • mozilla/browser, to be renamed Phoenix, then Firebird,...
2004                         mozilla           11Friday, April 29, 2011
2004           • The new roadmap split the community, with a bare majority supporting separate             browser and mai...
2004           • The new roadmap split the community, with a bare majority supporting separate             browser and mai...
2004           • The new roadmap split the community, with a bare majority supporting separate             browser and mai...
2004           • The new roadmap split the community, with a bare majority supporting separate             browser and mai...
2004           • The new roadmap split the community, with a bare majority supporting separate             browser and mai...
mozilla           12Friday, April 29, 2011
Mozilla’s contributions: community                                                mozilla           13Friday, April 29, 2011
Mozilla’s contributions: community           • In the modern Firefox era, 30-40% of patches come from volunteers          ...
Mozilla’s contributions: community           • In the modern Firefox era, 30-40% of patches come from volunteers          ...
Mozilla’s contributions: community           • In the modern Firefox era, 30-40% of patches come from volunteers          ...
Mozilla’s contributions: community           • In the modern Firefox era, 30-40% of patches come from volunteers          ...
Mozilla’s contributions: community           • In the modern Firefox era, 30-40% of patches come from volunteers          ...
Mozilla’s contributions: community           • In the modern Firefox era, 30-40% of patches come from volunteers          ...
mozilla           14Friday, April 29, 2011
mozilla           15Friday, April 29, 2011
Bugzilla                         mozilla           16Friday, April 29, 2011
Tinderbox, now “tinderboxpushlog”                                               mozilla           17Friday, April 29, 2011
Code review and testing                                     mozilla           18Friday, April 29, 2011
Code review and testing           • Code review before commit is required, in general                 • Large new modules ...
Code review and testing           • Code review before commit is required, in general                 • Large new modules ...
Fuzz-testing                          mozilla           19Friday, April 29, 2011
Fuzz-testing           • jsfunfuzz, Jesse Ruderman’s JavaScript travesty generator, has been             enormously produc...
Fuzz-testing           • jsfunfuzz, Jesse Ruderman’s JavaScript travesty generator, has been             enormously produc...
Automatic program analysis                                        mozilla           20Friday, April 29, 2011
Automatic program analysis           • Collaboration with UCB project Oink, ultimately forked as Mozilla Pork, for:       ...
Automatic program analysis           • Collaboration with UCB project Oink, ultimately forked as Mozilla Pork, for:       ...
Brian Hackett’s Sixgill                                     mozilla           21Friday, April 29, 2011
Rust and Project Servo                                    mozilla           22Friday, April 29, 2011
Rust and Project Servo                • A new, safer systems programming language, Rust, instead of C++                   ...
Rust and Project Servo                • A new, safer systems programming language, Rust, instead of C++                   ...
Rust and Project Servo                • A new, safer systems programming language, Rust, instead of C++                   ...
Rust and Project Servo                • A new, safer systems programming language, Rust, instead of C++                   ...
Rust and Project Servo                • A new, safer systems programming language, Rust, instead of C++                   ...
Rust and Project Servo                • A new, safer systems programming language, Rust, instead of C++                   ...
nbody.rs                         mozilla           23Friday, April 29, 2011
Rust performance results (nbody)                                              mozilla           24Friday, April 29, 2011
fannkuchredux.rs                              mozilla           25Friday, April 29, 2011
Rust performance results (fannkuchredux)           • Within 2x (much more to do) of clang optimized performance, with safe...
The big picture                             mozilla           27Friday, April 29, 2011
The big picture           • We at Mozilla like formal methods and invest in them, but incrementally.                 • Fie...
The big picture           • We at Mozilla like formal methods and invest in them, but incrementally.                 • Fie...
The big picture           • We at Mozilla like formal methods and invest in them, but incrementally.                 • Fie...
The big picture           • We at Mozilla like formal methods and invest in them, but incrementally.                 • Fie...
Where we are all headed                                     mozilla           28Friday, April 29, 2011
Where we are all headed           • When Mozilla began, CVS was the state of the art in open source version control       ...
Where we are all headed           • When Mozilla began, CVS was the state of the art in open source version control       ...
Where we are all headed           • When Mozilla began, CVS was the state of the art in open source version control       ...
Where we are all headed           • When Mozilla began, CVS was the state of the art in open source version control       ...
Where we are all headed           • When Mozilla began, CVS was the state of the art in open source version control       ...
mozilla           29Friday, April 29, 2011
mozilla           30Friday, April 29, 2011
mozilla           31Friday, April 29, 2011
mozilla           32Friday, April 29, 2011
mozilla           33Friday, April 29, 2011
mozilla           34Friday, April 29, 2011
mozilla           35Friday, April 29, 2011
Conclusions                         mozilla           36Friday, April 29, 2011
Conclusions           • Open source and distributed development obviously on the rise. Synergies             between the t...
Conclusions           • Open source and distributed development obviously on the rise. Synergies             between the t...
Upcoming SlideShare
Loading in...5
×

MSR Talk

2,222

Published on

My talk from the Microsoft Research Software Summit Paris 2011, on open source, software engineering, and Mozilla.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,222
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

MSR Talk

  1. 1. Software Engineering and Open Source (A few things I have learned doing Mozilla since 1998) Brendan Eich <brendan@mozilla.org> mozilla 1Friday, April 29, 2011
  2. 2. Happy to be here! mozilla 2Friday, April 29, 2011
  3. 3. Agenda • Problem statement • Mozilla’s history and contributions • The big picture • Where we are all headed mozilla 3Friday, April 29, 2011
  4. 4. Problem statement mozilla 4Friday, April 29, 2011
  5. 5. Problem statement • Software, especially “in the large”, remains quite challenging mozilla 4Friday, April 29, 2011
  6. 6. Problem statement • Software, especially “in the large”, remains quite challenging • “What makes code strange” - Jeremy Ashkenas, CodeConf 2011 mozilla 4Friday, April 29, 2011
  7. 7. Problem statement • Software, especially “in the large”, remains quite challenging • “What makes code strange” - Jeremy Ashkenas, CodeConf 2011 • “I do not know of any other technology covering a ratio of 10^10.” - Dijkstra mozilla 4Friday, April 29, 2011
  8. 8. Problem statement • Software, especially “in the large”, remains quite challenging • “What makes code strange” - Jeremy Ashkenas, CodeConf 2011 • “I do not know of any other technology covering a ratio of 10^10.” - Dijkstra • “By evoking the need for deep conceptual hierarchies, the automatic computer confronts us with a radically new intellectual challenge that has no precedent in our history.” - Dijkstra again mozilla 4Friday, April 29, 2011
  9. 9. Problem statement • Software, especially “in the large”, remains quite challenging • “What makes code strange” - Jeremy Ashkenas, CodeConf 2011 • “I do not know of any other technology covering a ratio of 10^10.” - Dijkstra • “By evoking the need for deep conceptual hierarchies, the automatic computer confronts us with a radically new intellectual challenge that has no precedent in our history.” - Dijkstra again • “Science is knowledge which we understand so well that we can teach it to a computer; and if we don’t understand something, it is an art to deal with it.” - Knuth mozilla 4Friday, April 29, 2011
  10. 10. The scale problem is worse over time mozilla 5Friday, April 29, 2011
  11. 11. The scale problem is worse over time • Path dependence in any large codebase creates “lock-in” (see sunk cost fallacy) mozilla 5Friday, April 29, 2011
  12. 12. The scale problem is worse over time • Path dependence in any large codebase creates “lock-in” (see sunk cost fallacy) • Rewriting is both a beginner’s error and something experienced programmers can defer for far too long... mozilla 5Friday, April 29, 2011
  13. 13. The scale problem is worse over time • Path dependence in any large codebase creates “lock-in” (see sunk cost fallacy) • Rewriting is both a beginner’s error and something experienced programmers can defer for far too long... • “Code by itself almost rots and it’s gotta be rewritten. Even when nothing has changed, for some reason it rots.” - Ken Thompson mozilla 5Friday, April 29, 2011
  14. 14. Mozilla’s history: 1998 mozilla 6Friday, April 29, 2011
  15. 15. Mozilla’s history: 1998 • An “escape pod” jettisoned by Netscape, the first big commercial open source conversion attempt - an act of desperation, a bit of executive legacy vanity mozilla 6Friday, April 29, 2011
  16. 16. Mozilla’s history: 1998 • An “escape pod” jettisoned by Netscape, the first big commercial open source conversion attempt - an act of desperation, a bit of executive legacy vanity • 1998-2003 mozilla.org was a "virtual organization" with no legal independence mozilla 6Friday, April 29, 2011
  17. 17. Mozilla’s history: 1998 • An “escape pod” jettisoned by Netscape, the first big commercial open source conversion attempt - an act of desperation, a bit of executive legacy vanity • 1998-2003 mozilla.org was a "virtual organization" with no legal independence • staff@mozilla.org drawn from part- and full-time employees at several companies mozilla 6Friday, April 29, 2011
  18. 18. Mozilla’s history: 1998 • An “escape pod” jettisoned by Netscape, the first big commercial open source conversion attempt - an act of desperation, a bit of executive legacy vanity • 1998-2003 mozilla.org was a "virtual organization" with no legal independence • staff@mozilla.org drawn from part- and full-time employees at several companies • Mitchell Baker (MPL creator): governance and process mozilla 6Friday, April 29, 2011
  19. 19. Mozilla’s history: 1998 • An “escape pod” jettisoned by Netscape, the first big commercial open source conversion attempt - an act of desperation, a bit of executive legacy vanity • 1998-2003 mozilla.org was a "virtual organization" with no legal independence • staff@mozilla.org drawn from part- and full-time employees at several companies • Mitchell Baker (MPL creator): governance and process • Brendan Eich: technical architecture and oversight mozilla 6Friday, April 29, 2011
  20. 20. Mozilla’s history: 1998 • An “escape pod” jettisoned by Netscape, the first big commercial open source conversion attempt - an act of desperation, a bit of executive legacy vanity • 1998-2003 mozilla.org was a "virtual organization" with no legal independence • staff@mozilla.org drawn from part- and full-time employees at several companies • Mitchell Baker (MPL creator): governance and process • Brendan Eich: technical architecture and oversight • March 31, 1998: all contributors were @netscape.com mozilla 6Friday, April 29, 2011
  21. 21. The beginner’s error: rewrite! mozilla 7Friday, April 29, 2011
  22. 22. The beginner’s error: rewrite! • October 1998: I declared it time to rebuild almost all of the code using: • A new, cross-platform XML-based user-interface widget-set and front end, customizable similar to how web pages can be customized • Gecko, a new, C++, standards-oriented HTML/CSS engine from Netscape, not nearly done but able to load pages and do basic layout • A scriptable C++ component model, complete with IDL, IDL-derived type libraries, automatic JS <=> C++ type and method bridging mozilla 7Friday, April 29, 2011
  23. 23. The beginner’s error: rewrite! • October 1998: I declared it time to rebuild almost all of the code using: • A new, cross-platform XML-based user-interface widget-set and front end, customizable similar to how web pages can be customized • Gecko, a new, C++, standards-oriented HTML/CSS engine from Netscape, not nearly done but able to load pages and do basic layout • A scriptable C++ component model, complete with IDL, IDL-derived type libraries, automatic JS <=> C++ type and method bridging • Big gamble to bring in new developers attracted by green-field development and a new focus on correctly implemented Web standards mozilla 7Friday, April 29, 2011
  24. 24. 2000 • Netscape 6 - “Black and Blue”, circular buttons, slow, crashy, AOL IM built in... • Based on “Mozilla 0.6” - our view was that the code was not nearly “1.0” mozilla 8Friday, April 29, 2011
  25. 25. 2001-2002 mozilla 9Friday, April 29, 2011
  26. 26. 2001-2002 • After Netscape 6 flopped, Mitchell and I leveled the community playing field: mozilla 9Friday, April 29, 2011
  27. 27. 2001-2002 • After Netscape 6 flopped, Mitchell and I leveled the community playing field: • New Netscape hires had to prove merit before getting “commit access” mozilla 9Friday, April 29, 2011
  28. 28. 2001-2002 • After Netscape 6 flopped, Mitchell and I leveled the community playing field: • New Netscape hires had to prove merit before getting “commit access” • Mozilla binaries (available to testers who did not have a compiler or time to do their own builds) became competitive with Netscape releases mozilla 9Friday, April 29, 2011
  29. 29. 2001-2002 • After Netscape 6 flopped, Mitchell and I leveled the community playing field: • New Netscape hires had to prove merit before getting “commit access” • Mozilla binaries (available to testers who did not have a compiler or time to do their own builds) became competitive with Netscape releases • A roadmap plan of milestone releases leading up to “Mozilla 1.0”, running from 2001 into 2002, independent of any Netscape product release plans mozilla 9Friday, April 29, 2011
  30. 30. 2001-2002 • After Netscape 6 flopped, Mitchell and I leveled the community playing field: • New Netscape hires had to prove merit before getting “commit access” • Mozilla binaries (available to testers who did not have a compiler or time to do their own builds) became competitive with Netscape releases • A roadmap plan of milestone releases leading up to “Mozilla 1.0”, running from 2001 into 2002, independent of any Netscape product release plans • The community of volunteers and contributors employed by other companies than Netscape grew in strength and numbers from 2000 to 2002 mozilla 9Friday, April 29, 2011
  31. 31. 2002-2003 mozilla 10Friday, April 29, 2011
  32. 32. 2002-2003 • Mozilla 1.0 released in June 2002 mozilla 10Friday, April 29, 2011
  33. 33. 2002-2003 • Mozilla 1.0 released in June 2002 • mozilla/browser, to be renamed Phoenix, then Firebird, was created mozilla 10Friday, April 29, 2011
  34. 34. 2002-2003 • Mozilla 1.0 released in June 2002 • mozilla/browser, to be renamed Phoenix, then Firebird, was created • Dave Hyatt and I wrote a new roadmap advocating unbundling the “Mozilla suite” into browser, mail, and other apps, each doing one thing well, with fewer features and an “add-ons” architecture so users could customize at will mozilla 10Friday, April 29, 2011
  35. 35. 2002-2003 • Mozilla 1.0 released in June 2002 • mozilla/browser, to be renamed Phoenix, then Firebird, was created • Dave Hyatt and I wrote a new roadmap advocating unbundling the “Mozilla suite” into browser, mail, and other apps, each doing one thing well, with fewer features and an “add-ons” architecture so users could customize at will • Mozilla downloads were at least as popular as Netscape product downloads mozilla 10Friday, April 29, 2011
  36. 36. 2002-2003 • Mozilla 1.0 released in June 2002 • mozilla/browser, to be renamed Phoenix, then Firebird, was created • Dave Hyatt and I wrote a new roadmap advocating unbundling the “Mozilla suite” into browser, mail, and other apps, each doing one thing well, with fewer features and an “add-ons” architecture so users could customize at will • Mozilla downloads were at least as popular as Netscape product downloads • July 2003: AOL laid off almost all Netscape employees and created the Mozilla Foundation, giving it initial funding, trademarks, and the project’s infrastructure mozilla 10Friday, April 29, 2011
  37. 37. 2002-2003 • Mozilla 1.0 released in June 2002 • mozilla/browser, to be renamed Phoenix, then Firebird, was created • Dave Hyatt and I wrote a new roadmap advocating unbundling the “Mozilla suite” into browser, mail, and other apps, each doing one thing well, with fewer features and an “add-ons” architecture so users could customize at will • Mozilla downloads were at least as popular as Netscape product downloads • July 2003: AOL laid off almost all Netscape employees and created the Mozilla Foundation, giving it initial funding, trademarks, and the project’s infrastructure • Mozilla Foundation initial headcount: 10, with money for ~1.5 years mozilla 10Friday, April 29, 2011
  38. 38. 2004 mozilla 11Friday, April 29, 2011
  39. 39. 2004 • The new roadmap split the community, with a bare majority supporting separate browser and mail apps mozilla 11Friday, April 29, 2011
  40. 40. 2004 • The new roadmap split the community, with a bare majority supporting separate browser and mail apps • Firefox (renamed from Firebird) started rapid gains in market share and buzz mozilla 11Friday, April 29, 2011
  41. 41. 2004 • The new roadmap split the community, with a bare majority supporting separate browser and mail apps • Firefox (renamed from Firebird) started rapid gains in market share and buzz • Focus on end-user usability, Windows integration over Linux, unusual for open source software mozilla 11Friday, April 29, 2011
  42. 42. 2004 • The new roadmap split the community, with a bare majority supporting separate browser and mail apps • Firefox (renamed from Firebird) started rapid gains in market share and buzz • Focus on end-user usability, Windows integration over Linux, unusual for open source software • June 2004: Mozilla, Opera, and (not announced at first) Apple founded the WHATWG to create HTML5; I blogged about it and against the W3C mozilla 11Friday, April 29, 2011
  43. 43. 2004 • The new roadmap split the community, with a bare majority supporting separate browser and mail apps • Firefox (renamed from Firebird) started rapid gains in market share and buzz • Focus on end-user usability, Windows integration over Linux, unusual for open source software • June 2004: Mozilla, Opera, and (not announced at first) Apple founded the WHATWG to create HTML5; I blogged about it and against the W3C • The rest is perhaps better-known history... mozilla 11Friday, April 29, 2011
  44. 44. mozilla 12Friday, April 29, 2011
  45. 45. Mozilla’s contributions: community mozilla 13Friday, April 29, 2011
  46. 46. Mozilla’s contributions: community • In the modern Firefox era, 30-40% of patches come from volunteers mozilla 13Friday, April 29, 2011
  47. 47. Mozilla’s contributions: community • In the modern Firefox era, 30-40% of patches come from volunteers • ~700 volunteers contributing patches over a year (outsize impact) mozilla 13Friday, April 29, 2011
  48. 48. Mozilla’s contributions: community • In the modern Firefox era, 30-40% of patches come from volunteers • ~700 volunteers contributing patches over a year (outsize impact) • Firefox 4 available in 82 locales and 74 languages, the work of many volunteers mozilla 13Friday, April 29, 2011
  49. 49. Mozilla’s contributions: community • In the modern Firefox era, 30-40% of patches come from volunteers • ~700 volunteers contributing patches over a year (outsize impact) • Firefox 4 available in 82 locales and 74 languages, the work of many volunteers • ~50,000 nightly build testers mozilla 13Friday, April 29, 2011
  50. 50. Mozilla’s contributions: community • In the modern Firefox era, 30-40% of patches come from volunteers • ~700 volunteers contributing patches over a year (outsize impact) • Firefox 4 available in 82 locales and 74 languages, the work of many volunteers • ~50,000 nightly build testers • 5M beta testers in the Firefox 4 release cycle mozilla 13Friday, April 29, 2011
  51. 51. Mozilla’s contributions: community • In the modern Firefox era, 30-40% of patches come from volunteers • ~700 volunteers contributing patches over a year (outsize impact) • Firefox 4 available in 82 locales and 74 languages, the work of many volunteers • ~50,000 nightly build testers • 5M beta testers in the Firefox 4 release cycle • 450M users mozilla 13Friday, April 29, 2011
  52. 52. mozilla 14Friday, April 29, 2011
  53. 53. mozilla 15Friday, April 29, 2011
  54. 54. Bugzilla mozilla 16Friday, April 29, 2011
  55. 55. Tinderbox, now “tinderboxpushlog” mozilla 17Friday, April 29, 2011
  56. 56. Code review and testing mozilla 18Friday, April 29, 2011
  57. 57. Code review and testing • Code review before commit is required, in general • Large new modules may plead for post-commit review • No formal design review; important to catch design mistakes early • Code review of more, smaller patches landing early and often is best mozilla 18Friday, April 29, 2011
  58. 58. Code review and testing • Code review before commit is required, in general • Large new modules may plead for post-commit review • No formal design review; important to catch design mistakes early • Code review of more, smaller patches landing early and often is best • Tests are required as part of the patch • Previously we lacked this rule and suffered lack of test coverage and (prior to 2006) even test automation mozilla • Valgrind [Nethercote, Seward] automated build test coverage 18Friday, April 29, 2011
  59. 59. Fuzz-testing mozilla 19Friday, April 29, 2011
  60. 60. Fuzz-testing • jsfunfuzz, Jesse Ruderman’s JavaScript travesty generator, has been enormously productive • More productive than static analysis frameworks (Coverity, e.g.) • Knows JS syntax and some semantics, bends and break rules, nests constructs in evil ways • Extremely effective when steered by an expert who tracks new code and language features, senses soft spots and attacks them with targeted fuzzer logic... mozilla 19Friday, April 29, 2011
  61. 61. Fuzz-testing • jsfunfuzz, Jesse Ruderman’s JavaScript travesty generator, has been enormously productive • More productive than static analysis frameworks (Coverity, e.g.) • Knows JS syntax and some semantics, bends and break rules, nests constructs in evil ways • Extremely effective when steered by an expert who tracks new code and language features, senses soft spots and attacks them with targeted fuzzer logic... • Variations perform random DOM manipulation, markup generation, etc. mozilla 19Friday, April 29, 2011
  62. 62. Automatic program analysis mozilla 20Friday, April 29, 2011
  63. 63. Automatic program analysis • Collaboration with UCB project Oink, ultimately forked as Mozilla Pork, for: • experiments with automatic patch generation mozilla 20Friday, April 29, 2011
  64. 64. Automatic program analysis • Collaboration with UCB project Oink, ultimately forked as Mozilla Pork, for: • experiments with automatic patch generation • Second-generation analysis tools based on the GNU C Compiler (GCC): • dxr, a semantic source code cross-referencer • Dehydra, a scriptable static analysis tool (GCC plugin) • Treehydra, heavy-duty static analysis version of Dehydra (binds to JS reflection of GCC’s GIMPLE AST representation) mozilla 20Friday, April 29, 2011
  65. 65. Brian Hackett’s Sixgill mozilla 21Friday, April 29, 2011
  66. 66. Rust and Project Servo mozilla 22Friday, April 29, 2011
  67. 67. Rust and Project Servo • A new, safer systems programming language, Rust, instead of C++ mozilla 22Friday, April 29, 2011
  68. 68. Rust and Project Servo • A new, safer systems programming language, Rust, instead of C++ • Rust is a multi-paradigm, compiled language focused on safety, efficiency, programming-in-the-large, and a high degree of concurrency mozilla 22Friday, April 29, 2011
  69. 69. Rust and Project Servo • A new, safer systems programming language, Rust, instead of C++ • Rust is a multi-paradigm, compiled language focused on safety, efficiency, programming-in-the-large, and a high degree of concurrency • Research building parallel browser engine stages in Rust mozilla 22Friday, April 29, 2011
  70. 70. Rust and Project Servo • A new, safer systems programming language, Rust, instead of C++ • Rust is a multi-paradigm, compiled language focused on safety, efficiency, programming-in-the-large, and a high degree of concurrency • Research building parallel browser engine stages in Rust • Experiment with Andreas Gal’s DOM implemented in JavaScript mozilla 22Friday, April 29, 2011
  71. 71. Rust and Project Servo • A new, safer systems programming language, Rust, instead of C++ • Rust is a multi-paradigm, compiled language focused on safety, efficiency, programming-in-the-large, and a high degree of concurrency • Research building parallel browser engine stages in Rust • Experiment with Andreas Gal’s DOM implemented in JavaScript • For a scalably-faster-on-manycore, much safer Browser from the Future mozilla 22Friday, April 29, 2011
  72. 72. Rust and Project Servo • A new, safer systems programming language, Rust, instead of C++ • Rust is a multi-paradigm, compiled language focused on safety, efficiency, programming-in-the-large, and a high degree of concurrency • Research building parallel browser engine stages in Rust • Experiment with Andreas Gal’s DOM implemented in JavaScript • For a scalably-faster-on-manycore, much safer Browser from the Future • Rust is good for Servers and other Software from the Future, too mozilla 22Friday, April 29, 2011
  73. 73. nbody.rs mozilla 23Friday, April 29, 2011
  74. 74. Rust performance results (nbody) mozilla 24Friday, April 29, 2011
  75. 75. fannkuchredux.rs mozilla 25Friday, April 29, 2011
  76. 76. Rust performance results (fannkuchredux) • Within 2x (much more to do) of clang optimized performance, with safety mozilla 26Friday, April 29, 2011
  77. 77. The big picture mozilla 27Friday, April 29, 2011
  78. 78. The big picture • We at Mozilla like formal methods and invest in them, but incrementally. • Fielding a competitive browser still entails use of unsafe languages such as C and C++. • Being safe but so slow you lose users does not help. mozilla 27Friday, April 29, 2011
  79. 79. The big picture • We at Mozilla like formal methods and invest in them, but incrementally. • Fielding a competitive browser still entails use of unsafe languages such as C and C++. • Being safe but so slow you lose users does not help. • Manycore and security challenges motivate pragmatic and hybrid methods mozilla 27Friday, April 29, 2011
  80. 80. The big picture • We at Mozilla like formal methods and invest in them, but incrementally. • Fielding a competitive browser still entails use of unsafe languages such as C and C++. • Being safe but so slow you lose users does not help. • Manycore and security challenges motivate pragmatic and hybrid methods • Art + Engineering require many hands at all layers of the system. Not all are ready for formal methods. mozilla 27Friday, April 29, 2011
  81. 81. The big picture • We at Mozilla like formal methods and invest in them, but incrementally. • Fielding a competitive browser still entails use of unsafe languages such as C and C++. • Being safe but so slow you lose users does not help. • Manycore and security challenges motivate pragmatic and hybrid methods • Art + Engineering require many hands at all layers of the system. Not all are ready for formal methods. • Jeremy Ashkenas: “Ruby is not used for software engineering reasons”, rather for aesthetic and programmer productivity reasons mozilla 27Friday, April 29, 2011
  82. 82. Where we are all headed mozilla 28Friday, April 29, 2011
  83. 83. Where we are all headed • When Mozilla began, CVS was the state of the art in open source version control mozilla 28Friday, April 29, 2011
  84. 84. Where we are all headed • When Mozilla began, CVS was the state of the art in open source version control • Since then (see http://revctrl.org/), distributed peer-to-peer version control systems have emerged and gained widespread use mozilla 28Friday, April 29, 2011
  85. 85. Where we are all headed • When Mozilla began, CVS was the state of the art in open source version control • Since then (see http://revctrl.org/), distributed peer-to-peer version control systems have emerged and gained widespread use • 15 years ago, forking was a dire thing, to be avoided; “branches” were costly and unmaintainable in parallel for very long mozilla 28Friday, April 29, 2011
  86. 86. Where we are all headed • When Mozilla began, CVS was the state of the art in open source version control • Since then (see http://revctrl.org/), distributed peer-to-peer version control systems have emerged and gained widespread use • 15 years ago, forking was a dire thing, to be avoided; “branches” were costly and unmaintainable in parallel for very long • Now with DVCSes such as Git and Mercurial, forking is not always a bad thing, and often a good thing mozilla 28Friday, April 29, 2011
  87. 87. Where we are all headed • When Mozilla began, CVS was the state of the art in open source version control • Since then (see http://revctrl.org/), distributed peer-to-peer version control systems have emerged and gained widespread use • 15 years ago, forking was a dire thing, to be avoided; “branches” were costly and unmaintainable in parallel for very long • Now with DVCSes such as Git and Mercurial, forking is not always a bad thing, and often a good thing • We are increasingly using http://github.com/, a Git-based “social hacking” site mozilla 28Friday, April 29, 2011
  88. 88. mozilla 29Friday, April 29, 2011
  89. 89. mozilla 30Friday, April 29, 2011
  90. 90. mozilla 31Friday, April 29, 2011
  91. 91. mozilla 32Friday, April 29, 2011
  92. 92. mozilla 33Friday, April 29, 2011
  93. 93. mozilla 34Friday, April 29, 2011
  94. 94. mozilla 35Friday, April 29, 2011
  95. 95. Conclusions mozilla 36Friday, April 29, 2011
  96. 96. Conclusions • Open source and distributed development obviously on the rise. Synergies between the two for social hacking, scaling up projects, recruiting new employees, and even partnering, all favor more open source and distributed development mozilla 36Friday, April 29, 2011
  97. 97. Conclusions • Open source and distributed development obviously on the rise. Synergies between the two for social hacking, scaling up projects, recruiting new employees, and even partnering, all favor more open source and distributed development • Formal methods for verifying correctness, static and dynamic, are coming, but there are two large gaps to close: • Between the github.com world of JS, Ruby, etc., and Mozilla’s C++ and Rust systems programming “kernel languages”. Contracts, gradual typing? • Between kernel languages we use at large scale, and the sound but hard to scale verification systems used on, e.g., device drivers (SLAM) mozilla 36Friday, April 29, 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×