Your SlideShare is downloading. ×
8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action


Published on

Keynote for eBay Classifieds TechCon 2013, Tues June 25, 2013. …

Keynote for eBay Classifieds TechCon 2013, Tues June 25, 2013.

This is a variation on previous lean engineering talks but focuses on 8 principles for enabling build/measure/learn.

Published in: Technology

  • Nice talk. But words are cheap, and this is very light on delivery. So you changed a dozen static marketing pages and made Express Checkout work in a modal (and apparently only for selected merchants from all I can see)... Over 2 years? I love how new people always think they're so much better than their predecessors. In 2 years, we used to do a redesign of pretty much the whole site, including checkout. With a lot fewer buzzwordy SlideShare presentations. FWIW.
    Are you sure you want to  Yes  No
    Your message goes here
  • Love this talk Bill. Esp #1 and #5 are the hardest to focus on.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. 8 principles for enabling build/measure/learnlean engineering in actioneBay Classifieds TechConJune 2013@billwscottSr. DirectorUser Interface Engineering@paypal
  • 2. continuous customer feedback (GOOB)customer metrics drive everythingthink it. build it. ship it. tweak itfail fast. learn fast.lots of experimentation... build/measure/learna different view ofengineering
  • 3. paypal vs netflixcontrast this with a large enterprise like paypal (circa 2011)
  • 4. guess what i found (in 2011)roll your own. disconnected deliveryexperience. culture of long shelf life.inward focus. risk averse.
  • 5. In 2011, even a simplecontent copy changecould take as much as 6weeks to get live to site
  • 6. new dna insertedjan 2012fleshed out ui layer that could support rapidexperimentationmarch 2012david Marcus becomes president of PayPalapril 2012formed lean ux team to reinvent checkoutexperience
  • 7. hermes project lean ux/engineering in actionfrom whiteboard to code from code to usabilitylearningsstart again
  • 8. change has started working its way out
  • 9. change has started working its way out
  • 10. lean startup movementfounded on build/measure/learn cycleget out of the building (GOOB)invalidate your risky assumptionsfail fast, learn fastget to the pivotgo for the minimal viable product (MVP)
  • 11. lean uxdesigning products for build/measure/learnrequires 3 rules to be followed at all timesget to & maintain a shared understandingform deep collaboration across disciplineskeep continuous customer feedback flowing
  • 12. purpose of lean engineeringbuildmeasurelearnLEANENGINEERINGEnablingBuild/Measure/LearnwithLeanStartupPrinciples
  • 13. buildembracecontinuous deliverymake mistakes fastmeasure learnthe etsy way. Kellan Elliott-McCrea, CTO etsyuse metrics drivendevelopmentknow that you made amistakeblameless postmortemlearn from yourmistakes
  • 14. LEANENGINEERING8principlesforenablingbuild/measure/learn
  • 15. 1. focus on learning, not delivery
  • 16. one of our biggest challenges is movingfrom a culture of delivery to aculture of learning
  • 17. too many teams can create silos within theexeriencecommon silos that can affect experience:• number of scrum teams• specialization of skills• device channels• regional adaptationsCE2don’t let delivery driveexperience
  • 18. 2. build a culture of rapid experimentation
  • 19. long shelf life for softwarewhen software is not dynamically updatablewhen it takes herculean effort to deliverresultengineers run the asylumdelivery dates drive the experienceBDUF & waterfall prevail
  • 20. 16 different test cells in the initial PS3 Launch (2010)focus is on build/measure/learnfour distinct PS3 experiences launched on same daytypical netflix release
  • 21. ramping vs experimentingramping model results in oneexperience (with some tweaks alongthe way) after a long ramp up timeexperimentation model results inmany experiences being tested allalong the way
  • 22. avoid disconnecteddelivery experiencecirca 1985deliver to disk then to usereverything was focused on getting itperfect for stamping on the diskno user in the loop. experiencehappened somewhere down thesupply chain
  • 23. 3. design for volatility
  • 24. the epiphany
  • 25. you have to engineerfor volatilitychange is the normexperimentation is not a one time eventlaunching a product is giving birth to theproduct. the product’s life just for throwaway-abilitymajority of theexperience codewritten is thrownaway in a yearthe ui layer is theexperimentation layer
  • 26. experiences must learnOur software is always tearing itself apart(or should be)Recognize that different layers change atdifferent velocitiesAll buildings are predictions.All predictions are wrong.Theres no escape from this grimsyllogism, but it can be softened.Stewart Brand
  • 27. 4. embrace open source
  • 28. building experiencescirca 1985merry band of three. dropped out ofcollege for semester. it was nirvana.however...
  • 29. roll your own “everything”(close your eyes & imagine)no internet. no google. no blogs. no email. noblogs. no stackoverflow. no github. no twitter.much of the software era has been aboutbuilding from scratch.of course open source was gaining momentum.unix. gnu. linux. perl. mozilla.
  • 30. use open source religiously
  • 31. work in open source modelinternal github revolutionizingour internal developmentrapidly replacing centralizedplatform teamsinnovation democratizedevery developer encouragedto experiment and generate reposto share as well as to fork/pull request
  • 32. give back to open sourcewe have projects that we will open sourcenode webcore (similar to yeoman)we are contributing back to open sourcecontributions to bootstrap (for accessibility)contributions to bootstrap (for internationalization)core committer on dustjs project
  • 33. using github for continuous *use github for continuous integrationstarting to use github repo model for continuous deploymentmarketing pagesproduct pagescontent updates & triggers into i18n, l10n, adaptationcomponents
  • 34. 5. map lean onto agile
  • 35. btw, agile doesn’t have a brain...agile has been a big step in the right directionbut is an engineering disciplinedoesn’t address the full life cycleagile has become big business and sometimes collapses under the weightof “ceremonies” (process)but agile is a good “engine” for delivery if you know what to roughly buildagile needs a brain...
  • 36. lean ux: enable a brain for agileuser interface engineering - agile scrum team (production)lean ux - lean team track (prototyping)engineering - agile scrum teamsprint 0usability usability usability usability usabilityrelease release release release{agile
  • 37. 6. make your product a living spec
  • 38. create a living specenabling the prototypelearning
  • 39. stack circa 2011/early 2012simple change could take minutesto seefollows an “enterprise application”model. ui gets built into the “app”javajsp***restrictedcapabilities*prototypingwas hard“ui bits” couldonly live here* assumed client developers were low-skill* required server side java eng for simple client changes** java server pages. server-side java templating solutionserver sidecomponents**clientserver
  • 40. we blended prototype & productionwe enabled the “ui bits” to beportable between the prototypingstack and the production stackjava (rhinoscript)node.js{dust}JS templateprototypestackproductionstack{dust}JS templateeither stack
  • 41. java (rhinoscript)productionstack{dust}JS templateone stack: prototype & productionnode.js{dust}JS templateprototypestackthe final step is we made theprototype stack and productionstack the same technologythroughout the application stack
  • 42. 7. refactor your way out of debt
  • 43. technical debtrarely do you have a clean slategenerally you will have to refactor yourway to a nimble framework
  • 44. we separated the ui bitscode = JS(backbone)templates =JS{dust}style = CSS(less)imagesre-engineered the userinterface stack so thatthe only artifacts are:• javascript• css• imagesditched the server-sidementality to creating UIs• no more server-sideonly templates• no more server-sidecomponents• no more server-sidemanaging the ui
  • 45. code = JS(backbone)templates =JS{dust}style = CSS(less)imageswe used javascript templatingtemplates get convertedto javascript<p>Hello {name}</p>JavaScript compiles to...javascriptexecutedto generate ui
  • 46. we used natural web artifacts - “web bits”server-side language independentserver/client agnosticCDN readycacheablerapid to createcode = JS(backbone)templates =JS{dust}style = CSS(less)images
  • 47. ensured we could run on new & legacyJS templating can be runin client browser orserver on the productionstackwe can drag & drop theui bits from prototypingstack to the productionstackjava(rhinoscript)node.js{dust}JS templateprototypestackproductionstack{dust}JS templateclient ORservereither stack
  • 48. experience debtdon’t just think about our technical debtconsider our “experience debt”cripples our ability to capture market andinhibits learning
  • 49. 8. learn across all channels
  • 50. mobile strategy ≠ just iOS appnative apps make it easier to create a richexperiencehowever, they are limited in reach and inlearning capabilityapp install rates will only be a subset of thecustomer baseyou need both a native and html5 strategy inorder to maximize learning
  • 51. html5 is critical to learning strategynew users will see your html5 experiencethe onramp to onboarding is the lowly linknetwork delivery makes a/b testingstraightforwardnetflix gambled on html5 for mobile (iOS,android) and for game consoles, bluray players,hdtvs, etc.why? build/measure/learn. network delivery.
  • 52. summaryrethink engineering. every dimension of your engineering needs to be aboutenabling build/measure/learntechnology. but not for tech sake. we are doing it for the experience tosupport lean startup principles.process. enabled lean ux and put a brain on agile.people. revitalizing our existing talent and started attracting new talent.
  • 53. before
  • 54. after
  • 55. designing web interfacesO’Reillypicture credits me on twitter@billwscottDesigning Web InterfacesO’ReillyBill Scott & Theresa Neil