Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action


Published on

Keynote for eBay Classifieds TechCon 2013, Tues June 25, 2013.

This is a variation on previous lean engineering talks but focuses on 8 principles for enabling build/measure/learn.

Published in: Technology
  • Nice talk. But words are cheap, and this is very light on delivery. So you changed a dozen static marketing pages and made Express Checkout work in a modal (and apparently only for selected merchants from all I can see)... Over 2 years? I love how new people always think they're so much better than their predecessors. In 2 years, we used to do a redesign of pretty much the whole site, including checkout. With a lot fewer buzzwordy SlideShare presentations. FWIW.
    Are you sure you want to  Yes  No
    Your message goes here
  • Love this talk Bill. Esp #1 and #5 are the hardest to focus on.
    Are you sure you want to  Yes  No
    Your message goes here

8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

  1. 8 principles for enabling build/measure/learnlean engineering in actioneBay Classifieds TechConJune 2013@billwscottSr. DirectorUser Interface Engineering@paypal
  2. continuous customer feedback (GOOB)customer metrics drive everythingthink it. build it. ship it. tweak itfail fast. learn fast.lots of experimentation... build/measure/learna different view ofengineering
  3. paypal vs netflixcontrast this with a large enterprise like paypal (circa 2011)
  4. guess what i found (in 2011)roll your own. disconnected deliveryexperience. culture of long shelf life.inward focus. risk averse.
  5. In 2011, even a simplecontent copy changecould take as much as 6weeks to get live to site
  6. new dna insertedjan 2012fleshed out ui layer that could support rapidexperimentationmarch 2012david Marcus becomes president of PayPalapril 2012formed lean ux team to reinvent checkoutexperience
  7. hermes project lean ux/engineering in actionfrom whiteboard to code from code to usabilitylearningsstart again
  8. change has started working its way out
  9. change has started working its way out
  10. lean startup movementfounded on build/measure/learn cycleget out of the building (GOOB)invalidate your risky assumptionsfail fast, learn fastget to the pivotgo for the minimal viable product (MVP)
  11. lean uxdesigning products for build/measure/learnrequires 3 rules to be followed at all timesget to & maintain a shared understandingform deep collaboration across disciplineskeep continuous customer feedback flowing
  12. purpose of lean engineeringbuildmeasurelearnLEANENGINEERINGEnablingBuild/Measure/LearnwithLeanStartupPrinciples
  13. buildembracecontinuous deliverymake mistakes fastmeasure learnthe etsy way. Kellan Elliott-McCrea, CTO etsyuse metrics drivendevelopmentknow that you made amistakeblameless postmortemlearn from yourmistakes
  14. LEANENGINEERING8principlesforenablingbuild/measure/learn
  15. 1. focus on learning, not delivery
  16. one of our biggest challenges is movingfrom a culture of delivery to aculture of learning
  17. too many teams can create silos within theexeriencecommon silos that can affect experience:• number of scrum teams• specialization of skills• device channels• regional adaptationsCE2don’t let delivery driveexperience
  18. 2. build a culture of rapid experimentation
  19. long shelf life for softwarewhen software is not dynamically updatablewhen it takes herculean effort to deliverresultengineers run the asylumdelivery dates drive the experienceBDUF & waterfall prevail
  20. 16 different test cells in the initial PS3 Launch (2010)focus is on build/measure/learnfour distinct PS3 experiences launched on same daytypical netflix release
  21. ramping vs experimentingramping model results in oneexperience (with some tweaks alongthe way) after a long ramp up timeexperimentation model results inmany experiences being tested allalong the way
  22. avoid disconnecteddelivery experiencecirca 1985deliver to disk then to usereverything was focused on getting itperfect for stamping on the diskno user in the loop. experiencehappened somewhere down thesupply chain
  23. 3. design for volatility
  24. the epiphany
  25. you have to engineerfor volatilitychange is the normexperimentation is not a one time eventlaunching a product is giving birth to theproduct. the product’s life just for throwaway-abilitymajority of theexperience codewritten is thrownaway in a yearthe ui layer is theexperimentation layer
  26. experiences must learnOur software is always tearing itself apart(or should be)Recognize that different layers change atdifferent velocitiesAll buildings are predictions.All predictions are wrong.Theres no escape from this grimsyllogism, but it can be softened.Stewart Brand
  27. 4. embrace open source
  28. building experiencescirca 1985merry band of three. dropped out ofcollege for semester. it was nirvana.however...
  29. roll your own “everything”(close your eyes & imagine)no internet. no google. no blogs. no email. noblogs. no stackoverflow. no github. no twitter.much of the software era has been aboutbuilding from scratch.of course open source was gaining momentum.unix. gnu. linux. perl. mozilla.
  30. use open source religiously
  31. work in open source modelinternal github revolutionizingour internal developmentrapidly replacing centralizedplatform teamsinnovation democratizedevery developer encouragedto experiment and generate reposto share as well as to fork/pull request
  32. give back to open sourcewe have projects that we will open sourcenode webcore (similar to yeoman)we are contributing back to open sourcecontributions to bootstrap (for accessibility)contributions to bootstrap (for internationalization)core committer on dustjs project
  33. using github for continuous *use github for continuous integrationstarting to use github repo model for continuous deploymentmarketing pagesproduct pagescontent updates & triggers into i18n, l10n, adaptationcomponents
  34. 5. map lean onto agile
  35. btw, agile doesn’t have a brain...agile has been a big step in the right directionbut is an engineering disciplinedoesn’t address the full life cycleagile has become big business and sometimes collapses under the weightof “ceremonies” (process)but agile is a good “engine” for delivery if you know what to roughly buildagile needs a brain...
  36. lean ux: enable a brain for agileuser interface engineering - agile scrum team (production)lean ux - lean team track (prototyping)engineering - agile scrum teamsprint 0usability usability usability usability usabilityrelease release release release{agile
  37. 6. make your product a living spec
  38. create a living specenabling the prototypelearning
  39. stack circa 2011/early 2012simple change could take minutesto seefollows an “enterprise application”model. ui gets built into the “app”javajsp***restrictedcapabilities*prototypingwas hard“ui bits” couldonly live here* assumed client developers were low-skill* required server side java eng for simple client changes** java server pages. server-side java templating solutionserver sidecomponents**clientserver
  40. we blended prototype & productionwe enabled the “ui bits” to beportable between the prototypingstack and the production stackjava (rhinoscript)node.js{dust}JS templateprototypestackproductionstack{dust}JS templateeither stack
  41. java (rhinoscript)productionstack{dust}JS templateone stack: prototype & productionnode.js{dust}JS templateprototypestackthe final step is we made theprototype stack and productionstack the same technologythroughout the application stack
  42. 7. refactor your way out of debt
  43. technical debtrarely do you have a clean slategenerally you will have to refactor yourway to a nimble framework
  44. we separated the ui bitscode = JS(backbone)templates =JS{dust}style = CSS(less)imagesre-engineered the userinterface stack so thatthe only artifacts are:• javascript• css• imagesditched the server-sidementality to creating UIs• no more server-sideonly templates• no more server-sidecomponents• no more server-sidemanaging the ui
  45. code = JS(backbone)templates =JS{dust}style = CSS(less)imageswe used javascript templatingtemplates get convertedto javascript<p>Hello {name}</p>JavaScript compiles to...javascriptexecutedto generate ui
  46. we used natural web artifacts - “web bits”server-side language independentserver/client agnosticCDN readycacheablerapid to createcode = JS(backbone)templates =JS{dust}style = CSS(less)images
  47. ensured we could run on new & legacyJS templating can be runin client browser orserver on the productionstackwe can drag & drop theui bits from prototypingstack to the productionstackjava(rhinoscript)node.js{dust}JS templateprototypestackproductionstack{dust}JS templateclient ORservereither stack
  48. experience debtdon’t just think about our technical debtconsider our “experience debt”cripples our ability to capture market andinhibits learning
  49. 8. learn across all channels
  50. mobile strategy ≠ just iOS appnative apps make it easier to create a richexperiencehowever, they are limited in reach and inlearning capabilityapp install rates will only be a subset of thecustomer baseyou need both a native and html5 strategy inorder to maximize learning
  51. html5 is critical to learning strategynew users will see your html5 experiencethe onramp to onboarding is the lowly linknetwork delivery makes a/b testingstraightforwardnetflix gambled on html5 for mobile (iOS,android) and for game consoles, bluray players,hdtvs, etc.why? build/measure/learn. network delivery.
  52. summaryrethink engineering. every dimension of your engineering needs to be aboutenabling build/measure/learntechnology. but not for tech sake. we are doing it for the experience tosupport lean startup principles.process. enabled lean ux and put a brain on agile.people. revitalizing our existing talent and started attracting new talent.
  53. before
  54. after
  55. designing web interfacesO’Reillypicture credits me on twitter@billwscottDesigning Web InterfacesO’ReillyBill Scott & Theresa Neil