6 principles for enabling build/measure/learn
lean engineering in action
Lean Day West
Portland, OR
Sept 17, 2013
@billwsc...
continuous customer feedback (GOOB)
customer metrics drive everything
think it. build it. ship it. tweak it
fail fast. lea...
paypal vs netflix
contrast this with a large enterprise like paypal (circa 2011)
two pizza team?
two burger team?
paypal way of engineering (in 2011)
roll your own. disconnected delivery
experience. culture of long shelf life.
inward fo...
In 2011, even a simple
content copy change
could take as much as 6
weeks to get live to site
new dna inserted
jan 2012
fleshed out ui layer that could support rapid
experimentation
march 2012
david Marcus becomes pre...
hermes project lean ux/engineering in action
hermes project lean ux/engineering in action
from whiteboard to code
hermes project lean ux/engineering in action
from whiteboard to code from code to usability
hermes project lean ux/engineering in action
from whiteboard to code from code to usability
learnings
hermes project lean ux/engineering in action
from whiteboard to code from code to usability
learningsstart again
before
after
change has started working its way out
change has started working its way out
change has started working its way out
rethink engineering in the
light of lean
shift the lens of engineering to
embrace the build/measure/learn
cycle
engineer f...
LEANENGINEERING
6principlesforenabling
build/measure/learn
1. engineer for learning, not delivery
most organizations biggest challenge is moving
from a culture of delivery to a
culture of learning
include engineering in customer learning
engineers should regularly be in usability
studies & customer visits
feedback fro...
include engineering in customer learning
engineers should regularly be in usability
studies & customer visits
feedback fro...
enable prototyping in the engineering stack
because engineering teams are
not trying to solve the learning
problem, they s...
stack circa 2011/early 2012
simple change could take minutes
to see
follows an “enterprise application”
model. ui gets bui...
we blended prototype & production
we enabled the “ui bits” to be
portable between the prototyping
stack and the production...
java (rhino js eng)
production
stack
{dust}
JS template
new single stack: prototype & production
node.js
{dust}
JS templat...
java (rhino js eng)
production
stack
{dust}
JS template
new single stack: prototype & production
node.js
{dust}
JS templat...
connect delivery to
learning
in 1985 I delivered software on a 3.5”
diskette
little or no feedback loop
everything was foc...
enable learning on mobile
native apps make it easier
to create a rich experience
however, they are limited in
reach and in...
html5 is critical to learning strategy
new users will see your html5 experience
the onramp to onboarding is the lowly link...
2. engineer for experimentation
16 different test cells in the initial PS3 Launch (2010)
focus is on experimentation
the netflix way
16 different test cells in the initial PS3 Launch (2010)
focus is on experimentation
four distinct PS3 experiences launche...
build
embrace
continuous delivery
make mistakes fast
measure learn
the etsy way. Kellan Elliott-McCrea, CTO etsy
use metri...
ramping vs experimenting
the big bet. ramping model results in
one experience (with some tweaks
along the way) after a lon...
long shelf life kills
experimentation
engineering has to make delivery a non-
event
result
delivery dates drive the experi...
a tale of two trains
departs infrequently
“gotta get on the train or I will have
to wait a long time”
a tale of two trains
departs infrequently
“gotta get on the train or I will have
to wait a long time”
departs all the time...
using git for continuous deployment
starting to use git repo model for continuous deployment
marketing pages
product pages...
3. design for volatility
the epiphany
you have to engineer
for volatility
change is the norm
experimentation is not a one time event
launching a product is givi...
you have to engineer
for volatility
change is the norm
experimentation is not a one time event
launching a product is givi...
experiences must adapt
Our software is always tearing itself apart
(or should be)
Recognize that different layers change a...
4. decentralize engineering
building experiences
circa 1985
merry band of three. dropped out of
college for semester. it was nirvana.
however...
roll your own “everything”
(close your eyes & imagine)
no internet. no google. no blogs. no email. no
blogs. no stackoverfl...
work in open source model
internal github revolutionizing
our internal development
rapidly replacing centralized
platform ...
give back to open source
we have projects that we will open source
node webcore (similar to yeoman)
we are contributing ba...
use open source religiously
nconf
async
q
supertest
5. put a brain on agile
credit: Krystal Higgins
http://bit.ly/18uP7N1
agile doesn’t have a brain...
agile is a hungry machine. it will crank out garbage or brilliance. and it will
do it iterat...
lean ux: enable a brain for agile
user interface engineering - agile scrum team (production)
lean ux - lean team track (pr...
lean ux: enable a brain for agile
user interface engineering - agile scrum team (production)
lean ux - lean team track (pr...
too many teams can create silos within the
exerience
what can fracture the experience
• number of scrum teams
• specializa...
as we have mapped lean onto agile it has
exposed mismatches between the way we
work
you run the risk:
• driving the experi...
6. refactor your way out of debt
technical debt
rarely do you have a clean slate
generally you will have to refactor your
way to a nimble framework
ensured we could run “ui bits” on new & legacy
we chose to make our
“ui bits” as close to
regular HTML/CSS as
possible
we ...
experience debt
don’t just think about our technical debt
consider the “experience debt”
cripples our ability to capture m...
rethink engineering in the
light of lean
shift the lens of engineering to
embrace the build/measure/learn
cycle
engineer f...
designing web interfaces
O’Reilly
picture creditshttp://www.flickr.com/photos/decade_null/2053134780/
http://www.flickr.com/...
6 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action
Upcoming SlideShare
Loading in...5
×

6 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

6,530

Published on

Presented at Lean Day West - Portland, OR. Sept. 17, 2013

How do you take a gigantic organization like PayPal and begin to transform the experiences? Engineering is often the key blocker in being able to achieve a high rate of innovation. In this talk, Bill Scott will give specific examples on implemented Lean UX in a 13,000 person company, re-factored the technology stack and changed the way engineers work with design & product partners. In addition, Bill will provide additional examples that go back to his early days writing one of the first Macintosh games to his more recent work at Netflix and the power of treating the user interface layer as the experimentation layer.

Published in: Technology, Business
1 Comment
15 Likes
Statistics
Notes
No Downloads
Views
Total Views
6,530
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
63
Comments
1
Likes
15
Embeds 0
No embeds

No notes for slide

6 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

  1. 1. 6 principles for enabling build/measure/learn lean engineering in action Lean Day West Portland, OR Sept 17, 2013 @billwscott Sr. Director User Interface Engineering @paypal
  2. 2. continuous customer feedback (GOOB) customer metrics drive everything think it. build it. ship it. tweak it fail fast. learn fast. lots of experimentation... build/measure/learn engineering for experimentation netflix view of engineering
  3. 3. paypal vs netflix contrast this with a large enterprise like paypal (circa 2011)
  4. 4. two pizza team?
  5. 5. two burger team?
  6. 6. paypal way of engineering (in 2011) roll your own. disconnected delivery experience. culture of long shelf life. inward focus. risk averse.
  7. 7. In 2011, even a simple content copy change could take as much as 6 weeks to get live to site
  8. 8. new dna inserted jan 2012 fleshed out ui layer that could support rapid experimentation march 2012 david Marcus becomes president of PayPal april 2012 formed lean ux team to reinvent checkout experience
  9. 9. hermes project lean ux/engineering in action
  10. 10. hermes project lean ux/engineering in action from whiteboard to code
  11. 11. hermes project lean ux/engineering in action from whiteboard to code from code to usability
  12. 12. hermes project lean ux/engineering in action from whiteboard to code from code to usability learnings
  13. 13. hermes project lean ux/engineering in action from whiteboard to code from code to usability learningsstart again
  14. 14. before
  15. 15. after
  16. 16. change has started working its way out
  17. 17. change has started working its way out
  18. 18. change has started working its way out
  19. 19. rethink engineering in the light of lean shift the lens of engineering to embrace the build/measure/learn cycle engineer for experimentation LEANENGINEERING Engineeringfor Experimentation withLeanStartup Principles
  20. 20. LEANENGINEERING 6principlesforenabling build/measure/learn
  21. 21. 1. engineer for learning, not delivery
  22. 22. most organizations biggest challenge is moving from a culture of delivery to a culture of learning
  23. 23. include engineering in customer learning engineers should regularly be in usability studies & customer visits feedback from “measure” phase should be regularly discussed in engineering you want to create an engineering culture that focuses on real customer problems
  24. 24. include engineering in customer learning engineers should regularly be in usability studies & customer visits feedback from “measure” phase should be regularly discussed in engineering you want to create an engineering culture that focuses on real customer problems engineer
  25. 25. enable prototyping in the engineering stack because engineering teams are not trying to solve the learning problem, they see prototyping as outside the engineering discipline this is a real engineering challenge engineer for the “living spec”
  26. 26. stack circa 2011/early 2012 simple change could take minutes to see follows an “enterprise application” model. ui gets built into the “app” java jsp*** restricted capabilities* prototyping was hard “ui bits” could only live here * assumed client developers were low-skill * required server side java eng for simple client changes ** java server pages. server-side java templating solution server side components** client server
  27. 27. we blended prototype & production we enabled the “ui bits” to be portable between the prototyping stack and the production stackjava (rhino JS eng)node.js {dust} JS template prototype stack production stack {dust} JS template either stack
  28. 28. java (rhino js eng) production stack {dust} JS template new single stack: prototype & production node.js {dust} JS template prototype stack the final step is we made the prototype stack and production stack the same technology throughout the application stack
  29. 29. java (rhino js eng) production stack {dust} JS template new single stack: prototype & production node.js {dust} JS template prototype stack the final step is we made the prototype stack and production stack the same technology throughout the application stack
  30. 30. connect delivery to learning in 1985 I delivered software on a 3.5” diskette little or no feedback loop everything was focused on getting it the one right experience on the disk no user in the loop. experience happened somewhere down the supply chain
  31. 31. enable learning on mobile native apps make it easier to create a rich experience however, they are limited in reach and in learning capability app install rates will only be a subset of the customer base you need both a native and html5 strategy in order to maximize learning
  32. 32. html5 is critical to learning strategy new users will see your html5 experience the onramp to onboarding is the lowly link network delivery makes a/b testing straightforward netflix gambled on html5 for mobile (iOS, android) and for game consoles, bluray players, hdtvs, etc. why? build/measure/learn. network delivery.
  33. 33. 2. engineer for experimentation
  34. 34. 16 different test cells in the initial PS3 Launch (2010) focus is on experimentation the netflix way
  35. 35. 16 different test cells in the initial PS3 Launch (2010) focus is on experimentation four distinct PS3 experiences launched on same day the netflix way
  36. 36. build embrace continuous delivery make mistakes fast measure learn the etsy way. Kellan Elliott-McCrea, CTO etsy use metrics driven development know that you made a mistake blameless post mortems learn from your mistakes
  37. 37. ramping vs experimenting the big bet. ramping model results in one experience (with some tweaks along the way) after a long ramp up time lots of little bets. experimentation model results in many experiences being tested all along the way vs
  38. 38. long shelf life kills experimentation engineering has to make delivery a non- event result delivery dates drive the experience feature-itus becomes prevalent BDUF & waterfall prevail little to no learning
  39. 39. a tale of two trains departs infrequently “gotta get on the train or I will have to wait a long time”
  40. 40. a tale of two trains departs infrequently “gotta get on the train or I will have to wait a long time” departs all the time “if I miss this train another one comes in a few minutes”
  41. 41. using git for continuous deployment starting to use git repo model for continuous deployment marketing pages product pages content updates & triggers into i18n, l10n, adaptation components works well with cloud deployment (devops model) enables the train to be leaving all the time
  42. 42. 3. design for volatility
  43. 43. the epiphany
  44. 44. you have to engineer for volatility change is the norm experimentation is not a one time event launching a product is giving birth to the product. the product’s life just begins. design for throwaway-ability majority of the experience code written is thrown away in a year
  45. 45. you have to engineer for volatility change is the norm experimentation is not a one time event launching a product is giving birth to the product. the product’s life just begins. design for throwaway-ability majority of the experience code written is thrown away in a year the ui layer is the experimentation layer
  46. 46. experiences must adapt Our software is always tearing itself apart (or should be) Recognize that different layers change at different velocities All buildings are predictions. All predictions are wrong. There's no escape from this grim syllogism, but it can be softened. Stewart Brand
  47. 47. 4. decentralize engineering
  48. 48. building experiences circa 1985 merry band of three. dropped out of college for semester. it was nirvana. however...
  49. 49. roll your own “everything” (close your eyes & imagine) no internet. no google. no blogs. no email. no blogs. no stackoverflow. no github. no twitter. much of the software era has been about building from scratch. of course open source was gaining momentum. unix. gnu. linux. perl. mozilla.
  50. 50. work in open source model internal github revolutionizing our internal development rapidly replacing centralized platform teams innovation democratized every developer encouraged to experiment and generate repos to share as well as to fork/pull request
  51. 51. give back to open source we have projects that we will open source node webcore (similar to yeoman) we are contributing back to open source contributions to bootstrap (for accessibility) contributions to bootstrap (for internationalization) core committer on dustjs project
  52. 52. use open source religiously nconf async q supertest
  53. 53. 5. put a brain on agile credit: Krystal Higgins http://bit.ly/18uP7N1
  54. 54. agile doesn’t have a brain... agile is a hungry machine. it will crank out garbage or brilliance. and it will do it iteratively. it is a travesty to waste this machine you have to get the experience “in the ballpark” to best use the machine it is imperative to make it easy to iterate designs ahead of the agile sprints leanux in the form of a “leanux scrum team” is one way to do this
  55. 55. lean ux: enable a brain for agile user interface engineering - agile scrum team (production) lean ux - lean team track (prototyping) engineering - agile scrum teamsprint 0 usability usability usability usability usability release release release release {agile
  56. 56. lean ux: enable a brain for agile user interface engineering - agile scrum team (production) lean ux - lean team track (prototyping) engineering - agile scrum teamsprint 0 usability usability usability usability usability release release release release {agile stories, user interface code come over from the leanux stream n+2 is more feasible because it actually bootstraps the agile stream
  57. 57. too many teams can create silos within the exerience what can fracture the experience • number of scrum teams • specialization of skills • device channels • regional adaptations CE2 watch out for fracturing the experience
  58. 58. as we have mapped lean onto agile it has exposed mismatches between the way we work you run the risk: • driving the experience based on eng teams • becoming a botteneck in almost all cases • either product is not aligned to the biz • or there are just too many people watch out for mismatch between teams
  59. 59. 6. refactor your way out of debt
  60. 60. technical debt rarely do you have a clean slate generally you will have to refactor your way to a nimble framework
  61. 61. ensured we could run “ui bits” on new & legacy we chose to make our “ui bits” as close to regular HTML/CSS as possible we enabled the same “ui bits” (templates) to run on any of our stacks java (rhino js eng) node.js prototype stack & production stack production stack client server any stack c++ (v8 js eng) production stack {dust} JS template
  62. 62. experience debt don’t just think about our technical debt consider the “experience debt” cripples our ability to capture market and inhibits learning key that engineering sees a chance to improve the experience whenever they are cleaning up technical debt
  63. 63. rethink engineering in the light of lean shift the lens of engineering to embrace the build/measure/learn cycle engineer for experimentation LEANENGINEERING Engineeringfor Experimentation withLeanStartup Principles
  64. 64. designing web interfaces O’Reilly picture creditshttp://www.flickr.com/photos/decade_null/2053134780/ http://www.flickr.com/photos/not_wise/182849352/ http://www.flickr.com/photos/37217398@N02/3442676067/ http://www.flickr.com/photos/hongiiv/4151964823/ Photo by Kim White: http://readwrite.com/2013/09/05/paypal-app-update-in-store- payments#awesm=~ohHUppP9dhMmMG http://www.flickr.com/photos/matthewpaulson/6176787688/ http://www.flickr.com/photos/olvrbrown/4542851399/ http://www.flickr.com/photos/juanpol/16287486/ http://www.flickr.com/photos/olvrbrown/4542851399/ http://www.flickr.com/photos/mbiskoping/6075387388/ http://www.flickr.com/photos/giesenbauer/4092794246/ http://www.flickr.com/photos/kowani/5565778790/ http://www.flickr.com/photos/ahockley/2657296577/ http://www.flickr.com/photos/90585146@N08/8222922317/ http://www.flickr.com/photos/therevsteve/3104267109/ Stewart Brand: How Buildings Learn (illustrations) http://www.flickr.com/photos/light_seeker/7444052000/ Krystal Higgins: http://www.kryshiggins.com/sketchnotes-of-bringing-design-to-life-with-lean-ux-lean-engineering/ http://www.flickr.com/photos/epsos/8463683689/ http://www.flickr.com/photos/proimos/3473264448/ http://www.flickr.com/photos/stuckincustoms/2380543038/ follow me on twitter @billwscott
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×