1. Thank you to our Sponsors
Doc Norton
Groupon - Global Director of Engineering Culture
doc@groupon.com
@DocOnDev
Media
Sponsor:
The Experimentation Mindset

2. The Experimentation Mindset
Doc Norton
Global Director of Engineering Culture
doc@groupon.com
@DocOnDev

3. Fixed Mindset
Redwood Bonsai - This is a life constrained

4. Growth (Agile) Mindset
Redwood Tree - This is a life freed

5. Dreyfus Model of Skills Acquisition

6. Novice
Novice
• Little or no knowledge
• Little or no experience
• Need rules
• Impatient for results
• focus on how over why
• Generally one teacher
Need a mentor and close monitoring

7. Advanced Beginner
Advanced Beginner
• Some experience
• Can find information
• Break free of some rules
• Can’t filter irrelevant information
• Can’t determine importance
Need experience in limited and controlled real-world situations

8. Competent
Competent
• Have a mental model
• Associations formed
• Can handle the unknown
• Methodical
• Thinking still steeped in “right” and “wrong”
Need a variety of real-world situations to form connections between already held ideas and models

9. Proficient
Proficient
• Interested in big picture
• Impatient with over-simplified information
• Grasp and apply maxims - YAGNI, Do the Simplest Thing That Can Possibly Work
• Internalization
• Potential to become an investigator / experimenter
Need a lot of practice, hindered as little as possible by policies or guidelines

10. Expert
Expert
• True Authority
• Developed Intuition
• Deep pool of knowledge
• Can Interlink Skills
• Tend to be inarticulate in how they arrive at conclusions
• Passionate Advocate for True Learning (having experienced it)
Continue to practice. Learn by teaching.

11. Where are You?

12. Most think they’re expert
Most think they’re expert

13. Most never progress past proficient
Most stay stuck at proficient

14. From Implementation
These first few stages are all similar in that …

15. Implementation
Mindset
Get it Right
A+ To progress from beginner to competence and into proficiency, our focus is on getting it right. Earning good grades. NOT making mistakes. This is a focus
on implementation.

16. From Implementation to Experimentation
But the next phase of growth requires a change…

17. Experimentation
Mindset
Explore Different Ways
But to progress from Proficiency into Mastery, we must experiment and learn unencumbered by rules and constraints. We need to get it wrong and then
make it better. We need to explore different ways.
Need a lot of practice, hindered as little as possible by policies or guidelines
MUST EXPERIMENT to move PAST PROFICIENT

18. From Implementation to Experimentation
To progress from beginner to competence and into proficiency, our focus is on getting it right. Earning good grades. NOT making mistakes. This is a focus
on implementation.
But to progress from Proficiency into Mastery, we must experiment and learn unencumbered by rules and constraints. We need to get it wrong and then
make it better.
Need a lot of practice, hindered as little as possible by policies or guidelines
MUST EXPERIMENT to move PAST PROFICIENT

19. Telephone Sales

20. Telephone Sales

21. Telephone Sales

22. Telephone Sales

23. Follow the Script
This is
gonna go
GREAT!

24. Weekly Sales Rankings
It didn’t go so GREAT
Ptolemy
Janet
Raphael
Susan
Kathy
Tommy
Doc
Chris P.
3 weeks in a row below the line and you’re out. 5 weeks in an 7 week period and you’re out.
The line was not a set sales target. This was strict rank and yank. 25% of people were ALWAYS below the line.

25. Hello (sGir/omoadd Damay),!my name is _P_t_o_lePma_ty _M Mici_ch_ha_ee_llaska_k_is_ …
Watch the Master

26. Experiment
Jokester

27. Experiment
Demanding

28. Experiment
Calmly Confident

29. Experiment
Played Dumb

30. Experiment
Was just friendly

31. Weekly Sales Rankings
Top in Sales!
Doc
Ptolemy
Janet
Kathy
Tommy
Raphael
Susan
Andy

32. I’d like to see you in my office.

33. Not the feedback I was expecting
“The script is proven. It is a best practice. We need to follow best practices to get the best results.”

34. My last day
On the day I decided was my last day, for my last phone call, I stood on my desk. Then Tommy stood, then Kathy, then Ptolemy, then others….
I closed that call. I sold a gentleman 3 years of Field and Stream Magazine.
And I gladly left.

35. “Best” Practices
Best Practices are a misnomer. The very notion is steeped in a Fixed and Implementation Mindset. They create artificial boundaries for our learning.

36. Implementation
Mindset
Get it Right
A+ They wanted me to get it right. Follow the script. Practice the script. Get better at the script. The script was their Best Practice. I was violating best
practices.
More organizations need an experimentation mindset

37. Single Loop
Learning
Assumptions Actions Outcomes
Get it Right
This is what Chris Argyris calls Single Loop Learning.
We hold fast to underlying assumptions, such as the notion of a best practice. Operating in this context, we prioritize how good we are at following the
practice over how well we’ve achieved the actual goal.
Incremental Improvement & Little innovation (if any at all)
A focus on getting it right creates an environment where failure is hidden. Want to look good.
Deceit => Suspicion => Contempt => Dark Side

38. Double Loop
Learning
Assumptions Actions Outcomes
Get it Right
Explore Different Ways
Double Loop Learning
Challenge our base assumptions - remember your purpose
• Don’t just get better at code reviews; consider how else you can
• Share knowledge, Enforce Standards, Maintain Quality
• What if we didn’t have - Annual goals, quarterly financials, managers or hierarchy…
• How do we KNOW the latest management/leadership/process trend will work here?

39. Experimentation
Mindset
Explore Different Ways
The experimentation mindset moves us from not only getting better at how we do things, but finding better ways to do them.

40. “Best” Practices
Instead of “Best” Practices

41. “Leading” Practices
Think of them as “Leading” Practices

42. Dreyfus Model of Skills Acquisition
Montessori Schools and other “alternative” educational formats

43. Experimentation

44. Experiment Configuration
Finch - homage to Darwin

45. Experiment Monitoring
Panopticon

46. Build Status

47. Application Performance

48. Cumulative Flow

49. Team Velocity

50. Code Quality

51. Code Quality

52. Retrospectives

53. Interest Leagues
Interest Leagues
Communities of people with common interests, related to work, but not directly about delivery of a Groupon project.
Java League, Node League, Ruby League, On-Boarding League, Speaker League
These groups create our standards. Standards come from the people who do the work. They come from a team self-selected and self-organized people
from all over the globe.

54. Internal Hack Fest
GeekOn
Two (or more) times per year - Engineering and Product take a week to hack.
Cannot be part of your regular work
Have to be able to squint at it and see Groupon
Projects are voted on and Executive team funds them for additional 20% time

55. Culture Clubs
Culture Clubs
Volunteers in each office. Plan office events - parties, charity drives, in-chair massage
Culture clubs meet to share ideas and tips. Coordinate events across offices for some occasions.

56. GEMs Program
GEMs program
Groupon Enrichment and Mentoring

57. 360 Feedback
360 Feedback
Several Teams were doing this already.
Launched multi-team pilot to see if we could spread this throughout the organization.

58. The Check-In
The Check-In
Form => Histograms => Overall
01 - I know what's expected of me at work.
02 - I have the materials and equipment I need to do my work right.
03 - At Groupon, I have the opportunity to do what I do best every day.
04 - In the last seven days, I have received recognition or praise for doing good work.
05 - My manager, or someone at Groupon, seems to care about me as a person.
06 - There is someone at Groupon who encourages my development.
07 - At Groupon, my opinions seem to count.
08 - The mission/purpose of Groupon makes me feel my job is important.
09 - My co-workers are committed to doing quality work.
10 - I have a best friend at work.
11 - In the last six months, someone at Groupon has talked to me about my progress.
12 - This last year, I've had opportunities at Groupon to learn and grow.

59. 4 things

60. Know your purpose
What problem are you trying to solve? What value are you trying to create?
Without this, you are not experimenting, you are meandering aimlessly.
Are you building educational software or are you making education available for all, regardless of their financial standing?
Are you selling coupons on the internet or are you empowering local business?
Tesla Motors recently open-sourced their patents.
Their purpose is to reduce carbon emissions and revolutionize the automotive industry. Knowing this, it was the obvious choice. They are blazing the trail,
but cannot do it alone.

61. “Failure is simply the
opportunity to begin again,
this time more intelligently.”
– Henry Ford
Make Failure Acceptable
When failure is acceptable, success is more likely

62. THINK BIG… start small…
Don’t scale too soon. Test in multiple ways. Iterate. Learn.

63. Assumptions Actions Outcomes
Get it Right
Explore Different Ways
Keep Experimenting.

64. The Experimentation Mindset
Doc Norton
Global Director of Engineering Culture
doc@groupon.com
@DocOnDev
Thank
You!

65. Images
• Mad Scientist - https://www.flickr.com/photos/dzingeek/4587871752/sizes/o/
• Redwood Bonsai - http://upload.wikimedia.org/wikipedia/commons/6/6e/Redwood_bonsai.JPG
• Redwood Canopy - https://c2.staticflickr.com/4/3548/3498180845_a2d653a323_b.jpg
• Child to Adult - http://1.bp.blogspot.com/-rBplVVYK1CQ/Uu-EDJPXSzI/AAAAAAAAAHs/ErtWY1kctiY/s1600/HiRes.jpg
• Phones -
• http://www.enumclawsda.org/uploads/1/3/3/9/13391741/1971539_orig.gif?119
• http://4.bp.blogspot.com/-qASUt2T7C7o/ULvJ8HVU3LI/AAAAAAAAACw/LufW5HYhCyM/s1600/Tel%C3%A8fon+fixe
+antic.png
• https://c2.staticflickr.com/8/7443/8758001086_2692f91620_h.jpg
• http://cdn2.hubspot.net/hub/147894/file-270530761-jpg/images/phone-large.jpg
• Phone Script - http://www.google.com/imgres?imgurl=http%3A%2F%2Fimg.docstoccdn.com%2Fthumb%2Forig
%2F6629075.png&imgrefurl=http%3A%2F%2Fwww.docstoc.com%2Fdocs%2F6629075%2FSample-Sales-
Script&h=1650&w=1275&tbnid=nhUALN2u__S9YM%3A&zoom=1&docid=-
yQKAu8BxDXbEM&ei=PhVFVIvsOcfRiQK9kYGABQ&tbm=isch&ved=0CB8QMygDMAM&iact=rc&uact=3&dur=1681&page=1&start
=0&ndsp=25

66. Reference Material
• Dreyfus Model of Skills Acquisition - http://en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition, http://
files.eric.ed.gov/fulltext/ED384695.pdf#page=130, file:///Users/mnorton/Documents/Dropbox/Presentations/
Groupon/Experimentation%20Mindset/ADA084551.pdf
• HBR: Collective Genius - http://hbr.org/2014/06/collective-genius/ar/1
• How Organizations Learn - http://www.areteadventures.com/articles/
the_challenge_of_entering_the_green_room.pdf
• Argyris: Teaching Smart People how to Learn - http://hbr.org/1991/05/teaching-smart-people-how-to-learn/ar/1,
http://pds8.egloos.com/pds/200805/20/87/chris_argyris_learning.pdf
• Argyris: Double Loop Learning in Organizations - http://hbr.org/1977/09/double-loop-learning-in-organizations/ar/3
• Argyris: Single-Loop and Double-Loop Learning in Research on Decision Making - https://
093fb1c5e28761e25b79cfc10de03f5a188778a0.googledrive.com/host/0B_RO2ClD0V7AZHpWRS1mNUc2d1k/
argyris.pdf
• Lean Change - http://www.lean.org/LeanPost/Posting.cfm?LeanPostId=225
• Scaling Up Excellence -
• Start With Why -