Successfully reported this slideshow.
Your SlideShare is downloading. ×

Forget methodology. Focus on what matters.

Check these out next

1 of 142 Ad
1 of 142 Ad

Forget methodology. Focus on what matters.

Download to read offline

Presentation given by Jack Humphrey (Engineering Director, Indeed) at Keep Austin Agile 2014. http://keepaustinagile2014.sched.org/event/7faaa9add53a23a59185a595bca50230#.UzCTx61dVt0

For more presentations from Keep Austin Agile 2014, see: http://2014conf.agileaustin.org/?q=presentations

For more presentations from Indeed, see http://engineering.indeed.com/talks.

Presentation given by Jack Humphrey (Engineering Director, Indeed) at Keep Austin Agile 2014. http://keepaustinagile2014.sched.org/event/7faaa9add53a23a59185a595bca50230#.UzCTx61dVt0

For more presentations from Keep Austin Agile 2014, see: http://2014conf.agileaustin.org/?q=presentations

For more presentations from Indeed, see http://engineering.indeed.com/talks.

Advertisement
Advertisement

More Related Content

Similar to Forget methodology. Focus on what matters. (20)

Advertisement

Forget methodology. Focus on what matters.

  1. 1. forget methodology focus on what matters Jack Humphrey Keep Austin Agile March 21, 2014
  2. 2. JACK HUMPHREY
  3. 3. About Me ● I’ve been… ● a software developer for 19 years, ● a manager for 10 of those years, ● at companies including: ○ Trilogy (3 years) ○ Coremetrics (3 years) ○ Google (5 years) ○ Indeed (5 years and counting)
  4. 4. About Me ● I’ve worked on products like: ○ a web content management system, ○ enterprise software development frameworks, ○ a large online data collection platform, ○ features for online advertisers, and ○ a few large-scale consumer web applications
  5. 5. I help people get jobs.
  6. 6. About Indeed More job searches worldwide than any other employment website ● Over 100 million unique users ● Over 3 billion searches/month ● Over 24 million jobs ● 50+ countries, 28 languages Jun-07 Jun-08 Jun-09 Jun-10 Jun-11 Jun-12 Jun-13 Unique Visitors / Month
  7. 7. focus on what matters…
  8. 8. focus on what matters… SUCCESS
  9. 9. success for software products
  10. 10. build a solution
  11. 11. build a solution to a real problem
  12. 12. build a solution to a real problem that people want to use
  13. 13. build a solution to a real problem that (enough) people want to use
  14. 14. ● Planning (what to do) ● Process (how you do it) Build a solution to a real problem…
  15. 15. ● Timing ● Pricing ● Competition ● Promotion ● Quality ● Random chance ● etc. … that enough people want to use.
  16. 16. We identified the problem, now where do we start? Build a solution to a real problem…
  17. 17. STEP 1 define success
  18. 18. STEP 1 define success
  19. 19. STEP 1 build a great team
  20. 20. talent + passion
  21. 21. talent + passion + opportunity
  22. 22. STEP 2 define success
  23. 23. What is the overall mission?
  24. 24. Does everyone understand it and agree? What is the overall mission?
  25. 25. Does everyone understand it and agree? Does project success line up with the mission? If not, then a) it’s not success, or b) wrong mission! What is the overall mission?
  26. 26. ● Not a subjective determination Success should be quantifiable
  27. 27. ● Not a subjective determination ● Shipping is not success Success should be quantifiable
  28. 28. ● Not a subjective determination ● Shipping is not success ● Prove success with data Success should be quantifiable
  29. 29. I help people get jobs.
  30. 30. Job Search helps people get jobs ...by helping them find relevant job postings. Success for Indeed Job Search
  31. 31. Success: Unique Visitors?
  32. 32. Success: Job Searches?
  33. 33. success?
  34. 34. ● Demonstrate success with data ● Evaluate data objectively (unique visitors?) Avoid Subjectivity
  35. 35. ● You may need short-term approximations. When Success is Hard To Measure
  36. 36. ● Best metric for “help people get jobs”: hires Success for Job Search, take 2
  37. 37. ● Best metric for “help people get jobs”: hires ● What’s better than unique visitors & searches? repeat visitors, bounce rate, click-through rate Success for Job Search, take 2
  38. 38. ● Best metric for “help people get jobs”: hires ● What’s better than unique visitors & searches? repeat visitors, bounce rate, click-through rate ● What’s even better? applies (IndeedApply) Success for Job Search, take 2
  39. 39. ● Measurement should be repeatable ● Analysis should be: ○ accurate: based on correct/complete data ○ consistent: metrics don’t change meaning ○ fast: seconds/minutes, not hours/days/weeks Measure and Learn
  40. 40. Google Analytics
  41. 41. In-House Analysis Tools
  42. 42. ● We use Google Analytics ● We build our own frameworks and tools ○ logrepo: capturing what happens ○ ramses, imhotep: detailed analysis in seconds ○ Demo of these tools @IndeedEng on April 30th http://engineering.indeed.com/talks Measurement & Analysis Tools
  43. 43. Use whatever tools work for you, but make sure: ● measurement is repeatable ● analysis is accurate, consistent, fast Measurement & Analysis Tools
  44. 44. STEP 1 build a great team
  45. 45. STEP 2 define success
  46. 46. STEP 3 turn the crank
  47. 47. process can never replace talent and passion
  48. 48. process won’t deliver a successful product
  49. 49. but the right process can help
  50. 50. Repeatable and predictable mechanisms for: How process can help
  51. 51. Repeatable and predictable mechanisms for: ● Communication: what, who, how, when How process can help
  52. 52. Repeatable and predictable mechanisms for: ● Communication: what, who, how, when ● Priority: do things in the right order How process can help
  53. 53. Repeatable and predictable mechanisms for: ● Communication: what, who, how, when ● Priority: do things in the right order ● Memory: what happened, what next How process can help
  54. 54. Repeatable and predictable mechanisms for: ● Communication: what, who, how, when ● Priority: do things in the right order ● Memory: what happened, what next ● Delivery: get it done as efficiently as possible How process can help
  55. 55. Repeatable and predictable mechanisms for: ● Communication: what, who, how, when ● Priority: do things in the right order ● Memory: what happened, what next ● Delivery: get it done as efficiently as possible ● Quality: it should work intended How process can help
  56. 56. Repeatable and predictable mechanisms for: ● Communication: what, who, how, when ● Priority: do things in the right order ● Memory: what happened, what next ● Delivery: get it done as efficiently as possible ● Quality: it should work intended ● Risk: minimize the chance of bad things How process can help
  57. 57. “forget about methodology” is an exaggeration How methodologies can help
  58. 58. Can be templates for improving: ● communication ● priority ● delivery How methodologies can help
  59. 59. Can be templates for improving: ● communication ● priority ● delivery ...helping your team iterate on process How methodologies can help
  60. 60. Can help you think about ● What is working well for us? ● What could be better? How methodologies can help
  61. 61. Can help you think about ● What is working well for us? ● What could be better? … and offer prescribed process changes How methodologies can help
  62. 62. Recently, an experienced engineering manager described a process change that he loved: An interesting anecdote
  63. 63. Recently, an experienced engineering manager described a process change that he loved: “we stopped tracking bugs” An interesting anecdote
  64. 64. Ask yourself, what techniques can help your team: Focus on what matters.
  65. 65. Ask yourself, what techniques can help your team: ● communicate Focus on what matters.
  66. 66. Ask yourself, what techniques can help your team: ● communicate ● prioritize Focus on what matters.
  67. 67. Ask yourself, what techniques can help your team: ● communicate ● prioritize ● remember what has been done Focus on what matters.
  68. 68. Ask yourself, what techniques can help your team: ● communicate ● prioritize ● remember what has been done ● know what needs to be done Focus on what matters.
  69. 69. Ask yourself, what techniques can help your team: ● communicate ● prioritize ● remember what has been done ● know what needs to be done ● build solutions that work, efficiently Focus on what matters.
  70. 70. Ask yourself, what techniques can help your team: ● communicate ● prioritize ● remember what has been done ● know what needs to be done ● build solutions that work, efficiently ● manage risk associated with changes Focus on what matters.
  71. 71. Does your process (or lack thereof) enable success or just get in the way? Focus on what matters.
  72. 72. Start small and iteratively build process. ...or start big and iteratively reduce/modify process. Focus on what matters.
  73. 73. the evolution of process at Indeed
  74. 74. Process can be summarized simply: GET STUFF DONE Started Lean
  75. 75. Started Lean Deliver MeasureLearn
  76. 76. Started Lean Quickly! Deliver MeasureLearn
  77. 77. Idea to implementation, as quickly as possible! ● Software Architecture (Service-Oriented) ● Tools, like automated translation ● Risk Management with Rapid Iteration Engineering Velocity @IndeedEng Talk: http://go.indeed.com/velocity
  78. 78. Rapid Iteration… to production (multiple times a week) Velocity and Delivery
  79. 79. Rapid Iteration… to production (multiple times a week) ● Don’t wait to fail: means no big failures Velocity and Delivery
  80. 80. Rapid Iteration… to production (multiple times a week) ● Don’t wait to fail: means no big failures ● Work that’s ready doesn’t have to wait Velocity and Delivery
  81. 81. Rapid Iteration… to production (multiple times a week) ● Don’t wait to fail: means no big failures ● Work that’s ready doesn’t have to wait ● More satisfying for developers Velocity and Delivery
  82. 82. Rapid Iteration… to production (multiple times a week) ● Don’t wait to fail: means no big failures ● Work that’s ready doesn’t have to wait ● More satisfying for developers ● Less deadline pressure Velocity and Delivery
  83. 83. Rapid Iteration… to production (multiple times a week) ● Don’t wait to fail: means no big failures ● Work that’s ready doesn’t have to wait ● More satisfying for developers ● Less deadline pressure ● Developers own quality Velocity and Delivery
  84. 84. Velocity and Quality
  85. 85. ● Encourages modular thinking Velocity and Quality
  86. 86. ● Encourages modular thinking ● Fewer integration issues Velocity and Quality
  87. 87. ● Encourages modular thinking ● Fewer integration issues ● Fewer changes → faster, comprehensive QA ○ Release multiple times a week for 12 weeks, vs. ○ Release once in 12 weeks Velocity and Quality
  88. 88. ● Encourages modular thinking ● Fewer integration issues ● Fewer changes → faster, comprehensive QA ○ Release multiple times a week for 12 weeks, vs. ○ Release once in 12 weeks ● Encourages automated testing Velocity and Quality
  89. 89. ● Team was small, talented, passionate, focused. ● They iterated quickly. ● Job search was successful. Success...
  90. 90. Process was extremely ad-hoc ● not very consistent (e.g. bug tracking) ● not very repeatable or predictable ● wasn’t creating a memory for the future ...and Mess
  91. 91. Process was extremely ad-hoc ● not very consistent (e.g. bug tracking) ● not very repeatable or predictable ● wasn’t creating a memory for the future … and that was fine. For a while. ...and Mess
  92. 92. ...and develop predictable, repeatable ways to: ● Coordinate effort ● Improve communication and transparency ● Create a memory for the future ● Manage risk ● Ensure quality Time to reflect on process...
  93. 93. Infrastructure for Process
  94. 94. Infrastructure for Process
  95. 95. Infrastructure for Process ...for everything!
  96. 96. Type: PRODUCT Product Management in JIRA Issue Type: Initiative requirements
  97. 97. Project Management in JIRA Type: PROJECT Issue Type: New Feature Issue Type: Bug Issue Type: Improvement Issue Type: Task developer work deployable component
  98. 98. “Incorporates” Service A Service B Feature Improvement Product Initiative Webapp X Feature incorporates
  99. 99. “Depends Upon” Service A Service B Feature Improvement Product Initiative Webapp X Feature depends upon
  100. 100. Links are bidirectional Service B Improvement Product Initiative Webapp X Feature is depended on by is incorporated by
  101. 101. When will the feature be ready? Webapp X Feature Fix Version: 2014-03-20 target release
  102. 102. When will the initiative complete? Product Initiative Fix Version: March 2014 completion timeframe
  103. 103. Self-Documenting Workflow ● Workflow states and actions should suggest a clear meaning ● Custom workflows for different types of issues
  104. 104. Example: Project Issue Workflow Pending Triage Pending Acceptance Accepted Pending Verification Pending Closure Closed Create Triage Accept Resolve Verify Close
  105. 105. ● Fix Version deployed to production → issues are closed → future code changes require new issues ● Every code commit references issue key Closing the Memory Loop
  106. 106. Closing the Memory Loop JASX-4643 Type: New Feature Fix Version: 2011-04-21 is incorporated by UPDOG-76 Type: Initiative Fix Version: June 2011
  107. 107. iterate on process. it’s never done.
  108. 108. translation process production deploys a/b testing
  109. 109. ● Planning rapid international growth Iteration: Translation Process
  110. 110. ● Planning rapid international growth ● Need a localization strategy that wouldn’t slow us down Iteration: Translation Process
  111. 111. ● Planning rapid international growth ● Need a localization strategy that wouldn’t slow us down ● Must coordinate many translators Iteration: Translation Process
  112. 112. ● Planning rapid international growth ● Need a localization strategy that wouldn’t slow us down ● Must coordinate many translators ● We modeled it in JIRA. Iteration: Translation Process
  113. 113. Custom Translation Issues Project Explosion Translation (French) Translation (Spanish) Translation (Japanese) incorporates Translation (*)
  114. 114. “Explosion” Issue Workflow Pending Triage Pending QA Instructions Pending Review Pending Issue Explosion Pending Translations Pending Verification Create Triage Submit Reviewed Explode Complete Verify …
  115. 115. “Explosion” Issue Workflow Pending Triage Pending QA Instructions Pending Review Pending Issue Explosion Pending Translations Pending Verification Create Triage Submit Reviewed Explode Complete Verify … ?
  116. 116. Translation Issue Workflow Pending Acceptance Accepted Pending Verification Pending Closure Closed Accept Resolve Verify Close 1 issue per language
  117. 117. ● Custom issue types and workflows ○ clear communication ○ smooth transitions How JIRA helped Translation
  118. 118. ● Custom issue types and workflows ● Automation → 1 issue per language How JIRA helped Translation
  119. 119. ● Custom issue types and workflows ● Automation → 1 issue per language ● Subsequent iterations: ○ machine translation ○ translation service integration How JIRA helped Translation
  120. 120. Iteration: Production Deploys
  121. 121. Iteration: Production Deploys 50/week
  122. 122. ● Require a lot of communication and coordination between roles: ○ Create the release candidate deployable ○ Get it into a testing environment ○ Find the bugs ○ Fix the bugs ○ Declare it ready for production deploy ○ Deploy it to production Iteration: Production Deploys
  123. 123. ● Require a lot of communication and coordination between roles ● Many people care about current deploy status Iteration: Production Deploys
  124. 124. ● Require a lot of communication and coordination between roles ● Many people care about current deploy status ● We modeled it in JIRA. Iteration: Production Deploys
  125. 125. Deploy Issue Workflow Pre-Deploy QA Ready In QA QA Complete Deploy Ready In Production Ready In QA QA Complete Ready Deploy Create
  126. 126. ● Clear communication, smooth hand-offs ● Transparency for all stakeholders ● Ability to look at trends in deploys How JIRA helped Deploys
  127. 127. ● Proctor: data-driven framework for A/B testing ○ No full code deploy required for test updates ○ Lighter-weight, but still requires a process for verification and deployment Iteration: A/B Testing
  128. 128. ● Proctor: data-driven framework for A/B testing ○ No full code deploy required for test updates ○ Lighter-weight, but still requires a process for verification and deployment ● We modeled it in JIRA. Iteration: A/B Testing
  129. 129. Managing Data-Driven A/B Tests
  130. 130. Experiments and “Pro(ctor)Tests” Service A ProTest Product Experiment Webapp X ProTest incorporates
  131. 131. ProTest Issue Workflow Pending Requirements Pending Triage Pending Acceptance Accepted Submit Triage Accept Resolve Create … QA Ready
  132. 132. Managing Data-Driven A/B Tests
  133. 133. Managing Data-Driven A/B Tests
  134. 134. Managing Data-Driven A/B Tests QA Ready In QA In QA
  135. 135. ProTest Issue Workflow (cont.) QA Ready In QA Production Ready In Production Pending Closure Closed Deploy Verify Deploy Verify Close Resolve …
  136. 136. ● Clear communication ● Tool integration saves time ● Manages risk through verification process How JIRA helped A/B Testing
  137. 137. Indeed’s approach: Iterate. Iterate. Iterate. Summary
  138. 138. Find the best process for your team’s success. Iterate on process as necessary. Stay focused on success. Focus on Success
  139. 139. good luck, and thanks!
  140. 140. Q & A for more about Indeed Engineering: engineering.indeed.com for more about me: indeed.com/me/JackHumphrey

×