[@IndeedEng] Engineering Velocity: Building Great Software Through Fast Iteration

3,959 views

Published on

Video available: http://www.youtube.com/watch?v=zCy077_dyJo&feature=youtu.be

Since 2005, Indeed has created and cultivated a strong engineering culture with a focus on ownership, real-world impact, and constant incremental delivery. Our experience has demonstrated that rapid iteration is essential to discovering the most valuable functionality for our users. In the next @IndeedEng talk, Dan Heller will share some of the architectural solutions, tools, and processes Indeed has created to support constant incremental delivery of new features and enhancements.

Speaker:
Dan Heller has been working in software development for 13 years including time at Google, IBM, and long-forgotten startups. He has been at Indeed for the last 4 years, helping people get jobs by building products for Indeed’s employers and advertisers.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,959
On SlideShare
0
From Embeds
0
Number of Embeds
2,951
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

[@IndeedEng] Engineering Velocity: Building Great Software Through Fast Iteration

  1. 1. Engineering Velocity Building Great Software Through Rapid Iteration
  2. 2. Dan Heller Engineering Manager Employer Applications
  3. 3. Indeed #1 Job Site Worldwide More people find jobs on Indeed than anywhere else Over 100M monthly visitors Over 3B monthly searches
  4. 4.
  5. 5. Rapid Iteration Leads to Better Products
  6. 6. Data-Driven Product Design go.indeed.com/ddpd
  7. 7. Indeed Product Development Principles Measure Everything Test Everything (A/B Experiments) Iterate Quickly
  8. 8. Build MeasureLearn Lean Startup Cycle
  9. 9. Indeed Product Development Cycle Deliver MeasureLearn
  10. 10. Indeed Product Development Cycle Quickly! Deliver MeasureLearn
  11. 11. Indeed Product Development Cycle MeasureLearn MeasureLearn Deliver
  12. 12. Engineering Velocity MeasureLearn MeasureLearn Deliver
  13. 13. Engineering Velocity (n) [en-juh-neer-ing vuh-los-i-tee] Our ability to quickly deliver implementations of ideas to job seekers & employers
  14. 14. Deliver Developer Productivity
  15. 15. Software Architecture Processes and Tools Risk Management Deliver
  16. 16. Software Architecture Processes and Tools Risk Management Deliver
  17. 17. Service Oriented Architecture
  18. 18. Job Search Web App Doc Retrieval Company Info Authentication Location Index Search
  19. 19. Job Search Web App Index Search Doc Retrieval Company Info Location Authentication
  20. 20. Doc Retrieval Company Info Job Search Web App Authentication Location Index Search
  21. 21. Technical Challenges High Performance ● Speed is a feature ● 100ms response time
  22. 22. Technical Challenges Forward & Backward Compatibility ● Any client and any service can be updated at any time
  23. 23. High Performance SOA Framework Boxcar Image used with permission of O Scale Trains Magazine (oscalemag.com), photo credit Don McFall of Old Line Graphics http://go.indeed.com/boxcar
  24. 24. http://go.indeed.com/boxcar Boxcar: Load Distribution without Middleware
  25. 25. Represent complex data types Extremely compact and performant Forwards and backwards compatible Boxcar: API Compatibility
  26. 26. Boxcar: API Compatibility Request and response data encoded as Protocol Buffers Compatible evolution of service interfaces Reduces release order dependencies
  27. 27. Doc Retrieval Company Info Job Search Web App Authentication Location Index Search Boxcar!
  28. 28. Service Oriented Architecture for Web Frontends
  29. 29. Each page has different features different operational requirements different owners Each page should be its own application
  30. 30. Trends Company Info Forums Salary Job Search Indeed
  31. 31. Technical Challenges Provide a consistent user experience One implementation of shared functionality
  32. 32. Option: Shared Code Libraries Reduced flexibility of technology decisions Requires a release of all projects Occasional inconsistent user experience
  33. 33. Navshell Application-level Rewriting HTTP Proxy
  34. 34. Navshell Request Web App HTTP
  35. 35. <head> <title>Johnson & Johnson Careers</title> <link type="text/css" href="common.css"> <link type="text/css" href="comp.css"> </head> <body> <div id="page-header"> ...... </div> <div id="page-body"> <div id="page-frame"> <div id="comp-header"> ...... </div> </div> </div> </body>
  36. 36. <head> <title>Johnson & Johnson Careers</title> <link type="text/css" href="common.css"> <link type="text/css" href="comp.css"> </head> <body> <div id="page-header"> ...... </div> <div id="page-body"> <div id="page-frame"> <div id="comp-header"> ...... </div> </div> </div> </body>
  37. 37. <head> <title>Johnson & Johnson Careers</title> <link type="text/css" href="common.css"> <link type="text/css" href="comp.css"> </head> <body> <div id="page-header"> ...... </div> <div id="page-body"> <div id="page-frame"> <div id="comp-header"> ...... </div> </div> </div> </body>
  38. 38. REQUEST HEADERS Cookie, User-Agent, Referer, etc X-Indeed-Shell: 1 X-Indeed-Logged-In: [01] X-Indeed-Email: chris@indeed.com X-Indeed-Locale: en_US X-Indeed-Geo: US Navshell Web App
  39. 39. RESPONSE HEADERS Set-Cookie X-Indeed-Page-Title: Indeed Careers X-Indeed-Include-Js-Footer: /js/app.js X-Indeed-Include-Css: /style/main.css Navshell Web App
  40. 40. Navshell No duplicate code Centralized navigation changes Plain HTTP with strong contract Decoupled front-ends
  41. 41. Navshell Requests Services Web Apps
  42. 42. Web-Based Delivery of Mobile App Features
  43. 43. Mobile Mobile Browser Mobile App
  44. 44. Mobile Mobile Browser Mobile App
  45. 45. Mobile Browsers Mobile Apps Desktop Browsers Job Search App
  46. 46. ● One code base for mobile web, iOS, Android ● Simultaneous release of features across platforms Shared Code Across Platforms
  47. 47. ● No waiting for App Store approval ● No waiting for job seekers to upgrade Release On Our Own Schedule
  48. 48. Easy To Run Experiments ● A/B testing in native apps extremely challenging ● Reuse existing infrastructure for performing and evaluating experiments
  49. 49. Software Architecture Processes and Tools Risk Management Deliver
  50. 50. <Indeed map of the world>
  51. 51. one search. all jobs. una búsqueda. todos los empleos. Jedno vyhledávání. Všechna pracovní místa. Ein Klick. Alle Jobs. 一站搜尋,工作齊全 tek arama. bütün işler. uma busca. todos os empregos. één klik. alle vacatures. あらゆる仕事 まとめて検索 una ricerca. tutti i lavori.
  52. 52. Implement Commit Deploy to QA Deploy to Production Verify Before Internationalization
  53. 53. Implement Commit Translate Deploy to ProductionVerify Deploy to QA CommitExtract Internationalization Process Overhead
  54. 54. +1 day (20%)
  55. 55. +1 day (20%)
  56. 56. <Indeed map of the world>
  57. 57. <Indeed map of the world> <but with only US in orange>
  58. 58. Machine Translation English Portuguese
  59. 59. Machine Translation Low quality translations are better than no translations But we needed something better
  60. 60. Human Translation Service Tuesday afternoon: Commit Code Fetch Machine Translations
  61. 61. Human Translation Service Tuesday afternoon: Commit Code Fetch Machine Translations Request Human Translations Wednesday morning: Retrieve Human Translations
  62. 62. Implement Commit Translate Deploy to ProductionVerify Deploy to QA CommitExtract Localization before Translation Automation
  63. 63. Implement Commit Deploy to ProductionVerify Deploy to QA Extract Localization with Translation Automation Retrieve
  64. 64. Software Architecture Processes and Tools Risk Management Deliver
  65. 65. Probability of issue Duration of issue Number of users affected Risk
  66. 66. Releases
  67. 67. Smaller Releases More releases → less functionality per release Probability of issue
  68. 68. Monitoring Duration of issue
  69. 69. Real-time monitoring with Datadog ● Agent reports summarized data ● System and customized business metrics ● Customizable dashboards
  70. 70. Healthcheck Framework ● Applications and services report internal state & health ● Divert traffic away from unhealthy instances ● Disable features for failing dependencies
  71. 71. Monitoring and Alerting ● Identify runtime issues ● Alert Ops when an issue arises
  72. 72. Proctor ● System for managing A/B experiments ● Segment visitors and assign test groups ● Manage feature visibility
  73. 73. Proctor Manage Experiments
  74. 74. Proctor Manage Experiments → Manage Behavior
  75. 75. Proctor "rule" : "loggedIn" "rule" : "${smartphone && 'JP' == country}" "rule" : "${(ua.OS.family == 'ios' && ua.OS.majorVersion >= 6)}" etc.
  76. 76. Proctor if (proctor.isNewThingEnabled()) { newThing.generateResults(); } else { oldThing.generateResults(); }
  77. 77. Feature Toggles Probability of issue "rule" : "fn:endsWith(confirmedUserEmail, '@indeed.com')"
  78. 78. Decouple Features from Releases Duration of issue Rollback features and releases independently
  79. 79. Number of users affected Small allocation changes 0% → 1% → 5% → 50% → 100%
  80. 80. Software Architecture Processes and Tools Risk Management Deliver
  81. 81. Increased impact on products
  82. 82. Increased chance of success
  83. 83. No Failed Projects
  84. 84. No Failed Projects
  85. 85. Fail Fast
  86. 86. No Deadline Crunches
  87. 87. There’s always another release in a few days
  88. 88. More Transparent Schedules
  89. 89. Large changes hidden behind feature toggles
  90. 90. Incremental progress validated in production environment
  91. 91. We built all of this incrementally, to meet our own challenges
  92. 92. Yours are different
  93. 93. v = d / t
  94. 94. With rapid iteration & quick delivery, we can cover more ground
  95. 95. Q&A

×