Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Journey to APIs and Microservices: Best Practices

883 views

Published on

Agility is enabled by systems that are flexible and extensible.

IEEE Standard Glossary of Software Engineering Technology defines Flexibility as the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed. and Extensibility as the ease with which a system or component can be modified to increase its functional capacity.

As customer needs evolve, flexibility helps with composing features and applications rapidly from using API, and extensibility helps in introducing the required changes quickly by making changes in services.

The challenge many companies have today is that they have built systems primarily as monoliths during a stage of their evolution, or have several services but they are tightly coupled. Satisfying a customer need even though the functionality already exists can take significant effort and cost.

Companies are therefore moving to APIs and microservices to address this business imperative. While many initiate this journey, most of them don’t end up with the intended outcomes - they falter or fail and go through many iterations.

This talk will share principles, best practices and approaches - across architecture, organizational and cultural - that have been applied in transforming companies such as eBay, PayPal and Intuit to APIs and Microservices.

Published in: Engineering
  • Be the first to comment

Journey to APIs and Microservices: Best Practices

  1. 1. Deepak Nadig Distinguished Architect,Intuit @deepak_nadig September 30,2016 Journey to APIs and Microservices: Best Practices
  2. 2. 2
  3. 3. 3 Consumers Small Businesses Accounting Professionals Who we serve:
  4. 4. 4 Where we started Intuit’s journey began over 30 years ago when our founder Scott Cook sat at his kitchen table and watched his wife as she balanced their checkbook and thought there must be a better way. Where we started:
  5. 5. 5 Era of the CloudEra of Windows Era of WebEra of DOS 2010+1980s 1990s 2000s Proven track record: capitalizing on change Intuit Founded Quicken DOS IPO Quicken Window s QuickBooks DOS TurboTax Online QuickBooks Windows TurboTax Windows QuickBooks MobileQuickBooks Online QuickBooks Online Global • Employees: 150 • Customers: 1.3M • Revenue: $33M • INTU + 1,400%, NASDAQ + ~500% • Employees: 4,500 • Customers: 5.6M • Revenue: $1.04B • INTU - 20%, NASDAQ - 40% • Employees: 7,700 • Customers: 50M • Revenue: $3.1B • INTU +160%, NASDAQ 90% INTUshareprice SnapTax Competitor A Competitor A Competitor C Competitor A Competitor A Competitor B Competitor D Competitor E
  6. 6. 6 Recognized as one of the world’s leading companies: FORTUNE 100 BESTCOMPANIES TO WORK FOR - 15 Years in a Row MOST ADMIRED: SOFTWARE INDUSTRY MOST INNOVATIVE COMPANIES 13 Years in a Row 2004 2005 2006 2007 2008 2009 2010 2011 2012 2015 WORLD’S MOST ADMIRED COMPANIES 2013 2014 2 Ranked # 2016 2016 34 Ranked #
  7. 7. 7 # 1 Small Business Management.7700 employees.$4.7 Billion in revenue. QuickBooks has more than 5 million customers. Online is 1.5 million (41% growth YoY). 26K developers. 2500 third-party apps.15% of customers use at least one third-party app. Yearly, 100 million invoices; $1.5 trillion in commerce; payroll for more than 1 million employees (1/12 in US). Small Business Owners spend 4 hours online every day. QuickBooks is in 112 countries. Intuit and QuickBooks by the numbers
  8. 8. 8 The capability of rapidly and cost efficiently adapting to changes - Wikipedia The ability of an organization to sense environment change and to respond efficiently and effectively to it - Gartner Challenge for businesses today = Agility
  9. 9. 9 Architecture Business Cycle organization (who, together, fashioned the quality-attribute requirements), the technical environment, and the architect’s experience. In our adaptation of the ABC, the “influences” have been replaced by a set of forces that collectively shape the requirements and, over time, the architect’s experience. The extended ABC is depicted in Figure 1. Figure 1: Extending the Architecture Business Cycle to Include All Design Forces Notice that this extended ABC shows the requirements as being categorized into three overlapping sets: quality-attribute requirements (e.g., the system should initialize itself in 60 seconds), business requirements (e.g., version 1.0 must be available to our resellers by August 1), and functional requirements (e.g., the system must provide monthly sales summaries, broken down by region). These requirements, along with the Software Engineering Institute http://sei.cmu.edu/ Architecture Business Cycle has impact on Architecture, Business and Culture
  10. 10. 10 • Architecture journey • Business journey • Culture journey • Current à Target ABC S-Curve Business Performance Time Kickoff High Growth Limits Reached ABC ABC ABC (*) ABC = Architecture Business Cycle
  11. 11. 11 Flexibility and Extensibility Flexibility Ease with which a system or component can be modifiedfor usein applicationsor environmentsother than thosefor which it was specificallydesigned Extensibility Ease with which a system or component can be modifiedto increaseits functionalcapacity IEEE Standard Glossary of Software Engineering Technology
  12. 12. 12 Agility is enabled by Flexibility and Extensibility APIs enable flexibility Microservices enable extensibility Why APIs and Microservices?
  13. 13. 13 Journey begins as one of the following … UI and Business Logic Web Browser Web UI Web Browser Business Logic Web UI Web Browser Business Logic Mobile UI Mobile Web UI Web Browser Business Logic Mobile UI Mobile We need microservices! Why are we still slow? We need mobile! No time to refactor Develop feature N times Just wantto tweak UI! How do we organize? Yay! Growingfast One language Slowing releases
  14. 14. 14 Target State API Definition Internal or External Universal API Discovery Painful Developer Portal API Design Project specific API as a Product Architecture Tightly coupled SOA Loosely coupled SOA Technology Proprietary Standards based Integration Expensive TTFHW* < 5 min (*) TTFHW = Time To First Hello World
  15. 15. 15 Portfolio of APIs aligned by business capabilities, realized by isolated and encapsulated services, that can be used by internal and external developers to develop applicationsand integrations quicklyand cost effectively API Platform
  16. 16. 16 API Platform Principles API First API Design • Think API before experience • API is a product • Externalize-able • Domain model & Integration scenarios Developer Experience • Easy to learn, integrate, diagnose • Be the ‘developer’ API Quality Attributes • Response-time • Availability Service Architecture • Encapsulated, Isolated • Craftsmanship
  17. 17. 17 • Know the end user • Work back from the developer and integration • Treat 1st party = 2nd party = 3rd party • Make it easy to integrate • Zero impact to integration as API evolves • Loosely coupled services Elements in building an API Platform
  18. 18. Best Practices API as a product API design Experience vs Capability API API specification API documentation Service granularity Service development API testing API status & support Developer Portal
  19. 19. 19 End users Understand Segments Their Experiences And Expectations Developers Know Tools and processes Their Integration needs Be Transparent on usage API as a Product
  20. 20. 20 API Portfolio Define bounded contexts/capabilities Organize cohesive concepts Use orthogonality or distance to separate API Design Use problem space for domain model Consider all interactions – Events, Batch Plan to dog food API (and events) Share designs (apistylebook.com) API Design id id /invoicing /payments /risk DomainModel à API • Entities, Attributes • Verbs • Relationships • State machine
  21. 21. 21 Capability API What API teams deliver Optimizedfor API provider REST, Batch, … Experience API Personalized for end user experience Make it self-service for developer Plan for multiple- GraphQL, Groovy Experience vs Capability API API Facade Invoicing Payments Customer Experience API Capability API
  22. 22. 22 Many choices OAS*, JSON Schema Capability,Experience API Automatic Docs, SDKgeneration Maintain as code GitHub, GitFlow(agile versioning) Pull Request (Governance) Share it API Specification (*) Open API Specification
  23. 23. 23 By developers for developers Developers contribute documentation Technical writers are editors Include in release criteria Maintain as code GitHub, GitFlow Should be tested like code Open it for contributions API Documentation
  24. 24. 24 Why you should care? Extensibility Granularity Principles Team boundary Cohesion Security Rate of change Scalability Service Granularity Credit: Spotify Engineering Culture
  25. 25. 25 Getting to microservices Code complexity vs. change Pick the first, complete, repeat Team autonomy, diagnosingissues New microservice Plan for multiple languages Plan for CI Team (4) > Team (2) (45% more time)* Service Development Complexity Low High High Low Least priority Low priority Impacting cycle time Easier to decompose Start here Impacting cycle time most Decompose, then refactor Change (*) The team scaling fallacy: Underestimating the declining efficiency of larger teams
  26. 26. 26 API Testing Test independently API specificationcanhelp bootstrap Complete scenarios (e.g. Webhooks) Sandbox Sandbox = Production For internal and external developers Support automatedtesting API Testing
  27. 27. 27 API Status Developers need to know Healthdashboard Report status promptly on issues Diagnosing issues Treat developers alike Provide API logs with messages Be proactive using API analytics API Status & Support
  28. 28. 28 Complete developer experience For 1st and 3rd party developers Time To First Hello World < 5 min Developer Portal
  29. 29. Culture
  30. 30. 30 A way of thinking,behaving, or working that exists in a place or organization - Merriam-Webster Behavior of humanswithin an organization and the meaning that people attach to those behaviors. - Wikipedia What is culture?
  31. 31. 31 Application development Web page design Process Site design/Portfolio Management Engineers work with page designs API development API DesignProcess API Portfolio Management Engineers work with API designs API Engineering Processes App. Product Manager UED* Application Engineering API Product Manager API Designer Service Engineering (*) UED = User Experience Design
  32. 32. 32 Be aware of Conway’s law Namespace != Organization Reuse = Coupling Agreements based on standards Teams evolve at different rates Use maturity model to measure Organization Evolution How do committees invent?
  33. 33. 33 Build something in 24 hours Internal hackathons Usability testing for APIs Pride of ownership for API teams Developer advocacy Hackathons
  34. 34. 34 • Agility is a business imperative • Agility comes from flexible and extensible architectures • APIs enable flexibility • Microservices enable extensibility • Architecture and culture evolve together Summary
  35. 35. Thank You

×