apidays LIVE Paris - Growing an API Culture by Saul Caganoff & Liz Douglass
•Download as PPTX, PDF•
0 likes•82 views
apidays LIVE Paris - Responding to the New Normal with APIs for Business, People and Society December 8, 9 & 10, 2020 Growing an API Culture Saul Caganoff, Principal at Deloitte Platform Engineering Liz Douglass, Partner at Deloitte
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to apidays LIVE Paris - Growing an API Culture by Saul Caganoff & Liz Douglass
Similar to apidays LIVE Paris - Growing an API Culture by Saul Caganoff & Liz Douglass (20)
More from apidays
More from apidays (20)
Recently uploaded
Recently uploaded (20)
apidays LIVE Paris - Growing an API Culture by Saul Caganoff & Liz Douglass
- 1. Growing an API Culture “T’aint what you do…” Liz Douglass, Partner at Deloitte Platform Engineering @lizdouglass Saul Caganoff, Principal & CTO at Deloitte Platform Engineering @scaganoff
- 2. Agenda Part 1: Common problems and anti- patterns Part 2: What does better look like? Part 3: Growing applications into domains
- 4. Bottlenecks ⊷ Consumers waiting on APIs ⊷ Waiting on central integration team ⊷ Waiting on shared infrastructure ⊷ System inertia due to technical debt 4 Silos ⊷ Lots of similar APIs ⊷ No benefits from reuse Contention ⊷ Version proliferation – APIs don’t get retired ⊷ Version churn – APIs change rapidly
- 5. 55
- 6. 66
- 7. 77
- 8. 9 Reference Henrik Kniberg: Spotify Engineering Culture (Part 1)
- 9. 10 Disconnected Projects Sweet Spot • Domain autonomy& durability • Governanceat the edges 😭 🚀 HighAlignment • Discovery • Consistency High Autonomy • Durability • Self-service LowAutonomy • Order-takers Delivery Model Governance Model Integration Factory 🏭
- 10. Part 2. What does better look like?
- 11. The flow model ⊷Features ⊷Defects ⊷Risk mitigation ⊷Technical debt 12 a zero-sum game Project to Product by Mik Kersten
- 12. The problem with (only) projects ⊷ Compartmentalised scope and time ⊶ “Just need to get it done” ⊶ Sometimes this is unavoidable – but technical debt is OK if you pay it back ⊷ Short-term incentives with no consequences for shortcuts ⊷ More convenient to look at things as evergreen ⊶ Easier to start from scratch than to re- use someone else’s stuff ⊷ Projects are hampered in their ability to promote durable, re-usable assets such as13
- 13. Product thinking ⊷ An antidote to some of the problems of projects ⊷ Product thinking looks like: ⊶ A roadmap of features extending into the future ⊶ Features driven by customer demand ⊶ The ability to push back (or say “no”) on customer demand – e.g. to maintain the integrity of the product ⊷ Typically manifest in a product manager who has resources, rights and longevity to deliver on a product roadmap 14
- 15. Many organisations struggle with product thinking ⊷ 50 years of the project paradigm: ⊶ Concepts ⊶ Processes ⊶ Funding models ⊶ Organisational structure ⊷ Procurement models based around COTS ⊶ Projects commission COTS “boxes” and join them up (integration) ⊶ Pick the product then pick an implementation partner (system integrator) ⊷ Attention centres on the box rather than the business capability 16
- 16. Integration becomes a thing we “do” 17 ⊷Delivered via projects ⊷Lines between boxes ⊷An obligation ⊷Inherits the problems of projects ⊶difficult to create durable, re-usable assets Application 1 Asset boundary Application 2 Asset boundary Integration Design Build Run Endpoints Data structures Mapping rules
- 17. Can we flip this model? 18 ⊷Most organisations already have durable asset teams ⊶Centred on COTS applications ⊶Stopping at the application boundaries ⊷What if we move the APIs into those asset boundaries? ⊷Asset teams become the durable owners of their domain APIs Application 1 Asset boundary Application 2 Asset boundary API Provider Design Build Run Endpoints Data structures API Consumer Mapping rules
- 18. 19 ⊷ Application boundaries expand to form domains Application 1 Asset boundary Application 2 Asset boundary
- 19. 20 Domain boundary Domain boundary ⊷ Application boundaries expand to form domains ⊷ Domain capabilities expressed via APIs & event streams
- 20. 21 Domain boundary Domain boundary ⊷ Application boundaries expand to form domains ⊷ Domain capabilities expressed via APIs & event streams ⊷ Core applications augmented by microservices
- 21. 22 Domain boundary Domain boundary Domain boundary ⊷ Application boundaries expand to form domains ⊷ Domain capabilities expressed via APIs & event streams ⊷ Core applications augmented by microservices ⊷ Domain boundaries become a scaffold for modernisation
- 22. 23 Domain boundary Domain boundary Domain boundary
- 23. Integration “emerges” from the act of consuming APIs Integration is no longer a thing you “do” in projects 24
- 24. 25 Disconnected Projects Sweet Spot • Domain autonomy& durability • Governanceat the edges 😭 🚀 HighAlignment • Discovery • Consistency High Autonomy • Durability • Self-service LowAutonomy • Order-takers Delivery Model Governance Model Integration Factory 🏭 Slide template courtesy of slidescarnival.com – licensed under Creative Commons
Editor's Notes
- Introductions – Liz first
- Saul: Our talk today is split into 3 parts - part 1 looks at common problems and anti-patterns that we see today - part 2 looks at what a better model might look like - part 3 presents a technique for achieving that better model within enterprises today
- Saul: so first – what’s the problem?
- Saul: In Platform Engineering we specialize in enterprise integration and we work with dozens of different clients across a range of industries and with different digital maturity. The common thread we see is that organisations want to deliver technology faster and at reduced cost. We’re in the position where we can see many different responses to this challenge and there are common anti-patterns that emerge from these responses Liz: Bottlenecks - Teams waiting for their APIs to be implemented often because of a dependency on a single centralised team. Alternatively this can occur when you're constrained by shared infrastructure, or because you're weighed down by technical debt. Saul: Silos We often see duplicate APIs – lots of slightly different copies of APIs doing essentially the same thing We also see organisations struggling with achieving time and cost savings through reuse of APIs Liz: The third class of anti-pattern we see is contention And in this context we're talking about contention between versions - - Either you have the domain expressed in lots of APIs, or The expression of the API is changing quickly the versions are churning. Saul: And it's important to note that these anti-patterns arise regardless of the technologies used in API programs: - API gateways, - developer portals, - microservices, - CI/CD etc. These things are helpful but not necessarily sufficient for success.
- Liz: Each of these problems can be traced back to intermediate causes: Ownership – not possible because of dependencies on tasks or infrastructure that belong to a different team Durability – or “attention span” – the ability of a team to evolve their ”product” over a period of time
- Liz prioritized roadmap that allows different teams to plan and collaborate over shared assets And finally having visibility of those shared assets – both the ones in production and the ones which are planned or under development
- Liz: And then we thought about it more we decided.... Ownership and Durability are driven by the level of Autonomy that teams have, which in turn is driven by the delivery model used. Roadmap and visibility are driven by how different teams achieve alignment across the broader organisation and ultimately that comes down to your governance model
- Liz: Each of these problems can be traced back to intermediate causes: Ownership – not possible because of dependencies on tasks or infrastructure that belong to a different team Durability – or “attention span” – the ability of a team to evolve their ”product” over a period of time prioritized roadmap that allows different teams to plan and collaborate over shared asset And finally having visibility of those shared assets – both the ones in production and the ones which are planned or under development And then we thought about it more we decided.... Ownership and Durability are driven by the level of Autonomy that teams have, which in turn is driven by the delivery model used. Roadmap and visibility are driven by how different teams achieve alignment across the broader organisation and ultimately that comes down to your governance model
- Saul: This is reminiscent of the way that Spotify have talked about the relationship between team autonomy and alignment and how technology teams need both in order to move faster in the same direction. Liz: We can overlay this diagram with an integration perspective and we see similar patterns of use occurring in organisations....
- Liz: We see most organisations that we speak to are in the top left hand quadrant Organisations in the quardrant have traditional approach to integration and a centralised team that does all the integration work. They have high level of alignment by virtue of the fact that they sit together terms of: - discovery, and - consistency But the team often have the symptoms of being a bottleneck that we spoke about at the start of this presentation. Saul: Similarly, if they have multiple project teams and a poor governance model, then we see many disconnected projects leading to the silo and contention antipatterns that we also spoke about at the start of this presentation. Liz: Now this being a good 2x2 means that we always want to get to the top right quadrant. In this state the teams working on the APIs have: - domain autonomy and durability - and a high level of governance at the domain boundaries So how do can we get to the top right without falling into the chaos state at the bottom right where everyone just does whatever they feel like?
- Saul:
- Liz: Let's start by going into some more detail into what good looks like. One of our favourite books at the moment is "Project to Product" by Mik Kersten. In the book he describes how teams need to balance four flow types There's the feature flows where you are building new capability Defect flow where you are correcting errors Risk mitigation flow where you are proactively preventing issues like a compliance or security incidents, and Technical debt flow where you are paying down the debt that you might have accumulated whilst working on one of the other flow types. The point that Mik makes in the book is that you need to constantly balance the four flow types. All are required and you must make tradeoffs so as to prevent any of them from stagnating, Though the balance of them changes at each stage of the product lifecycle.
- Saul: The problem is that projects generally only concentrate on two types of flow – features and defects Projects are particularly bad at dealing with technical debt which is an important flow type which leads to systems inertia if not addressed Projects are often built around short-term incentives with no consequences for short-cuts - it is the following projects or the enterprise at large that suffer for short-cuts Projects also find it difficult to re-use because often that adds risk and dependencies rather than taking them away. So overall, projects are hampered in their ability to promote durable, re-usable assets such as APIs Liz: - This is especially true in the API space because you can keep building them with impunity…
- Liz: The way to get to that top right quadrant is to take a product thinking approach. Teams that do that: - Have durability: a roadmap of what they are going to build that takes into consideration how to evolve what they already have - They know their customers and therefore their domain well and the domain is expressed in the API design. - And importantly, the team, and in particular the product manager, is in a position to maintain the integrity of the product Saul: <EXAMPLE – doing some logic in the integration layer that should be in in the app. Don't corrupt your domain> Liz: So to summarise what we've established so far: API anti-patterns are driven for the most part by the governance model and the delivery model To avoid these anti-patterns we're aiming for a governance model that promotes high alignment, and a delivery model that optimises for high autonomy ...the top right quadrant And we've made the case that that an essential element of getting there is to adopt product thinking.... …. So what does that practically look like? I'm sure some of you are thinking, well that's great, but it would be easier to cycle to the moon than make that happen in my organisation....
- Saul: transition
- Saul: …. And yes we acknowledge that the project paradigm is deeply entrenched in many organisations. The project paradigm is deeply entrenched in the way we do things and it ripples through lots of dimensions: - Concepts: our mental model make it hard to believe there are different ways <LANGUAGE, SKILLS> - Processes: including the processes for addressing (or not addressing) the four flow types, like risk management processes - Funding models, as well as - The organisational structure Liz: We often find that organisations where the project paradigm is most ingrained also tend to typically buy software. They buy boxes and then customise them The enterprise capability has a hard boundary at the edge of the box And you go out and get a 2 for 1 deal: you get a box from a vendor and a 'systems integrator' to wire it in – which is not exactly a durable team approach. This procurement approach also impacts how organisations also think about integration.....
- Saul: As a result of the fact that we orient around applications, integration has become a specific thing that we do Integration is delivered within projects It’s concerned with drawing lines between boxes And it’s often regarded as an obligation rather than something with inherent value Delivering integration within projects makes it difficult to create durable, reusable APIs as assets Liz: - Yes and design , build and run etc all fall between asset boundaries
- Liz: So what to change? The best way we've found is to give roadmap responsibility and ownership to the asset teams for the APIs that integrate their systems. The asset teams are more likely to have at least a core team that is consistently funded to be able to take up this responsibility And they have the best knowledge of the end-users and the business domain. This has the effect of expanding the asset boundary outside the COTs application boundary And has numerous benefits including: - Durability of the APIs, and - A better likelihood of visibility and therefore re-use for the APIs.
- Saul: - The first thing we want to do is expand application boundaries to form business domains
- Saul: domain capabilities (services) are expressed via business APIs and event streams
- Saul: our original applications still reside at the core of our domains – but they become augmented by auxilliary applications and microservices which take care of concerns like customisation or internal integrations
- Saul: and these tightly controlled domain boundaries become a scaffold for modernisation of our applications within the domain with minimal impact outside the domain
- Saul: Zooming out to the bigger picture we see digital channels are built on top of the underlying domain capabilities – delivering our business capabilities out to customers, employees and partners
- Liz: Integration is a byproduct
- Liz: To finish up, We've covered some common anti-patterns associated with operating in the left hand side of the matrix, specifically: - Bottlenecks, silos and contention We've looked at the underlying causes of these issues and proposed that the aim is to achieve both high alignment and high autonomy. We've spoken about product thinking and how it better positions teams to achieve the top right quadrant than project thinking. And we've proposed that that an achievable way to adopt a product approach is for asset teams to become the durable owners of their domain APIs.