Gareth Jones AllAboutTheAPI KeyNote 2016
•Download as PPTX, PDF•
0 likes•190 views
My keynote talk from AllAboutTheAPI.com in Las Vegas July 2016. Design practices to get you to mastery in the ways of the API. With a little help from Star Wars cosplayers along the way.
Recommended
More Related Content
More from Gareth Jones
More from Gareth Jones (9)
Recently uploaded
Recently uploaded (20)
Gareth Jones AllAboutTheAPI KeyNote 2016
- 3. Episode IV: A New API
- 9. Episode 5: The Docs Strike Back
- 13. Episode VI: Return of the Guidelines
- 19. Episode II: Attack of the Boundaries
- 23. Episode III: Revenge of the Pivot
- 28. Episode VII: The Citizen Awakens
- 30. @garethj_msft
- 32. @garethj_msft
- 33. Episode VIII: Your turn it is
Editor's Notes
- Thanks very much for the opportunity to talk to so many of you right at the start of this exciting new conference. It’s great to have another venue to bring the API community together with API consumers. My name’s Gareth Jones and I’m an API Architect at Microsoft and I want to take the next thirty minutes or so to talk about some of the best lessons I’ve learned whilst doing APIs at Microsoft.
- I’ve been at Microsoft for almost nineteen years now, which is a crazy amount of time – always working on something in the developer space. About 5 years ago, I started to get interested in building APIs for Visual Studio, where I was working, and I caught the API bug. Then I thought, well, if you’re going to work on APIs, you might as well go where the numbers are. So I moved to an API team in the group at Microsoft that produces Office. You can see for yourself from, these numbers, that Office’s journey from a desktop app to being an API-powered mobile and cloud app is well underway. So I want to share with you some of my experiences and lessons - and as last week was Star Wars Celebration in London, I thought I’d borrow a few Jedi metaphors to help out.
- So, to our first episode. Of course, everyone starts at Episode four, right?
- Here’s my question to you as a product manager? “Would you build an app before understanding your users’ goals?” I’m really hoping the answer is mostly “no” , of course not! For a mobile app, a website, pretty much any user experience, we wouldn’t dream of just throwing together a set of tasks and views of data and hoping someone found value in it. And yet, we don’t always hold ourselves to that standard when it comes to API design. You’d be amazed how many teams I talk to feel that simply projecting their product’s UI into their API surface is a good approach. Or even worse, projecting their backend datastore into their API.
- Here’s how it goes down – a team has heard all about the promise of the API economy. They have a product. They build an API around it, and hey presto – they hope the magic will happen. They will have new partnerships popping up all over the place and a steady stream of new users and revenue. But if they didn’t get super-clear about what problem they are solving and the developers don’t flock to it Then the ROI never shows up we have a problem. So how can we avoid this trap?
- Find your potential API customers – and be creative here – try and spread the net wide – not just the folks BizDev know will want the API, but folks at hackathons, conferences like this - you’ll be surprised how they think of their world intersecting with yours once you start talking. Brainstorm with them what your mutual customers would be achieving with an integration built on your two products and let that lead the API design. Who knows - maybe you’ll need to build new product features just to satisfy the really meaty integration scenarios, but then you’ll be adding real business value.
- And this is hard, cos when we’re focused on building APIs, we’re often not looking at the wider horizon of adding value to the product. My previous job at Microsoft was owning the API for OneNote - and I spent a lot of my first year meeting customers to encourage them to integrate with my new API. Most of them told me they wanted to have their product show up in OneNote’s UI, so customers could discover the value. And I spent that first year telling them, “Sorry, OneNote doesn’t have extensible UI. Please just use my REST API”. I spent much of the second year frantically adding that UI extensibility to enable to enable the compelling scenarios. Lesson learned.
- This can drive change in the way your whole team works. Try to build support at the top and bottom of your organization for partnerships and integrations as priorities in your product planning and go-to-market activities as well as simply constructing APIs. So I challenge you to start with a really open question when you meet potential partners – “If you could connect your product with my product, what would it look like?” <Pause> Build your key API priorities from there.
- So let’s say we now have some key scenarios validated with a range of partners. We need to crystalize those into something engineering can work on. Typically, at Microsoft, we’ve used a pretty traditional process of writing design specs, lots of internal reviewing, maybe sharing them back with a few partners, who frankly often found them pretty unparseable and full of Microsoft jargon. So we’ve shifted our functional speccing process to writing the developer docs for the APIs as the next step. Writing specs as docs simply forces you into taking an caller-centric view of your design. Poor usability or leaking internal concepts tends to stand out quite quickly if you’re trying to explain how to call your new creation. And then you don’t end up with monstrosities like my awful example here. This also has the huge benefit that they are easily reviewable by a lot of stakeholders.
- We want to take advantage of that ease of consumption by getting a really broad set of eyes on the docs, so we’ve started simply uploading them to github – we call this openspec. We just create a new branch in git and start to tell folks about it. We start with the scenarios written as user docs, then add more detailed reference pages. These are extremely cheap to iterate on and really easy for partners to review. It’s a process that’s starting to work really well for us.
- Now docs are great, but they lack the precision and detail of an API Description language like OAI or Swagger. Contract-first design with Swagger is something you’ll hear a lot about this week, but we wanted to keep our docs as the source of truth. They are just written in markdown, so we embedded the metadata in the docs to generate the API descriptions in either OData’s CSDL or OAI/Swagger. We use OData for rich querying a lot at Microsoft, so supporting both makes sense for us. This lets us validate that the docs are internally consistent and don’t have any basic errors in them, like two GET methods on the same URL. Then the generated API description lets us plug into all the great toolchains that API description languages enable. But we get to keep those original key scenarios in customer-readable docs as the ultimate source of truth.
- Return of the Guidelines with a motley crew of rebels
- So we have a nice clean flow through from the customer problem space to the engineering space of API description languages, but what other key design issues have we faced? Consistency. How do we get consistency between APIs designed by different teams and different people with strong opinions? At a conference a couple of years ago I asked a smart fellow named Jason Harman who was working for PayPal at the time how he’d managed to get different business units to align around guidelines. To paraphrase his answer, it was something like “By working really, really hard for two years.“. And there really isn’t any other way.
- Groups will always have reasons why they are special snowflakes, but customer feedback reflects that consistency across a company’s API surface removes frustration and keeps people engaged and prepared to learn about doing more with your products. So write your org’s guidelines for building APIs down and persevere with evangelizing them. It will pay back. One thing I’d note here - as an industry, we really, really need tools in this space.
- At Microsoft, we’ve been writing up our consistency rules as we learned them over the last couple of years and I’m really happy to announce that today we’re publishing our internal work as the Microsoft REST API Design Guidelines. We’ve been pulling together our collective experience building APIs for some of the largest cloud services in the world, and a lot of the customer feedback we’ve been given about them On top of that we’ve added a strong dose of accepted industry best practice building on those who’ve gone before us.
- Teams at Microsoft are using these guidelines every day. Internally, we have a number of API design coaching and review groups as part of the process we use to ship software, and these guidelines are our starting point across all of the core engineering groups in the company. Teams then add their own specializations. For example, the Azure team has an addendum to this which specifies things like how they apply versioning policies. We made sure we had buy-in and detailed agreement from technical leadership across all of those core groups, so the guidelines carried weight. It’s hard to just ignore a guideline if the technical leadership for your whole division has signed off on it.
- I encourage you to go to github and check the guidelines out and to come to our workshop which is bright and early at 8:30 tomorrow morning to dive in deeper. We believe the community will be able to gain value from reusing what we’ve learned and we’d love your feedback whether aligning on these guidelines makes our services easier and more effective for you to use.
- You’ll note there is no Episode One – there should be no episode one – what can I say. Jar-Jar. <shudder>
- No matter how consistent you APIs are, you can still show your product seams in ways that don’t benefit your customers. When we put product-focused API ideas and docs in front of customers, we find again and again that they talk to us about other Microsoft products – we talk about SQL, they bring up Azure Tables – we talk about Outlook, they bring up OneDrive - how annoying! Of course, customers simply aren’t invested in our product boundaries, and why on earth should they be? They just want to get at the value you provide. I’ve paraphrased Conway’s Law here – “Any piece of software reflects the organizational structure that produced it.” Don’t let your org structure constrain your customer’s potential for working with as much of your company’s value as possible.
- So to address that, let your API span your product portfolio, if at all possible. Make your API an explicit join point for your products. You can have a spanning API team - or make a virtual API team. Whatever works in your org. If you’re sitting there thinking about the energy required to work across teams in your organization, this may sound like a tall order to achieve. But even at Microsoft, and we’re a pretty large company, we’ve been able to make significant progress on a portfolio API.
- Our realization of this principle is an API we call the Microsoft Graph. We’ve pulled together data and insights from cloud services all across Microsoft into a coherent whole. You can see, we’ve got users, files, notes, email, calendars, spreadsheets, group conversations. All under one API. We find this incredibly freeing when talking with customers – it liberates us from our own silos and sparks our creativity. For example, you’re no longer doing just an email integration – you might pull a piece of email and then jump across to a file attached to that email on OneDrive and then put an invite on the calendar of the person who last modified the file. Its really quite a game changer for us. You can check it out at Graph.Microsoft.com to learn more and we also have a workshop tomorrow morning to learn more.
- OK, those red Sith eyes are a bit scary.
- Another huge challenge for APIs is managing product change, especially larger changes and pivots. Apps need our APIs to take a conservative approach to change in order to be stable. But what happens when the product simply must change? Firstly, when you make such changes, you need to increase your version number. And you need to run the older version of the API, so no apps actually break. But we also want integrations to take advantage of the new version without doing a huge rewrite. Is it possible to design for unknown future change? Or is it just an anti-agile waste of time?
- I believe a lot of significant change is relatively predictable. And removing assumptions from your APIs will help your customers deal with it. For example - new customer segments – business software embracing prosumers or even consumers and vice versa – don’t assume an org structure exists. Not everyone has a manager – not once you sell to home users – although my wife might disagree.
- - a product moving into adjacent markets – say a move from doctors into dentists – keep the domain-agnostic pieces distinct from the highly domain-specific and many integrations may still work - appointments are still appointments.
- As products mature, their data often becomes more customizable by end users– use an API stack that supports dynamic objects So quite a lot of low cost things you can do up front to provide for business change without forcing rewrites on your customers. The key is to drive out unnecessary assumptions from your API surface by thinking just a little ahead.
- And now to Rey, a humble citizen entrepreneur, who doesn’t know that she is strong in the way of APIs.
- Throughout the rise of the API economy, as API builders we’ve focused on two stakeholders. Business decision makers choosing to integrate with our products, and developers; with the vast majority of product energy going towards creating a great developer experience. This is on the cusp of huge change. We need to broaden from developer experience to a more inclusive consumer experience for APIs. Simply put, we need to expect our APIs to be consumed directly be a much wider set of people. A majority of enterprise products have an administrative component to them, ranging from setting them up when new users join an organization to auditing and regulatory compliance. APIs for these tasks bring the needs of IT pros to the fore.
- We’re well used to thinking of developer experience as being about portals where engineers learn about JSON structures or SDKs and try API calls in consoles. Examples in a range of programming languages are necessary. Here’s the OneDrive developer portal for example.
- But here’s the IT Pro’s natural home – not a code editor or an IDE, but the terminal. Her expectations are for pre-packaged scripts for bash or powershell. Having to learn the JSON to directly make her own curl calls is possible, but who has the time? We’re just starting to learn what IT Pros really want from APIs.
- And then the newest buzzwords – Citizen integrator and Citizen developer. This is the category that most analysts are predicting will grow fastest in the coming years as IT spend becomes more decentralized. Systems such as Microsoft Flow that I’m showing here, Zapier or StamPlay allow end users to connect APIs together like building blocks and assemble them into business workflows of increasing sophistication with zero-to-minimal coding skills. Low-code systems like Quickbase, or Mendix allow entire applications to be designed and built with only a little more expertise. As API producers, we need to broaden our developer experience models to include all of these new customers. There’s lots yet to learn to understand what kind of API experience these customers will prefer, but we need to engage them.
- We don’t know the title yet so it’s up to us to go out there and invent what comes next.
- So here’s my call to action for you. As an attendee at an API conference, you are among the API leaders in your organization. Whatever your job role, when you get back to work next week, be part of the drive for API consistency. If you build APIs and you don’t have API guidelines, start to push for some. Better still, as a leader, create some; build on the work other companies have published. If you already have guidelines, help evangelize them – make sure they’re being followed on your projects and run internal talks on them. If you primarily use APIs, then push your suppliers to be consistent across their APIs. It doesn’t matter if your APIs are public or internal - persuade people of the value of consistency to API consumers. For me, the best thing about working in the API arena is that there’s always something new to learn. This morning I’ve shared some of the lessons I’ve picked up over the last few years, and I’m hoping to learn many more from other practitioners here at the show this week. Thanks very much for your time and your patience with my terrible Star Wars puns. Enjoy the conference, and thanks to the following folk for the images: