VMware's Journey to Deliver Developer Portals as a Service
•Download as PPTX, PDF•
0 likes•80 views
For many enterprises, publishing API docs is a lot like herding cats. In this talk, we’ll show you how VMware herds cats at scale with a framework that delivers developer portals as-a-service and provides developers ubiquitous access to API docs and other resources on the web and in VMware products.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to VMware's Journey to Deliver Developer Portals as a Service
Similar to VMware's Journey to Deliver Developer Portals as a Service (20)
More from Pronovix
More from Pronovix (20)
Recently uploaded
Recently uploaded (20)
VMware's Journey to Deliver Developer Portals as a Service
- 1. Confidential │ ©2018 VMware, Inc. Delivering Developer Portals as a Service VMware’s Journey to Create an Enterprise Platform for Developer Engagement Richard Thomchick Product Manager, Developer Experience November 2018
- 3. Agenda 3Confidential │ ©2018 VMware, Inc. The Evolution of VMware APIs Dev Center Framework Overview Herding the Cats: API Explorer Embracing the Community: Sample Exchange Streamlining Content Management: DxPress What’s Next: VMware {code} 2.0 Lessons Learned
- 4. 4Confidential │ ©2018 VMware, Inc. • BA degree is Lit, Master degree is Library and Information Science (how did I get here?) • Have been serving enterprise developers since 1999 (RIP Linuxcare) • Beaten in razor scooter race by Linus Torvald in 2000 • At VMware since 2007 in various positions (marketing, customer experience, R&D) • AM NOT the person who owns API design or documentation • AM the person who presents the docs to end users (and takes their complaints) • Docs are critical and represent 50% of my site’s traffic Who Am I? I promise this is the only slide with bullets
- 5. 5Confidential │ ©2018 VMware, Inc. The Evolution of VMware APIs From vSphere to the Hybrid Cloud vSphere APIs VIC CLI On-Prem Plugins APIs Hybrid Cloud vRealize Cloud Foundry SaaS VMware Cloud AWS Docker Kubernetes Jenkins Azure Plugins EUC Identity Manager AirWatch NSX vSAN PKS VKE Horizon
- 6. 6Confidential │ ©2018 VMware, Inc. The Evolution of VMware APIs A Typical VMware Integration Project vRealize Lifecycle Manager Dell-EMC Storage Dell-EMC VIPRSRM VMware NSXvRealize Automation vRealize Orchestrator API Integration vRealize OperationsManager vRealize LogInsight API Integration Content Pack API Integration Dell-EMC DataProtection vSphereClient &vCenter API Integration Management Pack API Integration Dell-EMC VIPR Solution Pack Plugin Management Pack SiteRecoveryManager Solution Pack API Integration Plugin Management Pack API Integration Plugin Plugin API Integration Plugin Plugin Custom Custom API Integration Custom Custom API Integration Custom
- 7. 7Confidential │ ©2018 VMware, Inc. The Need for a More Scalable Solution Monthly average volume of content requests 25-30 Tickets Monthly average time spent on content tasks 80-100 Hours Average time to complete each content request 8-10 Days10x More Content Contributors 100% Increase in page views and visitors + xxx More pages on the website 6,000+ New Community Members Overhead for Engineering Growth Since January 2017 Launch VMware {code} has grown, and so has our overhead
- 8. 8Confidential │ ©2018 VMware, Inc. The Need for a More Scalable Solution Projecting the break point for sustainability 0 20 40 60 80 100 120 140 2014 2016 2018 2020 API Growth vs. Engineering Story Points Growth Capacity
- 9. 9Confidential │ ©2018 VMware, Inc. Dev Center Framework Overview Deliver All Developer Resources as a Service to Developers on Any Device DxPress Portal CICDPipeline Hybrid On-Prem SaaS Other products vSphere {code} VMC-AWS Access PointsContentSources 3rd-party services Blogs GitHub Internal repositories Developer Content API Docs Developer Guides Code Samples How-to Articles Video Tutorials UI/UX Frameworks Clarity + Angular (in-product)Bootstrap (website) Common Services API Explorer Sample Exchange Acrolinx Other web services DevCenter Framework
- 10. 10Confidential │ ©2018 VMware, Inc. Dev Center Framework in Action VMware Cloud on AWS
- 11. 11Confidential │ ©2018 VMware, Inc. Herding the Cats: API Explorer vSphere 12+ APIs REST & SOAP OSS & Proprietary vRealize Automation 30+ APIs 30+ Swagger Docs SaaS & On-Prem Workspace ONE 700+ APIs 5+ Swagger Docs 3+ Authorization Models
- 12. 12Confidential │ ©2018 VMware, Inc. Herding the Cats: API Explorer Open source. API driven. Container friendly. vSphere 12 APIs REST & SOAP OSS & Proprietary vRealize Automation 30+ APIs 30+ Swagger Docs SaaS & On-Prem Workspace ONE 700+ APIs 5+ Swagger Docs 3+ Authorization Models Clarity UX Framework Swagger UI REST API GitHub GitLabInternal Build System VMware{code} API Explorer Website V StandaloneApp ProductPlugin
- 13. 13Confidential │ ©2018 VMware, Inc. Herding the Cats: API Explorer Product plugin experience
- 14. 14Confidential │ ©2018 VMware, Inc. Herding the Cats: API Explorer API Explorer “2.0” on website
- 15. 15Confidential │ ©2018 VMware, Inc. Embracing the Community: Sample Exchange Community powered. API driven. GitHub friendly.
- 16. 16Confidential │ ©2018 VMware, Inc. Embracing the Community: Sample Exchange Syndication and curated collections
- 17. 17Confidential │ ©2018 VMware, Inc. Streamlining Content Management: DxPress DxPress is “Developer Experience Press” … it’s like WordPress for Developer Content
- 18. 18Confidential │ ©2018 VMware, Inc. Publishing Content Prior to DxPress Create Page on Staging Publish Page to Production 1-2 days 1-2 days6-8 days CMS task (web content) Eng. task (help) Email queryDAM task (digital assets) Review task Typical Tasks Involved: Old Workflow: Creating a New SDK Page (8-10 Days!) Content tasks were only a small part of the process New DxPress Workflow: Creating a New SDK Page (8-10 Minutes!) 1-2 days … including turnaround time for emails and reviews
- 19. 19Confidential │ ©2018 VMware, Inc. DxPress in Action 100% of content tasks performed by end users 0 Help Tickets Engineering team went out for lunch and movies 0 Engineering Hours vs. 3-4 weeks of prep and testing before DxPress 3 Days to Complete23 Users Accessed the DxPress portal 8-9 SDKs Updated for vSphere and vSAN 130 Docs Contributed via DxPress 28 APIs Updated for vSphere and vSAN Overhead for VMware R&D Usage Metrics for DxPress vSphere 6.7 U1 Release
- 20. 20Confidential │ ©2018 VMware, Inc. What’s Next: VMware {code} 2.0 One developer experience, accessible from any device, on the web and in-product
- 21. 21Confidential │ ©2018 VMware, Inc. Best Practices, Lessons Learned, Work Still to be Done Don’t Try This at Home Validated Research Lean UX Design Constant Contact Relentless Automation Beware of Dependency Hell Don’t Build a Rocket Ship for a Trip to the Greengrocer
- 22. 22Confidential │ ©2018 VMware, Inc. Join VMware {code} code.vmware.com/join Subscribe to the newsletter code.vmware.com/newsletter Read our blog blogs.vmware.com/code Follow us on Twitter twitter.com/vmwarecode Learn More Herd Them Kitties All the Way to the Link Farm ….
- 23. Confidential │ ©2018 VMware, Inc.
Editor's Notes
- Show of hands, how many people know where these photos came from? These are from the original vinyl press of the White Album … believe it or not, I found these under a bed while I was renovating a college dorm room. I recognized them immediately and snatched them up since their previous owner had departed. I originally had all four prints, but you’ll notice that George Harrison is conspicuously absent …. His photo went missing a while back, and sometimes my friends and I like to quip about what the Beatles would be without him. I mean, technically John played guitar, so theoretically they could have have made it as a three-man band... But imagine what it would be like, sound like, without George. It would be a lot harder and the results probably wouldn’t be as good. Or better yet, imagine you’re a trio and someone said to you hey, what if you had a fourth Beatle? What could you do, or play, or accomplish that you wouldn’t even try otherwise? Hi, I’m Richard Thomchick, the product manager for VMware {code}, and several months ago, at the outset of this project, I asked myself these same questions as I pondered what my team could do with an extra developer … not just any engineer… I’m talking about a rock star 10x developer … 2. in our case, his name is Max … he’s like the fourth Beatle on our engineering team, and the journey I’m going to describe today has a lot to do with enabling developers like Max – employees and ecosystem alike– to be more productive and spend more time doing what they do best: innovating.
- In the 15 or so minutes we have together, I’ll briefly discuss the evolution of VMware APIs and why we needed to build Dev Center solution and major components such as API Explorer and Sample Exchange. I’ll also walk you through the dev center experience on VMware Cloud, and give you a glimpse of where we are going next....
- In the beginning, there was vSphere, and we had a SOAP API and a command-line interface ... VMware grew a lot over the next few years, with new platforms and new ways for developers to extend and integrate, but vSphere was still the star of the show … But then came along SaaS, public cloud, containers, and on and on and on … and so we shifted our focus the public cloud… Today we are pursuing a hybrid cloud strategy and releasing co-developed solutions like Pivotal Container Service, VMware Kubernetes Engine, and VMware Cloud on AWS …
- For developers, integrating VMware technology can be quite daunting …. What you see here is for a “basic” software-defined data center, but it looks a bit more like a flying spaghetti monster, doesn’t it? And many real-world integration projects are far more complex. ...... Managing all the content that enables developers to do this integration work is no easy feat, either. In fact, this was one of the biggest challenges facing us at the outset of this project – how to manage the accelerating growth and complexity of our company and developer ecosystem.
- For context, VMware {code} has seen tremendous growth since our initial launch in 2017 in terms of web traffic and memberships, and also in terms of the amount of content and the number of contributors we support. This growth does have a cost. Content requests comprise a significant and growing percentage of our total help ticket volume, and GTIX engineering has seen content-related overhead climb steadily to the point where it takes nearly a full FTE of engineering time just to handle content requests. In fact, we now employ an offshore team of engineers in China just to focus on content.
- This is a slide my boss likes me to show ... When we started projecting the expected rate of growth, based on the number of acquisitions and new products in the pipeline, and mapped it against our funding model and projected bandwidth, we realized we were quickly reaching a break point after which we would not be able to sustain operations … I naively thought this slide would get us more funding … but instead it spurred us to innovate, automate, and streamline our operations.
- Our solution for managing this complexity at scale is called the Dev Center Framework…. In a nutshell, the Dev Center framework routes content through common API services and presents it to the user via a standardized look and feel …. Here is the basic content flow …. Contributors – tech writers, engineers, product managers, etc. – add and update content via the DxPress self-service portal or directly from their build chains …. The Dev Center framework automatically assembles that content into a developer portal via a set of API services, and makes it accessible to users on the website and in our products. There are a lot of moving parts involved, but today I’m going to focus on three components: API Explorer…. Sample Exchange …. And our DxPress self-service portal for content management.
- But first, let me show you what one of these portals looks like … Our first go-to-market partner for the Dev Center Framework was VMware Cloud on AWS (which basically lets you run vSphere in the AWS public cloud) …. For this project we, embedded a ”developer center” into the VMC-AWS interface …
- The first step in our journey was to create a single source of truth for VMware API documentation, and this really did feel like we were herding cats … every product team seemed to have their own approach to documenting their APIs and while their single-serving solutions worked well at first, many of the teams had built up some technical debt as they grew and converged with other … For example, take vSphere, our flagship enterprise product … there are at least a dozen different public APIs … SOAP APIs for hardware access, REST APIs for automation, some open-source, some proprietary …. Another example, vRealize Automation, is more like a litter of cats … they market a single API but in reality there are more than two dozen individual services, each one documented in a separate JSON file … And then there’s Workspace ONE…other products may be colorful but these guys are the freaking rainbow Many of these teams were accustomed to working in “startup mode” (silos), so it took a lot of wrangling to chase them down and add their docs to the service….
- This an architectural overview of our API Explorer … the solution itself has an API that retrieves doc from multiple source repositories, and presents it to users via either Swagger UI or our own Angular library … the system supports Open API and RAML, as well as older SOAP APIs that were documented with Javadoc …there are even a couple of PDFs that we display as iframes…. we also knew that we needed to architect the solution as a service that could consume all kinds of API docs from multiple repositories, and deliver the information not only to the website
- what you’re looking at is here is the “product plugin” edition of API Explorer …. It’s essentially an Angular library that leverages an API to retrieve local and remote resources. One of the features I love the most is the live search and deep linking .... When you start typing in a path name, method description, etc., the system will show you the specific methods in each API that match your query, and link you directly to that section of the doc … developers really like this feature and it is something we are about to release on the website in the next sprint or two.
- Here’s a look at the next rev of our API Explorer on the website … this is us iterating on our haystack … it’s still in testing but as you can see we are applying the features across the board so you can get that deep linking experience across all our APIs … this is especially useful for developers who may be working with multiple APIs, some of which do the same thing or have the same name …
- The next component of the Dev Center framework is Sample Exchange for code samples and automation scripts from VMware and the community …. This project came about because our developer community asked for a better way to share their code samples, which they had previously been doing in forums and Slack… We wanted to enhance, not replace these tools, so we developed Sample Exchange as an API service that integrates with forums, Slack, GitHub, and so forth … With Sample Exchange, contributors can upload their content from a local directory, or link to existing code in any GitHub repository they have access to … the system automatically updates the content when changes are made to the repository so there is no need to continuously update this listing ... Once your sample is in the exchange, you can embed it in a blog article, forum post, or Slack conversation … The system is also integrated with our Jive community platform, so you also get Community points any other developers can bookmark, favorite, or download your samples, and other users can view the reputation and expertise of a sample author….
- The next component of the Dev Center framework is Sample Exchange for code samples and automation scripts from VMware and the community …. This project came about because our developer community asked for a better way to share their code samples, which they had previously been doing in forums and Slack… We wanted to enhance, not replace these tools, so we developed Sample Exchange as an API service that integrates with forums, Slack, GitHub, and so forth … With Sample Exchange, contributors can upload their content from a local directory, or link to existing code in any GitHub repository they have access to … the system automatically updates the content when changes are made to the repository so there is no need to continuously update this listing ... Once your sample is in the exchange, you can embed it in a blog article, forum post, or Slack conversation … The system is also integrated with our Jive community platform, so you also get Community points any other developers can bookmark, favorite, or download your samples, and other users can view the reputation and expertise of a sample author….
- One of the most important aspects of the Dev Center Framework is one that end-user developers don’t even know about: Streamlined self-service content management for developer resources that’s as easy as publishing a WordPress blog … Here’s a quick walkthrough of the 5-minute process for publishing a API doc on VMware {code} ….
- So, why did we build DxPress and why does it matter? Prior to DxPress, it took about 2 weeks, 2 help tickets, 3 separate software interfaces, and at least 4 people to publish pretty much anything. When we analyzed the actual tasks involved, we quickly realized that in addition to adding web pages, attaching assets, and reviewing the content, a LOT of time was being spent on engineering tasks and email queries .... For example, to add a new SDK to the website with a download and a programming guide, I would need to file a minimum of two help tickets, and wait for the offshore team in China to respond each time … just this part can take almost a week, but it still doesn’t account for the other 6-8 days, even when you factor in ample time to create and review content … So where was the time-suck coming from? Well, when we dug deeper, we found most users got stuck … a lot … and were filing tickets to get help almost every step of the way … Which inevitably led to email exchanges with engineers before, during, and after the ticket is resolved … Then after the problem is resolved, the user STILL has to contribute their own content … assuming the engineer hasn’t volunteered to do it on their behalf … again usually due to some corner case … And then finally, we observed that this workflow caused users to review their work over and over again, which prolonged the process even further …. NOW, with DxPress, we've cut this process down by an order of magnitude by making it fast and simple for users to publish their own content ... Even when you factor in lengthy reviews and email chatter, the typical turnaround time is 1-2 days max....
- Last month, we put DxPress to the test last month for a new release of the vSphere platform ... here are the highlights of what we published ..... And here is the total overhead for our group to support the launch ...
- So, what's next for the framework? Well, now that we’ve been able to validate the basic market fit, our next step is to unleashing it on the website. code.vmware.com leverages Dev Center services to display APIs and sample code, but we are looking to have full integration to the point where the website a fully compliant best practices reference to showcase its capabilities and establish best practices for stakeholders who want to embed developer centers in their products and cloud services .... What this means for developers is that they will get ubiquitous access to API docs and other resources with a consistent user experience on any device.... So with that in mind, here is a walkthrough of the {code} 2.0 prototype ...
- Last but not least, I’d like to share with you some of the learnings, takeaways, and tips from our experiences along this journey … many of you may already be familiar with some of these, so the main point I want to make here is that even in a large enterprise like VMware, operating like a Lean startup can really help you get where you want to go faster: Validated research: This is a very common practice in startup product management, and the basic premise is to fail forward fast by validating your assumptions upfront to make sure you’re talking to the right people, that you’ve correctly identified their most pressing problems, and that your value proposition actually addresses their problems. Making design prototypes is a great way to do this because you can quickly put ideas in front of people before you commit to coding … Lean UX design: If you are familiar with user-centered design, you can think of Lean UX as UCD for lean startups ….. Basically how to do it faster … One of the concepts I really like but was ashamed to say out loud is “pain-driven design” which is about figuring out what is causing your users the most pain … this is very useful for brownfield projects where there are existing legacy tools as was the case with VMware, and believe it or not, this turned out to be my secret sauce at work … When I designed DxPress, I spent a lot of time with users but I also wanted to codify the results, so I made a survey but instead of sending it out and crossing my fingers, I actually went through the survey with each user, structured it as an interview and then charted the results in SurveyMonkey … there are some great books on Lean UX, UX Strategy, Running Lean et. Stay in Constant Contact with stakeholders and users … this is probably the MOST important principle we followed … most of the people I interviewed at the beginning became design partners and eventually advocates for the system because we had invested so much time to make sure the solution worked for them…. Never forget that you are building relationships and not just software!!! Automate all the things … because nobody has any bandwidth …even though we managed to automate 80% of the process, most of the product teams we talked to were even more stretched than our engineers, and a few of them scoffed at the notion of a 8-10 minute task “you can’t automate a task by giving it to someone else”)… even 3 clicks is 3 clicks too many. This led us to shift focus from pure contribution to curation … Avoid Dependency hell … this is an issue that comes up a lot with open source software and microservices … in our case, we have a dependency on an Angular library that another VMware team developed as a fully branded alternative to the default Swagger UI look and feel … and to be honest with you I’m starting to regret it … once you commit to using an upstream library, you are now a customer you have to file issues like everyone else … and if that library itself has dependencies, things start to get complicated … its really important to have a firm handshake upfront with other teams in your organization to avoid issues down the road. And last but not least … avoid over-engineering at all costs … if you look on the {code} website this can be particularly challenging if you work in an engineering-led environment, but prioritizing features and focusing on MVPs is essential to staying agile, moving quickly, and ensuring high adoption rates ….