Recommended
More Related Content
What's hot
What's hot (15)
Similar to What Does API Monitoring Mean for Product Managers?
Similar to What Does API Monitoring Mean for Product Managers? (20)
Recently uploaded
Recently uploaded (20)
What Does API Monitoring Mean for Product Managers?
- 1. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved moesif What Does API Monitoring Mean for Product Managers?
- 2. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Who am I? ● Co-founder and CEO of Moesif, an API analytics platform ● Focus on API strategy, platform growth, and analytics ● I love IPAs! derric@moesif.com
- 3. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Lots of infrastructure level metrics for engineering goals Uptime and SLA Resource Usage Average Latency Request Per Minute Errors Per Minute 1 2 3 4 5
- 4. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved However, product metrics should be aligned to one of three product/business goals: Adoption (Evangelism) Engagement Retention
- 5. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Similarly, many top of funnel metrics, but are these the best for product success Page views Sign ups/ Sign ins Requests Per Minute Average Latency Visits to a landing page is not engagement Can be faked by unqualified leads signing up Could be artificially high from simple health probes A consistently slow API can be fine
- 6. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Adoption: Map out your funnel to find barriers to adoption The developer funnel tracks a user’s journey from initial sign up to shipping a working app using your API Pre-Integration Test/Sandbox Production
- 7. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved What to measure? Unlike the mobile app acquisition funnel, an API user could stay in a single stage for days. You should track both: Conversion rate for each step Time to reach next step
- 8. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Track both conversion rate and time to next step Sign Up First API Call First Working App 30% 2 days 30% 6 hours
- 9. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Pre-integration stage: Get them to sign up Content, conferences, and 1:1 meetings Stellar and up to date docs Paid ads on developer focused networks
- 10. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Sandbox stage: Get them to their first hello world TTFHW (Time to First Hello World) is the time a new developer takes to: Sign up Create a simple test app Make first transaction and verify API fits their needs 1 2 3
- 11. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Lack of docs/simple onboarding slows TTFHW Ambiguous or lengthy onboarding No framework for their environment Lack of documentation
- 12. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Production stage: Get them to ship a working app TTFWA (Time to First Working App) or TTFPA (Time to First Paid App) is the time a new developer takes to ship a fully working solution to their users.
- 13. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Internal stakeholders slow TTFPA Legal and compliance risk Project priorities Functional & performance testing
- 14. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Let’s create example funnel goals for Algolia Pre-integration stage: Developers who signed-up and viewed onboarding docs or their API key Sandbox stage: Developers who completed at least: One POST /v1/indexes operation AND One POST /v1/indexes/{indexName}/query operation WHERE response.headers.Content-Length > 0 . Production stage: Developers who completed at least: 1,000 POST /v1/indexes/{indexName}/query operations WHERE response.headers.Content-Length > 0 .
- 15. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved API Engagement: Weekly active tokens can be a good north star metric The more accurate version is Weekly Active API Users. This is because each user can create multiple keys, such as for dev and prod.
- 16. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Once you define a north star metric, understand what’s driving it Which SDKs have higher or lower integration rates? Which marketing channels drive the most integrations? 2%30%20%2%
- 17. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved With proper attribution, you can measure the effectiveness of acquisition channels
- 18. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved It’s also important to look at API volume such as which partners are most active
- 19. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved For most APIs, inconsistent performance is worse for users than a high average latency
- 20. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved API Retention: What causes customers to become loyal?
- 21. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Programing languages or SDK API versions Features used Digging into your API Retention Find areas of improvement by grouping retention curves by: V 3.281.32 V 2.812.93 V 1.472.48
- 22. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved Finding SDK issues via retention curves
- 23. Confidential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved23 Derric Gilling