Download as PDF, PPTX
Uploaded byJohn Musser
KPIs for APIs (and how API Calls are the new Web Hits, and you may be measuring all wrong)
The document discusses the evolution of API metrics and the importance of measuring them effectively to enhance API performance and business value. It emphasizes the need for defining key performance indicators (KPIs) based on the purpose of the API and its target audience, while warning against common pitfalls such as vanity metrics. Additionally, it outlines different lenses for measurement, the significance of developer experience, and the integration of qualitative metrics for comprehensive evaluation.
More Related Content
Similar to KPIs for APIs (and how API Calls are the new Web Hits, and you may be measuring all wrong)
KPIs for APIs (and how API Calls are the new Web Hits, and you may be measuring all wrong)
- 1. KPIs for APIs or, why API Calls are the new Web Hits and you may be measuring all wrong John Musser / @johnmusser / API Science / apiscience.com Business of APIs Conference, NYC, 2014
- 2. Why do weneed a Web site? 1995 Of course we have a Web site Of course we have an API Why do we need an API? 2000 2005 2010
- 3. Why do weneed a Web site? 1995 Of course we have a Web site You are here Of course we have an API Why do we need an API? 2000 2005 2010
- 4.
- 5. Evolution of website metrics Hits Page Views Visitors Conversions
- 6. Evolution of APImetrics? Calls ? ? ?
- 7. Great APIs getmeasured
- 8.
- 9.
- 10. Why do youhave an API?
- 11. Marketing channel BizDev/ LeadGen New line of business Extend product Drive innovation User acquisition Upsell opportunity Device and mobile support Content acquisition API as Product Distribution channel Increase stickiness Accelerate internal projects Partner opportunities Drive traffic Increase footprint
- 12. Who is yourAPI for?
- 13. API consumer continuum You Your partners & customers Everyone else Internal Private Open
- 14. The answers to who & why should drive your KPIs
- 15. The answers to who & why should drive your KPIs
- 16. Many ways tothink about API KPIs… Traffic Total calls Top methods Call chains Quota faults Developers Total developers Active developers Top developers Trending apps Retention Service Performance Availability Error rates Code defects Marketing Dev registrations Dev portal funnel Traffic sources Event metrics Support Support tickets Response times Business Direct revenue Indirect revenue Market share Costs
- 17. More ways tothink about API KPIs… Customer Satisfaction NPS Churn Community Forum activity Social media (both for API and apps built on it) Quality Defects Error prone APIs Repair rate API Maturity Stability Maturity metric Change rate Innovation Number of products Number of apps Speed to market Channel Call volume: Mobile, web, others Revenue by channel
- 18. There are alot of API KPIs you could measure
- 19. Don’t drown inAPI KPIs
- 20. KPI SECRET #1 Great APIs prioritize what to measure
- 21. Choose your “KPIlens” CFO lens CMO lens CEO lens Ops lens QA lens CRO lens PM lens PR lens
- 22. CFO lens: financialmetrics Total direct and indirect revenue Margin ROI Marginal cost Capex / Opex investment CLTV per developer
- 23. DevOps lens: operationalmetrics Performance Availability Error rates Faults by API type Defect backlog API traffic trends
- 24. CMO lens: marketingmetrics Developer acquisition cost Developer demographics Portal visitor count & traffic sources SEO / SEM metrics Social media metrics Evangelism and event metrics
- 25. KPI SECRET #2 Know your Developer Funnel
- 26. KPI SECRET #2 Know your Developer Funnel (for public APIs)
- 27. Web sites havefunnels Acquisi9on Ac9va9on Reten9on Revenue Referral How do users find you? Does the user have a good first experience? Do users return? How do you make money? Do users tell others? Dave McClure’s “Startup Metrics for Pirates”
- 28. APIs have funnelstoo Acquisi9on Ac9va9on Reten9on Revenue Referral How do developers find you? Does the developer have a good first experience? Do developers return? How do you and developers make money? Do developers tell others?
- 29. Potential KPIs Acquisi9on Ac9va9on Reten9on Revenue Referral Dev portal site: sources, traffic vol, uniques # dev registraCons, API keys issued # acCve devs, API call volume by dev, # apps $ per dev, $ per app, # end users # dev referrals, dev/app growth, NPS
- 30. KPI SECRET #3 Measure your API ecosystem
- 31. App Users AppsDevelopers Your APIs You API 1 API 2 API 3
- 32. App Users AppsDevelopers Your APIs You API 1 API 2 API 3 Total app users Revenue per user User growth rate User churn
- 33. App Users AppsDevelopers Your APIs You API 1 API 2 API 3 Total apps Trending apps Revenue per app Channel: mobile/web
- 34. App Users AppsDevelopers Your APIs You API 1 API 2 API 3 Total developers Active developers Revenue per developer Top developers Stuck developers
- 35. App Users AppsDevelopers Your APIs You Total API calls Top API calls API 1 Top API packages Internal usage API 2 API 3
- 36. App Users AppsDevelopers Your APIs You API 1 API 2 API 3 Total revenue Partnerships Market share Innovation
- 37. KPI SECRET #4 Most APIs obey the Power Law
- 38. The Power Lawof APIs Developers API call volume
- 39. The Power Lawof APIs Most of your API traffic, revenue, success, etc…. Developers API call volume
- 40. Think quality, notjust quantity Use your API KPIs, to find who, how, where, when. Developers API call volume
- 41. KPI SECRET #5 API KPIs need buy-in
- 42. Don’t let APIKPIs live in a vacuum Who are your stakeholders? Who’s your sponsor? Get executive buy-in on your top 3-5 KPIs
- 43. KPI SECRET #6 Measuring internal API usage often just as important as external
- 44. Guardian: 70% internalcalls API usage from internal apps External usage
- 45. Evernote: 99% internalcalls API usage from internal apps External usage
- 46. Netflix: 99.9% internalcalls API usage from devices External usage
- 47. Netflix: 99.9% internalcalls Why Enable Netflix experience Who Netflix device and UI teams
- 48. KPI WARNING #1 Beware of API vanity metrics
- 49. API Billionaire Club 13 billion API calls / day (May 2011) 5 billion API calls / day (April 2010) 5 billion API calls / day (October 2009) 1.1 billion API calls / day (April 2011) 1 billion API calls / day (May 2012) 1 billion API calls / day (Q1 2012) 1 billion API calls / day (January 2012)
- 50. 5 Billion APIcalls/day
- 51. “What if, allthings being equal, instead of 30 billion API calls a month, we served 5 billion?” Daniel Jacobson, Netflix
- 52. KPI WARNING #2 Beware of API KPI traps
- 53. 3 common APIKPI traps Not having a metrics process (have a metrics plan & a feedback loop) Using the wrong metrics (don’t be afraid to change what you track) Unrealistic expectations (“Absolutely, we’ll have X developers on-board by X date”)
- 54. One important metric you may have missed...
- 55. What’s your TTFHW? Time To First “Hello World” aka: how long from zero to 60?
- 56. What’s your TTFHW? Developer experience (DX) can make or break your API.
- 57. What’s your TTFHW? Too subjective to measure?
- 58. What’s your TTFHW? Don’t forget about qualitative metrics
- 59. What’s your TTFHW? User testing. Surveys. Forums. Social media…
- 60. Real world APIKPIs
- 61. KPI: Conversions “Weare seeing a trend where a person who uses another app with Evernote is 50% more likely to move on to our premium (paid) service”
- 62. KPI: Engagement “Appsthat have an API integration with Evernote are seeing much more usage: Pocket readers normally read 15% of what they save, but Evernote users read 80% of what they save in Pocket.”
- 63. KPI: Revenue APInow accounts for 20-25% of the charity’s annual revenue
- 65.
- 66. KPI: Apps 3,500Apps Aviary, March, 2013
- 67. Goal: “more eyeballson our content” API calls (requests) Response volume (ex: # stories) Impressions (via image beacon) Loyalty More KPIs KPI: Impressions
- 68. KPI: Recipe delivery “We sell Vodka. But people buy drinks.” Goal: “ensure top quality drinks consistently delivered to people regardless of where people encounter Absolut”
- 69. KPI: Recipe delivery APIs let you be where your customers are. Measure how. “We sell Vodka. But people buy drinks.” Goal: “ensure top quality drinks consistently delivered to people regardless of where people encounter where people encounter Absolut”
- 70. Evolution of APImetrics? Calls ? ? ?
- 71. Who & whydrives KPIs Have a KPI lens & process Prioritize & repeat
- 72. Thank You john@apiscience.com @johnmusser
- 73. Photo Credits Race car: hCp://www.flickr.com/photos/lim_lik_wei/3270522646/ Stopwatch: hCp://www.flickr.com/photos/purplemaMish/3020016417/ Stop sign: hCps://www.flickr.com/photos/9toperez/5791194358 Power law: hCp://en.wikipedia.org/wiki/Power_law