Scaling the APIEconomyWith Scale-Free NetworksMike AmundsenPrincipal API ArchitectLayer 7 Technologies@mamund
I come as a messenger…
The Goddess Iris• Iris is associated with– communication– new endeavors
I bring hard news…
Iridium (lr)• Very hard & brittle• Second-densest element• Most corrosion-resistant• Often found withinimpact craters• Use...
My news is not good…
My news is not good…• The most commonapproach toInternet-based APIsruns counter to afundamentalproperty the Web
My news is not good…• At our current pace ofgrowth, we may notbe able to keep thisup for much longer.
My news is not good…• We may be headed for:“Scale-pocalypse!”
But wait,I’m getting ahead of myself…
Scale-pocalypse 1998
1973 – Lansing, Michigan• I start my first year at Michigan StateUniversity• The same year, same city,Larry Page is born.
1994 – Stanford University• Jerry Yang & David Filo• "Jerrys guide to theworld wide web”• YAHOO!•Curated, hierarchicalindex
1998 – Stanford University• Sergey Brin & Larry Page• The Anatomy of a Large-Scale Hypertextual WebSearch Engine (1998)• “...
What was different?• What did Brin &Page perceive thatothers had missed?• How did Googleavoid the“Scale-pocalypse”?
“Free” as in “Scale-Free”
Scale-Free Network• “A network whosedegree distributionfollows a power law.”
Scale-Free Network• “A network whosedegree distributionfollows a power law.”
Power Law
“Long Tail”
“Long Tail”• Not an equal distribution• A few nodes have many of links• Many nodes have a few links• “Preferential Attachm...
Power Law Web Search• Brin & Pagerecognized that• curation andaggregation• would not scale onthe WWW
Power Law Web Search• Instead, theydecided to• allow users• and the links theycreate• to inform the searchengine.
Power Law Success• Using this feature ofthe WWW allowedGoogle to• gain the leading rolein search• and build a multi-billio...
Have we forgotten so soon?
Hub-Vulnerability• Unexpected disasters cancause complex systems tofail.• DNS failures of the 2000s• Recent Amazon AWSfail...
Node-Resiliency• Highly distributedsystems reduce riskof system-widefailures.
Does power-law only apply to search?
Storage• Google Drive• Dropbox
Collaboration• SVN• git
Identity• Captive User Stores• Multiple OAuth Stores
APIs?• Do we really want toadd more vulnerabilityto the WWW?• Can we afford torely on curation andaggregation for APIs?
“We can rebuild [it].We have the technology.”
What if we changed our model?• Could we create node-based services?• What would we gain?• What would we lose?
Business models based on nodes• Powerful clientaggregators• User-centric– Discovery– Linking– Sharing• Allow users to iden...
Advantages of node-based models• Increased processingpower (SETI)• Access to more metadata(EXIF)• User-driven selectionmod...
Yes, but can you make money this way?
Models/Paradigms• IBM gladly let Microsoftown the OS (IBM knewhardware was the realprofit source)• Yahoo knew the value of...
Be a Node and not a Hub!
Node-based?• Twitter• Facebook• Google Docs• Wordpress
Node-based?• I post messagesto my own machine• The client sends itto one or more selectedservices of my choice• Publishers...
Avoiding the Scale-pocalypse• API providers:• Consider hub vulnerability problem• Explore node-based advantages• Empower u...
Avoiding the Scale-pocalypse• API consumers:• Is there a node-based provider available?• Build client-based aggregators• E...
And so…
So, my message is a positive one…• We have an opportunityfor new endeavors
So, my message is a positive one…• We can build strong,vulnerability-resistantsystems
The news is not so bad…
The news is not so bad…• There is a way tobuild systems thattake advantage of akey principal of theWWW
So, the news is no so bad…• Using power-law andnode-based systems,we can continue tomeet growingdemand.
The news is not so bad…• And we can avoid“Scale-pocalypse!”
Be a Node!
Scaling the APIEconomyWith Scale-Free NetworksMike AmundsenPrincipal API ArchitectLayer 7 Technologies@mamund
References• http://en.wikipedia.org/wiki/File:Winged_goddess_Cdm_Paris_392.jpg• http://en.wikipedia.org/wiki/File:Iridium-...
References• http://www.cesa.eu/images/timeline/pictures/09_2008.jpg• http://www.codeproject.com/KB/web-cache/AdvanceCachin...
Upcoming SlideShare
Loading in …5
×

Scaling the API Economy - with Scale-Free Networks API Days Keynote from Layer 7 Principal API Architect Mike Amundsen

2,218 views

Published on

The Web exhibits a feature found in many complex systems known as "Scale-Free" or "Power-Law" networks, sometimes called the "long tail" Most people think of the "long tail" as an economic and/or social property. However, it also represents physical and informational properties fundamental to the way the Web works. But the steady increase in major service outages indicate that many current Web APIs, services, and even client applications ignore this basic "law of the Web."
This talk explores the "Scale-Free" rule of complex systems and offers clear and simple advice to those planning to build and/or consume APIs for the Web. Such as what to avoid, what to plan for, what to build, and how to identify & steer clear of clients and services that fail to abide by the rules and, in the process, are making it harder for all of us to scale the API Economy.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,218
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
26
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Scaling the API Economy - with Scale-Free Networks API Days Keynote from Layer 7 Principal API Architect Mike Amundsen

  1. 1. Scaling the APIEconomyWith Scale-Free NetworksMike AmundsenPrincipal API ArchitectLayer 7 Technologies@mamund
  2. 2. I come as a messenger…
  3. 3. The Goddess Iris• Iris is associated with– communication– new endeavors
  4. 4. I bring hard news…
  5. 5. Iridium (lr)• Very hard & brittle• Second-densest element• Most corrosion-resistant• Often found withinimpact craters• Used in deep spacesatellites
  6. 6. My news is not good…
  7. 7. My news is not good…• The most commonapproach toInternet-based APIsruns counter to afundamentalproperty the Web
  8. 8. My news is not good…• At our current pace ofgrowth, we may notbe able to keep thisup for much longer.
  9. 9. My news is not good…• We may be headed for:“Scale-pocalypse!”
  10. 10. But wait,I’m getting ahead of myself…
  11. 11. Scale-pocalypse 1998
  12. 12. 1973 – Lansing, Michigan• I start my first year at Michigan StateUniversity• The same year, same city,Larry Page is born.
  13. 13. 1994 – Stanford University• Jerry Yang & David Filo• "Jerrys guide to theworld wide web”• YAHOO!•Curated, hierarchicalindex
  14. 14. 1998 – Stanford University• Sergey Brin & Larry Page• The Anatomy of a Large-Scale Hypertextual WebSearch Engine (1998)• “…due to rapid advance in technologyand web proliferation, creating a websearch engine today is very differentfrom three years ago.”
  15. 15. What was different?• What did Brin &Page perceive thatothers had missed?• How did Googleavoid the“Scale-pocalypse”?
  16. 16. “Free” as in “Scale-Free”
  17. 17. Scale-Free Network• “A network whosedegree distributionfollows a power law.”
  18. 18. Scale-Free Network• “A network whosedegree distributionfollows a power law.”
  19. 19. Power Law
  20. 20. “Long Tail”
  21. 21. “Long Tail”• Not an equal distribution• A few nodes have many of links• Many nodes have a few links• “Preferential Attachment”– Barabási–Albert (1999)
  22. 22. Power Law Web Search• Brin & Pagerecognized that• curation andaggregation• would not scale onthe WWW
  23. 23. Power Law Web Search• Instead, theydecided to• allow users• and the links theycreate• to inform the searchengine.
  24. 24. Power Law Success• Using this feature ofthe WWW allowedGoogle to• gain the leading rolein search• and build a multi-billion dollarcompany.
  25. 25. Have we forgotten so soon?
  26. 26. Hub-Vulnerability• Unexpected disasters cancause complex systems tofail.• DNS failures of the 2000s• Recent Amazon AWSfailures• 2008 Monetary systemfailures• “Too big to fail”
  27. 27. Node-Resiliency• Highly distributedsystems reduce riskof system-widefailures.
  28. 28. Does power-law only apply to search?
  29. 29. Storage• Google Drive• Dropbox
  30. 30. Collaboration• SVN• git
  31. 31. Identity• Captive User Stores• Multiple OAuth Stores
  32. 32. APIs?• Do we really want toadd more vulnerabilityto the WWW?• Can we afford torely on curation andaggregation for APIs?
  33. 33. “We can rebuild [it].We have the technology.”
  34. 34. What if we changed our model?• Could we create node-based services?• What would we gain?• What would we lose?
  35. 35. Business models based on nodes• Powerful clientaggregators• User-centric– Discovery– Linking– Sharing• Allow users to identifynew, popular services
  36. 36. Advantages of node-based models• Increased processingpower (SETI)• Access to more metadata(EXIF)• User-driven selectionmodels (reverse-flow)• Reverse scale/vulnerabilitychallenge• Increase “network”intelligence• Mix of “smart” and “dumb”clients (IoT)
  37. 37. Yes, but can you make money this way?
  38. 38. Models/Paradigms• IBM gladly let Microsoftown the OS (IBM knewhardware was the realprofit source)• Yahoo knew the value ofcurated indexes (Googlehad another idea)• Few thought there was abusiness in Open Sourcemovement.
  39. 39. Be a Node and not a Hub!
  40. 40. Node-based?• Twitter• Facebook• Google Docs• Wordpress
  41. 41. Node-based?• I post messagesto my own machine• The client sends itto one or more selectedservices of my choice• Publishers competefor my content• Publishers distributeto other nodes
  42. 42. Avoiding the Scale-pocalypse• API providers:• Consider hub vulnerability problem• Explore node-based advantages• Empower users andreap rewards
  43. 43. Avoiding the Scale-pocalypse• API consumers:• Is there a node-based provider available?• Build client-based aggregators• Empower users andreap rewards
  44. 44. And so…
  45. 45. So, my message is a positive one…• We have an opportunityfor new endeavors
  46. 46. So, my message is a positive one…• We can build strong,vulnerability-resistantsystems
  47. 47. The news is not so bad…
  48. 48. The news is not so bad…• There is a way tobuild systems thattake advantage of akey principal of theWWW
  49. 49. So, the news is no so bad…• Using power-law andnode-based systems,we can continue tomeet growingdemand.
  50. 50. The news is not so bad…• And we can avoid“Scale-pocalypse!”
  51. 51. Be a Node!
  52. 52. Scaling the APIEconomyWith Scale-Free NetworksMike AmundsenPrincipal API ArchitectLayer 7 Technologies@mamund
  53. 53. References• http://en.wikipedia.org/wiki/File:Winged_goddess_Cdm_Paris_392.jpg• http://en.wikipedia.org/wiki/File:Iridium-2.jpg• http://dynamicdecorator.files.wordpress.com/2013/01/fesvh.png• http://www.flickr.com/photos/luvataciousskull/3374223404/in/photolist-696AXi-69aM4A-69aM51-69aM7h-69aM8o-69aM95-69aM9S-69aMaj-69aMbC-69aMdW-69aMeC-69aMgf-69aMgE-69aMjj-6c5PSD-6kLGyn-6uuq6s-6CazEo-6CDvfQ-6PtMEG-6ZBpBx-7f9XeB-7keQgx-7kzMec-7kzMmB-7kDDmN-7kDDxb-7kDDDh-7rtS3a-ePziXc-9eQeak-9eTm8W-9eQdCr-9eQcYk-cwkcSy-eS88UX-8pb5fQ-9D4KUU-884Y12-eRZG7k-ejqjh3-cnWzM3-93qEWH-cu9GaC-84WtXY-88WN2g-bqxc3k-9LjgRw-8GtxN8-8Aw55k-9G6Hdb/• http://3.bp.blogspot.com/_VyTCyizqrHs/S-Lp0UmtOVI/AAAAAAAAHik/HSQ1KfRLRa0/s1600/ericsson1.jpg• http://www.flickr.com/photos/34206904@N02/6841249195/in/photolist-bqxc3k-9LjgRw-8GtxN8-8Aw55k-9G6Hdb• http://kinlane-productions.s3.amazonaws.com/api-evangelist-site/blog/apis-resource-to-last-mile.png• http://blog.adw.org/wp-content/uploads/question-mark.jpg• http://upload.wikimedia.org/wikipedia/commons/a/ac/Jerry_Yang_and_David_Filo.jpg• http://en.wikipedia.org/wiki/File:Larry_Page_in_the_European_Parliament,_17.06.2009.jpg• http://upload.wikimedia.org/wikipedia/commons/d/d2/Internet_map_1024.jpg• http://upload.wikimedia.org/wikipedia/commons/8/8a/Long_tail.svg• http://i.msdn.microsoft.com/dynimg/IC76141.gif• http://dangerouslyirrelevant.org/images/various-small/Linked_thumb1.jpg• http://tctechcrunch2011.files.wordpress.com/2009/08/google.jpg• http://farm5.staticflickr.com/4150/5018184807_60be8b669e_z.jpg• http://3.bp.blogspot.com/-C9jVV3v-0DY/T-ho44hgg7I/AAAAAAAAAgQ/an4mwa43XXM/s640/AWS+global+infrastructure.jpg
  54. 54. References• http://www.cesa.eu/images/timeline/pictures/09_2008.jpg• http://www.codeproject.com/KB/web-cache/AdvanceCaching/partioned.jpg• http://blog.mybackupbox.com/wp-content/uplo• https://si0.twimg.com/profile_images/480262550/box.png ads/2012/06/google_drive.jpg• http://svn.apache.org/repos/asf/subversion/trunk/notes/logo/256-colour/subversion_logo-384x332.pnghttps://secure.gravatar.com/avatar/878a0ea898da1701df8573ed64a5cc9d?s=420&d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-user-420.png• http://3.bp.blogspot.com/-ZeYZpE3tESU/UOz8-49WYvI/AAAAAAAAAP8/vt0-vDzXxwk/s72-c/database-developer-Eastbourne-Brighton-Sussex.png• http://hueniverse.com/wp-content/uploads/2009/09/OAuth-Shine-300x298.png• http://mindcommerce.files.wordpress.com/2013/05/telephonyapi_management.gif?w=286&h=300• http://blogs.bmj.com/bmj-journals-development-blog/files/2010/12/semantic-web.jpg• http://blog.caplinked.com/wp-content/uploads/2011/09/iStock_000009230566Large.jpg• http://studsovet.ipsa.kpi.ua/uploads/posts/2013-05/1368982043_ibm-intel-logo.jpg• http://www.shinyshiny.tv/microsoft-logo__111129012732.jpg• http://rack.2.mshcdn.com/media/ZgkyMDEyLzEyLzA0L2I1L3lhaG9vdm9pY2VzLmJrSC5qcGcKcAl0aHVtYgk5NTB4NTM0IwplCWpwZw/61883c7e/479/yahoo-voices-hacked-450-000-passwords-posted-online-7169a7e88d.jpg• http://nyocommercialobserver.files.wordpress.com/2012/06/google-logo.gif?w=400&h=283• http://upload.wikimedia.org/wikipedia/commons/thumb/4/42/Opensource.svg/220px-Opensource.svg.png• http://www.salesnet.com/images/xml-web-services-api.jpg

×