Punctuated Equilibrium, Celestial Navigation, and APIs
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Punctuated Equilibrium, Celestial Navigation, and APIs

on

  • 14,518 views

In evolutionary theory, punctuated equilibrium refers to a period of significant environmental stress resulting in rapid, dramatic changes among species. It is a powerful model for understanding the ...

In evolutionary theory, punctuated equilibrium refers to a period of significant environmental stress resulting in rapid, dramatic changes among species. It is a powerful model for understanding the changes in technology, business models and data over the last few decades - changes that have given rise to the age of APIs. In this talk, we'll look at three key themes in the API economy:
•The Evolution of Business Models: From 1st Party to Partner to Platform
•The Evolution of Architecture: From Mainframe to Mobile
•The Evolution of Data: From Silos to Social

Statistics

Views

Total Views
14,518
Views on SlideShare
7,167
Embed Views
7,351

Actions

Likes
13
Downloads
241
Comments
0

15 Embeds 7,351

http://blog.apigee.com 6511
http://www.web2expo.com 604
http://blog.sonoasystems.com 79
https://twitter.com 78
http://www.linkedin.com 28
https://blog.apigee.com 21
url_unknown 14
https://www.linkedin.com 4
http://webcache.googleusercontent.com 3
http://translate.googleusercontent.com 3
http://www.pearltrees.com 2
https://abs.twimg.com 1
http://www.twylah.com 1
http://dev.en.oreilly.com 1
http://kred.com 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Creative Commons Attribution-Share Alike 3.0 United States License
  • http://www.britannica.com/EBchecked/topic-art/197367/74641/Fourteen-species-of-Galapagos-finches-that-evolved-from-a-common
  • © Giuliano Gerra and Silvio Sommazzi - http://www.justbirds.it/
  • © Giuliano Gerra and Silvio Sommazzi - http://www.justbirds.it/
  • Camarhynchus Pauper© Nancy Bell
  • Camarhynchus Pauper© Nancy Bell
  • Geospiza Conirostris© Nancy Bell
  • Certhidea Olivacea© Giuliano Gerra and Silvio Sommazzi - http://www.justbirds.it/
  • http://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Punctuated-equilibrium.svg/1000px-Punctuated-equilibrium.svg.pnghttp://www.talkorigins.org/faqs/punc-eq.htmlThere are two models for the flow of evolution, each with its adherents. Phyletic gradualism says that things change slowly over time, day by day, until one day you realize there has been enough change that something new has arisen.The opposing model is Punctuated Equilibrium. This model says that species tend to remain in a homeostatic mode of stable equilibrium – but that every so often some significant new stress affects the environment, prompting rapid, dramatic changes that settle down into equilibrium again.For our purposes, punctuated equilibrium serves as a powerful model for viewing the changes in technology, business models, and data over the last few decades and shows us that the rapid, dramatic changes part of the cycle is coming again.
  • Adapted from http://www.csuchico.edu/~curbanowicz/DarwinSP2002Phil108.htm
  • http://www.csuchico.edu/~curbanowicz/DarwinSP2002Phil108.htm
  • Cherokee-class 10-gun brig-sloop of the Royal Navy downgraded to a 6-gun barquehttp://www.theora.com/msdb.php/OriginOfSpecies/orderBy/subject/msdbCat/subject/12%20-%20Geographical%20Distribution%20--%20continuedExploration - like evolutionary change - only looks smooth in hindsight. Not even the voyage of Darwin’s HMS Beagle was smoothly planned. Looking back he seems to have been predestined to arrive at the Galapagos islands, but this is not the case.
  • http://darwin-online.org.uk/people/chancellor.htmlAt some points the trip got pretty rough.
  • http://thebeagleproject.blogspot.com/2008_07_01_archive.htmlAnd the voyage itself was anything but linear.
  • http://digi.ub.uni-heidelberg.de/diglit/cpgraec398/0113
  • http://www.archive.org/details/cu31924031441847
  • http://en.wikipedia.org/wiki/File:Hannon_map-fr.svg
  • http://www.stedmundsbury.gov.uk/sebc/visit/precisionengineering.cfm
  • http://digi.ub.uni-heidelberg.de/diglit/cpgraec398/0113
  • http://www.britannica.com/EBchecked/topic-art/197367/74641/Fourteen-species-of-Galapagos-finches-that-evolved-from-a-common
  • http://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Punctuated-equilibrium.svg/1000px-Punctuated-equilibrium.svg.pnghttp://www.talkorigins.org/faqs/punc-eq.html
  • http://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Punctuated-equilibrium.svg/1000px-Punctuated-equilibrium.svg.pnghttp://www.talkorigins.org/faqs/punc-eq.html
  • Credit to Dick Hardt who formulated this in December 2010.
  • In 2010, Twitter announced at Chirp Conference that 75% of its total traffic comes from sources other than Twitter.com. This is part of a growing trend where new, emerging companies gain relevance on mobile and connected platforms other than the web.
  • Over time, we started adding our own devices to run off the API. They drove far more traffic than we saw from outside developers.
  • As a result, we are in the process now of redesigning the API with an emphasis towards supporting our devices, while trickling down API features to the 1000 flowers.
  • Another key metric going into our redesign is total requests per month. The growth of the API skyrocketed around October/November. Scale is one of our biggest issues moving forward and continuing on this trajectory will create more challenges.
  • REST is dominating APIs – expanding share of APIs even as the number of APIs is exploding (from just over 1000 in 2008 to over 3000 in 2010).
  • Netflix: Groups of entire movies will be stored together
  • Netflix: The same part of all movies will be stored together – such as frequently-used data about the movies such as actors, director, synopsis, first few minutes of video, etc.
  • Netflix: The same part of groups of movies will be stored together according to geography
  • Similar to the rapid changes we've seen in API-enabled business models and related architectures, we've witnessed four major shifts in how data is shared.Back in the mainframe days that Daniel spoke of, flat file data storage ruled and data sharing was fairly non-existent.  Allowing multiple applications to access the same file was impractical.Databases with concurrency controls through locks made sharing more possible, as you could now more easily let multiple apps manipulate the same data.  However, having applications work directly against data in the format with which it is stored introduced a very tight dependency that could be hard to manage.Middleware introduced another layer that allowed applications to share data via messages to applications, insulating the apps from the actual storage format of the data, effectively making those apps immune to changes in data storage format and schema.Web services improved data sharing even further.  The adoption of the internet-friendly HTTP protocol for exposing data now lets sharing occur beyond the boundaries of your business, while also lowering the technical expertise and software required for consumers of the data.  
  • Netflix: Groups of entire movies will be stored together
  • [Optimize your data delivery strategies]1. Consider allowing the download of larger datasets, providing an efficient way to get updates, and allowing liberal retention policies to improve availability, scalability and query-ability- Only allowing bite-sized data requests will cause developers to make many more queries which will affect your scalability- Retention restrictions, such as a one-day limit on data retention, will encourage developers to come back often to refresh their data- even if the data hasn't changed- Both result in higher scalability requirements for your API, which costs money- More frequent requests means an increase likelihood you'll break an app dependent on your data, assuming your service is anything less than 100% available- Push or publish incremental updates to consumers so they don't have to download the world regularly2. Consider limited access to richer queries to determine what new APIs to provide
  • Caveat structor – developer beware
  • Similar to the rapid changes we've seen in API-enabled business models and related architectures, we've witnessed four major shifts in how data is shared.Back in the mainframe days that Daniel spoke of, flat file data storage ruled and data sharing was fairly non-existent.  Allowing multiple applications to access the same file was impractical.Databases with concurrency controls through locks made sharing more possible, as you could now more easily let multiple apps manipulate the same data.  However, having applications work directly against data in the format with which it is stored introduced a very tight dependency that could be hard to manage.Middleware introduced another layer that allowed applications to share data via messages to applications, insulating the apps from the actual storage format of the data, effectively making those apps immune to changes in data storage format and schema.Web services improved data sharing even further.  The adoption of the internet-friendly HTTP protocol for exposing data now lets sharing occur beyond the boundaries of your business, while also lowering the technical expertise and software required for consumers of the data.  
  • Caveat structor – developer beware
  • http://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Punctuated-equilibrium.svg/1000px-Punctuated-equilibrium.svg.pnghttp://www.talkorigins.org/faqs/punc-eq.html
  • http://digi.ub.uni-heidelberg.de/diglit/cpgraec398/0113

Punctuated Equilibrium, Celestial Navigation, and APIs Presentation Transcript

  • 1. Punctuated Equilibrium,Celestial Navigation,and APIs
    Competing through dynamic adaptation
    Sam Ramji, Apigee @sramji
    Dan Jacobson, Netflix @daniel_jacobson
    Michael Hart, Netflix @michaelhart
  • 2. PUNCTUATED
    EQUILIBRIUM
  • 3. Darwin formulated his theory of evolution about 150 years ago
  • 4. Based on observations he made in the Galapagos Islands 15 years earlier
  • 5. A wild diversity of creatures existed in a new environment
  • 6.
  • 7. Starting from an ancestor which looked like this
  • 8. Geospiza Fulginosa
  • 9. Finches evolved that looked like this
  • 10. Geospiza Fortis
  • 11. and this
  • 12. Camarhynchus Pallidus
  • 13. and this
  • 14. Camarhynchus Pauper
  • 15. and this
  • 16. Geospiza Conirostris
  • 17. and this
  • 18. Certhidea Olivacea
  • 19. and this
  • 20. Geospiza Scandens
  • 21. For many years the belief was that this change happened slowly and gradually.
  • 22. In 1972, Stephen Jay Gould and Niles Eldredge proposed a new idea
  • 23. that evolution is not slow and gradual
  • 24. but sudden and violent.
  • 25.
  • 26. Applying this view to the observations of finches
  • 27. G. Scandens
    C. Pallidus
    G. Magnirostris
    G.
    Conirostris
    C. Heliobates
    G. Fortis
    G. Fulginosa
    C. Psittacula
    C. Pauper
    C. Olivacea
    C. Parvulus
    Certhidea
    Camarhynchus
    Geospiza
  • 28. “Thus, from the war of nature, from famine and death, the most exalted object of which we are capable of conceiving, namely the production of higher animals directly follows.”
    Charles Darwin
    On Origin of Species
  • 29. So while it may look slow and gradual in hindsight
  • 30. Evolution is experienced in punctuated bursts.
  • 31. If you’re living in a punctuated burst of evolution
    it feels like a revolution
  • 32. CELESTIAL
    NAVIGATION
  • 33. Exploration
  • 34.
  • 35. like evolutionary change
  • 36. only looks smooth in hindsight
  • 37.
  • 38. Living through it is usually chaotic
  • 39. Karen James
    The Beagle Project Blog
  • 40. To navigate, you need a map and instruments
  • 41.
  • 42. Maps exist for transferring knowledge
  • 43. and they too have evolved over time.
  • 44. They started as oral traditionand were written down in a form called a periplus
  • 45. Periplus of Hanno
    Courtesy of Heidelberg University
  • 46. Periplus of Hanno
    Courtesy of Cornell University
  • 47. Far less efficient knowledge transfer than
    a modern map of the same journey
  • 48. Map of Hanno’s Journey
    Courtesy of Bourrichon/Wikipedia
  • 49. Exploration was dramatically held back for want of a map
  • 50. In the two thousand years between Hanno’s journey on a Phoenician trireme
  • 51.
  • 52. And the Mediterranean caravel of the 15th century
  • 53.
  • 54. Maps had only evolved to be graphical descriptions of coastlines
  • 55.
  • 56. That was a map published a few years before
    Columbus crossed the Atlantic to find India
  • 57. After his crossing, his expedition shared their knowledge
    in a new map
  • 58.
  • 59. Still far from perfectbut much improved.
  • 60. The biggest challenge in this kind of exploration
    was determining their location on the Earth
  • 61. Instruments for measuring latitude had beenused and improved for centuries
  • 62.
  • 63. Longitude was the hard problem.
  • 64. You needed to know not just the angle of the sun and stars
  • 65.
  • 66. you also needed to know the precise time.
  • 67.
  • 68. Regardless of your sailing technology
    without the proper measurement
    you were lost
  • 69. We are not promising a perfect map of the new world
  • 70. But it should be more like this
  • 71.
  • 72. than this
  • 73. Periplus of Hanno
    Courtesy of Heidelberg University
  • 74. and we will show you what we know how to measure.
  • 75. APIS
  • 76. There are more niches today than we’ve seen before, so we need to borrow from nature
  • 77.
  • 78. If we start with an API we can explore all the niches around our business
  • 79. Visualization by Apigee
  • 80. The leaders of today’s Internet
  • 81.
  • 82. clearly understand this mechanism.
  • 83. They understand that the distribution model for value has changed in the Internet era.
  • 84. Packaged Goods
    Consumer
    Retail Store
    Producer
    Internet Services
    App
    Developer
    Consumer
    Provider
  • 85. Developers took their APIs and explored the niches for them
  • 86.
  • 87. The providers and the developers both benefited from this adaptation
  • 88.
  • 89.
  • 90. Suddenly this seems obvious to everyone.
  • 91. Data from Programmable Web
  • 92. And developers are racing to pack the niches.
  • 93. Data from Wikipedia
  • 94. This is a sudden, material shift in competition.
  • 95.
  • 96. It only looks gradual if you’re losing.
  • 97.
  • 98. BUSINESS MODELS
    From 1st Party to Partner to Platform
  • 99. We’ve seen punctuated equilibrium in business models over the last hundred years
  • 100. App Developer
    Device App
    Big Box Retailer
    Mobile App
    National Chain
    Web Retail
    Department Store
    Web Catalog
    Specialty Store
    Indirect Sales
    Direct Sales
  • 101. What’s the environmental stress driving the current rapid change?
  • 102. Hardt’s Theorem: The Internet Power Law
    The first wave of the Internet demonstrated the economic impact of web-based business models.
    99:1
    95:5
    The API-driven Internet is demonstrating the next concentration of power and is reflecting a “99:1” distribution (examples: Twitter, Facebook) due to the high switching costs and effective lock-in through software.
    The HTML-driven Internet showed new business models that focused revenue in the 3rd standard deviation (examples: Amazon, EBay). Reality reflected a “95:5” rule where 5% of companies dominated the transactions and profits.
    80:20
    Previous eras of business showed a normal distribution for revenue, with most firms getting most of the revenue. In the later half of the 20th century, business model innovations focused revenue in the 2nd standard deviation above the mean. The “80:20” rule became conventional wisdom.
    The next wave of the Internet is demonstrating the economic impact of API-based business models.
  • 103. But you need to tackle it in a way that fits your business
  • 104. 1st Party
    Open
    Open
    Open
    Platform
    Partner
  • 105. These are complementary and distinct.
    Open is different for each one.
  • 106. 1st Party Apps
    Partner Apps
    Platform Ecosystem
  • 107. 1st party
    1st party is about offering direct access to your core business via apps that you make or contract out
  • 108. 1st party
    Here open means all the business is accessible to internal developers and contract specialists
  • 109. partner
    Partner is about enabling directed development of apps that extend your business model towards your business partners
  • 110. partner
    Here open means existing partners have access to your business via APIs and can innovate asynchronously
  • 111. platform
    Platform is about enabling unknown developers to build brand new apps and businesses that will surprise and inform you
  • 112. platform
    Here open means enabling business models and allowing developers to support each other at massive scale
  • 113. open
    Open is attractive
  • 114.
  • 115. open
    Open is Biz Dev 2.0
  • 116. 1st Party
    Open
    Open
    Open
    Platform
    Partner
  • 117. open
    Open lets you navigate across the possible business models when your first model doesn’t work as planned
  • 118. 1st Party
    Open
    Open
    Open
    Platform
    Partner
  • 119. To get your API strategy properly grounded
  • 120. “
    John Musser
    Programmable Web
  • 121. But how?
  • 122. Let’s break it down
  • 123. Establish Target Segments
    Engage Developer Channel
    Set Industry Goal
  • 124. target segments
    An API should extend your core business
    into a new part of the market
  • 125. target segments
    Your core business already has
    key performance indicators
  • 126. target segments
    So apply your KPIs to the new market segment you’re targeting with your APIs
  • 127. target segments
    What is the market impact you need to create in order to succeed as a business?
  • 128. target segments
    What does the targetsegment need that it is not getting from you today?
  • 129. target segments
    The answer will be the foundation of your API strategy.
  • 130. developer channel
    In most cases the channel for your API will be developers, but what do they need?
  • 131. developer channel
    A profit motive.
  • 132. developer channel
    Here are the leading profit models for developers today
  • 133. developer channel
    In-app purchases
    Affiliate royalty
    Your advertising spend
    Market awareness of their offering
    App sales
  • 134. industry goal
    If you don’t know where you’re going, you definitely won’t get there
  • 135. industry goal
    Partnerships and platform businesses are very different things.
  • 136. industry goal
    Partnerships are formed to serve a known set of entities
  • 137. industry goal
    A partner API should be traceable to each partner’s relationship
  • 138. industry goal
    And support end-to-end business processes
  • 139. industry goal
    A platform exists to create massive and unpredictable opportunities
  • 140. industry goal
    All your technology, support, and community decisions will be about surviving the scale of adoption
  • 141. That’s the strategy dimension.
  • 142. The execution dimension is what you already know.
  • 143. Planning.
    Management.
    Organization.
  • 144. Putting all this in context gives us a map for our API strategy
  • 145.
  • 146. The instruments will be your KPIs and your core API metrics: performance and adoption
  • 147. ARCHITECTURE
    From Mainframe to Mobile
  • 148. Connected
    Devices
    Smartphone
    Web App
    N-tier
    Personal
    Computer
    DCOM
    CORBA
    Website
    Minicomputer
    Client/Server
    Mainframe
    Integrated
    Distributed
    Computing
  • 149. “
    The Web is Dead. Long Live the Internet.
    Chris Anderson
    Wired Magazine
  • 150. The Web is Dead. Long Live APIs!
  • 151. Twitter traffic distribution shows what he means
  • 152. Twitter Traffic in 2010
  • 153. Netflix traffic distribution is nearly the same.
  • 154. The majority of Netflix traffic comes from
    API-drivenconnected devices.
  • 155. Like Columbus, Netflix started with a map of the coastline
  • 156.
  • 157. Build an open API as a platform
    and let a thousand flowers bloom
  • 158.
  • 159. But they had left the coastline far behind
  • 160. And the instruments indicated that there were fewer flowers than expected
  • 161. Netflix API Requests by Segment
  • 162. But partners started building apps for connected devices and the business took off
  • 163. Wii
    iPad
    PS3
    iPhone
    Apple TV
    LG TVs
    Samsung TVs
    Roku
    XBox
  • 164. Architecture should
    reflect the business model
  • 165. So Netflix has drawn the following map
  • 166. Wii
    Roku
    PS3
    iPhone
    iPad
    Google TV
    Apple TV
    XBox
    LG TVs
    Samsung TVs
  • 167. Instruments show that API traffic has grown tremendously in a short time
  • 168. Growth of Netflix API
  • 169. 20,000,000,000 API requests per month.
    Is that a cause for celebration?
    Or for concern?
  • 170. When you’re navigating uncharted waters,
    speed is not your friend.
    Perhaps it’s time to slow down and avoid risking unknown reefs.
  • 171. Navigating this growth challenge for Netflix means that the next API revision will focus on reducing overall traffic.
  • 172. Part of this redesign is reviewing conventions
  • 173. Punctuated Equilibrium: REST
    Data sourced from
    ProgrammableWeb
  • 174. REST seems obvious but assess what makes sense for your business.
  • 175. Tiered architecture helps you navigate different problems with agility
  • 176. Controller
    Model
    View
    iPhone Wrapper
    Recommendations
    R12n Service
    Xbox Wrapper
    UNIFIED LIST/TITLE API
    Wii Wrapper
    User Info
    User Service
    PS3 Wrapper
    Similar Movies
    Similar Movie Service
    SHARED API INTERFACES
    Roku Wrapper
    Movie Metadata
    DataNormalization
    &
    Resiliency
    iPad Wrapper
    USER API
    Viewing History
    Apple TV Wrapper
    Ratings
    PC / Mac Wrapper
    TiVo Wrapper
    API Repository
    Layer
    Wrapper
    Layer
    Web Service
    Layer
    Shared
    Layer
    Source Data
    Layer
    App Layer
    API Layer
  • 177. Stable
    Flexible
    Agile
    iPhone Wrapper
    Recommendations
    R12n Service
    Xbox Wrapper
    UNIFIED LIST/TITLE API
    Wii Wrapper
    User Info
    User Service
    PS3 Wrapper
    Similar Movies
    Similar Movie Service
    SHARED API INTERFACES
    Roku Wrapper
    Movie Metadata
    DataNormalization
    &
    Resiliency
    iPad Wrapper
    USER API
    Viewing History
    Apple TV Wrapper
    Ratings
    PC / Mac Wrapper
    TiVo Wrapper
    API Repository
    Layer
    Wrapper
    Layer
    Web Service
    Layer
    Shared
    Layer
    Source Data
    Layer
    App Layer
    API Layer
  • 178. Server architecture should support both crests and troughs of the waves of demand
  • 179. Instance Architecture Based on Specialization
    List Creation
    Dependency Service
    ELASTIC INSTANCE LAYER
    API METADATA CACHING LAYER
    Meta
    Data
    Dependency Service
    METADATA SERVICE
  • 180. Instance Architecture Based on Specialization
    Handles Request/Response
    List Creation
    Dependency Service
    ELASTIC INSTANCE LAYER
    Caches Dependency Data
    API METADATA CACHING LAYER
    Meta
    Data
    Dependency Service
    Populates and Manages Cache
    METADATA SERVICE
  • 181. Map out your usage patterns and cache your data accordingly
  • 182. Vertical Caching
  • 183. Vertical Caching: Netflix Full Movie Data
  • 184. Horizontal Caching
  • 185. Horizontal Caching: Netflix Basic Data
  • 186. Combining horizontal and vertical caching may be the best approach when building for multiple geographies
  • 187. Two-Dimensional Caching
  • 188. Design for where you’re going
  • 189.
  • 190. Not for where you are
  • 191.
  • 192. You may be starting here
  • 193. Growth of Netflix API
  • 194. But you must design for here
  • 195. Growth of Netflix API
  • 196. You don’t need to implementfor massive scale
  • 197. But you must design for it or you will follow your successful ocean crossing with a massive shipwreck.
  • 198. Navigation also means constantly adjusting your course to ensure you arrive at your final goal
  • 199. Sometimes adjusting course on an API means you must change your version
  • 200. Versioning means supporting multiple applications, all of which basically do the same thing
  • 201. Versioning
    1.0
    1.5
    2.0
    3.0?
    4.0?
    5.0?
    Today
  • 202. If possible, go versionless
  • 203. Version-less API
    1.0
    1.5
    2.0
    Here there be dragons
    versionless
    Today
  • 204. Rules for going versionless
    in your production APIs
  • 205. Extend your API by extending data types
    Addition is not version-worthy
    Better to be incomplete than inaccurate
    Withhold implementation if you are unsure
  • 206. With APIs emerging, we need better tools help us navigate
  • 207.
  • 208. The service level agreement is your sextant
  • 209. Set reasonable service level agreements
  • 210. What is reasonable will vary by API and use case
    but you must communicate them to your users.
  • 211. Measure average latencies,
    error rates and types,
    and respond when SLAs are broken
  • 212.
  • 213. Having visibility into performance means you can tack immediatelyrather than after your users threaten to mutiny.
  • 214. Longer-term navigation requires higher-level metrics
  • 215. Request-based metrics such as what endpoints were called and what parameters were passed can show you what aspects of your API are popular.
    Response-based metrics such as what was delivered, how quickly, and whether the response was valid can show you what aspects of your API need work.
  • 216. System trace metrics that track what underlying systems were called and how they responded can show you where you need to evolve your internal architecture.
    Business performance metrics such as how much revenue or how much customer engagement is occurring through the API show you how close you are to delivering on the business plan.
  • 217. DATA
    From Silos to Social
  • 218. Domain-specific
    Data APIs
    Private Cloud DBs
    Caching DBs
    Data API
    Data
    Warehousing
    Mainframe
    RDBMS
    Flat file
    Shared
    Silos
    Data
  • 219. Punctuated equilibrium in data sharing
  • 220. App
    Org
    Cross-org
    Cross-business
    Mainframe
    Databases
    Middleware
    APIs
  • 221. So we are evolving to cross-business sharing
  • 222. There are a few challengesin making this journey
  • 223. Sharing your data via an API entails a different set of considerations than APIs which expose your services
  • 224. loss of control
    only recourse is legal
    enforcement is expensive
  • 225. That said, it may be time to get over
    your control issues
  • 226. Be honest about the value you could get from sharing your data outside your corporation
  • 227. instead of just the costs and risks of sharing it.
  • 228. In opening up its movie data warehouse
  • 229.
  • 230. Netflix found that the cost was the same as any API,
    the risks could be managed through rate limits and access control,
    but that now others could build great movie discovery experiences that led to increased Netflix viewing hours.
  • 231. Sharing data as a service means making a few course corrections on your API
  • 232. enable larger downloads for fewer queries
    more liberal retention policies means less API traffic, higher performance, and less cost
    push incremental updates
    limited access to richer queries
  • 233. Looking forward, how are we going to work through this wave of shared data?
  • 234. Two different dimensions are apparent
  • 235. First, no one wants all of your data,
    just some of it.
    Just the right parts of it for their particular need.
  • 236. Where have we seen this before?
  • 237. App
    Org
    Cross-org
    Cross-business
    Mainframe
    Databases
    Middleware
    APIs
  • 238. Last time the right solution was a
    query language
    Perhaps it’s the right solution now
  • 239. Instead of a static REST call, we could pose a query like “what are the highest rated movies from the 1980s?”
    http://odata.netflix.com/Catalog/Titles?$filter=ReleaseYear le 1989and ReleaseYearge1980and AverageRatinggt4
    &$expand=Awards
  • 240. Caveat structor does apply
    This is new ground and we haven’t seen anyone do this at massive scale
  • 241. Second, people want just the right parts of your data for their particular need
    But they need the right parts of other people’s data to have a complete context
  • 242. Where have we seen this before?
  • 243. App
    Org
    Cross-org
    Cross-business
    Mainframe
    Databases
    Middleware
    APIs
  • 244. Last time the right solution was middleware for distributed queries
    Perhaps it’s the right solution now
  • 245. But the rules have changed a bit since the data is laying all over the Internet
  • 246. [{
    "id": null,
    "name": null,
    "type": "/dining/restaurant",
    "/business/business_location/address": [{
    "street_address": [],
    "citytown": {
    "id": "/en/toronto"
    }
    }],
    "cuisine": [{
    "/dining/cuisine/region_of_origin": [{
    "!/film/film/featured_film_locations": [{
    "id": "/en/the_italian_job"
    }]
    }]
    Instead of a single domain query, we could ask for a list of “Toronto restaurants with cuisine from a filming location of ‘The Italian Job’”
  • 247. Caveat structor still applies.
    This is very new terrain indeed.
  • 248. The bigger question than what should you be sharing out
    Is what should you be sharing in
  • 249. What data APIs should your business be using?
  • 250. What could you offer your customers
    if you knew which of them were friends with each other?
  • 251.
  • 252. What does that logo mean to you today?
    Perhaps it’s time to think differently.
    Facebook is a massive data API
    that lets you correlate your customers
    with their true context
  • 253. You could move beyond a flat-earth view where all your customers are their own islands of data
    And each of them were connected in ways that makes your business more valuable to them
  • 254. What could you offer your customers
    if you knew where they like to go?
  • 255.
  • 256. The more of your customer’s context that you can understand
    The more time you can save them
    And that makes your business more valuable to them
  • 257. The good news is that there are already data APIs to get this context
    Now you need to focus on sharing in
  • 258. InCLOSING
  • 259. We are going through a period of rapid change in business models, architecture, and data
  • 260.
  • 261. Navigating based strictly on the stories of others
  • 262. Periplus of Hanno
    Courtesy of Heidelberg University
  • 263. Will not give you the clear map that you need
  • 264.
  • 265. Develop instruments around your API
    that help you understand where you’re going
  • 266.
  • 267. so that you can correct your course
    and beat your competition in the race for the future.
  • 268. THANK YOU
    Questions and ideas to:
    @michaelhart
    @daniel_jacobson
    @sramji