Overview from #apistrat conference about responsible data consumption and distribution, and a proposal for a new form of DRM: robots.json

1. Big Data*
*small print
*small print
Tyler Singletary
Director of Platform, Klout
@harmophone
1

2. My Perspective
• Klout uses GNIP,Twitter, Facebook, Foursquare, LinkedIn
and about a dozen other APIs. Consume terabytes/day.
• Distribute own data to partners (Salesforce,
CrimsonHexagon) & channels (Gnip, Datasift)
• Over 1,700 monthly active developers
2

3. Finding a great API
is only the first step
is only the first step
• The data is what you want!
• You get to it easily, it’s well organized!
• This is awesome! I’m going to do X,Y and Z!
• Did you read the TOS?
3

4. Constraints breed
Creativity
• Knowing what you can’t do opens doors
• Legalese is a programming language
• Bend rules, but build a rollback
• Respect the data of others as if it were your
own
4

5. Did you know?
• Facebook forbids you from distributing FBIDs
• Also from giving their data, or a derivative, to ad
networks, etc.
• Twitter restricts you from separating location
data from a tweet
• Nearly all require attribution, have cache and
storage restrictions

6. A Rant About Scraping
• If a service has data you
want, but doesn’t offer it in
an API, do you have a right
to it?
• Does this serve a long term
business relationship?
• Whenever possible: have a
relationship
• Robots.txt

7. Ubiquity
• If APIs are going to be the defacto way businesses communicate, then:
• The rules are important.
• They need to apply to innovators as well as established business
• Terms must scale and be programmatically delivered and executed
• When you judge an API you discover, think beyond your own need:
• If I’m taking this data, and 50 other companies are, what benefit do I
have? They have? The provider has?
• You serve at the provider’s pleasure.
• How can APIs be industry-wide if industry is afraid of abuse?

8. Distributing Your Data
• What do you want to encourage?
• What do you want discourage?
• Contingencies!
• Be upfront and clear: cucumber stories.
• legalese: thar be dragons! Test coverage.
• What do you give exceptions for?
8

9. DRM
• No, not that DRM.
• Okay, you probably still think it’s that DRM.
• Data Rights Management

10. Robots.json
• A standard way to describe
data rights
• Permissible use
• Data storage, caching
• Rate limits
• Copyrights, data stewardship
• License term, financial details

11. Alternatives
• Hypermedia?
• Requiring everyone to read and understand
TOS and implement
• Trust? ignorance?
• Headers

12. Thank you!
{can: republish,
must: attribute,
retention:
unlimited}
Tyler Singletary
Director of Platform, Klout
@harmophone
12