Noonan apistrat sf2013

114 views
96 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
114
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • MetadataMore specifically reference information or reference dataAPI payload scalability
  • Social media dataRealtimeMany publishersMany data points to look at, many variables in terms of size of payload, number of activities, etcAlways building in scalable sense
  • Twitter firehose5000 – 7000 activites/secondSee spikes regularly of 20k + for backfill from twitter after an issueLarge social media events like the superbowl and MJ’s DeathHistorical accessEntire twitter archiveStore 1 PETABYTE of historical data in S3Ingesting a new 2 TB / DayCLOUD and DEDICATED hardwareServe over 95% of the fortune 500 with social data – RELIABILITYOver 120 billion activities every month
  • Rest and Streaming APIsMostly streaming is consumed today, might be migrating towards more use of our REST APIs
  • Jack Dorsey’s first tweetGnip’s historical version vs Twitter 1.1 api version ~24% payload increaseJack dorsey’s first tweet vs my tweet today - ~50% payload increase
  • Increase in size Twitter -> GnipCompressed – 9% Uncompressed – 10%WordCount – 20%LineCount – 20%
  • Actor profile ~20% of payload
  • QUESTIONS?
  • Noonan apistrat sf2013

    1. 1. Drawing The Meta Line: Reference Information In APIs Andrew Noonan Software Engineer at Gnip @noonisms #apistrat @noonisms
    2. 2. #apistrat @noonisms
    3. 3. Social Data #apistrat @noonisms
    4. 4. #apistrat @noonisms
    5. 5. #apistrat @noonisms
    6. 6. #apistrat @noonisms
    7. 7. #apistrat @noonisms
    8. 8. #apistrat @noonisms
    9. 9. #apistrat @noonisms
    10. 10. #apistrat @noonisms
    11. 11. www.gnip.com @noonisms #apistrat @noonisms

    ×