Short URLs, Big Data:Learning about the world in realtime<br />Hilary Mason<br />h@bit.ly	@hmason<br />
[nyc]<br />
[NY: City of Data]<br />http://smrt.io/rmcZOU<br />
Dirty Secret of Social Media:It’s all GOSSIP<br />
Timely<br />Crosses networks<br />Tells us about the world<br />
[dystopia]<br />
realtime?!<br />
realtime, baby!<br />
Realfast?<br />
Infrastructure<br />
metadata > data<br />
What languages are in a page?<br />
raw data<br />"es"<br />"en-us,en;q=0.5"<br />"pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4"<br />"en-gb,en;q=0.5"<br />"en-US,en;q...
http://4sq.com/96kc1O<br />
What are the languages spoken by the people who are interested in the page?<br />
How long do people pay attention to things?<br />
How does one idea relate to the next?<br />
Gossip ... with queries<br />
Timely<br />Crosses networks<br />Tells us about the world<br />
Simple<br />Fast<br />Reduces friction<br />
Thank you!<br />h@bit.ly<br />@hmason<br />
Upcoming SlideShare
Loading in...5
×

Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

21,898

Published on

Published in: Technology, Sports
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
21,898
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
40
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide
  • Why did the data scientist cross the road?Two random variables were gossiping and thought they were discrete by whispering but I heard their chatter continuously.
  • Welcome to New York
  • Where do all of those identical black umbrellas COME FROM?Since you’ve been here you’ve probably seen some weird things.
  • http://www.flickr.com/photos/32443746@N07/4753829490/
  • http://bbs.keyhole.com/ubb/z0302a1700/aug07.htm
  • http://www.flickr.com/photos/74828596@N00/147208226/http://www.flickr.com/photos/spunkinator/3790112854/
  • Hadoop doesn’t work – it’s not realtime.
  • Hadoop is not realtime.
  • Creative commons: http://www.flickr.com/photos/13970152@N06/1471587677/We don’thave theluxury of time and iterations.
  • http://www.flickr.com/photos/95572727@N00/3892139852/
  • Dimensions dynamics of the network its shared on and the type of content.
  • http://www.flickr.com/photos/13668275@N07/3972173522/
  • Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

    1. 1. Short URLs, Big Data:Learning about the world in realtime<br />Hilary Mason<br />h@bit.ly @hmason<br />
    2. 2.
    3. 3. [nyc]<br />
    4. 4. [NY: City of Data]<br />http://smrt.io/rmcZOU<br />
    5. 5.
    6. 6.
    7. 7. Dirty Secret of Social Media:It’s all GOSSIP<br />
    8. 8. Timely<br />Crosses networks<br />Tells us about the world<br />
    9. 9.
    10. 10.
    11. 11. [dystopia]<br />
    12. 12.
    13. 13. realtime?!<br />
    14. 14.
    15. 15. realtime, baby!<br />
    16. 16.
    17. 17. Realfast?<br />
    18. 18.
    19. 19. Infrastructure<br />
    20. 20. metadata > data<br />
    21. 21. What languages are in a page?<br />
    22. 22. raw data<br />"es"<br />"en-us,en;q=0.5"<br />"pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4"<br />"en-gb,en;q=0.5"<br />"en-US,en;q=0.5"<br />"es-es,es;q=0.8,en-us;q=0.5,en;q=0.3”<br />"de, en-gb;q=0.9, en;q=0.8"<br />
    23. 23.
    24. 24. http://4sq.com/96kc1O<br />
    25. 25. What are the languages spoken by the people who are interested in the page?<br />
    26. 26. How long do people pay attention to things?<br />
    27. 27.
    28. 28.
    29. 29.
    30. 30.
    31. 31.
    32. 32. How does one idea relate to the next?<br />
    33. 33.
    34. 34. Gossip ... with queries<br />
    35. 35.
    36. 36.
    37. 37.
    38. 38. Timely<br />Crosses networks<br />Tells us about the world<br />
    39. 39. Simple<br />Fast<br />Reduces friction<br />
    40. 40.
    41. 41. Thank you!<br />h@bit.ly<br />@hmason<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×