Short URLs, Big Data:Learning about the world in realtime<br />Hilary Mason<br />h@bit.ly	@hmason<br />
[nyc]<br />
[NY: City of Data]<br />http://smrt.io/rmcZOU<br />
Dirty Secret of Social Media:It’s all GOSSIP<br />
Timely<br />Crosses networks<br />Tells us about the world<br />
[dystopia]<br />
realtime?!<br />
realtime, baby!<br />
Realfast?<br />
Infrastructure<br />
metadata > data<br />
What languages are in a page?<br />
raw data<br />"es"<br />"en-us,en;q=0.5"<br />"pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4"<br />"en-gb,en;q=0.5"<br />"en-US,en;q...
http://4sq.com/96kc1O<br />
What are the languages spoken by the people who are interested in the page?<br />
How long do people pay attention to things?<br />
How does one idea relate to the next?<br />
Gossip ... with queries<br />
Timely<br />Crosses networks<br />Tells us about the world<br />
Simple<br />Fast<br />Reduces friction<br />
Thank you!<br />h@bit.ly<br />@hmason<br />
Upcoming SlideShare
Loading in …5
×

Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

22,209
-1

Published on

Published in: Technology, Sports
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
22,209
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
41
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide
  • Why did the data scientist cross the road?Two random variables were gossiping and thought they were discrete by whispering but I heard their chatter continuously.
  • Welcome to New York
  • Where do all of those identical black umbrellas COME FROM?Since you’ve been here you’ve probably seen some weird things.
  • http://www.flickr.com/photos/32443746@N07/4753829490/
  • http://bbs.keyhole.com/ubb/z0302a1700/aug07.htm
  • http://www.flickr.com/photos/74828596@N00/147208226/http://www.flickr.com/photos/spunkinator/3790112854/
  • Hadoop doesn’t work – it’s not realtime.
  • Hadoop is not realtime.
  • Creative commons: http://www.flickr.com/photos/13970152@N06/1471587677/We don’thave theluxury of time and iterations.
  • http://www.flickr.com/photos/95572727@N00/3892139852/
  • Dimensions dynamics of the network its shared on and the type of content.
  • http://www.flickr.com/photos/13668275@N07/3972173522/
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×