Search

Smoke

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    8 Favorites

    Smoke - Presentation Transcript

    1. smoke Friday, 1 May 2009 Hi!
    2. smoke Ben Schwarz @benschwarz http://github.com/benschwarz Friday, 1 May 2009 Hi!
    3. Lunch time Friday, 1 May 2009 Starting with a story... It was lunch time.
    4. 2pm Friday, 1 May 2009 It was well and truly past eating time
    5. 1 Hour Friday, 1 May 2009 I had one hour. What do you do?
    6. Code lunch Friday, 1 May 2009 No need for eating. What will I do?
    7. Sinatra http://sinatrarb.com Friday, 1 May 2009 We use Sinatra!
    8. Yahoo Pipes http://pipes.yahoo.com Friday, 1 May 2009 Annd data from other peoples websites. No RDBMS for me!
    9. Yahoo Pipes • Get data from multiple web based sources • Mash it up a bit • Provide a single output in either JSON or XML Friday, 1 May 2009
    10. YQL Friday, 1 May 2009
    11. YQL? Friday, 1 May 2009 Yeah, its kinda SQL for the web. Most people don’t even really like writing SQL
    12. @lachlanhardy Friday, 1 May 2009 Lachlan convinced me to have a second look at YQL after slagging it o
    13. Time to start Friday, 1 May 2009
    14. Friday, 1 May 2009 The code isn’t important, but basically it just talks to a yahoo pipe that I made up.
    15. Friday, 1 May 2009 I made “munch”. It pulls from various recipe and cooking related web sites like taste.com.au and friends. It reads RSS and scrapes sites. Pretty simple.
    16. 2:46pm Friday, 1 May 2009 46 six minutes later. I’d been eected.
    17. “I need a new web site” - Me, probably avoiding real work Friday, 1 May 2009 I wanted to blog about this, sadly my blog is cached files sitting on a slicehost account. Mephisto isn’t even running any longer.
    18. Friday, 1 May 2009 I quickly threw a grid together.
    19. HTML5 Friday, 1 May 2009 I wanted to experiment.
    20. Sinatra Friday, 1 May 2009 a bit more sinatra.
    21. Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    22. Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    23. Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    24. Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    25. Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    26. Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    27. ? Friday, 1 May 2009 I knew that I didn’t want to write everything then link it in to my blog, I wanted to write it once and be able to keep it updated.
    28. Friday, 1 May 2009 of course, I could’ve just got a tumblr account. That didn’t let me use sinatra or mess around with HTML5.
    29. Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    30. Delicious Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    31. All Delicious Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    32. All Delicious Twitter Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    33. All Delicious When I wasn’t talking to @someone Twitter Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    34. All Delicious When I wasn’t talking to @someone Twitter Flickr Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    35. All Delicious When I wasn’t talking to @someone Twitter Selected (special tag) Flickr Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    36. All Delicious When I wasn’t talking to @someone Twitter Selected (special tag) Flickr ? Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    37. All Delicious When I wasn’t talking to @someone Twitter Selected (special tag) Flickr ? ? Friday, 1 May 2009 It was getting easier, I had constraints around the use of my data
    38. Friday, 1 May 2009 The yahoo pipe looked like this. A bit of a mental leap.
    39. Friday, 1 May 2009 Some things stood out. like union, I could join multiple sources of data together and output them as one
    40. Friday, 1 May 2009 or sort them by the publication date
    41. Friday, 1 May 2009 and filter out the tweets to people. Sure a regex would’ve been smarter, but this was 90% good enough
    42. Friday, 1 May 2009 I started thinking about my process, the data coming from all over the place.
    43. “What if I could write something like pipes?” Friday, 1 May 2009 I wanted to do something myself (again)
    44. “What if I could write a ruby block based DSL?” Friday, 1 May 2009 I’d never written one, totally a good time to jump right in head first.
    45. “What if you got plain old Ruby objects?” Friday, 1 May 2009 If the data returned could be ruby, irrespective of what format it started as... it would be awesome
    46. Or JSON Friday, 1 May 2009
    47. Friday, 1 May 2009 Enter, “Smoke.”
    48. Smoke Friday, 1 May 2009 Enter, “Smoke.”
    49. Friday, 1 May 2009 A search, which I named “ruby” uses yql and does a web based (search engine) search for the keyword of “ruby”
    50. Friday, 1 May 2009 A search, which I named “ruby” uses yql and does a web based (search engine) search for the keyword of “ruby”
    51. Friday, 1 May 2009 A search, which I named “ruby” uses yql and does a web based (search engine) search for the keyword of “ruby”
    52. Friday, 1 May 2009 A search, which I named “ruby” uses yql and does a web based (search engine) search for the keyword of “ruby”
    53. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    54. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    55. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    56. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    57. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    58. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    59. Friday, 1 May 2009 and hey, I’m gonna grab the RSS feed from slashdot. “emit” will transform the objects that my feed returns. I’ll rename any keys of link to url, this will make my data mind-map better
    60. Friday, 1 May 2009 The whole point of this was to bring these multiple feeds together, re interpret them as one common feed.
    61. Friday, 1 May 2009 This creates a new “source”, it’ll be named “ruby_useless_joined”. It can be refiltered if required
    62. Creates Friday, 1 May 2009 This creates a new “source”, it’ll be named “ruby_useless_joined”. It can be refiltered if required
    63. Creates Friday, 1 May 2009 This creates a new “source”, it’ll be named “ruby_useless_joined”. It can be refiltered if required
    64. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    65. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    66. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    67. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    68. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    69. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    70. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    71. Friday, 1 May 2009 That can be extended and the items from the two sources can have some transformations as well.
    72. Friday, 1 May 2009 That accessor
    73. Friday, 1 May 2009 Get the first item output (output is an array of the items from the source)
    74. Friday, 1 May 2009 It gives us this
    75. What just happened? Friday, 1 May 2009 Thinking a little more about what actually just happened. It parsed the data to a ruby object, irrespective of the format it came in as.
    76. What just happened? • Queried the web Friday, 1 May 2009 Thinking a little more about what actually just happened. It parsed the data to a ruby object, irrespective of the format it came in as.
    77. What just happened? • Queried the web • Auto-magically parsed the data Friday, 1 May 2009 Thinking a little more about what actually just happened. It parsed the data to a ruby object, irrespective of the format it came in as.
    78. What just happened? • Queried the web • Auto-magically parsed the data • Returned a clean object Friday, 1 May 2009 Thinking a little more about what actually just happened. It parsed the data to a ruby object, irrespective of the format it came in as.
    79. Friday, 1 May 2009 We can also output JSON
    80. Friday, 1 May 2009 Its string escaped and not worth reading, but you could pass this through a web based app straight down to javascript
    81. Method chaining Friday, 1 May 2009 Methods can also be chained without invoking a new source or joining sources together
    82. Friday, 1 May 2009 Here we can get a predefined source, shorten the result set and re sort it.
    83. Friday, 1 May 2009 Here we can get a predefined source, shorten the result set and re sort it.
    84. Friday, 1 May 2009 Here we can get a predefined source, shorten the result set and re sort it.
    85. Transformations Amount of items Truncate By a single key Sort One key to another Rename Items that match regex Keep Items that match regex Discard Ruby (default) or JSON Output Friday, 1 May 2009 Right now, you can do these things
    86. YQL Definitions Friday, 1 May 2009 I’m looking at adding support for YQL definitions
    87. Friday, 1 May 2009 They look like this, I’ve never written one, but it doesn’t look too hard. You can add search to your own preexisting site / rss etc
    88. http://github.com/spullara/yql-tables Friday, 1 May 2009 There are a collection of community provided yql definitions hosted on github
    89. All these fucking web sites Friday, 1 May 2009 Need I say more?
    90. My message Friday, 1 May 2009 Forgetting all the stu about pipes, yql and smoke, I want you to try something
    91. “Re-interpret the web” Friday, 1 May 2009 Reconsider data, re-represent it to your own needs.
    92. Thanks Friday, 1 May 2009
    93. Thanks github.com/benschwarz/smoke http://github.com/benschwarz/munch http://sinatrarb.com http://pipes.yahoo.com/pipes/pipe.edit?_id=fHK7LuUW3hGZHgSfbbsjiw http://twitter.com/lachlanhardy http://pipes.yahoo.com/pipes http://developer.yahoo.com/yql http://github.com/spullara/yql-tables http://www.flickr.com/photos/95579828@N00/71240905 http://www.flickr.com/photos/hockadilly/2910737768 http://www.flickr.com/photos/maxblack/2655837607 Friday, 1 May 2009

    + Ben SchwarzBen Schwarz, 7 months ago

    custom

    1219 views, 8 favs, 0 embeds more stats

    My presentation from melbourne roro showing my new more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1219
      • 1219 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 8
    • Downloads 36
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    more
    -->
    Search
    RSS Feed
    What's new?

    © 2009 SlideShare Inc. All Rights Reserved