Superglue: Web Hooks and the Future of the Web

4,675 views

Published on

This is the talk I gave at GlueCon. Based on same deck as the Pivotal Labs talk, but slightly different and updated notes.

Published in: Technology, Education
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,675
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
67
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • me: nasa, hacker dojo, web hooks
    questions on twitter
  • perhaps a pretentious title, i’ve called this superglue.
    the point of webhooks is to really get web app connecting in a way
    that’s not possible with REST or soap api’s alone
  • if that wasn’t pretentious enough ...
    although it’s really about webhooks, then future.
    webhooks may be the future of the web, but just a small part.


  • so just to jumpstart things. here’s what they are.
    i’ll be explaining this and the reason i’m here evangelizing this idea
  • in reality, webhooks aren’t the glue. they’re the bottle the glue comes in.
    we’ll come back to this, but think about that...
    so what are people saying about webhooks? i watch webhooks on twitter search...
  • lightning bolts of cloud computing.
  • tim bray just says they’re the next big thing.
  • i don’t know how this started. it had nothing to do with my talk titles...
  • this guy’s not so sure.
  • and then there’s this guy. apparently he’s french.
  • this is what my talk is going to feel like. i’m going to talk about...
  • before we get to what they are... what problem do they solve?
    people haven’t been asking for them, so what’s the point?


  • like a good engineer i came up with the problems after the solution.
    like a good programmer i came up with a solution that is very generalized and can be used for lots of stuff.
    these are the rough problems that webhooks solve, most of which haven’t been done well or much at all on the web.
  • notifications are the big pull these days. which is cool i suppose. but really just the tip of the iceberg
  • i use three web applications that have “projects.” i use them all for slightly different things, but none of them share data.
    twitter to facebook updates is sort of the same thing. both are updates, i like both apps. they should just be about the same data.
  • this is like the pipes for the web metaphor. this is about composing a system of applications to do more than the parts individually.
  • there is no open source equivalent for the concept of a SaaS. we don’t have the freedom to change code for things we use in the cloud.
    think of how many lame projects this would eliminate: “it’s like twitter, but it does INSERT MINOR IMPROVEMENT”
    what if you could just make it do that?
    this was brought up during one of the last breakout sessions: what do you do if a service doesn’t do what you want?
  • the most popular desktop apps we use: office, firefox, itunes, photoshop... even cult favorites like quicksilver, winamp, vlc... they all have plugins.
    how many web apps do you know with plugins?
  • to me, these ARE all part of the same problem. the web is not programmable enough. programmable web is a misnomer.
    programmatic web. and if people do try to solve these problems, they reinvent for each one... just lay proper infrastructure.


  • aaron fulkerson, web oriented architecture. rest + web
    said things like “extend” “compose” “bootstrap development”. certainly the dream.
    mashups aggregate, they don’t integrate.
  • socialcast, tim young
    pivotal tracker 2-way integration. could be done without socialcast...
    could also make socialcast extendable via CODE. programmability...
  • but obviously... we need webhooks. right?
  • we know what web apps are... callbacks is a bit curious--wait user defined? like end users?
  • i think of three classes of users. developers, power users, machines, and average users.
    so far, web hooks are for developers, but part of all this is about bridging the gap between their power and the average user. for the moment, we mostly talk about the first two here
  • flow through functions
  • flow through functions
  • flow through functions
  • flow through functions
  • flow through functions
  • flow through functions
  • flow through functions
  • compelx. use libraries. they have functions, but they’re black boxes
  • compelx. use libraries. they have functions, but they’re black boxes
  • we use them like black boxes most of the time
  • we use them like black boxes most of the time
  • we use them like black boxes most of the time
  • we use them like black boxes most of the time
  • we use them like black boxes most of the time
  • we use them like black boxes most of the time
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking
  • unless they have callbacks. here we can modify their behavior!
    this is also called hooking


  • devjavu, paypal ... before functional programming even?










  • look at those extra files in the repo!








  • code can do anything
  • all transparent. only see the effects


  • maybe later this
  • ipn is a webhook. started as just a real-time ping of a payment, but more events came up...
  • including events that didn’t involve a user at all. ex: subscription payment failed
  • simple. register a callback url.
  • used that to expose svn hooks in devjavu.
  • simple. too simple? heard disappointment after discovering it was HTTP POST.


  • came up with this tongue in cheek tagline.
    but simple isn’t bad. it’s usually great.
  • simple mechanics, if done right, yield rich, emergent dynamics.
  • so here’s a regular web app.
  • so here’s a regular web app.
  • so here’s a regular web app.
  • so here’s a regular web app.
  • so here’s a regular web app.
  • so here’s a regular web app.
  • just have the events, stuff your code already does, trigger a callback url using POST.
    the user will have a callback...
  • ..registers with you... and now it gets run when events happen
  • all the app needs to know is its a url. it shouldn’t care about much else.
  • so what is the callback? it’s just something to handle the post data.
    cheap php hosting, app engine, appjet, scriptlets...
    because it’s just a url that runs cgi, it can be any language on any machine...
  • so what is the callback? it’s just something to handle the post data.
    cheap php hosting, app engine, appjet, scriptlets...
    because it’s just a url that runs cgi, it can be any language on any machine...
  • so what is the callback? it’s just something to handle the post data.
    cheap php hosting, app engine, appjet, scriptlets...
    because it’s just a url that runs cgi, it can be any language on any machine...
  • so what is the callback? it’s just something to handle the post data.
    cheap php hosting, app engine, appjet, scriptlets...
    because it’s just a url that runs cgi, it can be any language on any machine...
  • so what is the callback? it’s just something to handle the post data.
    cheap php hosting, app engine, appjet, scriptlets...
    because it’s just a url that runs cgi, it can be any language on any machine...






  • jon is building a web app. writes code, deploys to server.
  • jon starts working with a team
  • jon starts working with a team
  • jon starts working with a team
  • jon starts working with a team
  • jon starts working with a team
  • jon starts working with a team
  • jon starts working with a team
  • jon starts working with a team
  • gets repetitive


  • puts a script on his server
  • registers it as a callback on github for post-recieve
  • as he pushes, it runs the script
  • as he pushes, it runs the script
  • automates his previous manual announcement
  • and even...
  • deploys to itself automatically. all he has to do is write code and push.
    could take it further, he owns the script... maybe testing before deploy?


  • the issue is that while interacting with amazon, the user picks options that
    could affect shipping, promotion discounts, and taxes.
    needs to call out back to you (the store owner) to calculate these.










  • here’s what they look like. just post params, key value pairs. you can see what i did.
    they trigger on a lot of events. like login...






  • verticals: ecommerce
  • another big vertical
  • more of a particular use case
  • more of a particular use case
  • this is another use case, but varies a lot in details
  • this is another use case, but varies a lot in details




  • let users decide how they will be notified
  • let users manage data from where they want
  • let users use your app as part of a system
  • let users tweak your app to their needs
  • let users build new functionality for your app.
    user contributed functionality...
  • This is real value: empowering your users to do more with your app than what you created it for...
    with one, simple mechanism


  • observer pattern: subscribe to subjects
  • getpingd, Fethr






  • as a user (power user or otherwise), all hookable apps are part of the ecosystem.
    and part of this new programmable web includes all the existing apis and services
  • scrobbld is an example of a webhook consumer made specifically for paypal ipn.
    really quite amazing they extended paypal without using their api... just ipn.
    this is really more of a mashup and doesn’t necessarily show the power of webhooks
  • here we get into chaining or piping. runcoderun is a webhook consumer for github.
    they run your tests automatically when you push to github.
  • this is more general and infrastructural. it’s a webhook router that was a side project
    of some awesome guys in the uk. input/output handlers... github style extending...


  • cloud code hosting/running is a major part of this infrastructure. you say webhooks
    needs a server? we’re abstracting servers away with the cloud...
    i envisioned a service like this 3 years ago, but it was more like this...
  • more like a pastebin. super simple. write web code, hit save, get a url to run it.
    made for hook scripts, but is good for lots of little tools.
    looking for a generic form poster tool, so i tweeted it
  • and an hour later somebody made it for me.... using scriptlets.




  • like i said, all existing apis are part of this infrastructure. you use these apis
    from hook scripts ... and you can do almost anything these days. order pizza,
    make phone calls, ship stuff from a warehouse...
  • hookah is a tool that tries to abstract away some of the details of implementing
    webhooks. it’s like the smtp daemon of webhooks.
  • in code in the cloud environments, you usually are limited to web requests for obvious reasons.
    protocol droid is going to get around that.


  • this makes working with email mailboxes way easier in the context of the web
  • it’s neat to see it in netnewswire. looks like mail.app



    point is to make more protocols easier to work with from web scripts in fairly limited environments... because there will be more of them as the cloud grows




  • do my own parsing on tasks... extend natural language, or add special codes
  • for example, all these apps share data about todos. they each have respective specialized talents,
    but all work with todos. by putting hooks on todo CRUD,
    you can use their apis to keep them synced pretty well. magically. real-time.


  • could use hooks to help with their magic, maybe... but could also use them to let people contribute connectors... reference github
  • monitoring hooks. run some code to restart your server if the site is down?




  • one thing i’ve been working on is an extension to integrate these ideas.
  • by detecting some markup in a page, it discovers hooks.
    like say for new photos from contacts.
    you want to do something when that happens, click it
  • and write some code. hit save, it posts to AppJet (or wherever),
    registers the handler (assuming a standard protocol), and done. all inline.
    go back and change the code. SCRIPTLETS API
  • also “why am i doing this?”














  • botanicalls, camera with webhooks




  • Any sufficiently advanced technology is indistinguishable from magic.
  • the glue is code... programming. it always will be.
    webhooks make the web more programmable
  • programmability, webhooks: user-defined callbacks over http,
    github, amazon, google, paypal, pbworks, big and little...
    simple implementation with a growing ecosystem of tools to make it easier and more valuable
    some ideas of creative what-ifs, and why i’m doing this, why i want programmability
  • comes back to that dream to “extend” “compose” “bootstrap development”.
    obviously enterprise is a big deal, startups its a big deal. somebody it will be people.
    end users WILL learn to do amazing things if the infrastructure is there...




  • Superglue: Web Hooks and the Future of the Web

    1. 1. webhooks jeff lindsay pro tip: @progrium for questions
    2. 2. webhooks jeff lindsay pro tip: @progrium for questions
    3. 3. superglue
    4. 4. WEBHOOKS AND THE FUTURE OF THE WEB
    5. 5. web•hooks |ˈwebˌhoŏks| pattern of enabling user-defined callbacks in web applications
    6. 6. problem solution examples implementation inspiration future
    7. 7. problem?
    8. 8. ‣ notifications ‣ data syncing ‣ chaining ‣ modification ‣ plugins
    9. 9. ‣ notifications ‣ data syncing ‣ chaining ‣ modification ‣ plugins
    10. 10. ‣ notifications “I wish I could get notifications exactly ‣ data syncing how I want.” ‣ chaining ‣ modification ‣ plugins
    11. 11. ‣ notifications ‣ data syncing “I wish my web apps would keep overlapping ‣ chaining data in sync.” ‣ modification ‣ plugins
    12. 12. ‣ notifications ‣ data syncing ‣ chaining “I wish I could make another app do something ‣ modification when I use this app.” ‣ plugins
    13. 13. ‣ notifications ‣ data syncing ‣ chaining ‣ modification “I wish I could make ‣ plugins this app work just slightly differently.”
    14. 14. ‣ notifications ‣ data syncing ‣ chaining ‣ modification “I wish I could let my ‣ plugins users extend/build plugins for my app.”
    15. 15. } ‣ notifications ‣ data syncing ‣ chaining programmability ‣ modification ‣ plugins
    16. 16. WEBHOOKS
    17. 17. WEBHOOKS Huh?
    18. 18. web•hooks |ˈwebˌhoŏks| pattern of enabling user-defined callbacks in web applications
    19. 19. us•er |ˈyoōzər|
    20. 20. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    21. 21. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    22. 22. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    23. 23. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    24. 24. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    25. 25. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    26. 26. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    27. 27. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    28. 28. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    29. 29. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    30. 30. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    31. 31. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    32. 32. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    33. 33. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    34. 34. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    35. 35. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    36. 36. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    37. 37. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    38. 38. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    39. 39. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    40. 40. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    41. 41. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    42. 42. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    43. 43. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    44. 44. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    45. 45. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    46. 46. call•back |ˈkôlˌbak| Program ƒ(x) ƒ(x) ƒ(x) Library ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x) ƒ(x)
    47. 47. “In computer programming, hooking is a technique used to alter or augment the behavior of [a program], often without having access to its source code.”
    48. 48. webhooks origin
    49. 49. subversion hooks repository working copy foo
    50. 50. subversion hooks repository working copy foo foo
    51. 51. subversion hooks repository foo
    52. 52. subversion hooks repository bar foo
    53. 53. subversion hooks repository bar foo
    54. 54. subversion hooks repository bar bar
    55. 55. subversion hooks repository pre-commit foo bar post-commit
    56. 56. subversion hooks repository pre-commit foo bar post-commit
    57. 57. subversion hooks Does this user have repository permission? pre-commit Is this change valid? foo bar post-commit
    58. 58. subversion hooks repository pre-commit foo bar post-commit
    59. 59. subversion hooks repository pre-commit bar bar post-commit
    60. 60. subversion hooks repository pre-commit bar bar Send a notice to IRC post-commit Try to build the project
    61. 61. subversion hooks repository bar bar
    62. 62. subversion hooks repository A new change was committed by Jeff. bar bar
    63. 63. subversion hooks repository bar bar Damn it Jeff, you broke the build!
    64. 64. paypal ipn
    65. 65. paypal ipn
    66. 66. paypal ipn
    67. 67. webhooks SO SIMPLE YOUʼLL THINK ITʼS STUPID
    68. 68. webhooks anatomy web app
    69. 69. webhooks anatomy web app db
    70. 70. webhooks anatomy ui web app db
    71. 71. webhooks anatomy ui web app db
    72. 72. webhooks anatomy ui web app event event db event
    73. 73. webhooks anatomy ui api web app event event db event
    74. 74. webhooks anatomy ui api web app event event db event
    75. 75. webhooks anatomy ui api web app event event db event
    76. 76. webhooks anatomy ui api web app event event db event callback
    77. 77. webhooks anatomy ui api web app event event db event callback
    78. 78. webhooks anatomy ui api web app event event db event
    79. 79. webhooks anatomy
    80. 80. webhooks anatomy callback
    81. 81. webhooks anatomy callback
    82. 82. webhooks anatomy callback
    83. 83. webhooks anatomy callback
    84. 84. webhooks anatomy callback
    85. 85. examples
    86. 86. DEPLOY
    87. 87. PUSH
    88. 88. Hey guys, pull.
    89. 89. k
    90. 90. DEPLOY
    91. 91. PUSH
    92. 92. Hey guys, pull.
    93. 93. k
    94. 94. DEPLOY
    95. 95. Hooks?
    96. 96. http://myserver.com/script
    97. 97. http://myserver.com/script R ISTE REG
    98. 98. PUSH
    99. 99. PUSH
    100. 100. Hey guys, pull.
    101. 101. k
    102. 102. OY PL E D
    103. 103. 1. define achievements here 2. add trigger pings to your backend 3. paste javascript into your layout
    104. 104. not done yet, but... QUESTIONS?
    105. 105. more examples ecommerce project hosting notifications plugins / platform
    106. 106. more examples ecommerce project hosting notifications plugins / platform
    107. 107. more examples ecommerce project hosting notifications plugins / platform
    108. 108. more examples ecommerce project hosting notifications plugins / platform
    109. 109. more examples ecommerce project hosting notifications plugins / platform
    110. 110. more examples ecommerce project hosting notifications plugins / platform
    111. 111. payoff notifications data syncing FOR THE DEVELOPER chaining modifications plugins
    112. 112. payoff notifications data syncing FOR THE DEVELOPER chaining modifications plugins
    113. 113. payoff notifications data syncing FOR THE DEVELOPER chaining modifications plugins
    114. 114. payoff notifications data syncing FOR THE DEVELOPER chaining modifications plugins
    115. 115. payoff notifications data syncing FOR THE DEVELOPER chaining modifications plugins
    116. 116. payoff notifications data syncing FOR THE DEVELOPER chaining modifications plugins one, simple solution
    117. 117. how to implement
    118. 118. 1. design callback channels per noun or per verb on noun wiki crud? project install/uninstall account pre-commit/post-commit plugin vs
    119. 119. 2. create registration method web ui or api both? :)
    120. 120. 3. choose payload format POST parameters JSON XML
    121. 121. 4. add trigger code to events
    122. 122. 5. consider these options queuing posts HMAC verification retry return data filtering
    123. 123. ecosystem
    124. 124. protocol-droid
    125. 125. protocol-droid coming soon...
    126. 126. inspiration
    127. 127. Todos Basecamp
    128. 128. Hey, there’s an event hook here!
    129. 129. Save
    130. 130. future
    131. 131. what to expect from webhooks
    132. 132. what to expect from webhooks sharing
    133. 133. what to expect from webhooks sharing standardization
    134. 134. what to expect from webhooks sharing standardization
    135. 135. what to expect from webhooks sharing standardization tools
    136. 136. what to expect from webhooks sharing standardization integration tools
    137. 137. what to expect from webhooks sharing standardization integration tools
    138. 138. what to expect from webhooks
    139. 139. web of things embedded devices are getting more powerful and connected
    140. 140. web of things “API for everything”
    141. 141. programming literacy As programming becomes more important, it will leave the back room and become a key skill and attribute of our top intellectual and social classes, just as reading and writing did in the past. Marc Prensky
    142. 142. program the world cloud computing + == near real “magic” web of things + easier programmability
    143. 143. use webhooks! join the community http://webhooks.org
    144. 144. ok, itʼs over... QUESTIONS?

    ×