Web 2.0: Making Email a Useful Web App

14,474 views

Published on

I gave this talk at Web 2.0 Expo in San Francisco on April 23, 2008. The presentation covers historical uses of email in applications as well as some of the new and innovative ways that companies such as TripIt are integrating email in to their applications. The presentation also goes over some of the practical concerns and implementation issues you will likely encounter while building an email based web application.

Published in: Technology, Business
  • Be the first to comment

Web 2.0: Making Email a Useful Web App

  1. Making Email a Useful Web App <ul><ul><li>Web 2.0 Conference </li></ul></ul><ul><ul><li>April 23, 2008 </li></ul></ul><ul><ul><li>San Francisco </li></ul></ul><ul><ul><li>Andy Denmark </li></ul></ul><ul><ul><li>TripIt, Co-Founder and VP Engineering </li></ul></ul>
  2. Email Bankruptcy “ I routinely declare email bankruptcy and simply delete my entire inbox.” - Michael Arrington, TechCrunch “ Wondering if I can get a Bear Sterns bailout for my email bankruptcy.” - willpate on Twitter “ (He) just declared email bankruptcy. I've read those declarations enough to know it means our email tools aren't usable.” - garrickvanburen on Twitter So, why is email still around despite all the challenges and criticism… ?
  3. Email Is Still The “Killer App” <ul><li>Email is a natural extension of a metaphor for communication that humans have used forever </li></ul><ul><li>Everyone has email and the overwhelming majority of all person to person Internet communication takes place there </li></ul><ul><li>No matter what else people use to communicate, they all use email too </li></ul><ul><li>Email contains an amazing amount of context and data about our lives </li></ul><ul><ul><li>Our relationships </li></ul></ul><ul><ul><li>Our purchases </li></ul></ul><ul><ul><li>Our thoughts and the thoughts of people we know </li></ul></ul>
  4. What More Can We Do With Email? <ul><li>Spam and mass email marketing may be contributing to the problems outlined earlier </li></ul><ul><li>But think of all the things we can do to make our applications better by making better use of email: </li></ul><ul><ul><li>Broadcasting data </li></ul></ul><ul><ul><li>Query interfaces </li></ul></ul><ul><ul><li>Data transfer </li></ul></ul><ul><ul><li>Intelligent parsing of data </li></ul></ul><ul><li>Mining of social connections (“Inbox 2.0”) </li></ul>
  5. Does An Email App Make Sense? <ul><li>PROS </li></ul><ul><li>Ubiquity – everyone has email </li></ul><ul><li>Exposure – email is on all day and captures a lot of attention </li></ul><ul><li>Simplicity – easy to send email, everyone knows how to do it </li></ul><ul><li>Embedded identity </li></ul><ul><li>CONS </li></ul><ul><li>May just not fit your service </li></ul><ul><li>May be too complex for your users to understand </li></ul><ul><li>It’s challenging to properly support all email clients, versions and formats </li></ul><ul><li>Building and supporting a system that sends, receives, and reads email is a pain </li></ul>
  6. Prehistoric Email Apps
  7. Domain Registration Email Sign-Up <ul><li>Email to: [email_address] </li></ul><ul><li>From address authentication for new sign-ups </li></ul><ul><li>Choose to receive your bill via email or USPS </li></ul><ul><li>Authentication schemes for modify/delete were CRYPT-PW and PGP </li></ul>
  8. Archie File Query Email Interface <ul><li>Available Interfaces </li></ul><ul><li>Text-based client </li></ul><ul><li>Graphical client </li></ul><ul><li>Telnet client </li></ul><ul><li>Email </li></ul>
  9. Browsing The Web Via Email “ For personal reasons, I do not browse the web from my computer. (I also have no net connection much of the time.) To look at page I send mail to a demon which runs wget and mails the page back to me. It is very efficient use of my time, but it is slow in real time.” -- Richard Stallman (2007-12-15)
  10. About TripIt ?
  11.  
  12.  
  13. ***CONFIDENTIAL***
  14.  
  15. Forward Any Confirmation Email Automated Online Master Itinerary Automated Weather, Maps, Guides… Trip Sharing & Social Alerts iCal & Atom Support Email & Mobile Access Add Meetings, Activities, Restaurants, etc.
  16. Why Email Made Sense For TripIt <ul><li>The native format of travel data is varied and unfriendly </li></ul><ul><li>The travel industry isn’t moving to expose that data </li></ul><ul><li>Partnering with big travel companies and building APIs would take too long </li></ul><ul><li>Users already have all their travel data in email, but that data is “dead” </li></ul><ul><li>Auto-account creation using the embedded identity of an email message </li></ul><ul><li>It has become an “instant delight” factor for users </li></ul>
  17. Types Of Email App Services Command Line Interface Data Transfer Semantic Parsing Broadcast ?
  18. Types Of Email App Services Command Line Interface Data Transfer Semantic Parsing Broadcast <ul><li>One Way </li></ul><ul><li>User requests email information to be sent to them </li></ul><ul><li>Typically triggered by an action or an alert </li></ul><ul><li>Examples </li></ul><ul><li>Mailing Lists </li></ul><ul><li>Google Alerts </li></ul>
  19. Example: Mailing Lists Searchable knowledge base built as a by-product of natural human communication
  20. Example: Google Alerts <ul><li>“ Real time” alerts delivered right to my desktop </li></ul><ul><li>Useful way to asynchronously broadcast data I’ve requested as it’s available </li></ul><ul><li>With linkage to the web, alerts are actionable and controllable </li></ul>
  21. Types Of Email App Services Command Line Interface Data Transfer Semantic Parsing Broadcast <ul><li>CLI </li></ul><ul><li>Users emails a command and Email App replies with a response </li></ul><ul><li>Typically a standard list of commands and responses </li></ul><ul><li>Examples </li></ul><ul><li>I Want Sandy </li></ul><ul><li>TripIt </li></ul><ul><li>Twitter </li></ul>
  22. Example: I Want Sandy http://iwantsandy.com/
  23. Example: TripIt
  24. Example: Twitter <ul><li>Messaging platform with multiple UIs </li></ul><ul><ul><li>Web (http://twitter.com/) </li></ul></ul><ul><ul><li>SMS (40404) </li></ul></ul><ul><ul><li>IM (Jabber: twitter@twitter.com) </li></ul></ul><ul><ul><li>Facebook (http://www.facebook.com/) </li></ul></ul><ul><ul><li>FriendFeed ( http://friendfeed.com/ ) </li></ul></ul><ul><ul><li>Desktop (e.g. Twhirl) </li></ul></ul><ul><ul><li>Email </li></ul></ul><ul><li>I can tweet by email?!? </li></ul><ul><ul><li>Yes, it’s just another way to use the messaging platform’s CLI </li></ul></ul><ul><ul><li>Very natural, ubiquitous and free (no SMS charges)! </li></ul></ul><ul><ul><li>See: http://emailtwitter.com/ and http://twittermail.com/ </li></ul></ul>
  25. Types Of Email App Services Command Line Interface Data Transfer Semantic Parsing <ul><li>Email Forwarding </li></ul><ul><li>Users forward emails and Email App recognizes and parses them </li></ul><ul><li>Typically converts “unstructured” data into “structured” data making it more usable to the app and user </li></ul><ul><li>Examples </li></ul><ul><li>Flickr </li></ul><ul><li>Blogger </li></ul>Broadcast
  26. Example: Flickr
  27. Example: Blogger
  28. Types Of Email App Services Command Line Interface Data Transfer Semantic Parsing Broadcast <ul><li>Intelligent Agent </li></ul><ul><li>Structured data from email is enriched by the Email App by retrieving related information from other sites </li></ul><ul><li>Typically leverages “keys” such as geo-coding to capture relevant data </li></ul><ul><li>Examples </li></ul><ul><li>TrackMyShipments </li></ul><ul><li>TripIt </li></ul>
  29. Example: TrackMyShipments
  30. Example: TripIt Timezone Online Checkin Flight Status Seat Advice Flight Details Weather
  31. Types Of Email App Services Command Line Interface Data Transfer Semantic Parsing <ul><li>Email Forwarding </li></ul><ul><li>Users forward emails and Email App recognizes and parses them </li></ul><ul><li>Typically converts “unstructured” data into “structured” data making it more usable to the app and user </li></ul><ul><li>Examples </li></ul><ul><li>Flickr </li></ul><ul><li>Blogger </li></ul>Broadcast <ul><li>CLI </li></ul><ul><li>Users emails a command and Email App replies with a response </li></ul><ul><li>Typically a standard list of commands and responses </li></ul><ul><li>Examples </li></ul><ul><li>I Want Sandy </li></ul><ul><li>TripIt </li></ul><ul><li>Twitter </li></ul><ul><li>One Way </li></ul><ul><li>User requests email information to be sent to them </li></ul><ul><li>Typically triggered by an action or an alert </li></ul><ul><li>Examples </li></ul><ul><li>Listserv </li></ul><ul><li>Google Alerts </li></ul><ul><li>Intelligent Agent </li></ul><ul><li>Structured data from email is enriched by the Email App by retrieving related information from other sites </li></ul><ul><li>Typically leverages “keys” such as geo-coding to capture relevant data </li></ul><ul><li>Examples </li></ul><ul><li>TrackMyShipments </li></ul><ul><li>TripIt </li></ul>
  32. Caution: Email Is Full Of Problems CAUTION STANDARDS CAUTION CAUTION SPAM CAUTION CAUTION PRIVACY CAUTION CAUTION DELIVERABILITY CAUTION
  33. Deliverability Issues <ul><li>Your reputation is attached to your IPs </li></ul><ul><li>Monitor your logs, it’s a dynamic system </li></ul><ul><li>Work with the ESPs, they are as much invested in this as you are </li></ul><ul><li>Even if you're a good citizen, there are no guarantees email will get through </li></ul><ul><li>Tips for getting email delivered… </li></ul><ul><ul><li>No Open Relays </li></ul></ul><ul><ul><li>Proper DNS Configuration (forward + reverse resolution) </li></ul></ul><ul><ul><li>SPF/DKIM </li></ul></ul><ul><ul><li>Privacy Policy </li></ul></ul><ul><ul><li>RFC Compliance </li></ul></ul><ul><ul><li>ESP “Whitelists” </li></ul></ul><ul><ul><li>Bounce Handling </li></ul></ul>CAUTION DELIVERABILITY CAUTION
  34. Spam Issues <ul><li>No silver bullet solutions, but a suite of counter-measures… </li></ul><ul><ul><li>Spam assessment tools </li></ul></ul><ul><ul><li>Spam Firewalls (Hardware/Software) ‏ </li></ul></ul><ul><ul><li>Improper/forged headers are key factor </li></ul></ul><ul><ul><li>Non-RFC Compliant Mailers (don't retry, don't try all your MXs, etc.) ‏ </li></ul></ul><ul><ul><li>DHCP/home IPs w/ no reverse DNS </li></ul></ul><ul><ul><li>Blacklists (be very careful) ‏ </li></ul></ul><ul><li>But all of these cause you to run the risk of catching legitimate email in the net </li></ul><ul><li>Spammers like to go fast, so slowing things down in an RFC compliant way can help </li></ul>CAUTION SPAM CAUTION
  35. Privacy Issues <ul><li>People are very protective of their Inbox, it’s their private space </li></ul><ul><li>Laws vary wildly but there’s a baseline in common sense </li></ul><ul><li>Make sure people can opt-out/in easily </li></ul><ul><li>Build mechanisms that prevent you from contacting people who don’t wish to be contacted </li></ul><ul><li>Listen to your communities, you’ll get some of this wrong, but you can fix it </li></ul><ul><li>Be transparent and responsive </li></ul>CAUTION PRIVACY CAUTION
  36. Standards Issues <ul><li>There are no standards that are strictly followed across mail clients </li></ul><ul><li>Pretty much every mail-related RFC has been outright violated or creatively interpreted by at least one major mail client </li></ul><ul><li>White space, HTML markup, attachments are all mangled and re-interpreted depending on which mail client is working on the message </li></ul><ul><li>If you build a mail application that reads email you will have to deal with this, it’s expensive and painful </li></ul>CAUTION STANDARDS CAUTION
  37. Key Take-aways <ul><li>Email is not dead! </li></ul><ul><ul><li>Treasure-trove of data waiting to be mined </li></ul></ul><ul><li>At least four different types of apps </li></ul><ul><ul><li>Broadcast, CLI, Data transfer, Semantic parsing </li></ul></ul><ul><li>Remember the gotchas: </li></ul><ul><ul><li>Deliverability, Spam, Privacy, Standards </li></ul></ul><ul><li>If you get it right, huge payoff for you and your users </li></ul>
  38. Questions <ul><ul><li>Andy Denmark </li></ul></ul><ul><ul><li>TripIt, Co-Founder and VP Engineering </li></ul></ul>
  39. Appendix <ul><li>ESP Whitelists/Resources </li></ul><ul><ul><li>Microsoft Hotmail/MSN: </li></ul></ul><ul><ul><ul><li>http://postmaster.msn.com/Services.aspx </li></ul></ul></ul><ul><ul><ul><li>https://postmaster.live.com/snds/ </li></ul></ul></ul><ul><ul><ul><li>http://postmaster.live.com/Troubleshooting.aspx </li></ul></ul></ul><ul><ul><ul><li>http://download.microsoft.com/download/e/3/3/e3397e7c-17a6-497d-9693-78f80be272fb/enhance_deliver.pdf </li></ul></ul></ul><ul><ul><ul><li>http://www.senderscorecertified.com/ by ReturnPath </li></ul></ul></ul><ul><ul><li>Yahoo </li></ul></ul><ul><ul><ul><li>http://help.yahoo.com/l/us/yahoo/mail/postmaster/ </li></ul></ul></ul><ul><ul><li>Google </li></ul></ul><ul><ul><ul><li>http://www.google.com/mail/help/bulk_mail.html </li></ul></ul></ul><ul><ul><li>AOL </li></ul></ul><ul><ul><ul><li>http://postmaster.aol.com/ </li></ul></ul></ul>
  40. Appendix cont’d <ul><li>Mail Server Setup </li></ul><ul><ul><li>http://domainkeys.sourceforge.net/ </li></ul></ul><ul><ul><li>http://www.openspf.org/ </li></ul></ul><ul><ul><li>http://www.maawg.org/about/MAAWG_Sender_BCP/MAAWG_Senders_BCP_Combine.pdf </li></ul></ul><ul><li>CLI Examples </li></ul><ul><ul><li>http://microformats.org/wiki/picoformats </li></ul></ul><ul><li>Relevant RFCs </li></ul><ul><ul><li>http://rfc.net/rfc2821.html (SMTP) </li></ul></ul><ul><ul><li>http://rfc.net/rfc2822.html (Mail Headers) </li></ul></ul><ul><ul><li>http://rfc.net/rfc2045.html (MIME part 1) </li></ul></ul><ul><ul><li>http://rfc.net/rfc2046.html (MIME part 2) </li></ul></ul>

×