Avoiding the Slog of Real-time Data Distribution

1,340 views

Published on

If your site is working with third-party sites, there's a lot that can go wrong. Let us count the ways ... and offer some suggestions.

Published in: Technology, Business
  • Be the first to comment

Avoiding the Slog of Real-time Data Distribution

  1. 1. Avoiding the Slog of Real-time Data Distribution Eric Lunt Glue 2012
  2. 2. Eric Lunt (@elunt)CTO, BrightTag (@BrightTag)Backchannel: #glueslog 2
  3. 3. Minimizing Avoiding the Slog of Real-time Data Distribution or at ,Least Understanding Why It’s a Slog to Begin With 3
  4. 4. The Players Brokered Site Owner Vendor Publisher Vendors Site Visitor 4
  5. 5. What kind of data? Site Owner Vendor ? Analytics Ad Networks Social/Sharing Retargeting Networks 5
  6. 6. What kind of data? Site Owner 6
  7. 7. How does the data get shared? I would like to use your services please. Great! All we need to get started is some data about your site visitors. Okay, how do we do that? Just add these tags to your site. 7
  8. 8. How does data get shared? Tags = HTML Fragments<script src="http://vendy <img height="1" width="1" <iframe width="1"corp.com/tag.js"></script> src="http://vendycorp.com/ height="1" frameborder="0"<script> tag?id=1234"> scrolling="no" vendy.id=1234; marginheight="0" fireTag(); marginwidth="0"</script> topmargin="0" leftmargin="0" src="http://vendycorp.com/ tag?id=1234"></iframe> JavaScript Image “Beacon” IFrame 8
  9. 9. Meanwhile, inside Shoemart… Oh, benevolent mighty IT group, we need to add these tags to our site. No. Why not? Itll slow the site down. But this tag will increase our ROI CPM SEO B2B WTF BBQ! My bonus is based on site performance. 9
  10. 10. Meanwhile, inside Shoemart… Heres an email from VP of marketing saying we need to add it. And heres the email from the vendor with the tag code. And some spring rolls. Okay, our next build is in two weeks. Well try to get it in there. 10
  11. 11. Meanwhile, inside Shoemart… Three weeks later… 11
  12. 12. Meanwhile, inside Shoemart… Okay, we pushed your tag. Victory! 12
  13. 13. Meanwhile, inside Shoemart… 24 hours later… 13
  14. 14. Meanwhile, inside Shoemart… Wait, none of these reports make sense. Whats wrong? We need to change the tag. You didnt implement the tag correctly. The Okay, our next build is in segment id should use a two weeks. Well try to get pipe delimiter instead of it in there. colons, youre not URI encoding the product name, and the revenue should be an integer in pennies. 14
  15. 15. What are the problems with tags?• Can slow the site down 15
  16. 16. What are the problems with tags? Weve all heard the stat… Every 100ms increase in latency means losing a BILLION dollars in revenue!* *Not true in all cases 16
  17. 17. What are the problems with tags?• Can slow the site down – <script> tag blocks the rendering thread – Most vendor tags use the <script> tag – Many vendors have lousy JavaScript I HATE tags! 17
  18. 18. What can we do about slow tags? Understand client performance at an intimate level READ THESE BOOKS! 18
  19. 19. What can we do about slow tags? Move the tags to the bottom of the page <html> <html> <head> <head> ... Bad ... <script src= </head> "http://vendycorp.com/tag.js"> <body> </script> ... </head> <script src= <body> "http://vendycorp.com/tag.js"> ... </script> </body> </body> Better 19
  20. 20. What can we do about slow tags? Make tags asynchronous or post-load Evolution of the Google Analytics markup:<script type="text/javascript">var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");document.write(unescape("%3Cscript src=" + gaJsHost + "google-analytics.com/ga.jstype=text/javascript%3E%3C/script%3E"));</script><script type="text/javascript">try{var pageTracker = _gat._getTracker("UA-xxxxxx-x");pageTracker._trackPageview();} catch(err) {}</script> Old Blocking Markup 20
  21. 21. What can we do about slow tags? Make tags asynchronous or post-load Evolution of the Google Analytics markup:<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push([_setAccount, UA-XXXXX-X]); _gaq.push([_trackPageview]); (function() { var ga = document.createElement(script); ga.type = text/javascript; ga.async = true; ga.src = (https: == document.location.protocol ? https://ssl : http://www) + .google-analytics.com/ga.js; var s = document.getElementsByTagName(script)[0]; s.parentNode.insertBefore(ga, s); })();</script> New Async Markup 21
  22. 22. What can we do about slow tags? Make tags asynchronous or post-load <script src= "http://vendycorp.com/tag.js"> </script> Better <script> (function() { var s = document.createElement(script); s.type = text/javascript; s.async = true; s.src = http://vendycorp.com/tag.js; var me = document.getElementsByTagName(script)[0]; me.parentNode.insertBefore(s, me); })(); </script> 22
  23. 23. What can we do about slow tags? Make tags asynchronous or post-load Or use a JS loader like LABjs: <script> $LAB.script(http://vendycorp.com/tag.js); </script> 23
  24. 24. SIDEBARWhat can we do about slow tags? Dont do this: Make tags asynchronous or post-load <script type="text/javascript"> var prot = (("https:" == document.location.protocol) ? "https:" : "http:"); new Image().src = prot+//vendycorp.com/img.cgi?id=123; </script> Or use a JS loader like LABjs: <script> Do this: $LAB.script(http://vendycorp.com/tag.js); </script> <script type="text/javascript"> new Image().src = //vendycorp.com/img.cgi?id=123; </script> 24
  25. 25. What can we do about slow tags? Make tags asynchronous or post-load What are the problems? document.write() Listening for DOM events 25
  26. 26. What are the problems with tags?• Can slow the site down• Incomplete data collection• Data integrity 26
  27. 27. What are the problems with tags?• Data integrity Report For Shoemark SKU Sales 123456 $82034.32 589483 90459.34 Site Owner Vendor sku=123456 price=339.25 qty=1 sku=123456 price=339.25 qty=1 Site Visitor 27
  28. 28. What are the problems with tags?• Data integrity Report For Shoemark SKU Sales 123456 $82034.32 589483 90459.34 Site Owner Vendor sku=123456 price=339.25 qty=1 sku=123456 price=3392.50 qty=10 Site Visitor 28
  29. 29. What are the problems with tags?• Can slow the site down• Incomplete data collection• Data integrity 29
  30. 30. Can we cut the browser out?• Asynchronous server-side call, so no slowdown• 100% data collection• No chance for data manipulation Site Owner sku=123456 Vendor price=339.25 qty=1 Site Visitor 30
  31. 31. Can we cut the browser out? So whats the problem? Site Owner sku=123456 Vendor price=339.25 qty=1 Site Visitor 31
  32. 32. Can we cut the browser out?• Server-side integrations are much more difficult• COOKIES! Site Owner sku=123456 Vendor price=339.25 qty=1 Site Visitor 32
  33. 33. Cookies are the real challengeSite Owner Publisher Vendor Cookie: <script name=Pat <script src="vendycorp.com"> src="vendycorp.com"> Set-Cookie: "Hi Pat!" name=Pat Later…Site Visitor 33
  34. 34. Cookies are the real challenge• No way to assert site visitor identity to the vendor• Actually, there is a way… Site Owner sku=123456 Vendor price=339.25 qty=1 name=??? Site Visitor 34
  35. 35. Cookie Sync1. Establish the first-party id Site Owner Set-Cookie: id=shoe321 Site Visitor 35
  36. 36. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor Cookie: Site Owner id=shoe321 Vendycorp id for id=shoe321? Nope Site Visitor 36
  37. 37. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities 302 Location: http://vendycorp.com/sync?redir=http://shoemark.com/sync%3Fvendycorpid%3D%[[id]] Site Owner Site Visitor 37
  38. 38. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities Cookie: Vendor name=Pat 302 Location: http://shoemark.com/sync?vendycorpid=Pat Site Visitor 38
  39. 39. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities Cookie: Site Owner id=shoe321 Vendycorp id for id=shoe321 is http://shoemark. "Pat"! com/sync? vendycorpid=Pat Site Visitor 39
  40. 40. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities4. Now we can pass along the vendor id server-side Cookie: Site Owner sku=123456 Vendor id=shoe321 price=339.25 qty=1 name=Pat Site Visitor 40
  41. 41. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities4. Now we can pass along the vendor id server-side5. Repeat for each vendor! Cookie: Site Owner sku=123456 Vendor id=shoe321 price=339.25 qty=1 name=Pat Site Visitor 41
  42. 42. Mo cookies, mo problemsSite Owner Publisher Vendor Cookie: Cookie: name=Pat <script name=Pat wants=boots <script src="vendycorp.com"> src="vendycorp.com"> Set-Cookie: "Hi Pat! Heres an wants=boots ad for some boots!" Later…Site Visitor 42
  43. 43. So what can we do?• Just accept it – Live with the problems – Pretend Safari doesnt exist 43
  44. 44. No third-party cookiesSite Owner Publisher Vendor Cookie: Cookie: name=Pat <script name=Pat wants=boots <script src="vendycorp.com"> src="vendycorp.com"> Set-Cookie: "Hi stranger!" wants=bootsSite Visitor 44
  45. 45. So what can we do?• Just accept it – Live with the problems – Pretend Safari doesnt exist – Ignore the EU cookie directive 45
  46. 46. EU Cookies 46
  47. 47. So what can we do?• Just accept it – Live with the problems – Pretend Safari doesnt exist – Ignore the EU cookie directive• Forge new paths – RTB (real-time bidding) is a fore-runner – Update user information outside the context of a HTTP transaction – Beyond the browser 47
  48. 48. So what can we do?I actually dont have the answers, but wanted you to beaware of all the moving parts, so the next time you seesome JavaScript that looks brain-dead and kicks off aseries of six 302 redirects to different domains thatpiggyback in forty new tags, you have some context.You dont have to like it, though. 48
  49. 49. Thank You!Eric Lunt (@elunt)CTO, BrightTag (@BrightTag) 49

×