Avoiding the Slog of Real-time Data Distribution

1,309 views

Published on

If your site is working with third-party sites, there's a lot that can go wrong. Let us count the ways ... and offer some suggestions.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,309
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • We are in Chicago, 2 years old, we are essentially middleware for data distribution. My background is in building large distributed scalable low-latency systems, not digital marketing.
  • List the kinds of services, build into the big diagram
  • List the kinds of services, build into the big diagram
  • List the kinds of services, build into the big diagram
  • This is kind of a general statement. What is really important is to understand the relationship between the site markup and the browser rendering thread.
  • Okay, maybe not in all cases, but there are several studies that show abandonment rate is directly related to latency, and that does actually tie to revenue.
  • IT hates this situation because it represents a part of the site that they don't control.
  • Steve Souders is the man. Talks in detail about what blocks the browser rendering thread and what doesn't.
  • Go as low as you can so as much of the page will render before you make the third-party calls.Oftentimes, vendors will suggest you go high in the page to minimize data loss, though.
  • There are all sorts of things that we haven't gotten into here, such as dependencies between scripts. LABjs helps with that, so you can wait for a script to load before executing functions in that script file.
  • Talk about how an http reference on an https page is bad. Use protocol relative URLs.
  • For those of you keeping track, we're now talking about the problems to the solution to the problem.
  • Avoiding the Slog of Real-time Data Distribution

    1. 1. Avoiding the Slog of Real-time Data Distribution Eric Lunt Glue 2012
    2. 2. Eric Lunt (@elunt)CTO, BrightTag (@BrightTag)Backchannel: #glueslog 2
    3. 3. Minimizing Avoiding the Slog of Real-time Data Distribution or at ,Least Understanding Why It’s a Slog to Begin With 3
    4. 4. The Players Brokered Site Owner Vendor Publisher Vendors Site Visitor 4
    5. 5. What kind of data? Site Owner Vendor ? Analytics Ad Networks Social/Sharing Retargeting Networks 5
    6. 6. What kind of data? Site Owner 6
    7. 7. How does the data get shared? I would like to use your services please. Great! All we need to get started is some data about your site visitors. Okay, how do we do that? Just add these tags to your site. 7
    8. 8. How does data get shared? Tags = HTML Fragments<script src="http://vendy <img height="1" width="1" <iframe width="1"corp.com/tag.js"></script> src="http://vendycorp.com/ height="1" frameborder="0"<script> tag?id=1234"> scrolling="no" vendy.id=1234; marginheight="0" fireTag(); marginwidth="0"</script> topmargin="0" leftmargin="0" src="http://vendycorp.com/ tag?id=1234"></iframe> JavaScript Image “Beacon” IFrame 8
    9. 9. Meanwhile, inside Shoemart… Oh, benevolent mighty IT group, we need to add these tags to our site. No. Why not? Itll slow the site down. But this tag will increase our ROI CPM SEO B2B WTF BBQ! My bonus is based on site performance. 9
    10. 10. Meanwhile, inside Shoemart… Heres an email from VP of marketing saying we need to add it. And heres the email from the vendor with the tag code. And some spring rolls. Okay, our next build is in two weeks. Well try to get it in there. 10
    11. 11. Meanwhile, inside Shoemart… Three weeks later… 11
    12. 12. Meanwhile, inside Shoemart… Okay, we pushed your tag. Victory! 12
    13. 13. Meanwhile, inside Shoemart… 24 hours later… 13
    14. 14. Meanwhile, inside Shoemart… Wait, none of these reports make sense. Whats wrong? We need to change the tag. You didnt implement the tag correctly. The Okay, our next build is in segment id should use a two weeks. Well try to get pipe delimiter instead of it in there. colons, youre not URI encoding the product name, and the revenue should be an integer in pennies. 14
    15. 15. What are the problems with tags?• Can slow the site down 15
    16. 16. What are the problems with tags? Weve all heard the stat… Every 100ms increase in latency means losing a BILLION dollars in revenue!* *Not true in all cases 16
    17. 17. What are the problems with tags?• Can slow the site down – <script> tag blocks the rendering thread – Most vendor tags use the <script> tag – Many vendors have lousy JavaScript I HATE tags! 17
    18. 18. What can we do about slow tags? Understand client performance at an intimate level READ THESE BOOKS! 18
    19. 19. What can we do about slow tags? Move the tags to the bottom of the page <html> <html> <head> <head> ... Bad ... <script src= </head> "http://vendycorp.com/tag.js"> <body> </script> ... </head> <script src= <body> "http://vendycorp.com/tag.js"> ... </script> </body> </body> Better 19
    20. 20. What can we do about slow tags? Make tags asynchronous or post-load Evolution of the Google Analytics markup:<script type="text/javascript">var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");document.write(unescape("%3Cscript src=" + gaJsHost + "google-analytics.com/ga.jstype=text/javascript%3E%3C/script%3E"));</script><script type="text/javascript">try{var pageTracker = _gat._getTracker("UA-xxxxxx-x");pageTracker._trackPageview();} catch(err) {}</script> Old Blocking Markup 20
    21. 21. What can we do about slow tags? Make tags asynchronous or post-load Evolution of the Google Analytics markup:<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push([_setAccount, UA-XXXXX-X]); _gaq.push([_trackPageview]); (function() { var ga = document.createElement(script); ga.type = text/javascript; ga.async = true; ga.src = (https: == document.location.protocol ? https://ssl : http://www) + .google-analytics.com/ga.js; var s = document.getElementsByTagName(script)[0]; s.parentNode.insertBefore(ga, s); })();</script> New Async Markup 21
    22. 22. What can we do about slow tags? Make tags asynchronous or post-load <script src= "http://vendycorp.com/tag.js"> </script> Better <script> (function() { var s = document.createElement(script); s.type = text/javascript; s.async = true; s.src = http://vendycorp.com/tag.js; var me = document.getElementsByTagName(script)[0]; me.parentNode.insertBefore(s, me); })(); </script> 22
    23. 23. What can we do about slow tags? Make tags asynchronous or post-load Or use a JS loader like LABjs: <script> $LAB.script(http://vendycorp.com/tag.js); </script> 23
    24. 24. SIDEBARWhat can we do about slow tags? Dont do this: Make tags asynchronous or post-load <script type="text/javascript"> var prot = (("https:" == document.location.protocol) ? "https:" : "http:"); new Image().src = prot+//vendycorp.com/img.cgi?id=123; </script> Or use a JS loader like LABjs: <script> Do this: $LAB.script(http://vendycorp.com/tag.js); </script> <script type="text/javascript"> new Image().src = //vendycorp.com/img.cgi?id=123; </script> 24
    25. 25. What can we do about slow tags? Make tags asynchronous or post-load What are the problems? document.write() Listening for DOM events 25
    26. 26. What are the problems with tags?• Can slow the site down• Incomplete data collection• Data integrity 26
    27. 27. What are the problems with tags?• Data integrity Report For Shoemark SKU Sales 123456 $82034.32 589483 90459.34 Site Owner Vendor sku=123456 price=339.25 qty=1 sku=123456 price=339.25 qty=1 Site Visitor 27
    28. 28. What are the problems with tags?• Data integrity Report For Shoemark SKU Sales 123456 $82034.32 589483 90459.34 Site Owner Vendor sku=123456 price=339.25 qty=1 sku=123456 price=3392.50 qty=10 Site Visitor 28
    29. 29. What are the problems with tags?• Can slow the site down• Incomplete data collection• Data integrity 29
    30. 30. Can we cut the browser out?• Asynchronous server-side call, so no slowdown• 100% data collection• No chance for data manipulation Site Owner sku=123456 Vendor price=339.25 qty=1 Site Visitor 30
    31. 31. Can we cut the browser out? So whats the problem? Site Owner sku=123456 Vendor price=339.25 qty=1 Site Visitor 31
    32. 32. Can we cut the browser out?• Server-side integrations are much more difficult• COOKIES! Site Owner sku=123456 Vendor price=339.25 qty=1 Site Visitor 32
    33. 33. Cookies are the real challengeSite Owner Publisher Vendor Cookie: <script name=Pat <script src="vendycorp.com"> src="vendycorp.com"> Set-Cookie: "Hi Pat!" name=Pat Later…Site Visitor 33
    34. 34. Cookies are the real challenge• No way to assert site visitor identity to the vendor• Actually, there is a way… Site Owner sku=123456 Vendor price=339.25 qty=1 name=??? Site Visitor 34
    35. 35. Cookie Sync1. Establish the first-party id Site Owner Set-Cookie: id=shoe321 Site Visitor 35
    36. 36. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor Cookie: Site Owner id=shoe321 Vendycorp id for id=shoe321? Nope Site Visitor 36
    37. 37. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities 302 Location: http://vendycorp.com/sync?redir=http://shoemark.com/sync%3Fvendycorpid%3D%[[id]] Site Owner Site Visitor 37
    38. 38. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities Cookie: Vendor name=Pat 302 Location: http://shoemark.com/sync?vendycorpid=Pat Site Visitor 38
    39. 39. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities Cookie: Site Owner id=shoe321 Vendycorp id for id=shoe321 is http://shoemark. "Pat"! com/sync? vendycorpid=Pat Site Visitor 39
    40. 40. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities4. Now we can pass along the vendor id server-side Cookie: Site Owner sku=123456 Vendor id=shoe321 price=339.25 qty=1 name=Pat Site Visitor 40
    41. 41. Cookie Sync1. Establish the first-party id2. Check to see if we know the vendor id for visitor3. Initiate a cookie sync to map identities4. Now we can pass along the vendor id server-side5. Repeat for each vendor! Cookie: Site Owner sku=123456 Vendor id=shoe321 price=339.25 qty=1 name=Pat Site Visitor 41
    42. 42. Mo cookies, mo problemsSite Owner Publisher Vendor Cookie: Cookie: name=Pat <script name=Pat wants=boots <script src="vendycorp.com"> src="vendycorp.com"> Set-Cookie: "Hi Pat! Heres an wants=boots ad for some boots!" Later…Site Visitor 42
    43. 43. So what can we do?• Just accept it – Live with the problems – Pretend Safari doesnt exist 43
    44. 44. No third-party cookiesSite Owner Publisher Vendor Cookie: Cookie: name=Pat <script name=Pat wants=boots <script src="vendycorp.com"> src="vendycorp.com"> Set-Cookie: "Hi stranger!" wants=bootsSite Visitor 44
    45. 45. So what can we do?• Just accept it – Live with the problems – Pretend Safari doesnt exist – Ignore the EU cookie directive 45
    46. 46. EU Cookies 46
    47. 47. So what can we do?• Just accept it – Live with the problems – Pretend Safari doesnt exist – Ignore the EU cookie directive• Forge new paths – RTB (real-time bidding) is a fore-runner – Update user information outside the context of a HTTP transaction – Beyond the browser 47
    48. 48. So what can we do?I actually dont have the answers, but wanted you to beaware of all the moving parts, so the next time you seesome JavaScript that looks brain-dead and kicks off aseries of six 302 redirects to different domains thatpiggyback in forty new tags, you have some context.You dont have to like it, though. 48
    49. 49. Thank You!Eric Lunt (@elunt)CTO, BrightTag (@BrightTag) 49

    ×