Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Weebly use Looker and Snowplow to do funnel analysis

1,703 views

Published on

Overview of how Weebly use Snowplow + Looker followed by a deep dive into the different techniques used by the Weebly team to develop funnel reports

Published in: Business
  • Login to see the comments

How Weebly use Looker and Snowplow to do funnel analysis

  1. 1. Weebly Looker Reports Powered by Snowplow Audrey Carstensen - Product Analytics Lead Bo Han - Data Engineer (With help from the entire Weebly Data team!)
  2. 2. Who is Weebly? ● Drag n’ Drop Website Builder and Tools to Support Entrepreneurs Online ● ~30 million users, 16MM websites, ~10B Pageviews per year ● 3 data engineers & 7 data analysts supporting 200+ employee company ● Use Snowplow’s data-pipeline for events on Weebly.com and Weebly built sites
  3. 3. What are we going to show you? ● Journey to Snowplow + Looker ○ Why Event Tracking? ○ Snowplow Implementation ● What we implemented in Looker (Usage / Visuals / LookML) ○ Event and User Counts ○ Funnels ○ Retention ○ Onboarding timing ● What we learned
  4. 4. Why Event Tracking? ● Product needs to track user behavior... ○ Product dashboards showing cohorts, retention, funnel ● Marketing needs visitor/pre-signup data... ○ Visitor / campaign level breakdown ● Analysts need granular/raw data... ○ AB testing, ad-hoc SQL queries into raw event stream ● Data scientists need features and need it real-time ○ Story for another meetup... Our needs are beyond what Google Analytics offers ...
  5. 5. Journey to Snowplow (Engineering Perspective) ● Product hosted on on-premise data center; analytics on AWS ● Small data engineering team with very limited DevOps support ○ Better not reinvent the wheel ● Need for a large scale event tracking solution capable of handling 100MMs events per day ○ $$$ Too expensive for Mixpanel ○ And need it fast.. Think 12 weeks of development time ● Long term goals ○ Need to store the underlying raw data ○ Real-time consumers Analysts Data Engineers Snowplow
  6. 6. Example Summary Tables ● Events rollup ○ General event volumes by date (no user data) ● Last 60 days ○ Base many dashboards / looks off this ○ Faster than all time table ● Users ○ User facts based on Snowplow events ● User action times ○ Keep track of onboarding timestamps ○ Quickly pull “time to X” metrics Pro-tip: Store frequently used views as summary tables for better performance
  7. 7. Rollup Explore: Usage and Visualization ● Date / Category / Action / Label / Page Type (Weebly-specific based on URL) ● Usage: troubleshooting events as an engineer or data analyst, quick estimates as a data analyst
  8. 8. Rollup Explore: LookML
  9. 9. Events Explore: Usage ● Growth - interaction volume ○ Weekly users performing specific steps in onboarding ○ Monthly active users ● Core Editor - editor actions and upgrade views ○ Weekly users adding pages to their website ○ Most common elements added to sites ○ Upgrade modal views ● Product - use of specific features ○ # of users enabling / disabling a given feature (especially new ones!) ○ Differing rates of feature usage by AB test
  10. 10. Events Explore: Visualization (Example Data) ● Count by events, sessions, users ● Many dimensions and ways to segment (below just snowplow segments)
  11. 11. Events Explore: LookML
  12. 12. Funnel Explore: Usage ● Growth - onboarding and conversion ○ Landing Page → Signup → Themes Choice → Editor View ○ Domain Modal → Domain Selection ○ Upgrade Modal → Transaction ● Core Editor - initial editor interaction ○ Signup → Editor Interaction ○ Signup → Viewing / Adding Pages ● App Center - app installation ○ App Center Page View → View app → Install app
  13. 13. Funnel Explore: Filters
  14. 14. Funnel Explore: Visualization (Example Data)
  15. 15. Funnel Explore: LookML (truncated) See Appendix for more detailed code samples, this slide for the special filter for exclusions Based off: https://looker.com/blog/modeling-conversion-funnels-in-looker-a-mysql-subselect-approach view: user_funnel { derived_table: { sql: select s0.blended_user_id, s0.s0_tstamp, min(s1.derived_tstamp) as s1_tstamp from ( select blended_user_id, min(derived_tstamp) as s0_tstamp from snowplow_pivots.events_last_60 where {% condition step_1_event_type %} event {% endcondition %} … and {more conditions here} and blended_user_id NOT IN {special filter for exclusions here} group by 1 ) s0 left join snowplow_pivots.events_last_60 s1 on s0.blended_user_id = s1.blended_user_id and s1.derived_tstamp >= s0.s0_tstamp and {% condition conversion_window %} and {% condition step_2_event_type %} event {% endcondition %} and …. {more conditions here} group by 1,2;; }
  16. 16. Retention Explore: Usage ● Growth - track early retention ○ % of signups visiting on day 2 ○ % of signups visiting in the first week ● Analytics - deep dives ○ Impact of product launches on retention ○ Differential retention between SEM/SEO/etc ○ Differential retention between AB tests
  17. 17. Retention Explore: Filters
  18. 18. Retention Explore: Visualization (Example Data) Looker automatically does the unique calculation % of signups that visited in days 2-7 is the blue box divided by the red box Calculated fields allow a “single value” to populate a dashboard
  19. 19. Retention Explore: LookML (truncated) See Appendix for full code derived_table: { sql: with xxx as ( select blended_user_id, min(derived_tstamp) as first_tstamp from snowplow_pivots.events_last_60 where {% condition initial_event_type %} event {% endcondition %} and {more conditions} group by 1) select e.blended_user_id, coalesce(e.derived_tstamp, xxx.first_tstamp) as derived_tstamp, xxx.first_tstamp, event, page_type, category, action, label from snowplow_pivots.events_last_60 e left join xxx on e.blended_user_id = xxx.blended_user_id where ( {% condition recurring_event_type %} event {% endcondition %} and {more conditions} and xxx.first_tstamp - '0 second'::interval < e.derived_tstamp ) or ( {% condition initial_event_type %} event {% endcondition %} and {more conditions} and xxx.first_tstamp = e.derived_tstamp) ;; }
  20. 20. First Actions Explore: Usage ● Growth - onboarding timing ○ Signup → Template Selection ○ Signup → Publish ● Core Editor - editor timing ○ First Editor View → Publish ● Analytics - differential timing by AB test ○ Signup → First Editor View for different onboarding flows
  21. 21. First Actions Explore: Visualization (Example Data)
  22. 22. First Actions Explore: LookML
  23. 23. Learnings - the Good ● Users huge on refreshable dashboards ● See events data and other data side by side in dashboards ● Join to AB segmentation data ○ Funnel drop off by AB segment ○ Event rate by AB segment ● Join to user data ○ Exclude banned users ○ Has best guess to country / location data
  24. 24. Learnings - the Challenges ● Timing has been tricky for us since Snowplow starts at 5 PM PST. Often set up dashboards to hide the partial day of data / tell users data is 2 days behind. ● Events can be slightly out of order - added some “slop” to our funnels to account for this ● Looks can take minutes each to run (depends on Redshift load). Opening a large dashboard causes a sudden large load on the servers. ○ We persist queries for 8 hours to help with this ○ Still have outstanding user requests to reduce latency ● Explores are complicated - working on getting all PMs creating their own looks
  25. 25. We’re hiring! https://careers.weebly.com/ Questions? audrey@weebly.com bo@weebly.com
  26. 26. Appendix: LookML for User Funnels & Retention Funnels ● Copied directly from Looker ● Tables / fields to know ○ snowplow_pivots.events_last_60 - enriched table with data for past 60 days ○ blended_user_id - internal user id to the best of our knowledge, otherwise cookie-based user id ○ derived_tstamp - snowplow “best guess” timestamp, see http://snowplowanalytics.com/blog/2015/09/15/improving-snowplows-understan ding-of-time/
  27. 27. User Funnel LookML: Main derived table
  28. 28. + 2 additional joins (s3, which represents 4 steps) User Funnel LookML: Example join to following step
  29. 29. User Funnel LookML: Excluding users based on events This is the “
  30. 30. User Retention LookML

×