Insight Demo

1. Analyzing Analysts
Dustin McIntosh
Mode Analytics
Understanding the Queriers Behind the Queries

2. Goal: Understand user base and how
they interact with the product
Novices
Power Users
● Who are the users?
● Which SQL errors do they make?
● How can we make users'
experience better?
● Why do users churn?
● Can we classify users' ability
level based on their queries?

3. Goal: Understand user base and how
they interact with the product
● Who are the users?
● Which SQL errors do they make?
● How can we make users'
experience better?
● Why do users churn?
● Can we classify users' ability
level based on their queries?
Novices
Power Users

4. Time
Churn – Are users getting frustrated?

5. Churn – Are users getting frustrated?

6. Churn – Are users getting frustrated?
No!

7. Churning Users = Tutorial Users
Recommendation: prompt tutorial users to connect their
own data at tutorial conclusion

8. Classifying Users Based on the
Content of their Queries
Bag of
SQL
keywords
Formatting based:
– White space
– Parentheses
Keyword diversity
Is it an Error?
Error type
Random
Forest

9. Classifying Users Based on the
Content of their Queries
Bag of
SQL
keywords
Formatting based:
– White space
– Parentheses
Keyword diversity
Is it an Error?
Error type
Random
Forest
AUC = 0.66

10. The differences between experts
and novices
Deliverable: model that classifies users' ability based on queries

11. Dustin McIntosh
EFavDB.com

12. Extra Slides

13. What errors do they make?
Recommendation: prominently display or auto-correct table/column
names

14. Top-heavy nature of SQL

15. Non-tutorial users also not getting
frustrated
All churned users Churned users that have
connected their own data

16. Correlations of the Features

17. Compare Using top 5 features
All features
Top 5 features

18. Histograms of important variables
novices
experts