This document describes a system to summarize a Stack Overflow question and provide recommendations to help answer it more quickly. It uses natural language processing and machine learning models like logistic regression and associate learning. The system extracts data from Stack Overflow, analyzes questions to predict response times, and recommends related questions based on common words to help users get answers within 24 hours.

1. ASK
STACK OVERFLOW
Eric Yun Ding
Louisiana State University

2. Want to get your question answered quicker?
Stack Overflow public data
2016-02-01 ~ 2016-03-06
30%
45%
26%
never-answered
answered
within 24 hours
answered
after 24 hours

3. Web Application
http://www.stackflows.space/

4. Extraction Transform and Load
python SQL
parse 66.5G filter
Luigi
Beautiful Soup

5. Data
1 if < 24 hours else 0
Features

6. Models
Prediction engine

7. Models
Prediction engine
Recommendation
engine

8. Prediction Engine
Words
Stemming
Tfidf
Truncated SVD

9. Prediction Engine
Words
Stemming
Tfidf
Truncated SVD
Supervised learning
(Logistic Regression)
Probability

10. Recommendation engine
Baskets of words

11. Recommendation engine
Baskets of words
Associate Learning
1. regex python numpy
2. javascript python
3. …

12. Recommendation engine
Baskets of words
Associate Learning
1. regex python numpy
2. javascript python
3. …
Too many?
How to rank?

13. Recommendation engine
Baskets of words
Associate Learning
1. regex python numpy
2. javascript python
3. …
Too many?
Rank by probability
Probability
engine

14. Model Performance
AUC = 0.671

15. Eric Yun Ding
Louisiana State University

16. Backup slides

21. Distribution of Response Time

22. Accuracy of Linear Regression

23. Accuracy of Random Forest Regressor

24. Model Performance
AUC of various classification models
Title, Body, Tags
Tags
Badges
0 0.25 0.5 0.75 1

25. Models
Words
Stemming
Tfidf
Supervised learning
(Logistic Regression)
Truncated SVD
Probability
Baskets of words
Associate Learning
1. regex python numpy
2. javascript python
3. …
Too many?
Ranked by
Recommendations