Presentation in the Web Science track at WWW'17. Full paper https://ingmarweber.de/wp-content/uploads/2017/04/A-Warm-Welcome-Matters-The-Link-Between-Social-Feedback-and-Weight-Loss-in-r-loseit.pdf. Work led by Tiago Cunha (https://twitter.com/tocunha). Abstract of paper: Social feedback has long been recognized as an important element of successful health-related behavior change. However, most of the existing studies look at the effect that online social feedback has. This paper fills gaps in the literature by proposing a framework to study the causal effect that receiving social support in the form of comments in an online weight loss community has on (i) the probability of the user to return to the forum, and, more importantly, on (ii) the weight loss reported by the user. Using a matching approach for causal inference we observe a difference of 9 lbs lost between users who do or do not receive comments. Surprisingly, this effect is mediated by neither an increase in lifetime in the community nor by an increased activity level of the user. Our results show the importance that a "warm welcome" has when using online support forums to achieve health outcomes.

1. A Warm Welcome Matters!
The Link Between Social Feedback and
Weight Loss in /r/loseit
Tiago Cunha1 Ingmar Weber2 Gisele Pappa1
1Federal University of Minas Gerais
2Qatar Computer Research Institute
@tocunha @ingmarweber @glpappa

2. Obesity Epidemic
• 30% of world population is overweight or obese.
• Obesity increases the probability of chronic diseases.
• Previous studies have tied successful weight loss to social support.

3. Measuring Importance of Social Support
We test the effect using
observational data.
• Needs access to large platform
• Would have to withhold social support

4. Matching in a Nutshell
Among given “organic” data (e.g. human trace data), can we
find a subset that looks like generated by an experiment?
matching == pruning

5. Ho, Daniel, Kosuke Imai, Gary King, and Elizabeth Stuart. 2007. “Matching as Nonparametric Preprocessing for Reducing Model Dependence
in Parametric Causal Inference.” Political Analysis 15: 199–236. Copy at http://j.mp/jPupwz
Position
education (in years)
Imbalance => Model Dependence
Position
education (in years)
Q: Are the T’s higher than the C’s,
while correcting for education?
A: It depends on your model!

6. Solution: Prune to Get Balance
position(p)
education (e)
Use simple model to estimate expected average treatment effect (EATE):
100 * (Average(T) – Average(C)) / Average(C)
Matching has reduced
model dependence!

7. Framework for Causal Inference
Step 0: Obtain data!

8. Differences to Standard Approaches
• We do not use Propensity Score Matching
– See King and Nielsen (2015) for reasons
• Our matching occurs directly on the variables
– Ensures balance, but “curse of dimensionality”
• We apply a mediation analysis (Sobel Test)
– X causes Y, but is that via Z?

9. /r/loseit Community

10. Support for Newcomers
So I've been working on losing weight since December, but since June I've been in a rut :(
3 points 0 comments submitted 4 years ago by moonyDP to r/loseit
Okay, so I was diagnosed back in December with GERD, and my doctor told me it would help to lose weight. I'm 5' 8" and, at the time, was
around 175-180. So, kind of embarrassed that my doctor told me I needed to lose weight, I went along with her diet with great enthusiasm -
no fats, no eating 2-3 hours before bed, no alcohol, etc. etc. So, by the end of the school year, I had gotten down to 150!…
I'm 23 and weigh 550lbs. Please help
455 points 204 comments submitted 2 months ago by Ecurtis936'5" 550Lbs Male to r/loseit
Starting weight: 550lbs Goal Weight: 250lbs
Just to tell you a little about myself; I'm 23 years old 6'5" and sadly weigh 560lbs. I work at a call center, sitting in a desk for 10 hours a day.
I leave work to go home and play video games until its time for me to go to bed. Today my 3 year old son ask me " Daddy.. Why are you so
fat?" I couldn't answer. I went into my room and cried. Now i'm writing this post. I'm not asking for you to feel bad for me, Because I know I
put myself in this situation.
I guess what I'm really asking, is for guidance. I have so many questions about workouts, dieting, nutrition, schedules, etc...

11. Data Collection (Step 0)
• 5 years of data (August 2010 to October 2014)
• 107,886 unique users
• 70,949 posts and 922,245 comments
• Metadata (timestamp, user name, voting score and
history of badges)

12. Define Treatment and Control (Step 1)
• Look at first post of a user in the community
• Treatment = received comments
• Sparsity: 96% of posts received a comment
• Re-Define:
– Treatment = received at least 4 comments
– 4,657 treatment and 1,468 control

13. Covariates Choice (Step 2)
• Matching only balances matched variables
– Important choice of what to match on
• Build LASSO regularized model to predict receiving “treatment”
• Use LDA topics, LIWC, Question words, posts size, sentiment
– 98 variables in total
• Final model 20 variables (selected by LASSO)
• Use coefficient values as covariates weights

14. Prune by Matching (Step 3)
• Use cosine similarity for matching
– Weighted by LASSO coefficients
• Use 1-to-Many matching
– To avoid throwing out data
• Use a caliper to only keep “similar enough” matches
– Extreme case: exact match

15. Balance Check (Step 4)
• Compute standardized mean difference
Small dc = similar values of c in treatment and control group
Remaining bias for variable c is considered to be insignificant if dc is smaller than 0.1
Note: don’t use a significance test! Else “too little data => no significant difference”

16. Estimate Effect Size (Step 5)
• Effect on return rate
25,647 users present in Group 1. 18,000 treatment and 7,647 control.
Balance check Effect size

17. Estimate Effect Size (Step 5)
• Effect on weight loss
6,143 users present in Group 2. 4,657 treatment and 1,468 control.
26%, or an absolute mean difference of 9 lbs.
Balance check Effect size

18. Mediation Analysis (Step 6)
• Used a Sobel Test to check for mediation
• No statistically significant mediation effect found
Social Feedback Weight Loss
Engagement in
Community

19. Qualitative Evidence
Community survey: “What do you like about /r/loseit?”

20. After Our Study

21. Summary
• To the best of our knowledge, first large scale study
using observational data to estimate causal effects
of social support in an online health community
• Correcting for confounding factors, users who
received social support:
– Were 66% more likely to return for a future activity
– Reported on average 9 lb more in weight loss
• This effect is not mediated by:
– an increased level of activity in the community
– a longer lifespan in the community

22. Limitations
• Using badges to track weight loss
– What if they don’t update badges?
• Determining the start of weight loss journey
– What if lost weight before first post?
• Our choice of covariates
– Can only correct for known covariates
• Observability of returning users
– No return does not equal no weight loss

24. Thanks!
Go diving on Rottnest if you have a chance.