Artificial intelligence can help address societal problems in areas like healthcare, education, financial inclusion and more. The document discusses two examples: (1) Using machine learning to help tuberculosis health workers prioritize patients by predicting adherence and outcomes, and (2) Analyzing online mental health forums to understand effective peer support and identify "moments of change". While correlations can be misleading, focused work with domain experts can produce models that achieve accuracy over 70% in predicting moments of change. Overall, AI is an amplifier that must be developed responsibly with societal impact in mind.

1. AI FOR SOCIETAL
IMPACT
Amit Sharma
Researcher, Microsoft Research India
@amt_shrma
http://www.amitsharma.in
MICROSOFT AXLE, AI FOR GOOD

2. Artificial Intelligence is having a
transformative impact

5. AI for Societal Impact?
Can we use recent advances in Machine learning and data science
to help with societal problems?
• Healthcare
• Education
• Financial Inclusion
• Governance
• Public awareness
• Environmental Sustainability

6. Yes, but correlations and predictions can lead us astray.
Can we use recent advances in Machine learning and data science
to help with societal problems?
•Medication adherence forTuberculosis
•Counselling support in mental health forums

7. • Suffers from same limitations
Image credit: Emre Kiciman

8. PART I: HELPINGTB HEALTH
WORKERS BE MORE EFFECTIVE
In collaboration with Everwell, a startup based on a Microsoft Research
project, 99Dots.
Learning to Prescribe Interventions forTuberculosis Patients using
Digital Adherence Data. Killian et al. (2019)
https://arxiv.org/abs/1902.01506

9. Helping patients adhere toTB treatment
is important but difficult
•TB is the leading infectious cause of death globally
•TB treatment takes 6 months or more
•Poor adherence to treatment increases risk of relapse, drug
resistance, and death
•India’s governmentTB program has used Directly Observed
Treatment (DOT) to monitor adherence, but effort-intensive
for patients and providers

10. Possible solution: 99Dots, a digital
adherence technology
https://www.99dots.org/

11. Background: How 99Dots works
* Slide content sourced from Everwell.

12. Combination of Caller
ID and numbers called
shows that doses are
in patient’s hands.
Background: How 99Dots works
* Slide content sourced from Everwell.

13. Background: Rapid growth with Govt. of India
* Slide content sourced from Everwell.

14. Background: Reminders & analytics on 99Dots
ICT enables remote observation + analytics:
 Reduces patient burden
 Increases provider efficiency
 Enables differentiated care
Two of your
patients have
missed doses
today: Raj & Om
SMS Message:
Two of your
patients have
missed doses
today: Raj &
Om
[0000]
Please
take pills
SMS Message:
[0000] Please
take pills
Reminders to Patients Alerts to Providers Analytics for Supervisors
* Slide content sourced from Everwell.

15. How canAI/Machine learning be useful?

16. Q1: Can we predict a
patient’s adherence in
advance?
Q2: Can we predict final
treatment outcome
(after 6 months) based
on initial patient
adherence (first few
weeks)?
GOAL: help health
workers prioritize
their visits to
patients.

17. Data
• Accessed anonymized data on pastTB patients who had used 99Dots
• For each patient, collected their:
• Adherence pattern: Calls made to 99Dots
• Treatment outcome at the end of 6 months:
• Positive: Cured/Treatment Completed
• Negative: Lost to follow up,Treatment Failure, Death
• Preprocessing
• Many people change numbers
• Some calls are manually added
• Some patients’ treatment ends early
• Real-world data is messy!

19. What does adherence look like for patients
with a positive treatment outcome?

20. What does adherence look like for patients
with a negative treatment outcome?

21. 2 Key questions
• How to help health workers reprioritize their interventions?
• Looking at a week’s data, can we predict adherence for the next week?
• Looking at first few weeks, can we predict the final treatment
outcome?

22. Machine learning task
• Input (t-7,t)
• demographic features (age, gender, location)
• Call details (number of calls, time of calls, days between calls,
etc.)
• Output (t, t+7)
• Number of calls in the next week

23. Obtain nearly 0.85 AUC.

24. Tale ofTwo worlds
• Person makes no calls in
week 1, intervention, starts
making calls in week 2
• Person makes no calls in
week 1, intervention, no calls
in week 2

25. A causal model for interventions
Person’s
Behavior (t)
Health worker’s
intervention
Call to 99Dots
(t)
Person’s
Behavior (t-1)
Call to 99Dots
(t-1)

26. Domain-based filtering solution
• 99Dots records suggested attention level for each patient
• High: 4 or more calls missed in the last week
• Medium: 1 to 4 calls missed in the last week
• Low: No missed calls
• Medium -> High?
• Given last week’s data, can we predict whether a person moves from
Medium to High attention ?

27. Three models:
1. Number of calls
missed
2. Random Forest model
3. Deep neural network
(LSTM+dense layer)

28. Complex model, but are able to save
more missed doses

30. PART II: COUNSELLING
SUPPORT FOR MENTAL
HEALTH
In collaboration withTalklife, a global mental health forum
Moments of Change: Analyzing Peer-Based Cognitive Support in Online
Mental Health Forums. Pruksachatkun, Pendse and Sharma (ACM CHI 2019)

32. Mental health:A neglected problem
• 10-15% of Indians are suffering from common mental
disorders
• Depresssion, Anxiety, Post-Traumatic Stress Disorder, …
• 5000 psychiatrists and 2000 clinical psychologists, almost all in
urban India
• 0.06% of govt. health budget spent on mental health
National Mental Health Survey of India 2015–2016. Murthy.
Prevalence of Mental Disorders in India and Other South Asian Countries (2017). Ranjan and
Asthana.

33. Varied reasons
Agrarian distress
Violence against women and
adolescents
Traumatic experiences
Urban lifestyle
Work pressure, exam
pressure
Lack of job opportunities, …

34. How can AI/machine learning help?

35. How can AI/Machine learning help?
• Help us understand how people support each other for mental
health issues
• What are generalizable signs of effective support?
• How can we route experienced counsellors to people who need the
most help?

36. Talklife: thousands of “counselling”
conversations online
• A social network for peer
support
• People experiencing mental
distress can post onTalklife and
get support from their peers.
• Global network, but also has
Indian users
Can we identify patterns of successful peer support conversations?
“Moments of cognitive change”

38. Ground-truth: Moments of change
• Various definitions in clinical psychology.
• Need a quantitative definition.
Moment of change: A change in sentiment for the original poster
on a topic that they initially talked about.

39. How do we get labelled data?
"Thank you very much sir! But every now and then somebody asksWhat's there in what you are
doing?You at least had IIT label. I may be negative a bit but that's a bitter truth that IIT and DU can't
be compared in any way "
Sentiment
-3 -2 -1 0 1 2 3
Topics
___________________________(Inter-rater reliability was 0.4437 Fleiss’ Kappa)
Also collected high-precision, large-scale labels based on common phrases, e.g.,
“Thanks, I feel better now!”

40. Can we predict forum threads that have a
moment of change?

41. What Factors May Feed into a MOC? (Baseline)
● LIWC (Linguistic Inquiry andWord Count)-based features [1]
● Punctuation-based Features
● Mental Health Language Based (swear words, n-grams for anxiety,
depression, and suicide). [2]
● Metadata-based
References
[1] Pennebaker, J.W., Boyd, R. L., Jordan, K., and Blackburn, K.The development and psychometric properties of liwc2015.Tech. rep., 2015
[2] Baumgartner, J.Complete public reddit comments corpus.

42. Can identify moments of change with accuracy = 0.90
Wait, correlations can lead us astray!

43. What happens with different
demographics?
Indicates a difference in how people from different cultures express mental health,
supported by past medical anthropological work.

44. Do we really understand the mechanisms of
what is going on in these threads?

45. Post: “I don’t understand why my mom, or any of
my friends like me. I’m such a bad person. I don’t
deserve their love.”
Collected label: [“mother”, “boyfriend”, “
“sister”, “dog”, “father”, “relationship”]
Here’s an example

46. Take 2: Sense2Vec + Clustering-Based

47. Step 1: Merging
Ears
Ach
e
Eyes
Ears
Cold
Legs
Ears
Legs
Cold
Eyes
Ache

48. Step 2: Pruning; Iterate.
Ears
Teacher
Eyes Ears
Eyes

49. Detect topics and sentiment simultaneously

50. “Senti-Topic” can detect moments of
change with 70-80% accuracy
Given a thread conversation,
does it contain a moment of
change?
• Accuracy > 0.8
Given a post, does it contain a
moment of change?
Do not use text of the post, but
posts in the same thread before it
• Accuracy > 0.7

51. Practical implications
• Routing support: Experienced supporters or counsellors can be routed in
real-time to conversations that are not going well.
• Helping supporters be effective: Can identify supporters who are
participating in conversations with successful moments of change and help
to design personalized training for those who are less effective.
• Cross-cultural implications: Need culture-specific models of prediction.
Same model unlikely to work.
• Currently trying to interpret the models to understand what topics, and
linguistic markers of peer support are more associated with moments of
change.

52. CONCLUDINGTHOUGHTS

53. What’s different this time with AI?
Technology and Societal Impact
• Computers[1990s]
• Internet [2000s]
• Mobile phones [2010s]
At Microsoft Research India, we have been working on
technology for societal problems for more than a decade.
“Technology is an amplifier of social forces.”
-- KentaroToyoma, Geek Heresy

54. How to makeAI/ML useful: Our efforts at
Microsoft Research
-- Finding the right problem where ML can have impact
--Working with domain experts/organizations that have deep expertise
-- Focus on “Implement, Deploy, test”
-- Have the right expectation of “success”
MSR Collaborative projects with academia, social enterprises and NGOs.
https://www.microsoft.com/en-us/research/event/msr-india-call-for-
collaborative-projects-on-cloud-and-ai-technologies/

55. Thank you!
Amit Sharma
@amt_shrma
http://amitsharma.in
MSR India Collaborative Projects 2018-19
Papers:
1. Learning to Prescribe Interventions for
Tuberculosis Patients using Digital
Adherence Data. Killian et al. (2019)
https://arxiv.org/abs/1902.01506
2. Moments of Change: Analyzing Peer-Based
Cognitive Support in Online Mental Health
Forums. Pruksachatkun, Pendse and Sharma
(ACM CHI 2019)