Halim Abbas

1. Early Screening for Autism
By Combining Question-Based and Video-Based Predictors
Halim Abbas
VP of Data Science at Cognoa
Predictive Analytics World – Chicago 2017

2. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Autism in America
Latest estimates: 1 in 67 children, 1 in 44 boys
Sources: Autism Speaks, CDC
Prevalence is on the rise

3. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Autism in America
• 18 month wait for specialist visit
• 4.17 Average age of diagnosis
• Earlier intervention during critical
neurodevelopment time window is shown
to improve lifelong outcomes for affected
children (ESDM 12-48months)
Sources: Autism Speaks, CDC, ESDM

4. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Problems with current screening
methods
• Clinical setting
• Expensive
• Time intensive
• Low tech (pencil and paper)
• Too simplistic (sum-codes > threshold)
• Fall short in predictive value

5. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Predictive modeling can help
• Train screeners that are:
– Cheap
– Quick
– Easy to apply
– Scalable
• But need to be at least as reliable as
standard of care screeners out there

6. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Our approach
+
Parent questionnaire
Short home video uploads

7. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Our approach
Item level clinical scoresheets
Corresponding clinical diagnoses
Predictive
modeling
Parent
questionnaire
predictive
model
Item level clinical scoresheets
Corresponding clinical diagnoses
Predictive
modeling
Video analyst
predictive
model
Model
combinator
training

8. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
But there are obstacles
• Training set bias, sparsity, and
shallowness
• Parent and analyst subjectivity
• Training vs. application contexts
• Subjective diagnosis
• False positives are a big deal
• False negatives are even more big of a
deal!

9. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
… which we worked our way around
• Combining parent questionnaire with video
analysis for added confidence
• Support for indeterminate outcome
• Guided clinical data acquisition
• Creative feature selection techniques
• Novel feature encoding technique

10. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Indeterminate outcome
• Autism is a complex multifaceted condition
• Presents in a spectrum of different ways
• Some cases amenable to quick reliable
screening, some not
 Impossible to screen reliably all the time

11. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Indeterminate outcome

12. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Indeterminate outcome
false positives
false
negatives
Typical classifier

13. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Indeterminate outcome
Our classifierinconclusive

14. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Indeterminate Outcome
All samples
Incorrectly classified samples
Learn binary
classifier
Learn autism
classifier
Learn
indeterminate
classifierCorrectly classified samples
-
+
• Tunable for more coverage vs. higher accuracy
• Feature selection/encoding potentially independent per module
• Autism classifier module can focus on easy-to-classify samples
Indeterminate
classifier
Autism
classifier
-
inconclusive
+
-
autism
+
not
autism
Offline Runtime

15. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
… which we worked our way around
• Combining parent questionnaire with video
analysis for added confidence
• Support for indeterminate outcome
• Guided clinical data acquisition
• Creative feature selection techniques
• Novel feature encoding technique

16. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Guided clinical data acquisition
Clinical
instrument
data
Cognoa
user base
data
Large (105 - 106)
Cheap to acquire
Mostly unlabeled
Noisy
Unreliable
We decide what to capture
Small (102 - 103)
Expensive to acquire
Mostly labeled
Clean
Reliable
Standardized info capture

17. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Guided clinical data acquisition
Clinical
instrument
data
Cognoa
user base
data
Can only model with this!
But would like to leverage that!

18. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Guided clinical data acquisition
Clinical
instrument
data
Cognoa
user base
data
Clustering
Imputation
Noise Reduction
Outlier Removal
Infer labels, expose inconsistencies
Infer clusters,
create silos,
expose under-sampled demographics

19. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Validation
Blinded clinical study
162 children (18-72 months old)
Each referred to a participating autism center for a question
of ASD
• University of Missouri Thompson Center
• Vanderbilt University
• Medical University of South Carolina
Each child received full diagnostic tests
Diagnosis ultimately ascertained by a licensed psychologist

20. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Validation

21. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Validation

22. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Future Improvement
Incorporating indirect signals:
– Eye tracking
– Facial affect
– Speech patterns
– Touch interface

23. Early Screening for Autism By Combining Question-Based and Video-Based Predictors Predictive Analytics World 2017
Future Improvement
Multi-stage prediction with fallback
Interactive
questionnaire
+
-
?
Autism
ID
No social/behavioral delay
+
-
?
+
-
?
+
-
?
+
-
?
ADHD
Autism or ADHD
Social/behavioral delay
Social/behavioral delay but not Autism or ADHD
Not sure
Social/
Behavioral
Delay
Autism &
ADHD
Autism
Vs
ADHD
Speech &
Language
Delay
Intellectual
Disability
Not sure Speech &
Language
delay but
no ID
Speech &
Language
delay
No speech or language delay

24. Early Screening for Autism
By Combining Question-Based and Video-Based Predictors
Halim Abbas
VP of Data Science at Cognoa
Predictive Analytics World – Chicago 2017
Questions?