The document discusses the interaction between machine learning and causal inference, advocating for sensitivity analysis as a means to address non-identification issues in latent variable models. It emphasizes the importance of understanding confounders and presents various statistical strategies to improve identifiability in analytical models. Key references and prior works are cited to support its arguments and the exploration of sensitivity analysis is highlighted as crucial for future research.

1. Latent Variable Models,
Causal Inference,
and Sensitivity Analysis
Alex D’Amour
Google Brain
SAMAI Causal Inference Opening Workshop
Dec 8, 2019

2. Goal
Summarize a recent conversation at the intersection of ML
and Causal Inference communities.
Advocate for sensitivity analysis as a constructive way
forward.

4. The Deconfounder Debate

5. Causes Outcome
A = (A(1)
, ... , A(m)
)

9. P(Y | do(A)) is not non-parametrically identified.

12. U factorizes P(A)

13. U factorizes P(A)

14. Factor Model as “Soft” Observation

15. Factor Model as “Soft” Observation

16. Factor Model as “Soft” Observation

17. Apply standard causal adjustment
Lots of ML tools for fitting the factor
model.
Factor Model as “Soft” Observation

18. Methods of this type appear across science, and are
standard procedure (e.g., Price et al 2006 in GWAS).
This paper brings them under the causal
microscope.

19. Methods of this type appear across science, and are
standard procedure (e.g., Price et al 2006 in GWAS).
This paper finally brings them under the causal
microscope.

20. Methods of this type appear across science, and are
standard procedure (e.g., Price et al 2006 in GWAS).
This paper makes the causal argument explicit.

21. The factor model setting buys useful information,
but not sufficient for identification.

22. Identification and Sensitivity

23. Knowing the Audience
The ML community is optimistic.
Presumption: Identified until proven otherwise (this is changing).

24. Knowing the Audience
The ML community is optimistic.
Presumption: Identified until proven otherwise (this is changing).
Proofs of identification.

25. Knowing the Audience
The ML community is optimistic.
Presumption: Identified until proven otherwise (this is changing).
Proofs of identification.
Counter-examples with non-identification.

26. Causal Inference Optimism
In every badly identified model,
there lies a great sensitivity analysis.

27. Causal Inference Optimism
In every badly identified model,
there lies a great* sensitivity analysis.
*Here, “great” means a sensitivity analysis that maps out a true
region of partial identification. See, e.g.,
https://arxiv.org/abs/1809.00399 and cited work.

28. Sensitivity Analysis
Constructive proof of non-identification.
Identify indeterminacy in the model for U, A, Y.
Ignorance region: the set of causal parameters compatible
with observed data distribution.

29. For fancy machine learning, consider
2 x 2 tables

30. Binary with Multiple Causes

31. Binary with Multiple Causes

32. Binary with Multiple Causes

33. Binary with Multiple Causes

34. Binary with Multiple Causes

35. Binary with Multiple Causes

36. Binary with Multiple Causes

37. How much do we know about the confounding?
1. P(U | A) ambiguous.MoreinformationaboutU

38. How much do we know about the confounding?
1. P(U | A) ambiguous.
2. P(U | A) identified.
MoreinformationaboutU

39. How much do we know about the confounding?
1. P(U | A) ambiguous.
2. P(U | A) identified.
3. P(U | A) degenerate.
MoreinformationaboutU

40. Rosenbaum and Rubin 1983
Single cause setting.
All binary.
Parameterize violations of
ignorability by 𝛿 and 𝛼.
Examine range of conclusions
as parameters vary.
Rosenbaum and Rubin 1983, Assessing Sensitivity to an Unobserved Binary
Covariate in an Observational Study with Binary Outcome. JRSS-B.
𝛿𝛼

41. Binary with Multiple Causes
Note: m ≥ 3 and pA
(U) non-trivial ⇒ P(U, A) unique.

42. Rosenbaum and Rubin 1983
Rosenbaum and Rubin 1983, Assessing Sensitivity to an Unobserved Binary
Covariate in an Observational Study with Binary Outcome. JRSS-B.
𝛿𝛼

43. Rosenbaum and Rubin 1983
Rosenbaum and Rubin 1983, Assessing Sensitivity to an Unobserved Binary
Covariate in an Observational Study with Binary Outcome. JRSS-B.
𝛿𝛼

44. P(U, Y | A = a)

45. What if there is uncertainty about the
unobserved confounder?

46. Non-Degenerate Case
Constrained by
observed data
Constrained by
identified
factorization
Sums to 1

47. Non-Degenerate Case
Constrained by
observed data
Constrained by
identified
factorization
Sums to 1

48. Non-Degenerate Case
Constrained by
observed data
Constrained by
identified
factorization
Sums to 1

50. (0,0,0,0,0,0)
(1,0,0,0,0,0)
…
(0,0,0,0,0,1)

54. What if we can recover the confounder?

55. Degenerate Case
U can be reconstructed
from A.

56. Cause Projections

57. Cause Projections
Mean of A across dimensions.

58. Cause Projections
Mean of A across dimensions.
Orthogonal direction

59. Cause Projections
Color: U-value
Opacity: Probability

60. Cause Projections

61. Cause Projections

62. Cause Projections
Confounder can be recovered exactly!

63. Cause Projections
What if we move a red point (U = 0) to the right?

65. Bridging the Gap with Parametric
Assumptions?

66. Beyond Binary

67. Beyond Binary

68. Non-Degenerate Case
Copula c(U, Y | A) is not
identified without further
modeling assumptions.

69. Non-Degenerate Case
Copula c(U, Y | A) is not
identified without further
modeling assumptions.

70. Non-Degenerate Case
Copula c(U, Y | A) is not
identified without further
modeling assumptions.

71. Non-Degenerate Case
Parametric assumptions
about outcome model
necessary to fix the
copula.

72. Degenerate Case

73. Degenerate Case
Left or Right

74. Degenerate Case
Left or Right
Parametric assumptions
needed to break tie.

75. Implications: Non-overlap Regime

77. Moving Forward with Sensitivity Analysis

78. A Template

79. Where do the assumptions live?
ML lives here

80. Where do the assumptions live?
Parameterize for
sensitivity analysis

81. Where do the assumptions live?
Interpretation
hinges on this

82. Going Forward
In every badly identified model, there lies a great sensitivity analysis.
Many threats to identification not covered here.
E.g., partitioning variation between confounders and mediators;
measurement error models (for some of these concerns, see Ogburn et al
comment).

83. Thanks!
AISTATS paper: https://arxiv.org/abs/1902.10286.
JASA comment: https://arxiv.org/abs/1910.08042.
Ogburn et al comment: https://arxiv.org/abs/1910.05438.
Contact:
alexdamour@google.com
@alexdamour

84. More Modest Goals

85. Apologies, U will now be denoted by Z.

86. Two Approaches to “Downsizing”
Effects on the treated
Subset of causes

87. “Consistent” Confounder Recovery (no overlap)
Require
Then missing outcome distributions are identifiable for values of A that map to
same value of Z. Specifically, for the above estimands

88. “Consistent” Confounder Recovery
Require
Then missing outcome distributions are identifiable for values of A that map to
same value of Z. Specifically, for the above estimands
Where’d the factor
model go?

89. The Deconfounder as Detector
Suppose we obtain a function
That renders the causes A mutually
conditionally independent. Then,
Note: Requires overlap in hat z(A, X) to be useful.

90. Alternative Settings

91. Strategy 1: Change the Estimand
Give up on estimating effect of all causes simultaneously.
Set some causes aside. Call these W.
Simpler estimand,
harder to interpret.

92. Strategy 1: Change the Estimand
If P(U | W) degenerate, identified! (Theorem 7).
Technically, no unobserved confounding.
Factor model provides useful
structure.
(Also approach in Price et al 2006)

93. Add a negative control exposure Z.
Strategy 2: Augment the Data
Miao W, Geng Z, Tchetgen Tchetgen EJ. Identifying causal effects with proxy
variables of an unmeasured confounder. Biometrika. 2018.

94. Strategy 2: Augment the Data
Miao W, Geng Z, Tchetgen Tchetgen EJ. Identifying causal effects with proxy
variables of an unmeasured confounder. Biometrika. 2018.
Under completeness conditions, conditional independence
between Z and Y identifies the copula.
Completeness conditions
put strong constraints on
how Z, W represent U.

95. An Identification Result
Kong, Yang, and Wang 2019. Multi-causal causal inference
with unmeasured confounding and binary outcome.
https://arxiv.org/abs/1907.13323.

96. An Identification Result
Kong, Yang, and Wang 2019. Multi-causal causal inference
with unmeasured confounding and binary outcome.
https://arxiv.org/abs/1907.13323.
✔Identified!

97. An Identification Result?
Kong, Yang, and Wang 2019. Multi-causal causal inference
with unmeasured confounding and binary outcome.
https://arxiv.org/abs/1907.13323.

98. An Identification Result?
Kong, Yang, and Wang 2019. Multi-causal causal inference
with unmeasured confounding and binary outcome.
https://arxiv.org/abs/1907.13323.
✘Not identified!

99. Genome <- Geo Ancestry -> Education

100. Thanks!
AISTATS paper: https://arxiv.org/abs/1902.10286.
Thanks to Yixin and Dave for open conversations.
Contact:
alexdamour@google.com
@alexdamour