Bounding Knowledge Decay From Temporal Extrapolation

Causal generalization is essential to contemporary political science practice. We argue that recent methodological advances in causal generalization pay insufficient attention to issues which arise from generalization over time. For assumptions of varying degrees of strictness, we derive novel statistical bounds of the growing uncertainty of a given causal estimate into the future. We derive these bounds using the Wasserstein divergence which allows us to weaken assumptions of positivity which are not typically met in practice.

In two empirical examples, we demonstrate that actual variation in treatment effects over time tends to dominate reported statistical uncertainty. The theoretical bounds we derive are shown to be quite wide, indicating that the uncertainty inflation that comes from temporal extrapolation is a major problem. 

Once implicit and untenable generalization assumptions about unchanging covariate distribution and conditional treatment effects are made explicit and relaxed, the value of individual research designs which make weaker/fewer assumptions becomes unclear. We discuss implications for research practice in political science, and argue that richer descriptive knowledge is essential for reducing the uncertainty from temporal causal generalization.