Self-supervised Causal Effects Estimation

Targeted maximum likelihood estimation for psychological research: From causal identification to statistical inference.

Causal questions have long been central to psychological research, particularly in randomized experiments, while formal causal-inference methods are increasingly being applied to observational and quasi-experimental data. Common outcome-regression and propensity-score approaches can be sensitive to nuisance-model misspecification, whereas flexible machine learning alone does not ensure valid target-parameter estimation or inference. Targeted maximum likelihood estimation (TMLE) addresses this estimation problem by combining outcome and treatment information through an efficient-influence-function-based update, yielding doubly robust estimation and influence-function-based inference under appropriate conditions. TMLE targets a parameter of the observed-data distribution; causal interpretation additionally requires a well-defined intervention, causal model, and identification assumptions. Although the Causal Roadmap and targeted-learning literature establish the general framework, guidance for modern applied implementation remains fragmented. This research integrates causal-to-statistical parameter mapping, accessible semiparametric explanation, reproducible Python code, positivity diagnostics, fully nested cross-fitting, and TMLE-specific reporting guidance for psychological researchers. The formal exposition and simulations use the point-treatment average treatment effect for a binary treatment or exposure as an illustrative target. The two empirical examples are instead framed as statistical implementation exercises targeting covariate-standardized observed-data risk differences, with the assumptions under which those parameters would equal causal effects stated separately. The examples contrast transparent parametric estimation with cross-fitted Super Learner TMLE, while the simulations examine nuisance-model misspecification, limited overlap, propensity-score truncation, and flexible TMLE with and without cross-fitting. The contribution is translational rather than theoretical: TMLE is presented as the estimation stage of a broader causal-inference workflow, not as a procedure that independently establishes causality.

A Data-Centric Decomposition of Estimator Performance in Continuous Treatment Effect Estimation

This work analyzes current benchmarking practices and introduces a novel decomposition framework that disentangles the contribution of distinct data-generating components, such as confounding, dose distribution non-uniformity, and response surface complexity, to estimator performance.

Causal State-Space Model for Causal Inference: Estimating Longitudinal Individual Treatment Effects

Estimating counterfactual outcomes over time from longitudinal observational data is central to clinical decision support. Existing methods rely on domain confusion -- adversarial training that renders representations invariant to treatment assignment -- yet this invariance creates a mutual information conflict: it suppresses treatment-correlated covariate signals necessary for accurate outcome prediction. We formalise this tension via a Jensen-Shannon divergence bound on counterfactual prediction error and develop two complementary models. CSSD (Causal State-Space model with Direct decoder) adapts selective State Space Models with a parallel multi-step decoder that eliminates accumulated rollout error by producing all prediction horizons simultaneously in a single forward pass. CSSPD (Causal State-Space model with Predictive regularisation and Direct decoder) augments CSSD with Contrastive Predictive Coding and Local Information Maximisation to reinforce temporal predictability in the balancing representation and recover local covariate information destroyed by domain confusion. On MIMIC-III, CSSPD achieves lower counterfactual RMSE than the Causal Transformer at every horizon tau>= 2 at O(T) encoder cost, with gains from 0.02 (2-step) to 0.07 (6-step). On Cancer Simulation across confounding strengths gamma in {0,1,2,3,4}, CSSPD outperforms CT at gamma<= 3 (margins 25.9%--37.0%), and CSSD achieves the lowest overall average RMSE (12.7% reduction over CT), confirming the MI conflict analysis. To our knowledge, this is the first work to formalise the balancing-prediction MI conflict and propose a structured resolution through complementary predictive and information-theoretic training objectives.

When and Why LLM Causal Priors Help: Closed-Loop Prior Selection for Amortized Causal Inference

Causal effect estimation asks how an outcome would change under an intervention, and medicine, economics, and public policy all treat it as a foundational task. Prior-data fitted networks (PFNs) amortize the task: a model trained on large numbers of programmatically generated synthetic causal tasks reads a new problem's observational data into context and returns an interventional-effect estimate in a single forward pass. The capability of such models is largely determined by the synthetic training prior, which is currently designed by hand, a bottleneck acknowledged by both Do-PFN and CausalPFN. Large language models (LLMs) can now ``draw''plausible causal graphs for a given domain, suggesting that LLM-distilled graphs could serve as prior material. Whether injecting such graphs helps at all, where any gain comes from, and when injection helps. Practice has so far relied on manual trial and error. We propose a \emph{closed-loop prior selection framework} that casts prior injection as a budget-constrained optimization over a candidate prior pool. Candidates undergo cheap post-training and are scored by a composite metric dominated by real-domain generalization; the winner then receives full training and paired statistical validation. On a 7.34M-parameter Do-PFN, the framework's winner attains a formally significant $2.75\times$ gain on the primary evaluation domain, and its error falls below that of the uninjected official base. Generalization on an adjacent monitoring domain improves significantly, and no monitored capability degrades. Mechanism experiments show that the gain depends on the semantic content of the distilled graph rather than its structural diversity alone does not produce it (directional evidence). With this framework and this regularity in hand, the use of LLM causal priors stops being manual trial and error and becomes an empirically verifiable selection problem.

Assessing Unmeasured Confounding in Observational Research via Bayesian Semiparametric Modeling

Information-Directed Sampling for Causal Bandits

Causal bandits exploit structural relationships among variables to share information across interventions and accelerate the identification of high-reward decisions. In many applications, however, some variables cannot be directly manipulated, even though they influence the reward and provide useful information about the underlying causal system. We study contextual causal bandits with non-manipulable variables, where context variables are observed before action selection and additional variables are observed after each intervention. Assuming a known causal graph without latent confounding, we adopt a Bayesian formulation in which the conditional probability tables of the observational distribution constitute the unknown parameter. This representation allows observations collected under one intervention to update reward estimates for other interventions through their shared causal mechanisms. We develop causal variants of Thompson Sampling and Information-Directed Sampling (IDS) for this setting. For Thompson Sampling, we establish an entropy-dependent sublinear Bayesian regret bound. For IDS, we derive an entropy-dependent regret bound that explicitly quantifies the additional error introduced by Monte Carlo approximation of the expected regret and information gain; when these quantities are available exactly, the bound recovers the standard sublinear IDS rate. We further provide high-probability confidence bounds for the Monte Carlo estimates used by the algorithm. Experiments on several synthetic causal bandit tasks show that the proposed methods outperform causal and non-causal baselines by more effectively exploiting information shared across interventions.