σ StatsDoge Causal inference workflows
59peers Log in Sign up
11
Workflow·6 steps·branched

Assessing heterogeneity with RATE (AUTOC & Qini)

@grf ✓ P D · Jun 2, 2026 · 1k views
Causal ForestHeterogeneous Effects
Source grf — Athey, Tibshirani & Wager
Summary by StatsDoge

Causal forest → train/eval split → RATE with both AUTOC and Qini → TOC plot.

1

Input · what goes in

A fitted causal forest plus a held-out evaluation sample (X, W, Y).

Show data format & exampleHide example

Format — one row per unit. A covariate matrix X (numeric), a binary treatment W ∈ {0,1}, and an outcome Y.

  X1     X2    X3    W    Y
 0.42  -1.1   0     1   3.10
-0.07   0.6   1     0   1.85
 1.20   0.3   0     1   4.02
2

Pipeline · the recipe ⑂ has parallel branches

↑ Click any step in the diagram to read its logic, code, assumptions & discussion.

1
Data prep

[GRF] Regression forest

Data preparation — shapes the raw inputs into what the estimator expects.

What happens here

Cross-fit nuisances.

Formula
\hat\mu(x)=\mathbb{E}\!\left[\,Y\mid X=x\, ight]
The estimator

Regression forest — Honest non-parametric regression for E[Y|X], with out-of-bag predictions and pointwise CIs.

OTHER Code @ v2.6.1 ↗ Paper ↗
Reads from the input data Feeds into #2
Key code 
rf <- regression_forest(X, Y)
Y.hat <- predict(rf)$predictions
Discussion on this step (0)
  • No comments on this step yet — be the first.
2
Estimation

[GRF] Causal forest

The core estimate — where the causal quantity itself is computed.

What happens here

Causal forest on the training half; produce CATE priorities.

Formula
au(x)=\mathbb{E}\!\left[\,Y(1)-Y(0)\mid X=x\, ight]
The estimator

Causal forest — Honest random forest for heterogeneous treatment effects — CATE for a binary treatment via GRF moment conditions.

CATE Code @ v2.6.1 ↗ Paper ↗
Reads from #1 Feeds into #3
Key code 
cf <- causal_forest(X, Y, W)          # Y.hat, W.hat cross-fit
tau.hat <- predict(cf)$predictions    # OOB CATEs
Discussion on this step (0)
  • No comments on this step yet — be the first.
3
Data prep

Train / evaluation split

Data preparation — shapes the raw inputs into what the estimator expects.

What happens here

Hold out an independent half to fit the evaluation forest — RATE needs out-of-sample priorities.

Reads from #2 Feeds into #4#5
Key code 
idx <- sample(n, n / 2)            # independent train / eval split
train <- data[idx, ]; eval <- data[-idx, ]
Discussion on this step (0)
  • No comments on this step yet — be the first.
4
Heterogeneity

[GRF] Rank-weighted ATE — RATE / AUTOC / Qini

Heterogeneity — who is affected, and by how much, not just on average.

What happens here

AUTOC: area under the TOC, sensitive to concentrated benefit.

Formula
\mathrm{AUTOC}=\int_0^1\!\mathrm{TOC}(q)\,dq,\quad \mathrm{TOC}(q)=\mathbb{E}\!\left[ au(X)\mid \hat S(X)\ge \hat F^{-1}(1-q) ight]-\mathbb{E}[ au(X)]
The estimator

Rank-weighted ATE — RATE / AUTOC / Qini — Evaluate how well a CATE estimate prioritizes treatment: TOC curve, AUTOC and Qini with confidence intervals.

OTHER Code @ v2.6.1 ↗ Paper ↗
Reads from #3 Feeds into #6
Key code 
rate <- rank_average_treatment_effect(eval.forest, priorities = tau.hat)
plot(rate)                          # TOC curve
rate$estimate / rate$std.err        # AUTOC z-stat
Discussion on this step (0)
  • No comments on this step yet — be the first.
5
Heterogeneity

[GRF] Rank-weighted ATE — RATE / AUTOC / Qini

Heterogeneity — who is affected, and by how much, not just on average.

What happens here

Qini: weights by treated fraction, sensitive to broad benefit.

Formula
\mathrm{AUTOC}=\int_0^1\!\mathrm{TOC}(q)\,dq,\quad \mathrm{TOC}(q)=\mathbb{E}\!\left[ au(X)\mid \hat S(X)\ge \hat F^{-1}(1-q) ight]-\mathbb{E}[ au(X)]
The estimator

Rank-weighted ATE — RATE / AUTOC / Qini — Evaluate how well a CATE estimate prioritizes treatment: TOC curve, AUTOC and Qini with confidence intervals.

OTHER Code @ v2.6.1 ↗ Paper ↗
Reads from #3 Feeds into #6
Key code 
rate <- rank_average_treatment_effect(eval.forest, priorities = tau.hat)
plot(rate)                          # TOC curve
rate$estimate / rate$std.err        # AUTOC z-stat
Discussion on this step (0)
  • No comments on this step yet — be the first.
6
Reporting

TOC plot + AUTOC/Qini table

Reporting — turn the numbers into a figure or table a reader can act on.

What happens here

Plot the TOC with CIs; report both summaries and whether each clears zero.

Reads from #4#5 Feeds into the final output
Discussion on this step (0)
  • No comments on this step yet — be the first.
3

Output · what you get 4 figures

A sample TOC curve: the average benefit as you treat the highest-ranked fraction of units.
Fig 1A sample TOC curve: the average benefit as you treat the highest-ranked fraction of units.
AUTOC is the shaded area under the TOC — one number for how much heterogeneity you can exploit.
Fig 2AUTOC is the shaded area under the TOC — one number for how much heterogeneity you can exploit.
Event times in the SPRINT/ACCORD trials, showing the right-censoring the estimator has to handle.
Fig 3Event times in the SPRINT/ACCORD trials, showing the right-censoring the estimator has to handle.
TOC curves estimated across trials — does the same prioritisation rule transport?
Fig 4TOC curves estimated across trials — does the same prioritisation rule transport?

Figures reproduced from grf — Athey, Tibshirani & Wager — unofficial community showcase; all credit to the original authors.

The GRF 'Assessing heterogeneity with RATE' tutorial. Fit on one half, evaluate prioritization on the other; report AUTOC (rewards concentrated heterogeneity) and Qini (rewards broad effects) with CIs. Unofficial summary.

Discussion (2)

  • 3
    @rate_rosa · Jun 2, 2026

    Running both AUTOC and Qini side by side is the right move — they answer different targeting questions and disagreeing is informative.

    2
    @qini_quinn · Jun 2, 2026

    Yep. If AUTOC fires but Qini doesn't, your benefit is concentrated in a small group.

  • 3
    @kfold_kenji · Jun 2, 2026

    The independent train/eval split is non-negotiable here. Priorities have to be out-of-sample or RATE is optimistic.