Standard errors¶

fairlex returns weights, not variances. The way to get standard errors for estimates built on those weights is the one survey statisticians use for any calibration: replicate weights.

Generate replicate base weights with your survey package: a bootstrap, jackknife or BRR from svy (weighting.create_bs_wgts) or R survey (as.svrepdesign).

Calibrate every replicate exactly as you calibrated the full sample:

from fairlex import calibrate, calibrate_replicates

full = calibrate(df, targets, base_weight="w0", total=N, bounds=(0.5, 2))
reps = calibrate_replicates(df, targets, rep_cols, total=N, bounds=(0.5, 2))

Compute your estimate with full.weights and with each column of reps, and combine the replicate estimates with the formula for your replicate type. For a bootstrap, that is the root mean square of the replicate estimates around the full-sample estimate.

Match the bootstrap to the design¶

When the weight bounds bind, calibration can no longer absorb the chance variation in how many people respond. The estimate then moves with the realised sample size, and a replicate scheme that holds the sample size fixed misses that variation. Under Poisson sampling, where the sample size is random, use svy’s kind="poisson" bootstrap rather than Rao-Wu. This is a general rule for calibrated estimates, not something specific to fairlex; the bounds are what expose it.

Evidence¶

benchmarks/se_study.py checks the recipe with simcheck. That library’s pass bands are set by the number of simulated samples, not chosen by hand.

Population. 20,000 people. The outcome depends on age, education and region, plus an age-by-education interaction that the margins miss.
Sample. Poisson sampling that favours older, more educated people, with an expected size of 1,000. Nonresponse then depends on age, with response rates from 35% to 80%. The base weights undo the selection but not the nonresponse.
Replication. 400 simulated samples per regime, each with 200 bootstrap replicates. The estimand is the population total of the outcome.

With correct targets and loose bounds, the estimate is unbiased, its standard error is right (1.05 to 1.06 times the true spread) and 95% intervals cover 96%. Both bootstrap schemes pass every gate.

With binding bounds (0.7x to 1.3x), the weights cannot undo the nonresponse, so the estimate is 17% low. That bias is why no interval covers. The Rao-Wu bootstrap reports a standard error only 0.36 times the true spread. The estimate tracks the realised sample size almost exactly (correlation 0.98), and Rao-Wu holds that size fixed. The Poisson bootstrap reports 1.02 times the true spread.

With disagreeing targets, the estimate inherits the targets’ 1.5% error. Both schemes still measure its sampling spread correctly (1.05 to 1.06); coverage falls to 84% because of the bias, not because of the standard error.

400 simulated samples per regime, 200 bootstrap replicates each¶
regime	bootstrap	bias, % of truth	reported se / true spread	95% coverage, %	se calibrated (simcheck)	unbiased (simcheck)	coverage ok (simcheck)
exact targets, loose bounds	rao-wu	0.0	1.05	96.0	pass	pass	pass
exact targets, loose bounds	poisson	0.0	1.06	96.2	pass	pass	pass
exact targets, binding bounds	rao-wu	-17.3	0.36	0.0	fail	not tested	not tested
exact targets, binding bounds	poisson	-17.3	1.02	0.0	pass	not tested	not tested
disagreeing targets	rao-wu	-1.5	1.05	83.5	pass	not tested	not tested
disagreeing targets	poisson	-1.5	1.06	84.0	pass	not tested	not tested