stable_cart.linear_frontier

stable_cart.linear_frontier(X, y, n_points=50, mu_max=1000.0, beta=None, sigma=None, signal='pooled')[source]

Trace an oracle or plug-in frontier for fixed-design linear prediction.

Among estimators that shrink each singular direction of the design, the one with the least squared bias at a given prediction variance is

\[s_j(\mu) = \frac{d_j^2\theta_j^2}{d_j^2\theta_j^2 + \mu\sigma^2},\]

with \(\mu\) the Lagrange multiplier on the variance budget. Sweeping \(\mu\) from 0 to \(\infty\) traces the whole frontier, from the minimum-variance zero-bias endpoint to the zero-variance constant. The first point equals least squares when every singular direction carries signal; it drops exactly null directions otherwise.

Two consequences are worth having in front of you. The slope of the frontier is \(dB/dV = -\mu\) exactly, so \(\mu\) is the exchange rate — the units of squared bias you pay per unit of variance you buy. And since risk is \(\sigma^2 + B + V\), risk is minimized exactly at \(\mu = 1\). Everything to the stable side of that point costs strictly more accuracy than it saves; that is not a matter of taste, it is where the slope crosses one.

Parameters

X

Full-column-rank design matrix of shape (n_samples, n_features). Include a column of ones for an intercept.

y

Targets of shape (n_samples,).

n_points

Number of points along the frontier, spaced geometrically in mu.

mu_max

Largest mu to trace. The frontier approaches the constant predictor as mu grows.

beta

True coefficients, if known — used by tests and by simulation studies to get the oracle fixed-design frontier rather than the plug-in curve.

sigma

True noise level, if known.

signal

How the unknown signal strength is supplied when beta is omitted; see shrinkage_coefficients(). 'pooled' (the default) makes the traced path the ridge path.

Returns

dict[str, Any]

points — a list of dicts, each with mu, bias2, variance, risk (excess over the noise floor, i.e. bias2 + variance), s1 and s2 instability, exchange_rate (equal to mu), and shrinkage (the per-direction factors); risk_optimal — the point at mu = 1; sigma — the noise level used, estimated if it was not supplied.

Raises

ValueError

If n_points is below 2, mu_max is not finite and positive, sigma is negative or nonfinite, an array input is nonfinite, or the design is not full column rank.

Notes

The separability that makes the closed form exact needs the evaluation metric to be diagonal in the design’s singular basis. Risk here is therefore in-sample prediction risk, weighted by \(X'X/n\) — the standard choice, and the one under which \(s_j(\mu)\) above is exactly optimal.

When beta and sigma are not supplied they are estimated from the same data, so the returned frontier is optimistic near \(\mu = 0\) in the same way any in-sample curve is.

Examples

>>> import numpy as np
>>> from stable_cart import linear_frontier
>>> rng = np.random.default_rng(0)
>>> X = rng.normal(size=(200, 5)); y = X @ np.arange(5.0) + rng.normal(size=200)
>>> out = linear_frontier(X, y, n_points=20)
>>> out["risk_optimal"]["mu"]
1.0
Parameters:

  • X (NDArray[floating])

  • y (NDArray[floating])

  • n_points (int)

  • mu_max (float)

  • beta (NDArray[floating] | None)

  • sigma (float | None)

  • signal (str)

Return type:

dict[str, Any]