stable_cart.stability_frontier

stable_cart.stability_frontier(estimator_factory, param_grid, X, y, task='continuous', n_bootstrap=200, test_size=0.3, random_state=None, *, X_eval=None, y_eval=None, prediction_method='predict', instability_metric='pairwise', groups=None)[source]

Sweep a parameter grid and return the validation-score/instability tradeoff.

Parameters

estimator_factory

Callable taking the grid’s keyword arguments and returning a fresh, unfitted estimator — e.g. lambda **kw: DecisionTreeRegressor(**kw).

param_grid

Grid in scikit-learn’s ParameterGrid form.

X

Feature matrix used to fit the models. When X_eval and y_eval are omitted, it is split once into fitting and validation parts.

y

Training targets, or targets to split alongside X.

task

'continuous' or 'categorical'.

n_bootstrap

Resamples per configuration. The returned Monte Carlo standard error is the guide to whether this is enough.

test_size

Fraction held out for evaluation.

random_state

Seed for resampling and the internal validation split. The same resampled index sets are reused for every configuration, so the data comparison is paired. Estimator randomness remains under estimator_factory.

X_eval

Optional explicit validation features. Supply with y_eval.

y_eval

Optional explicit validation targets. Supply with X_eval. The score is a model-selection score, not a final test-set performance estimate.

prediction_method

'predict' or, for classification, 'predict_proba'. See bootstrap_predictions().

instability_metric

Quantity minimized on the frontier: 'pairwise' compares two independently refitted models; 'mape' compares each refit with the model fitted on all training data.

groups

Cluster label per row of X, for correlated data; see bootstrap_predictions(). The internal validation split becomes a grouped split, so no cluster lands on both sides of it. For classification, up to 100 candidate splits are tried to retain every class in training, preferring splits that also retain every class in validation. If none of the candidates retains all training classes, a ValueError asks for a different split size or explicit validation data. This search does not guarantee a feasible split will be found.

Returns

dict[str, Any]

points — every configuration with score (validation accuracy or R²), the selected instability, its Monte Carlo standard error, pairwise, mape, resampling counts, and params; frontier — the non-dominated subset; n_fits and seconds — what the answer cost.

Raises

ValueError

If an argument is invalid.

Examples

>>> from sklearn.datasets import make_regression
>>> from sklearn.tree import DecisionTreeRegressor
>>> from stable_cart import stability_frontier
>>> X, y = make_regression(n_samples=300, n_features=5, noise=5.0, random_state=0)
>>> result = stability_frontier(
...     lambda **kw: DecisionTreeRegressor(random_state=0, **kw),
...     {"max_depth": [2, 5, 8]},
...     X, y, task="continuous", n_bootstrap=8, random_state=0,
... )
>>> len(result["points"]), len(result["frontier"]) <= len(result["points"])
(3, True)
Parameters:
Return type:

dict[str, Any]