Correctness audit

Findings resolved for the next release

Finding	Consequence	Resolution
update returned predictions after learning their own outcomes	Easy in-sample leakage	Separate calibrate from update; test outcome independence
NumPy broadcasting accepted unequal metric inputs	Plausible but wrong scores	Central equal-length validation
Probabilities, outcomes, and tuning parameters lacked domain checks	Silent clipping, NaNs, or downstream errors	Validate every public boundary
Streaming PAV weighted every bin equally	Sparse and dense bins influenced the fit equally	Weighted sufficient statistics and weighted PAV
EMA averaged batch rates without their counts	Results depended on arbitrary batch composition	Decayed counts and positive mass
Nearly-isotonic SGD was not the cited solution-path algorithm	The name overstated method fidelity	Remove it rather than preserve an unsupported API
Temperature scaling duplicated an established method	Extra numerical implementation to maintain	Delegate to scikit-learn’s public temperature calibration
ECE name hid estimator dependence on bins and sample size	Encouraged overinterpretation	Rename it and document it as a binned diagnostic
PyPI 0.1 and main exposed different algorithm families	Install instructions could imply the wrong API	Mark the unreleased API and add migration notes

Supported contracts

• StreamingIsotonicCalibrator estimates a binary monotone calibration map from fixed-width bin sufficient statistics with optional exponential forgetting and identity shrinkage.

• Batch references delegate to scikit-learn and expose accumulating, rolling, infrequent-refit, and frozen policies. Sigmoid uses scikit-learn’s probability score convention, replacing the old logistic-on-logits wrapper.

• Proper scores evaluate forecast usefulness. Calibration diagnostics are not used alone because a constant base-rate forecast can appear marginally calibrated while destroying resolution.

The implementation is compared with scikit-learn’s official isotonic and logistic APIs. Temperature scaling follows the one-parameter logit scaling studied by Guo et al. (2017). The removed nearly-isotonic name referred to Tibshirani, Höfling, and Tibshirani (2011), whose path algorithm was not implemented by the former fixed-step routine.

Implementation inventory

Operation	Implementation	Streamcal responsibility
Weighted isotonic optimization	sklearn.isotonic.IsotonicRegression	Accumulate the exact binned sufficient statistics
Batch isotonic	sklearn.isotonic.IsotonicRegression	Retained window, refit cadence, freezing
Batch sigmoid, temperature	sklearn.calibration.CalibratedClassifierCV and FrozenEstimator	Score adapter and the same history/refit policies
Brier, log loss, AUROC	sklearn.metrics	Strict aligned binary input validation
Calibration curve	sklearn.calibration.calibration_curve	Weight upstream bin gaps by matching bin counts
Interpolation, bin accumulation	NumPy	Define fixed-width bins and state semantics
Paired dependent-data intervals	arch.bootstrap.CircularBlockBootstrap	Paired losses and observation-weighted statistic
Delayed label order	river.stream.simulate_qa	Forward original probability and timestamps
Online logistic benchmark	River logistic regression and SGD	Translate one probability into a logit feature
Peak process memory	Standard resource and psutil	Run isolated measurement processes

The frozen score adapter has no learned base model. Its one-fold splitter supplies all available rows for calibration and stores no row-index history. Sharing these train/test indices is safe only because the adapter is stateless and frozen; base-model training occurs on an earlier, disjoint period.

The custom mathematical behavior is the summary-state recurrence, prediction-age weighting, and fixed identity prior. Their objective and equivalence conditions are proved in the formalization. Retention, scheduling adapters, and budget selection are integration policies, not new solvers.

Input-boundary and exactly computable expected-result tests are the primary correctness checks. Known-truth simulations and held-out comparisons assess the statistical choices. Latency budgets use observed p95 whole-call times; they are measurements, not worst-case execution guarantees.