API reference¶

The package exposes a single public estimator.

from popsregression import POPSRegression

Object	Description
`POPSRegression`	Bayesian regression with misspecification uncertainty
`popsregression.__version__`	Installed package version

POPSRegression¶

POPSRegression ¶

POPSRegression(
    *,
    max_iter=300,
    tol=0.001,
    alpha_1=1e-06,
    alpha_2=1e-06,
    lambda_1=1e-06,
    lambda_2=1e-06,
    alpha_init=None,
    lambda_init=None,
    compute_score=False,
    fit_intercept=False,
    copy_X=True,
    verbose=False,
    mode_threshold=1e-08,
    resample_density=1.0,
    resampling_method="uniform",
    percentile_clipping=0.0,
    minimum_relative_error=0.01,
    posterior="hypercube",
    leverage_percentile="deprecated"
)

Bases: BayesianRidge

Bayesian regression for low-noise data with misspecification uncertainty.

Fits a linear model using BayesianRidge, then estimates weight uncertainties accounting for model misspecification using the POPS (Pointwise Optimal Parameter Sets) algorithm [1]_. Unlike standard Bayesian regression, the aleatoric noise precision alpha_ is not used for predictions, as it should be negligible in the low-noise regime.

Standard Bayesian regression can only estimate epistemic and aleatoric uncertainties. In the low-noise limit, weight uncertainties (sigma_ in :class:BayesianRidge) are significantly underestimated as they only account for epistemic uncertainties that decay with increasing data. POPS corrects this by estimating misspecification uncertainty from pointwise optimal parameter sets.

Parameters:

Name	Type	Description	Default
`max_iter`	`int`	Maximum number of iterations for the BayesianRidge convergence loop.	`300`
`tol`	`float`	Convergence threshold. Stop the algorithm if the coefficient vector has converged.	`1e-3`
`alpha_1`	`float`	Shape parameter for the Gamma distribution prior over `alpha_`.	`1e-6`
`alpha_2`	`float`	Inverse scale (rate) parameter for the Gamma distribution prior over `alpha_`.	`1e-6`
`lambda_1`	`float`	Shape parameter for the Gamma distribution prior over `lambda_`.	`1e-6`
`lambda_2`	`float`	Inverse scale (rate) parameter for the Gamma distribution prior over `lambda_`.	`1e-6`
`alpha_init`	`float`	Initial value for `alpha_` (precision of the noise). If None, `alpha_init` is `1 / Var(y)`.	`None`
`lambda_init`	`float`	Initial value for `lambda_` (precision of the weights). If None, `lambda_init` is 1.	`None`
`compute_score`	`bool`	If True, compute the log marginal likelihood at each step.	`False`
`fit_intercept`	`bool`	Whether to fit an intercept. If True, a constant column is appended to X (rather than centering) so that the intercept participates in the POPS posterior estimation.	`False`
`copy_X`	`bool`	If True, X will be copied; else, it may be overwritten.	`True`
`verbose`	`bool`	Verbose mode when fitting the model.	`False`
`mode_threshold`	`float`	Eigenvalue threshold (relative to max) for determining the effective dimensionality of the POPS posterior. Eigenvalues below `mode_threshold * max_eigenvalue` are discarded.	`1e-8`
`resample_density`	`float`	Number of resampled points per training point. The actual number of samples is `max(100, int(resample_density * n_samples))`.	`1.0`
`resampling_method`	`(uniform, sobol, latin, halton)`	Quasi-random sampling method for generating points within the POPS hypercube posterior.	`'uniform'`
`percentile_clipping`	`float`	Percentile to clip from each end when determining hypercube bounds. The hypercube spans the `[percentile_clipping, 100 - percentile_clipping]` range. Should be between 0 and 50.	`0.0`
`minimum_relative_error`	`float`	Relative residual threshold for training point selection, in units of the root-mean-square error of the mean fit. Only training points whose residual satisfies `\|y - X @ coef_\| >= minimum_relative_error * rmse` contribute to the POPS posterior; a value of `0.01` thus discards points fit a hundred times better than the typical training point. Larger values accelerate fitting by focusing on the points the model fits worst, which are the ones that carry misspecification information. Use `0.0` to keep every training point. If no point passes the threshold, all points are used.	`0.01`
`posterior`	`(hypercube, ensemble)`	Form of the POPS parameter posterior: `'hypercube'`: fit an axis-aligned box in PCA space (default). `'ensemble'`: use raw pointwise corrections directly.	`'hypercube'`
`leverage_percentile`	`float`	Deprecated. Training points used to be selected by leverage score percentile; they are now selected by residual magnitude through `minimum_relative_error`. Passing this parameter raises a :class:`FutureWarning` and has no effect. .. deprecated:: 0.5 `leverage_percentile` is deprecated and will be removed in 0.7. Use `minimum_relative_error` instead.	`'deprecated'`

Attributes:

Name	Type	Description
`coef_`	`ndarray of shape (n_features,)`	Coefficients of the regression model (posterior mean).
`intercept_`	`float`	Independent term in the decision function. Set to 0.0 if `fit_intercept=False`.
`alpha_`	`float`	Estimated precision of the noise. Not used for prediction.
`lambda_`	`float`	Estimated precision of the weights.
`sigma_`	`ndarray of shape (n_features, n_features)`	Estimated epistemic variance-covariance matrix of the weights.
`misspecification_sigma_`	`ndarray of shape (n_features, n_features)`	Estimated misspecification variance-covariance matrix from POPS.
`posterior_samples_`	`ndarray of shape (n_features, n_posterior_samples)`	Samples from the POPS posterior, representing plausible weight perturbations.
`scores_`	`ndarray of shape (n_iter_,)`	Value of the log marginal likelihood at each iteration. Only available if `compute_score=True`.
`n_iter_`	`int`	The actual number of iterations to reach convergence.
`n_features_in_`	`int`	Number of features seen during :term:`fit`.
`feature_names_in_`	ndarray of shape (`n_features_in_`,)	Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings.

Methods¶

fit¶

fit ¶

fit(X, y, sample_weight=None)

Fit the POPS regression model.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Training data.	required
`y`	`array-like of shape (n_samples,)`	Target values.	required
`sample_weight`	`array-like of shape (n_samples,)`	Individual weights for each sample.	`None`

Returns:

Name	Type	Description
`self`	`object`	Returns the instance itself.

predict¶

predict ¶

predict(
    X,
    return_std=False,
    return_bounds=False,
    return_epistemic_std=False,
)

Predict using the POPS regression model.

In addition to the standard return_std from :class:BayesianRidge, this method can return prediction bounds (min/max over the posterior) and epistemic-only uncertainty.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Samples to predict for.	required
`return_std`	`bool`	If True, return the combined (misspecification + epistemic) standard deviation.	`False`
`return_bounds`	`bool`	If True, return the max and min predictions over the POPS posterior samples.	`False`
`return_epistemic_std`	`bool`	If True, return the epistemic-only standard deviation (from `sigma_`, excluding misspecification).	`False`

Returns:

Name	Type	Description
`y_mean`	`ndarray of shape (n_samples,)`	Predicted mean values.
`y_std`	`ndarray of shape (n_samples,)`	Combined standard deviation. Only returned if `return_std=True`.
`y_max`	`ndarray of shape (n_samples,)`	Upper bound from posterior samples. Only returned if `return_bounds=True`.
`y_min`	`ndarray of shape (n_samples,)`	Lower bound from posterior samples. Only returned if `return_bounds=True`.
`y_epistemic_std`	`ndarray of shape (n_samples,)`	Epistemic-only standard deviation. Only returned if `return_epistemic_std=True`.

Inherited methods¶

POPSRegression subclasses BayesianRidge and inherits the standard scikit-learn estimator methods unchanged:

Method	Description
`get_params(deep=True)`	Parameters of this estimator
`set_params(**params)`	Set parameters of this estimator
`score(X, y, sample_weight=None)`	Coefficient of determination R² of the prediction
`get_metadata_routing()`	Metadata routing of this object

score uses the mean prediction only; uncertainty outputs are available through predict.

Deprecated parameters¶

Since	Removed in	Parameter	Replacement
0.5	0.7	`leverage_percentile`	`minimum_relative_error`

Passing leverage_percentile raises a FutureWarning on fit and has no effect on the fitted model.

The top-level POPSRegression module shim (import POPSRegression) is deprecated in favour of from popsregression import POPSRegression and raises a DeprecationWarning on import.