API Reference¶

Model¶

POPSRegression ¶

POPSRegression(
    *,
    max_iter=300,
    tol=0.001,
    alpha_1=1e-06,
    alpha_2=1e-06,
    lambda_1=1e-06,
    lambda_2=1e-06,
    alpha_init=None,
    lambda_init=None,
    compute_score=False,
    fit_intercept=False,
    copy_X=True,
    verbose=False,
    mode_threshold=1e-08,
    resample_density=1.0,
    resampling_method="uniform",
    percentile_clipping=0.0,
    leverage_percentile=50.0,
    posterior="hypercube"
)

Bases: BayesianRidge

Bayesian regression for low-noise data with misspecification uncertainty.

Fits a linear model using BayesianRidge, then estimates weight uncertainties accounting for model misspecification using the POPS (Pointwise Optimal Parameter Sets) algorithm [1]_. Unlike standard Bayesian regression, the aleatoric noise precision alpha_ is not used for predictions, as it should be negligible in the low-noise regime.

Standard Bayesian regression can only estimate epistemic and aleatoric uncertainties. In the low-noise limit, weight uncertainties (sigma_ in :class:BayesianRidge) are significantly underestimated as they only account for epistemic uncertainties that decay with increasing data. POPS corrects this by estimating misspecification uncertainty from pointwise optimal parameter sets.

Parameters:

Name	Type	Description	Default
`max_iter`	`int`	Maximum number of iterations for the BayesianRidge convergence loop.	`300`
`tol`	`float`	Convergence threshold. Stop the algorithm if the coefficient vector has converged.	`1e-3`
`alpha_1`	`float`	Shape parameter for the Gamma distribution prior over `alpha_`.	`1e-6`
`alpha_2`	`float`	Inverse scale (rate) parameter for the Gamma distribution prior over `alpha_`.	`1e-6`
`lambda_1`	`float`	Shape parameter for the Gamma distribution prior over `lambda_`.	`1e-6`
`lambda_2`	`float`	Inverse scale (rate) parameter for the Gamma distribution prior over `lambda_`.	`1e-6`
`alpha_init`	`float`	Initial value for `alpha_` (precision of the noise). If None, `alpha_init` is `1 / Var(y)`.	`None`
`lambda_init`	`float`	Initial value for `lambda_` (precision of the weights). If None, `lambda_init` is 1.	`None`
`compute_score`	`bool`	If True, compute the log marginal likelihood at each step.	`False`
`fit_intercept`	`bool`	Whether to fit an intercept. If True, a constant column is appended to X (rather than centering) so that the intercept participates in the POPS posterior estimation.	`False`
`copy_X`	`bool`	If True, X will be copied; else, it may be overwritten.	`True`
`verbose`	`bool`	Verbose mode when fitting the model.	`False`
`mode_threshold`	`float`	Eigenvalue threshold (relative to max) for determining the effective dimensionality of the POPS posterior. Eigenvalues below `mode_threshold * max_eigenvalue` are discarded.	`1e-8`
`resample_density`	`float`	Number of resampled points per training point. The actual number of samples is `max(100, int(resample_density * n_samples))`.	`1.0`
`resampling_method`	`(uniform, sobol, latin, halton)`	Quasi-random sampling method for generating points within the POPS hypercube posterior.	`'uniform'`
`percentile_clipping`	`float`	Percentile to clip from each end when determining hypercube bounds. The hypercube spans the `[percentile_clipping, 100 - percentile_clipping]` range. Should be between 0 and 50.	`0.0`
`leverage_percentile`	`float`	Only training points with leverage scores above this percentile are used for POPS posterior estimation. Higher values accelerate fitting by focusing on high-leverage points.	`50.0`
`posterior`	`(hypercube, ensemble)`	Form of the POPS parameter posterior: `'hypercube'`: fit an axis-aligned box in PCA space (default). `'ensemble'`: use raw pointwise corrections directly.	`'hypercube'`

Attributes:

Name	Type	Description
`coef_`	`ndarray of shape (n_features,)`	Coefficients of the regression model (posterior mean).
`intercept_`	`float`	Independent term in the decision function. Set to 0.0 if `fit_intercept=False`.
`alpha_`	`float`	Estimated precision of the noise. Not used for prediction.
`lambda_`	`float`	Estimated precision of the weights.
`sigma_`	`ndarray of shape (n_features, n_features)`	Estimated epistemic variance-covariance matrix of the weights.
`misspecification_sigma_`	`ndarray of shape (n_features, n_features)`	Estimated misspecification variance-covariance matrix from POPS.
`posterior_samples_`	`ndarray of shape (n_features, n_posterior_samples)`	Samples from the POPS posterior, representing plausible weight perturbations.
`scores_`	`ndarray of shape (n_iter_,)`	Value of the log marginal likelihood at each iteration. Only available if `compute_score=True`.
`n_iter_`	`int`	The actual number of iterations to reach convergence.
`n_features_in_`	`int`	Number of features seen during :term:`fit`.
`feature_names_in_`	ndarray of shape (`n_features_in_`,)	Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings.

Fitting¶

fit ¶

fit(X, y, sample_weight=None)

Fit the POPS regression model.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Training data.	required
`y`	`array-like of shape (n_samples,)`	Target values.	required
`sample_weight`	`array-like of shape (n_samples,)`	Individual weights for each sample.	`None`

Returns:

Name	Type	Description
`self`	`object`	Returns the instance itself.

Prediction¶

predict ¶

predict(
    X,
    return_std=False,
    return_bounds=False,
    return_epistemic_std=False,
)

Predict using the POPS regression model.

In addition to the standard return_std from :class:BayesianRidge, this method can return prediction bounds (min/max over the posterior) and epistemic-only uncertainty.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Samples to predict for.	required
`return_std`	`bool`	If True, return the combined (misspecification + epistemic) standard deviation.	`False`
`return_bounds`	`bool`	If True, return the max and min predictions over the POPS posterior samples.	`False`
`return_epistemic_std`	`bool`	If True, return the epistemic-only standard deviation (from `sigma_`, excluding misspecification).	`False`

Returns:

Name	Type	Description
`y_mean`	`ndarray of shape (n_samples,)`	Predicted mean values.
`y_std`	`ndarray of shape (n_samples,)`	Combined standard deviation. Only returned if `return_std=True`.
`y_max`	`ndarray of shape (n_samples,)`	Upper bound from posterior samples. Only returned if `return_bounds=True`.
`y_min`	`ndarray of shape (n_samples,)`	Lower bound from posterior samples. Only returned if `return_bounds=True`.
`y_epistemic_std`	`ndarray of shape (n_samples,)`	Epistemic-only standard deviation. Only returned if `return_epistemic_std=True`.