Unified Lf-Norm Robust Fitting for Linear Models

Abstract

In statistical learning, accurately estimating model parameters is crucial for reliable predictions. Managing residuals, the differences between observed and predicted values, is a key challenge. In regression, the residual penalty choice strongly affects model performance. The L<inf>2</inf>-norm penalty aligns with the least-squares approach, while the L<inf>1</inf>-norm provides robust fitting by minimizing the influence of outliers. To generalize models, the weights can be regularized using either the L<inf>2</inf>-norm or L<inf>1</inf>-norm, corresponding to Ridge and LASSO regularization, respectively. Many methods have been developed to penalize residuals and model weights, resulting in diverse cost functions optimized by specific numerical solvers. In this study, we propose the smooth L<inf>f</inf>-norm, a quasi-norm, as a unified framework for penalizing both residuals and model weights in linear models. Our efficient and robust numerical minimization scheme ensures fast and accurate fitting by minimizing our novel cost function. © 2025 Elsevier B.V., All rights reserved.

In statistical learning, accurately estimating model parameters is crucial for reliable predictions. Managing residuals, the differences between observed and predicted values, is a key challenge. In regression, the residual penalty choice strongly affects model performance. The L-2-norm penalty aligns with the least-squares approach, while the L-1-norm provides robust fitting by minimizing the influence of outliers. To generalize models, the weights can be regularized using either the L-2-norm or L-1-norm, corresponding to Ridge and LASSO regularization, respectively. Many methods have been developed to penalize residuals and model weights, resulting in diverse cost functions optimized by specific numerical solvers. In this study, we propose the smooth L-f-norm, a quasi-norm, as a unified framework for penalizing both residuals and model weights in linear models. Our efficient and robust numerical minimization scheme ensures fast and accurate fitting by minimizing our novel cost function.