IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
Online ISSN : 1745-1337
Print ISSN : 0916-8508
Empirical Bayes Estimation for Lasso-Type Regularizers and its Consistency
Tsukasa YOSHIDAKazuho WATANABE
Author information
JOURNAL FREE ACCESS Advance online publication

Article ID: 2025TAP0008

Details
Abstract

This paper focuses on linear regression models with non-conjugate sparsity-inducing regularizers such as lasso and group lasso. Although the empirical Bayes approach enables us to estimate the regularization parameter, little is known on the properties of the estimators. In particular, many aspects regarding the specific conditions under which the mechanism of automatic relevance determination (ARD) occurs remain unexplained. In this paper, we derive the empirical Bayes estimators for the group lasso regularized linear regression models with limited parameters. It is shown that the estimators diverge under a specific condition, giving rise to the ARD mechanism. In addition, we demonstrate that group lasso solutions with the empirical Bayes estimators yield characteristics similar to those of the adaptive lasso, suggesting that such solutions exhibit consistency. Furthermore, we prove their consistency in variable selection. We also prove that empirical Bayes methods can produce the ARD mechanism in general regularized linear regression models and clarify the conditions under which models such as ridge, lasso, and group lasso can do.

Content from these authors
© 2025 The Institute of Electronics, Information and Communication Engineers
Previous article Next article
feedback
Top