Abstract
In regression problems, some of the most important goals are (i) to obtain a lower prediction error, and (ii) to interpret regression relationships. Friedman's Multivariate Adaptive Regression Splines (MARS) method which constructs basis functions with interaction effects is a very powerful data-driven technique in the viewpoint of (i), and the single tree-based structure built by MARS contributes to approach to (ii). Also, to address (i) and (ii), the better estimation and variable selection of the model are inevitable issues, and then shrinkage estimators contribute to resolve such issues. Recently, especially in the context of linear regression, Breiman's non-negative garrote (NNG) estimator and Tibshirani's Lasso estimator are shown to be a stable estimation and variable selection that often outperform theirs competitors. In this paper, we focus on Breiman's NNG as a foundation to incorporate the shrinkage estimators into the tree-based regression model and propose a new version of the MARS with the NNG (NNG-MARS). Then, we evaluated some performances of the NNG-MARS via an examination of a literature example and several small scale simulations. As a conclusion, the NNG-MARS, which holds the interpretable tree-based structure, has achieved much lower prediction error than the ordinary MARS.