Switching regression models can output multiple clusters and regression models. However, there is one problem: the results have a strong dependency on the predefined number of clusters. To avoid these drawbacks, we have researched sequential extractions. In sequential extractions process, one cluster is extracted at a time using a method of noise-detection, and the number of clusters are determined automatically. We propose semi-supervised sequential kernel regression models with penalty functions. Additionally, we also find that the sensitivity against the regularization parameter λ can be alleviated by semi-supervisions using penalty functions. We show the effectiveness of the proposed method by using numerical examples.