Elastic Net Regularization via Iterative Soft Thresholding

matlab code

Sofia Mosci and Lorenzo Rosasco


Description of the algorithm

Elastic Net Regularization is an algorithm for learning and variable selection. It is based on a regularized least square procedure with a penalty which is the sum of an L1 penalty (like Lasso) and an L2 penalty (like ridge regression). The first term enforces the sparsity of the solution, whereas the second term ensures democracy among groups of correlated variables. The second term has also a smoothing effect that stabilizes the obtained solution. 

The algorithm has two parameters to be tuned. The first is a regularization parameter controlling overfitting and  can be tuned via cross validation. The second is a correlation parameter preserving correlation among variables against sparsity. This parameter should be fixed according to the goal of the user. The solutions obtained for different values of the correlation parameter have the same prediction properties but different feature representation. 

The optimization problem underlying elastic net regularization is solved using an iterative soft thresholding technique. The iteration converges to the elastic net estimator with a prescribed rate of convergence and in practice is stopped when some user defined precision is reached.

Description of the code

The code includes three matlab functions for training elastic net, test the obtained estimator and also a k-fold cross validation procedure to choose the regularization parameter. A detailed matlab help is included for each function and includes examples.


(the code we provided in this page is meant for reasearch and teaching purpose)


  1. De Mol, C., De Vito E. and Rosasco L.
    "Elastic Net Regularization in Learning Theory",
    CBCL paper #273/ CSAILTechnical Report #TR-2008-046,
    Massachusetts Institute of Technology, Cambridge, MA, July 24, 2008.
    arXiv:0807.3423 (to appear in the Journal of Complexity).
  2. De Mol C., Mosci S., Traskine M., Verri A. "A Regularized Method for Selecting Nested Groups of Relevant Genes from Microarray Data", accepted for publication on Journal of Computational Biology in November,2008. 
  3. Barla, A., Mosci, S., Rosasco, L. and Verri, A.
    "A method for robust variable selection with significance assessments"
    16th European Symposium on Artificial Neural Networks.