Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014;7(4):571-582.
doi: 10.4310/SII.2014.v7.n4.a12.

A New Bayesian Lasso

Himel Mallick  1 , Nengjun Yi  1
Affiliations

A New Bayesian Lasso

Himel Mallick et al. Stat Interface. 2014.

Abstract

Park and Casella (2008) provided the Bayesian lasso for linear models by assigning scale mixture of normal (SMN) priors on the parameters and independent exponential priors on their variances. In this paper, we propose an alternative Bayesian analysis of the lasso problem. A different hierarchical formulation of Bayesian lasso is introduced by utilizing the scale mixture of uniform (SMU) representation of the Laplace density. We consider a fully Bayesian treatment that leads to a new Gibbs sampler with tractable full conditional posterior distributions. Empirical results and real data analyses show that the new algorithm has good mixing property and performs comparably to the existing Bayesian method in terms of both prediction accuracy and variable selection. An ECM algorithm is provided to compute the MAP estimates of the parameters. Easy extension to general models is also briefly discussed.

Keywords: Bayesian Lasso; Gibbs Sampler; Lasso; MCMC; Scale Mixture of Uniform.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Posterior mean Bayesian lasso estimates (computed over a grid of λ values, using 10,000 samples after burn-in) and corresponding 95% credible intervals (equal-tailed) of Diabetes data (n = 442) covariates. The hyperprior parameters were chosen as a = 1, b = 0.1. OLS estimates with corresponding 95% confidence intervals are also reported. For the lasso estimates, the tuning parameter was chosen by 10-fold CV of the LARS algorithm.
Figure 2
Figure 2
ACF plots and histograms based on posterior samples of Diabetes data covariates.
Figure 3
Figure 3
Trace plots of Diabetes data covariates.
Figure 4
Figure 4
Posterior mean Bayesian lasso estimates (computed over a grid of λ values, using 10,000 samples after burn-in) and corresponding 95% credible intervals (equal-tailed) of Prostate data (n = 67) covariates. The hyperprior parameters were chosen as a = 1, b = 0.1. OLS estimates with corresponding 95% confidence intervals are also reported. For the lasso estimates, the tuning parameter was chosen by 10-fold CV of the LARS algorithm.
Figure 5
Figure 5
Trace plots of Prostate data covariates.
Figure 6
Figure 6
ACF plots and histograms based on posterior samples of Prostate data covariates.
See this image and copyright information in PMC

References

    1. Andrews DF, Mallows CL. Scale mixtures of normal distributions. Journal of the Royal Statistical Society. Series B (Methodological) 1974;36:99–102.
    1. Bae K, Mallick B. Gene selection using a two-level hierarchical bayesian model. Bioinformatics. 2004;20(18):3423–3430. - PubMed
    1. Chen X, Wang JZ, McKeown JM. A bayesian lasso via reversible-jump mcmc. Signal Processing. 2011;91(8):1920–1932.
    1. Choy STB, Wan W, Chan C. Bayesian student-t stochastic volatility models via scale mixtures. Advances in Econometrics. 2008;23:595–618.
    1. Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression. The Annals of Statistics. 2004;32(2):407–499.

LinkOut - more resources