Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2011 Jan 4;21(2):261-273.
doi: 10.1007/s11222-009-9166-3.

A quasi-Newton acceleration for high-dimensional optimization algorithms

Hua Zhou  1 , David Alexander, Kenneth Lange
Affiliations

A quasi-Newton acceleration for high-dimensional optimization algorithms

Hua Zhou et al. Stat Comput. .

Abstract

In many statistical problems, maximum likelihood estimation by an EM or MM algorithm suffers from excruciatingly slow convergence. This tendency limits the application of these algorithms to modern high-dimensional problems in data mining, genomics, and imaging. Unfortunately, most existing acceleration techniques are ill-suited to complicated models involving large numbers of parameters. The squared iterative methods (SQUAREM) recently proposed by Varadhan and Roland constitute one notable exception. This paper presents a new quasi-Newton acceleration scheme that requires only modest increments in computation per iteration and overall storage and rivals or surpasses the performance of SQUAREM on several representative test problems.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Ascent of the different algorithms for the Lidwell and Somerville household type (a) data starting from (π0, α0) = (0.5, 1) with stopping criterion ε = 10−9 Top left: naive MM; Top right: q = 1; Middle left: q = 2; Middle right: SqS1; Bottom left: SqS2; Bottom right: SqS3
Fig. 2
Fig. 2
EM acceleration for the Poisson admixture example
See this image and copyright information in PMC

References

    1. Alexander DH, Novembre J, Lange KL. Fast model-based estimation of ancestry in unrelated individuals. Genome Res. 2009;19:1655–1664. - PMC - PubMed
    1. Becker MP, Young I, Lange KL. EM algorithms without missing data. Stat. Methods Med. Res. 1997;6:37–53. - PubMed
    1. de Leeuw J. Block relaxation algorithms in statistics. In: Bock HH, Lenski W, Richter MM, editors. Information Systems and Data Analysis. New York: Springer; 1994. pp. 308–325.
    1. Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm (with discussion) J. R. Stat. Soc. Ser. B. 1977;39(1):1–38.
    1. Golub GH, Van Loan CF. Matrix Computations. 3rd edn. Baltimore: Johns Hopkins University Press; 1996.

LinkOut - more resources