Array programming with NumPy

Charles R Harris¹, K Jarrod Millman^{2

3

4}, Stéfan J van der Walt^{5

6

7}, Ralf Gommers⁸, Pauli Virtanen^{9

10}, David Cournapeau¹¹, Eric Wieser¹², Julian Taylor¹³, Sebastian Berg¹⁴, Nathaniel J Smith¹⁵, Robert Kern¹⁶, Matti Picus¹⁴, Stephan Hoyer¹⁷, Marten H van Kerkwijk¹⁸, Matthew Brett^{19

20}, Allan Haldane²¹, Jaime Fernández Del Río²², Mark Wiebe^{23

24}, Pearu Peterson^{25

26

27}, Pierre Gérard-Marchant^{28

29}, Kevin Sheppard³⁰, Tyler Reddy³¹, Warren Weckesser¹⁴, Hameer Abbasi²⁵, Christoph Gohlke³², Travis E Oliphant²⁵

Affiliations

¹ Independent researcher, Logan, UT, USA.
² Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA. millman@berkeley.edu.
³ Division of Biostatistics, University of California, Berkeley, Berkeley, CA, USA. millman@berkeley.edu.
⁴ Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA. millman@berkeley.edu.
⁵ Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA. stefanv@berkeley.edu.
⁶ Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA. stefanv@berkeley.edu.
⁷ Applied Mathematics, Stellenbosch University, Stellenbosch, South Africa. stefanv@berkeley.edu.
⁸ Quansight, Austin, TX, USA. ralf.gommers@gmail.com.
⁹ Department of Physics, University of Jyväskylä, Jyväskylä, Finland.
¹⁰ Nanoscience Center, University of Jyväskylä, Jyväskylä, Finland.
¹¹ Mercari JP, Tokyo, Japan.
¹² Department of Engineering, University of Cambridge, Cambridge, UK.
¹³ Independent researcher, Karlsruhe, Germany.
¹⁴ Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA.
¹⁵ Independent researcher, Berkeley, CA, USA.
¹⁶ Enthought, Austin, TX, USA.
¹⁷ Google Research, Mountain View, CA, USA.
¹⁸ Department of Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, Canada.
¹⁹ Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA.
²⁰ School of Psychology, University of Birmingham, Edgbaston, Birmingham, UK.
²¹ Department of Physics, Temple University, Philadelphia, PA, USA.
²² Google, Zurich, Switzerland.
²³ Department of Physics and Astronomy, The University of British Columbia, Vancouver, British Columbia, Canada.
²⁴ Amazon, Seattle, WA, USA.
²⁵ Quansight, Austin, TX, USA.
²⁶ Independent researcher, Saue, Estonia.
²⁷ Department of Mechanics and Applied Mathematics, Institute of Cybernetics at Tallinn Technical University, Tallinn, Estonia.
²⁸ Department of Biological and Agricultural Engineering, University of Georgia, Athens, GA, USA.
²⁹ France-IX Services, Paris, France.
³⁰ Department of Economics, University of Oxford, Oxford, UK.
³¹ CCS-7, Los Alamos National Laboratory, Los Alamos, NM, USA.
³² Laboratory for Fluorescence Dynamics, Biomedical Engineering Department, University of California, Irvine, Irvine, CA, USA.

PMID: 32939066
PMCID: PMC7759461
DOI: 10.1038/s41586-020-2649-2

Review

Array programming with NumPy

Charles R Harris et al. Nature. 2020 Sep.

. 2020 Sep;585(7825):357-362.

doi: 10.1038/s41586-020-2649-2. Epub 2020 Sep 16.

Authors

Affiliations

¹ Independent researcher, Logan, UT, USA.
² Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA. millman@berkeley.edu.
³ Division of Biostatistics, University of California, Berkeley, Berkeley, CA, USA. millman@berkeley.edu.
⁴ Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA. millman@berkeley.edu.
⁵ Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA. stefanv@berkeley.edu.
⁶ Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA. stefanv@berkeley.edu.
⁷ Applied Mathematics, Stellenbosch University, Stellenbosch, South Africa. stefanv@berkeley.edu.
⁸ Quansight, Austin, TX, USA. ralf.gommers@gmail.com.
⁹ Department of Physics, University of Jyväskylä, Jyväskylä, Finland.
¹⁰ Nanoscience Center, University of Jyväskylä, Jyväskylä, Finland.
¹¹ Mercari JP, Tokyo, Japan.
¹² Department of Engineering, University of Cambridge, Cambridge, UK.
¹³ Independent researcher, Karlsruhe, Germany.
¹⁴ Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA.
¹⁵ Independent researcher, Berkeley, CA, USA.
¹⁶ Enthought, Austin, TX, USA.
¹⁷ Google Research, Mountain View, CA, USA.
¹⁸ Department of Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, Canada.
¹⁹ Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA.
²⁰ School of Psychology, University of Birmingham, Edgbaston, Birmingham, UK.
²¹ Department of Physics, Temple University, Philadelphia, PA, USA.
²² Google, Zurich, Switzerland.
²³ Department of Physics and Astronomy, The University of British Columbia, Vancouver, British Columbia, Canada.
²⁴ Amazon, Seattle, WA, USA.
²⁵ Quansight, Austin, TX, USA.
²⁶ Independent researcher, Saue, Estonia.
²⁷ Department of Mechanics and Applied Mathematics, Institute of Cybernetics at Tallinn Technical University, Tallinn, Estonia.
²⁸ Department of Biological and Agricultural Engineering, University of Georgia, Athens, GA, USA.
²⁹ France-IX Services, Paris, France.
³⁰ Department of Economics, University of Oxford, Oxford, UK.
³¹ CCS-7, Los Alamos National Laboratory, Los Alamos, NM, USA.
³² Laboratory for Fluorescence Dynamics, Biomedical Engineering Department, University of California, Irvine, Irvine, CA, USA.

PMID: 32939066
PMCID: PMC7759461
DOI: 10.1038/s41586-020-2649-2

Abstract

Array programming provides a powerful, compact and expressive syntax for accessing, manipulating and operating on data in vectors, matrices and higher-dimensional arrays. NumPy is the primary array programming library for the Python language. It has an essential role in research analysis pipelines in fields as diverse as physics, chemistry, astronomy, geoscience, biology, psychology, materials science, engineering, finance and economics. For example, in astronomy, NumPy was an important part of the software stack used in the discovery of gravitational waves¹ and in the first imaging of a black hole². Here we review how a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring and analysing scientific data. NumPy is the foundation upon which the scientific Python ecosystem is constructed. It is so pervasive that several projects, targeting audiences with specialized needs, have developed their own NumPy-like interfaces and array objects. Owing to its central position in the ecosystem, NumPy increasingly acts as an interoperability layer between such array computation libraries and, together with its application programming interface (API), provides a flexible framework to support the next decade of scientific and industrial analysis.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

**Fig. 1. The NumPy array incorporates several fundamental array concepts.**
a, The NumPy array data structure and its associated metadata fields. b, Indexing an array with slices and steps. These operations return a ‘view’ of the original data. c, Indexing an array with masks, scalar coordinates or other arrays, so that it returns a ‘copy’ of the original data. In the bottom example, an array is indexed with other arrays; this broadcasts the indexing arguments before performing the lookup. d, Vectorization efficiently applies operations to groups of elements. e, Broadcasting in the multiplication of two-dimensional arrays. f, Reduction operations act along one or more axes. In this example, an array is summed along select axes to produce a vector, or along two axes consecutively to produce a scalar. g, Example NumPy code, illustrating some of these concepts.

**Fig. 2. NumPy is the base of the scientific Python ecosystem.**
Essential libraries and projects that depend on NumPy’s API gain access to new array implementations that support NumPy’s array protocols (Fig. 3).

**Fig. 3. NumPy’s API and array protocols expose new arrays to the ecosystem.**
In this example, NumPy’s ‘mean’ function is called on a Dask array. The call succeeds by dispatching to the appropriate library implementation (in this case, Dask) and results in a new Dask array. Compare this code to the example code in Fig. 1g.

See this image and copyright information in PMC

References

1. Abbott BP, et al. Observation of gravitational waves from a binary black hole merger. Phys. Rev. Lett. 2016;116:061102. doi: 10.1103/PhysRevLett.116.061102. - DOI - PubMed
1. Chael A, et al. High-resolution linear polarimetric imaging for the Event Horizon Telescope. Astrophys. J. 2016;286:11. doi: 10.3847/0004-637X/829/1/11. - DOI
1. Dubois PF, Hinsen K, Hugunin J. Numerical Python. Comput. Phys. 1996;10:262–267. doi: 10.1063/1.4822400. - DOI
1. Ascher, D., Dubois, P. F., Hinsen, K., Hugunin, J. & Oliphant, T. E. An Open Source Project: Numerical Python (Lawrence Livermore National Laboratory, 2001).
1. Yang, T.-Y., Furnish, G. & Dubois, P. F. Steering object-oriented scientific computations. In Proc. TOOLS USA 97. Intl Conf. Technology of Object Oriented Systems and Languages (eds Ege, R., Singh, M. & Meyer, B.) 112–119 (IEEE, 1997).

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
Other Literature Sources
- H1 Connect - Access expert opinions and insights on biomedical research.
- The Lens - Patent Citations Database

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Array programming with NumPy

Affiliations

Array programming with NumPy

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources

Other Literature Sources