Structured Open Urban Data: Understanding the Landscape

Abstract

A growing number of cities are now making urban data freely available to the public. Besides promoting transparency, these data can have a transformative effect in social science research as well as in how citizens participate in governance. These initiatives, however, are fairly recent and the landscape of open urban data is not well known. In this study, we try to shed some light on this through a detailed study of over 9,000 open data sets from 20 cities in North America. We start by presenting general statistics about the content, size, nature, and popularity of the different data sets, and then examine in more detail structured data sets that contain tabular data. Since a key benefit of having a large number of data sets available is the ability to fuse information, we investigate opportunities for data integration. We also study data quality issues and time-related aspects, namely, recency and change frequency. Our findings are encouraging in that most of the data are structured and published in standard formats that are easy to parse; there is ample opportunity to integrate different data sets; and the volume of data is increasing steadily. But they also uncovered a number of challenges that need to be addressed to enable these data to be fully leveraged. We discuss both our findings and issues involved in using open urban data.

FIG. 1.

Population size versus number of data sets.

FIG. 2.

Proportion of format types.

FIG. 3.

Proportion of data in tabular format.

FIG. 4.

Tag clouds derived from the keywords associated with the data sets.

FIG. 5.

Distribution of data set views and downloads.

FIG. 6.

Tag clouds derived from the keywords associated with the most popular data sets, that is, data sets with the largest number of downloads.

FIG. 7.

Distribution of tables' age in months. The inclined horizontal line is the trend line for this distribution.

FIG. 8.

Change frequency ratio of tables over 30 days.

FIG. 9.

Distribution of schema sizes.

FIG. 10.

Schema diversity for tables in five cities.

FIG. 11.

Similarity among data sets taking into account their schemata and overlap of attribute names.

FIG. 12.

Proportion of different types in data sets for 10 cities.

FIG. 13.

Distribution of table sparseness: proportion of null values in tables.

FIG. 14.

Degree of informativeness: proportion of columns whose names contain words in the English dictionary.

FIG. 15.

Heatmaps of the geographical coverage of the data sets for (a) Chicago and (b) NYC.

FIG. 16.

Population of zip code regions versus references to the zip codes in Chicago data sets.