The Problem with Big Data: Operating on Smaller Datasets to Bridge the Implementation Gap

Abstract

Big datasets have the potential to revolutionize public health. However, there is a mismatch between the political and scientific optimism surrounding big data and the public's perception of its benefit. We suggest a systematic and concerted emphasis on developing models derived from smaller datasets to illustrate to the public how big data can produce tangible benefits in the long term. In order to highlight the immediate value of a small data approach, we produced a proof-of-concept model predicting hospital length of stay. The results demonstrate that existing small datasets can be used to create models that generate a reasonable prediction, facilitating health-care delivery. We propose that greater attention (and funding) needs to be directed toward the utilization of existing information resources in parallel with current efforts to create and exploit "big data."

Keywords: big data; health economics; length of stay; small data; surgery.

Figure 1

Predicting patient stay duration. The results from the model are presented using line plots (the median prediction is represented by a solid black line and the gray region represents the 95% quantile). (A) These data show a steady downwards trend in stay duration over the nine relevant years; (B) the duration of stay is longer for operations on Saturdays – most likely due to weekday discharge; (C) for patients above 55 years of age, stay duration rapidly increases with age; and finally, (D) stay duration also increases with operation time – presumably an indicator of complications in surgery or intrinsically more difficult cases. Interestingly, stay duration reaches a plateau for operations over ~3 h, though there are relatively few data points for surgeries of this length – and as such, this relationship should be treated with caution.

Figure 2

Upper 95% confidence interval of predicted stay duration. This figure shows that while, for younger patients, the effect of surgery length on length of stay is relatively weak up to the 3-h mark, for older patients (>55 years), stay duration increases strongly with duration length across all time scales.