Staying Ahead of the Epidemiologic Curve: Evaluation of the British Columbia Asthma Prediction System (BCAPS) During the Unprecedented 2018 Wildfire Season

Abstract

Background: The modular British Columbia Asthma Prediction System (BCAPS) is designed to reduce information burden during wildfire smoke events by automatically gathering, integrating, generating, and visualizing data for public health users. The BCAPS framework comprises five flexible and geographically scalable modules: (1) historic data on fine particulate matter (PM_2.5) concentrations; (2) historic data on relevant health indicator counts; (3) PM_2.5 forecasts for the upcoming days; (4) a health forecasting model that uses the relationship between (1) and (2) to predict the impacts of (3); and (5) a reporting mechanism. Methods: The 2018 wildfire season was the most extreme in British Columbia history. Every morning BCAPS generated forecasts of salbutamol sulfate (e.g., Ventolin) inhaler dispensations for the upcoming days in 16 Health Service Delivery Areas (HSDAs) using random forest machine learning. These forecasts were compared with observations over a 63-day study period using different methods including the index of agreement (IOA), which ranges from 0 (no agreement) to 1 (perfect agreement). Some observations were compared with the same period in the milder wildfire season of 2016 for context. Results: The mean province-wide population-weighted PM_2.5 concentration over the study period was 22.0 μg/m³, compared with 4.2 μg/m³ during the milder wildfire season of 2016. The PM_2.5 forecasts underpredicted the severe smoke impacts, but the IOA was relatively strong with a population-weighted average of 0.85, ranging from 0.65 to 0.95 among the HSDAs. Inhaler dispensations increased by 30% over 2016 values. Forecasted dispensations were within 20% of the observed value in 71% of cases, and the IOA was strong with a population-weighted average of 0.95, ranging from 0.92 to 0.98. All measures of agreement were correlated with HSDA population, where BCAPS performance was better in the larger populations with more moderate smoke impacts. The accuracy of the health forecasts was partially dependent on the accuracy of the PM_2.5 forecasts, but they were robust to over- and underpredictions of PM_2.5 exposure. Conclusions: Daily reports from the BCAPS framework provided timely and reasonable insight into the population health impacts of predicted smoke exposures, though more work is necessary to improve the PM_2.5 and health indicator forecasts.

Keywords: data integration; forecasting; public health; surveillance; wildfire smoke.

Figure 1

Illustration of the process for public health decision-making during wildfire smoke events without the British Columbia Asthma Prediction System (BCAPS) framework (A) and with the BCAPS framework described here (B).

Figure 2

Map of British Columbia showing the 16 Health Service Delivery Areas (HSDAs) in the study area. The HSDAs are color-coded to show the population-weighted average of fine particulate matter (PM_2.5) concentrations during the study period (15 July−15 September 2018) from the Optimized Statistical Smoke Exposure Model (OSSEM). Locations of the 62 regulatory air quality monitoring stations are also shown.

Figure 3

The upper time series (A) shows the daily population-weighted average of particulate matter (PM_2.5) concentrations in British Columbia during the summer of 2018 from the Optimized Statistical Smoke Exposure Model (OSSEM). The lower time series (B) shows the daily province-wide counts of salbutamol sulfate (i.e., Ventolin®) inhaler dispensations, illustrating clear day-of-week differences. The 63-day study period for evaluation of the British Columbia Asthma Prediction System (BCAPS) is indicated by the dashed lines (15 July−15 September 2018).

Figure 4

Four pages extracted from the British Columbia Asthma Prediction System (BCAPS) report disseminated to public health users on August 21, 2018. The maps show the blended forecast for fine particulate matter (PM_2.5) concentrations on August 21, 2018 (i.e., today) and August 22, 2018 (i.e., tomorrow) across all populated 5 × 5 km grid cells in the province. The range of PM_2.5 forecasts within the two Health Service Delivery Areas (HSDAs) is shown as vertical gray lines on the far right-hand side of the lower charts. The top of the range was used to make the health forecasts on the far right-hand side of the upper charts. The historical information on the left-hand side of the lower and upper charts shows how BCAPS forecasts have compared with observations over the previous 3 weeks, where the observed PM_2.5 concentrations are taken from the provincial air quality monitoring network.

Figure 5

Scatter plots showing the day₀ (i.e., today) agreement between observed and forecast values for (A) fine particulate matter (PM_2.5) and (B) inhaler dispensation counts for each of the 16 Health Service Delivery Areas (HSDAs). The points are color coded to group HSDAs by health region, and each plot shows a dashed 1:1 line for reference. For (B), differences in population sizes between the HSDAs are made evident by differences in the range of inhaler dispensations counts. Bi- or tri-modal distributions in the larger HSDAs are due to differences between weekdays and weekends, when many pharmacies are closed.

Figure 6

The index of agreement (IOA) was used to compare daily blended forecasts for fine particulate matter (PM_2.5) concentrations and salbutamol sulfate (i.e., Ventolin®) inhaler dispensations. Values are shown for the forecasts made for today (day₀, top) and tomorrow (day₊₁, bottom).

Figure 7

The root mean squared error (RMSE) for the inhaler dispensation forecasts per 10 000 population in each Health Service Delivery Area (HSDA) during the study period (15 July−15 September 2018). Summary includes all forecasts for day₀ (i.e., today) and day₊₁ (i.e., tomorrow).