Benchmarking performance of annual burn probability modeling against subsequent wildfire activity in California

Abstract

Wildfire simulation is deployed extensively to support risk management, and in the US has driven billions in federal investment. Foundational to strategic risk analysis is spatial information on the likelihood of burning in a fire year, typically provided by burn probability (BP) models. The recency of BP maps is a key driver of their accuracy, especially in disturbed landscapes that have experienced changes in fire spread potential. Few published examples exist comparing BP values against subsequent fire activity, and none to our knowledge evaluate annually updated BP maps. Here, we present a novel performance evaluation of the operational wildfire simulation system FSim, confronting updated BP maps with subsequent fire activity across the state of California over a 4-year period (2020-2023). Results show strong predictive ability: across 5 equal-area BP classes, 56.7-79.8% of the burned area occurred in the top 20% of mapped area; mean (median) BP values in burned areas were 238.5-348.8% (551.4-880.7%) greater than in unburned areas; differences in empirical cumulative distribution functions of BP for burned/unburned areas were statistically significant; Logarithmic Skill Scores ranged from - 0.072 to 0.389 against two reference models. Findings indicate reliable forecast performance and useful application of up-to-date BP maps, critical to support ongoing wildfire risk mitigation.

Keywords: Decision support; Forecast verification; Hazard; Risk; Simulation; Wildfire.

Fig. 1

Burn probability values and observed fire perimeters. The state map shows 2020 BP and 2020–2023 fire perimeters. Inset panels contain BP and fire perimeters for respective years. The chronological inset map shows burned areas that are accounted for in subsequent BP maps. Map created in QGIS 3.34 using Esri World Terrain Base © Esri, USGS, NOAA. State boundaries from U.S. Census Bureau TIGER/Line® Shapefiles (public domain).

Fig. 2

Interannual variability in simulated BP due to prior large fire disturbance; location matches inset map from Fig. 1. Panels (a) and (b) show observed 2020 wildfires with pre-and post-simulation BPs (2020 and 2021 BP maps), respectively. Panels (c) and (d) show observed 2021 wildfires with pre-and post-simulation BPs (2021 and 2022 BP maps), respectively. The map reflects both the impact of wildfires prior to 2020 as well as wildfires in 2020 and 2021 on subsequent fire spread potential and BP values. Map created in QGIS 3.34 using Esri World Terrain Base © Esri, USGS, NOAA.

Fig. 3

Proportional comparison of expected area burned (eAB) and observed area burned (oAB) across five equal area BP classes. BP class definitions vary by year; exact delineations of the BP bins are provided in supplementary materials.

Fig. 4

Violin plots comparing the distribution of burn probability values for burned and unburned areas. The horizontal lines represent the median (black) and mean (blue) within each respective class. Panels (a–d) correspond to years 2020–2023, respectively.

Fig. 5

Cumulative distribution plots comparing burn probability values in burned versus unburned areas. Panels (a–d) correspond to years 2020–2023, respectively.