Workload predictions from a biomathematical model compared to top-of-descent NASA Task Load Index scores in commercial pilots

Abstract

Biomathematical models of fatigue (BMMFs) are commonly used to predict cognitive alertness in commercial aviation. Accounting for workload in association with routine job tasks may help BMMFs to more accurately predict fatigue in real world operations. This study compared the accuracy of BMMF workload predictions (SF Workload) against pilot self-report of workload during normal flight operations. Ninety-nine (N=99) pilots from a major Asia-based airline completed the NASA Task Load Index (TLX) at top of descent (TOD) during a multiple-flight three-day roster that consisted of daytime flying. SF Workload predictions and TLX scores were normalized to a 100-point scale and compared using equivalence testing. SF Workload predictions were statistically non-different from pilot TLX scores at the same TOD (64 ± 7 vs. 65 ± 15; both t=1.56, p=0.06) using the two one-sided t-test (TOST) approach, indicating high workload and that BMMF predictions are non-inferior to pilot self-report as a means of estimating workload. Establishing the accuracy of workload predictions against real-world reports in a commercial pilot population is an important step towards risk management in situations where high workload may create a safety risk.

Keywords: Aviation; Biomathematical modeling; Fatigue; National Aeronautics and Space Administration Task Load Index (NASA TLX); Workload.

Fig. 1.

Average workload ratings by National Aeronautics and Space Administration Task Load Index (NASA TLX) Domain. A) Rosters and Days and B) Pilot Rank. Workload ratings are shown on a scale of Very Low to Very High on the y-axis. A) Rosters are shown as clustered bars organized by Day 1 and Day 3 across NASA TLX (mean and standard deviation) sub-domains on the x-axis. B) Rank is shown as clustered bars across NASA TLX (mean and standard deviation) sub-domains on the x-axis. * represents significance at p≤0.05.

Fig. 2.

Predicted Sleep Activity Fatigue Task Effectiveness (SAFTE)-FAST Workload Compared to Global Scores. Predicted SF Workload (mean and standard deviation) are depicted as clustered bars across roster days and flights on the x-axis. National Aeronautics and Space Administration Task Load Index (NASA TLX) global scores (means and standard deviations) are shown as markers above Flight 3 for Days 1 and 3 by roster. Workload ratings are shown on a scale of Very Low to Very High on the y-axis.

Fig. 3.

Equivalence Plot of Mean Difference in SFC Workload and National Aeronautics and Space Administration Task Load Index (NASA TLX) at TOD ± 95% Confidence Interval. Graphical non-inferiority equivalence plot depicting the difference between SF Workload predictions and NASA TLX at TOD with 95% CIs. Lower equivalence bound (LEB) (−3.23) and upper equivalence bound (UEB) (3.23) are depicted by the dashed vertical lines, and the equivalence limit (0) is depicted by a solid vertical line. The solid horizontal line depicts 95% CIs surrounding the mean difference between SF Workload and NASA TLX global scores. The mean difference between means (1.86) is indicated by the black circular marker at the center of the horizontal line.