Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Dec 10;19(24):16601.
doi: 10.3390/ijerph192416601.

Medical Experts' Agreement on Risk Assessment Based on All Possible Combinations of the COVID-19 Predictors-A Novel Approach for Public Health Screening and Surveillance

Mohd Salami Ibrahim  1 , Nyi Nyi Naing  1 , Aniza Abd Aziz  1 , Mokhairi Makhtar  2 , Harmy Mohamed Yusoff  1 , Nor Kamaruzaman Esa  1 , Nor Iza A Rahman  1 , Myat Moe Thwe Aung  1 , San San Oo  1 , Samhani Ismail  1 , Ras Azira Ramli  1
Affiliations

Medical Experts' Agreement on Risk Assessment Based on All Possible Combinations of the COVID-19 Predictors-A Novel Approach for Public Health Screening and Surveillance

Mohd Salami Ibrahim et al. Int J Environ Res Public Health. .

Abstract

During the initial phase of the coronavirus disease 2019 (COVID-19) pandemic, there was a critical need to create a valid and reliable screening and surveillance for university staff and students. Consequently, 11 medical experts participated in this cross-sectional study to judge three risk categories of either low, medium, or high, for all 1536 possible combinations of 11 key COVID-19 predictors. The independent experts' judgement on each combination was recorded via a novel dashboard-based rating method which presented combinations of these predictors in a dynamic display within Microsoft Excel. The validated instrument also incorporated an innovative algorithm-derived deduction for efficient rating tasks. The results of the study revealed an ordinal-weighted agreement coefficient of 0.81 (0.79 to 0.82, p-value < 0.001) that reached a substantial class of inferential benchmarking. Meanwhile, on average, the novel algorithm eliminated 76.0% of rating tasks by deducing risk categories based on experts' ratings for prior combinations. As a result, this study reported a valid, complete, practical, and efficient method for COVID-19 health screening via a reliable combinatorial-based experts' judgement. The new method to risk assessment may also prove applicable for wider fields of practice whenever a high-stakes decision-making relies on experts' agreement on combinations of important criteria.

Keywords: agreement; combinations; dashboard-based rating; novel method; public health screening; public health surveillance; risk assessment.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Figures

Figure 1
Figure 1
Conceptual framework for the overall methodology of a combinatorial-based prediction model for the COVID-19 Public Health Screening and Surveillance.
Figure 2
Figure 2
Determining and proving 2048 unique combinations of 11 predictors.
Figure 3
Figure 3
Turning unique bits into unique combinations of alphabetical code of predictors.
Figure 4
Figure 4
Removing the combinations which contain the mutually exclusive predictors.
Figure 5
Figure 5
Linking the table of unique combinations with the dynamic display of the rating dashboard.
Figure 6
Figure 6
The final layout of the dashboard-based rating.
Figure 7
Figure 7
Validation of a section of the dashboard-based rating via comparison and inspection with the manual record.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Rajani R., Klein J.L. Infective endocarditis: A contemporary update. Clin. Med. 2020;20:31–35. doi: 10.7861/clinmed.cme.20.1.1. - DOI - PMC - PubMed
    1. Kirsch B., Aziz M., Kumar S., Burke M., Webster T., Immadi A., Sam M., Lal A., Estrada-Y-Martin R.M., Cherian S., et al. Wells Score to Predict Pulmonary Embolism in Patients with Coronavirus Disease 2019. Am. J. Med. 2021;134:688–690. doi: 10.1016/j.amjmed.2020.10.044. - DOI - PMC - PubMed
    1. Patel S. Calculated decisions: CURB-65 score for pneumonia severity. Emerg. Med. 2021;23((Suppl. 2)):CD1–CD2. - PubMed
    1. Ismail A., Ibrahim M.S., Ismail S., Aziz A.A., Yusoff H.M., Mokhtar M., Juahir H. Development of COVID-19 Health-Risk Assessment and Self-Evaluation (CHaSe): A health screening system for university students and staff during the movement control order (MCO) Netw. Model. Anal. Health Inform. Bioinform. 2022;11:21. doi: 10.1007/s13721-022-00357-3. - DOI - PMC - PubMed
    1. Ibrahim M.S., Hassan N.M., Aziz A.A., Yusoff H.M., Esa N.K., Makhtar M., Rahman N.I.A., Ismail S., Ramli R.A., Naing N.N., et al. Clinical manifestations of COVID-19; what have we learned from the global database? Bangladesh J. Med. Sci. 2022;21:702–709. doi: 10.3329/bjms.v21i3.59588. - DOI

Publication types