Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 May 22;20(5):e0322854.
doi: 10.1371/journal.pone.0322854. eCollection 2025.

AISIM: evaluating impacts of user interface elements of an AI assisting tool

Kannika Wiratchawa  1 , Yupaporn Wanna  1 , Prem Junsawang  1 , Attapol Titapun  2   3 , Anchalee Techasen  4   3 , Arunnit Boonrod  5 , Vallop Laopaiboon  5   3 , Nittaya Chamadol  5   3 , Sahan Bulathwela  6 , Thanapong Intharah  1
Affiliations

AISIM: evaluating impacts of user interface elements of an AI assisting tool

Kannika Wiratchawa et al. PLoS One. .

Abstract

While Artificial Intelligence (AI) has demonstrated human-level capabilities in many prediction tasks, collaboration between humans and machines is crucial in mission-critical applications, especially in the healthcare sector. An important factor that enables successful human-AI collaboration is the user interface (UI). This paper evaluated the UI of BiTNet, an intelligent assisting tool for human biliary tract diagnosis via ultrasound images. We evaluated the UI of the assisting tool with 11 healthcare professionals through two main research questions: 1) did the assisting tool help improve the diagnosis performance of the healthcare professionals who use the tool? and 2) how did different UI elements of the assisting tool influence the users' decisions? To analyze the impacts of different UI elements without multiple rounds of experiments, we propose the novel AISIM strategy. We demonstrated that our proposed strategy, AISIM, can be used to analyze the influence of different elements in the user interface in one go. Our main findings show that the assisting tool improved the diagnostic performance of healthcare professionals from different levels of experience (OR = 3.326, p-value <10-15). In addition, high AI prediction confidence and correct AI attention area provided higher than twice the odds that the users would follow the AI suggestion. Finally, the interview results agreed with the experimental result that BiTNet boosted the users' confidence when they were assigned to diagnose abnormality in the biliary tract from the ultrasound images.

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. Explaining the user interface elements of the intelligence assisting tool.
Fig 2
Fig 2. Architecture of the BiTNet model.
Fig 3
Fig 3. The experiment was designed to evaluate the participants’s performance when assisted vs unassisted.
Group 1 diagnosed 150 test images without the assisting tool in the first session, then diagnosed the same test set with the assisting tool in the first session. Group 2 diagnosed 150 test images with an assisting tool in the first session, and then diagnosed the same test set without the assisting tool in the second session.
Fig 4
Fig 4. Comparison between accuracies of participants from different levels of experience when assisted and unassisted.
See this image and copyright information in PMC

References

    1. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, et al.. Intriguing properties of neural networks. arXiv, preprint, arXiv:13126199. 2013.
    1. Goodfellow I, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. arXiv, preprint, arXiv:14126572. 2014.
    1. Su J, Vargas DV, Sakurai K. One pixel attack for fooling deep neural networks. IEEE Trans Evol Computat. 2019;23(5):828–41. doi: 10.1109/tevc.2019.2890858 - DOI
    1. Penpong N, Wanna Y, Kamjanlard C, Techasen A, Intharah T. Attacking the out-of-domain problem of a parasite egg detection in-the-wild. Heliyon. 2024;10(4):e26153. doi: 10.1016/j.heliyon.2024.e26153 - DOI - PMC - PubMed
    1. Bonagiri K, VS NM, Gopalsamy M, Iyswariya A, Sultanuddin S, et al. AI-driven healthcare cyber-security: protecting patient data and medical devices. In: 2024 Second International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI). IEEE; 2024, pp. 107–12.

LinkOut - more resources