A review of existing and potential computer user interfaces for modern radiology

Abstract

The digitalization of modern imaging has led radiologists to become very familiar with computers and their user interfaces (UI). New options for display and command offer expanded possibilities, but the mouse and keyboard remain the most commonly utilized, for usability reasons. In this work, we review and discuss different UI and their possible application in radiology. We consider two-dimensional and three-dimensional imaging displays in the context of interventional radiology, and discuss interest in touchscreens, kinetic sensors, eye detection, and augmented or virtual reality. We show that UI design specifically for radiologists is key for future use and adoption of such new interfaces. Next-generation UI must fulfil professional needs, while considering contextual constraints. TEACHING POINTS: • The mouse and keyboard remain the most utilized user interfaces for radiologists. • Touchscreen, holographic, kinetic sensors and eye tracking offer new possibilities for interaction. • 3D and 2D imaging require specific user interfaces. • Holographic display and augmented reality provide a third dimension to volume imaging. • Good usability is essential for adoption of new user interfaces by radiologists.

Keywords: Computed tomodensitometry; Computer user interface; Interventional radiology; Virtual reality; Volume rendering.

Fig. 1

The usability of a computer user interface (CUI) in radiology is evaluated by three indicators. The UI is designed to maximize the usability in a specified context of use. In medical imaging, the usage context can be defined as a user (the reader of the images) inside his environment

Fig. 2

The human–machine interaction is constrained by human and contextual factors. Typically, the display is around 60 cm from the radiologist. At this distance, the field of view is around 50 cm (around 21 in.). Considering the maximum angular resolution of the eye, the display can have maximum pitch of 0.21 mm. This corresponds to a 3-megapixel screen. The human retina contains two types of photoreceptors, rods and cones. The cones are densely packed in a central yellow spot called the “macula” and provide maximum visual acuity. Visual examination of small detail involves focusing light from that detail onto the macula. Peripheral vision and rods are responsible for night vision and motion detection

Fig. 3

History of common computer user interfaces. The most common user interface (UI) is the graphical UI (GUI) used by operating systems (OS) of popular personal computers in the 1980s. It is designed with a mouse input to point to icons, menus and windows. Recently, new OS with specific interfaces for touchscreens have emerged, known as natural user interfaces (NUI)

Fig. 4

Potential use of manual and gaze input cascaded (MAGIC) pointing for diagnostic radiology. The radiologist is examining lung parenchyma. When he focuses on an anomaly, the eye tracking device automatically moves the pointer around the region of interest. A large amount of mouse movement is eliminated (dotted arrow), and is limited to fine pointing and zooming. This cascade follows the observer's examination, making the interaction more natural

Fig. 5

Tactile version of the Anywhere viewer (Therapixel, France). Cloud computing allows powerful post-processing with online PACS

Fig. 6

Potential use of Surface Studio^® (Microsoft, Redmond, WA, USA). The screen is tactile (a). The stylet would be handy for direct measurement and annotation of the image (b). The wheel could be used to select functions such as Windows and for scrolling of images (c). A properly designed software interface could replace traditional mouse and computer workstations

Fig. 7

Touchless image viewer for operating rooms, Fluid (Therapixel, Paris, France). The surgeon or the interventional radiologist interacts in sterile conditions with gloved hands. The viewer interface is redesigned without a pointer; the tools are selected with lateral movements

Fig. 8

Current possibility for 3D volume rendering (VR). VR post-processed from MRI acquisition (a, b). Hyper-realistic cinematic VR processed from CT scan acquisition (c, d)

Fig. 9

True 3D viewer (EchoPixel, Mountain View, CA, USA). This is the first holographic imaging viewer approved by the FDA as a tool for diagnosis as well as surgical planning. A stylet can interact with the displayed volume of the colon, providing an accurate three-dimensional representation of patient anatomy

Fig. 10

Augmented reality using 3D medical images is employed for planning and guiding surgical procedures. Surgeons wear a head-mounted optical device to create augmented reality (a). They can interact with the volume of the patient’s liver during surgery. Spatial augmented reality obtained by projection of the volume rendering on the patient (b, c). This see-thru visualization helps in guiding surgery

Fig. 11

Virtual reality headset with medical images. The user wears a head-mounted display that immerses him in the simulated environments. A hand device allows him to interact with the virtual objects (a). The user experiences a first-person view inside and interacts with the 3D volume of the medical images (a). The volume can be sliced as a CT scan in any reformatted axis (b)

Fig. 12

User-centred design for diagnostic imaging viewer. To maximize usability, the design of the interface needs to integrate the user and environmental requirements and constraints. Automatization and CAD have to facilitate use in order to minimize time and effort for image analysis