Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Mar 22;21(6):2212.
doi: 10.3390/s21062212.

Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments

Alberto Brunete  1 , Ernesto Gambao  1 , Miguel Hernando  1 , Raquel Cedazo  2
Affiliations

Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments

Alberto Brunete et al. Sensors (Basel). .

Abstract

This paper presents a new architecture that integrates Internet of Things (IoT) devices, service robots, and users in a smart assistive environment. A new intuitive and multimodal interaction system supporting people with disabilities and bedbound patients is presented. This interaction system allows the user to control service robots and devices inside the room in five different ways: touch control, eye control, gesture control, voice control, and augmented reality control. The interaction system is comprised of an assistive robotic arm holding a tablet PC. The robotic arm can place the tablet PC in front of the user. A demonstration of the developed technology, a prototype of a smart room equipped with home automation devices, and the robotic assistive arm are presented. The results obtained from the use of the various interfaces and technologies are presented in the article. The results include user preference with regard to eye-base control (performing clicks, and using winks or gaze) and the use of mobile phones over augmented reality glasses, among others.

Keywords: AAL; IoT; assistive; eye tracking; low mobility; multimodal interaction; robotics; smart environments.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Network architecture.
Figure 2
Figure 2
Block diagram of the main controller.
Figure 3
Figure 3
Different views of the tablet interface.
Figure 4
Figure 4
Robotic arm joints in detail: (a) joint 1, (b) motors and pulleys, (c) joint 2, and (d) joint 3.
Figure 5
Figure 5
Assistive robotic arm.
Figure 6
Figure 6
Eye-tracking coordinate system and wink detection.
Figure 7
Figure 7
Eye closure detection flow.
Figure 8
Figure 8
Gestures.
Figure 9
Figure 9
Augmented reality interface.
Figure 10
Figure 10
Room.
Figure 11
Figure 11
Answers for questions three to nine: one means hardest, none, and worst while 5 means easiest, very, and best.
See this image and copyright information in PMC

References

    1. Raffaeli L., Montanini L., Gambi E., Spinsante S. Handbook of Research on Human-Computer Interfaces, Developments, and Applications. IGI Global; Pennsylvania, PA, USA: 2016. User interfaces in smart assistive environments: Requirements, devices, applications; pp. 420–443. - DOI
    1. Bui H., Chong N.Y. An Integrated Approach to Human-Robot-Smart Environment Interaction Interface for Ambient Assisted Living; Proceedings of the IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO); Genova, Italy. 27–29 September 2018; pp. 32–37. - DOI
    1. Sacristan M., Brunete A., Hernando M., Gambao E. Robohealth: Smart Room Interface for Hospital Patients; Proceedings of the Robots for Assisted Living Workshop, IEEE/RSJ International Conference on Intelligent Robots and Systems; Madrid, Spain. 5 October 2018.
    1. Wada K., Shibata T., Saito T., Tanie K. Effects of robot-assisted activity for elderly people and nurses at a day service center. Proc. IEEE. 2004;92:1780–1788. doi: 10.1109/JPROC.2004.835378. - DOI
    1. Chang W., Šabanovic S. Potential use of robots in Taiwanese nursing homes; Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI); Tokyo, Japan. 3–6 March 2013; pp. 99–100. - DOI

LinkOut - more resources