Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jul 12;22(14):5202.
doi: 10.3390/s22145202.

An IoT Machine Learning-Based Mobile Sensors Unit for Visually Impaired People

Salam Dhou  1 , Ahmad Alnabulsi  1 , A R Al-Ali  1 , Mariam Arshi  1 , Fatima Darwish  1 , Sara Almaazmi  1 , Reem Alameeri  1
Affiliations

An IoT Machine Learning-Based Mobile Sensors Unit for Visually Impaired People

Salam Dhou et al. Sensors (Basel). .

Abstract

Visually impaired people face many challenges that limit their ability to perform daily tasks and interact with the surrounding world. Navigating around places is one of the biggest challenges that face visually impaired people, especially those with complete loss of vision. As the Internet of Things (IoT) concept starts to play a major role in smart cities applications, visually impaired people can be one of the benefitted clients. In this paper, we propose a smart IoT-based mobile sensors unit that can be attached to an off-the-shelf cane, hereafter a smart cane, to facilitate independent movement for visually impaired people. The proposed mobile sensors unit consists of a six-axis accelerometer/gyro, ultrasonic sensors, GPS sensor, cameras, a digital motion processor and a single credit-card-sized single-board microcomputer. The unit is used to collect information about the cane user and the surrounding obstacles while on the move. An embedded machine learning algorithm is developed and stored in the microcomputer memory to identify the detected obstacles and alarm the user about their nature. In addition, in case of emergencies such as a cane fall, the unit alerts the cane user and their guardian. Moreover, a mobile application is developed to be used by the guardian to track the cane user via Google Maps using a mobile handset to ensure safety. To validate the system, a prototype was developed and tested.

Keywords: IoT; machine learning; sensors; smartphone; visually impaired people; walking assistants.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
System hardware architecture.
Figure 2
Figure 2
Layered software architecture.
Figure 3
Figure 3
Proposed obstacle classification algorithm.
Figure 4
Figure 4
Sequence diagram of the proposed system.
Figure 5
Figure 5
Actual prototype model.
Figure 6
Figure 6
Ultrasonic (US) sensors and cameras.
Figure 7
Figure 7
HOG feature descriptor before and after feature extraction.
Figure 8
Figure 8
Confusion Matrix of Model 1 using SVM (doors vs. upward stairs).
Figure 9
Figure 9
Confusion Matrix of Model 2 using SVM (downward stairs vs. hollow pits).
Figure 10
Figure 10
GPS tracking in mobile app.
Figure 11
Figure 11
Notifications in the mobile app.
See this image and copyright information in PMC

References

    1. World Report on Vision. World Health Organization. 2019. [(accessed on 7 May 2022)]. Available online: https://www.who.int/publications/i/item/9789241516570.
    1. Bourne R., Steinmetz J.D., Flaxman S., Briant P.S., Taylor H.R., Casson R., Bikbov M., Bottone M., Braithwaite T., Bron A.M., et al. Trends in prevalence of blindness and distance and near vision impairment over 30 years: An analysis for the Global Burden of Disease Study. Lancet Glob. Health. 2021;9:e130–e143. doi: 10.1016/S2214-109X(20)30425-3. - DOI - PMC - PubMed
    1. Chang W.-J., Chen L.-B., Chen M.-C., Su J.-P., Sie C.-Y., Yang C.-H. Design and Implementation of an Intelligent Assistive System for Visually Impaired People for Aerial Obstacle Avoidance and Fall Detection. IEEE Sens. J. 2020;20:10199–10210. doi: 10.1109/JSEN.2020.2990609. - DOI
    1. Oviedo-Cáceres M.D.P., Arias-Pineda K.N., Yepes-Camacho M.D.R., Falla P.M. COVID-19 Pandemic: Experiences of People with Visual Impairment. Investig. Educ. Enfermería. 2021;39:e09. doi: 10.17533/udea.iee.v39n1e09. - DOI - PMC - PubMed
    1. Senjam S.S. Impact of COVID-19 pandemic on people living with visual disability. Indian J. Ophthalmol. 2020;68:1367–1370. doi: 10.4103/ijo.IJO_1513_20. - DOI - PMC - PubMed