Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Mar 26;13(4):521.
doi: 10.3390/mi13040521.

Control of a Drone in Virtual Reality Using MEMS Sensor Technology and Machine Learning

Florin Covaciu  1 , Anca-Elena Iordan  2
Affiliations

Control of a Drone in Virtual Reality Using MEMS Sensor Technology and Machine Learning

Florin Covaciu et al. Micromachines (Basel). .

Abstract

In recent years, drones have been widely used in various applications, from entertainment, agriculture, their use in photo and video services, military applications and so on. The risk of accidents while using a drone is quite high. To meet this risk, the most important solution is to use a device that helps and simplifies the control of a drone; in addition, the training of drone pilots is very important. To train the drone pilots, both physical and virtual environments can be used, but the probability of an accident is higher for beginners, so the safest method is to train in a virtual environment. The aim of this study is to develop a new device for controlling a drone in a virtual environment. This device is attached to the upper limb of the person involved in the control of that drone. For precise control, the newly created device uses MEMS sensor technology and artificial intelligence-specific methods.

Keywords: MEMS sensors; accelerometer; gyroscope; machine learning; magnetometer; sensors fusion; virtual reality.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Microstructure of an accelerometer.
Figure 2
Figure 2
(a) MEMS magnetometer—(b) Hall effect.
Figure 3
Figure 3
MEMS magnetometer—Magneto-resistive effect.
Figure 4
Figure 4
Gyroscope.
Figure 5
Figure 5
Microstructure of a gyroscope.
Figure 6
Figure 6
Interconnection of components.
Figure 7
Figure 7
Device for receiving and transmitting data from sensors.
Figure 8
Figure 8
Software architecture.
Figure 9
Figure 9
UML use cases diagram.
Figure 10
Figure 10
UML class diagram.
Figure 11
Figure 11
User Interface—manual control.
Figure 12
Figure 12
User Interface—automatic control.
Figure 13
Figure 13
(a) LSTM architecture. (b) The modified MLP architecture.
Figure 14
Figure 14
Accuracy corresponding to the first architecture.
Figure 15
Figure 15
Accuracy corresponding to the second architecture.
Figure 16
Figure 16
Drone control positions.
Figure 17
Figure 17
(a) Drone control for obstacles crossing. (b) Passing the drone through obstacles.
Figure 18
Figure 18
Comparing results.
See this image and copyright information in PMC

References

    1. Zhu J., Liu X., Shi Q., He T., Sun Z., Guo X., Liu W., Sulaiman O.B., Dong B., Lee C. Development Trends and Perspectives of Future Sensors and MEMS/NEMS. Micromachines. 2020;11:7. doi: 10.3390/mi11010007. - DOI - PMC - PubMed
    1. Fraune M.R., Khalaf A.S., Zemedie M., Pianpak P., Mianji Z.N., Alharthi S.A., Dolgov I., Hamilton B., Tran S., Toups Z.O. Developing Future Wearable Interfaces for Human-Drone Teams through a Virtual Drone Search Game. Int. J. Hum. Comput. Stud. 2021;147:102573. doi: 10.1016/j.ijhcs.2020.102573. - DOI
    1. Ullo S.L., Del Rosso M.P., Sebastianelli A., Puglisi E., Bernardi M.L., Cimitile M. How to develop your network with Python and Keras. Artif. Intell. Appl. Satell. Based Remote Sens. Data Earth Obs. 2021;98:131–158.
    1. Covaciu F., Pisla A., Vaida C., Gherman B., Pisla D. Development of a Virtual Reality Simulator for a Lower Limb Rehabilitation; Proceedings of the 2020 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR); Cluj-Napoca, Romania. 21–23 May 2020; pp. 1–6.
    1. Covaciu F., Gherman B., Pisla A., Carbone G., Pisla D. New Trends in Mechanism and Machine Science, Proceedings of the European Conference on Mechanism Science (EuCoMeS) Springer; Cham, Switzerland: 2020. Rehabilitation System with Integrated Visual Stimulation; pp. 131–137.

LinkOut - more resources