-
CiteScore
-
Impact Factor
IECE Transactions on Sensing, Communication, and Control, 2024, Volume 1, Issue 2: 89-100

Research Article | 30 October 2024
1 School of Computer and Information Engineering, Beijing Technology and Business University, Beijing 100048, China
2 Department of Information Engineering, University of Padua, Italy
* Corresponding author: Shenglun Yi, email: [email protected]
Received: 11 September 2024, Accepted: 15 October 2024, Published: 30 October 2024  

Abstract
Humanoid robots have much weight in many fields. Their efficient and intuitive control input is critically important and, in many cases, requires remote operation. In this paper, we investigate the potential advantages of inertial sensors as a key element of command signal generation for humanoid robot control systems. The goal is to use inertial sensors to detect precisely when the user is moving which enables precise control commands. The finger gestures are initially captured as signals coming from the inertial sensor. Movement commands are extracted from these signals using filtering and recognition. These commands are subsequently translated into robot movements according to the attitude angle of the inertial sensor. The accuracy and effectiveness of the finger movements using this method are experimentally demonstrated. The implementation of inertial sensors for gesture recognition simplifies the process of sending control inputs, paving the way for more user-friendly and efficient interfaces in humanoid robot operations. This approach not only enhances the precision of control commands but also significantly improves the practicality of deploying humanoid robots in real-world scenarios.

Graphical Abstract
Enhanced Recognition for Finger Gesture-Based Control in Humanoid Robots Using Inertial Sensors

Keywords
inertial sensor
finger gesture
NAO humanoid robot
quaternions
motion capture

References

[1] Katona, J. (2021). A review of human–computer interaction and virtual reality research fields in cognitive InfoCommunications. Applied Sciences, 11(6), 2646.

[2] Bhame, V., Sreemathy, R., & Dhumal, H. (2014, September). Vision based hand gesture recognition using eccentric approach for human computer interaction. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 949-953). IEEE.

[3] Chakravarthi, S. S., Rao, B., Challa, N. P., Ranjana, R., & Rai, A. (2023). Gesture Recognition for Enhancing Human Computer Interaction. Journal of Scientific & Industrial Research, 82(04), 438-443.

[4] Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015). Hand gesture recognition with 3D convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 1-7).

[5] Devineau, G., Moutarde, F., Xi, W., & Yang, J. (2018, May). Deep learning for hand gesture recognition on skeletal data. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) (pp. 106-113). IEEE.

[6] Tran, D. S., Ho, N. H., Yang, H. J., Baek, E. T., Kim, S. H., & Lee, G. (2020). Real-time hand gesture spotting and recognition using RGB-D camera and 3D convolutional neural network. Applied Sciences, 10(2), 722.

[7] Jaramillo-Yánez, A., Benalcázar, M. E., & Mena-Maldonado, E. (2020). Real-time hand gesture recognition using surface electromyography and machine learning: A systematic literature review. Sensors, 20(9), 2467.

[8] Pan, M., Tang, Y., & Li, H. (2023). State-of-the-art in data gloves: A review of hardware, algorithms, and applications. IEEE Transactions on Instrumentation and Measurement, 72, 1-15.

[9] Kim, B. K., Jang, M., Kim, J. S., Kang, K., Kim, D. E., & Kim, J. (2022). Investigation of FBG linear/angular acceleration sensor for novel typeinertial measurement. IEEE Transactions on Industrial Electronics, 70(6), 6377-6385.

[10] Sonchan, P., Ratchatanantakit, N., O-larnnithipong, N., Adjouadi, M., & Barreto, A. (2023, July). A Self-contained Approach to MEMS MARG Orientation Estimation for Hand Gesture Tracking in Magnetically Distorted Environments. In International Conference on Human-Computer Interaction (pp. 585-602). Cham: Springer Nature Switzerland.

[11] Wang, Y., & Zhao, Y. (2023). Handwriting recognition under natural writing habits based on a low-cost inertial sensor. IEEE Sensors Journal.

[12] Nguyen, V., Rupavatharam, S., Liu, L., Howard, R., & Gruteser, M. (2019, November). HandSense: capacitive coupling-based dynamic, micro finger gesture recognition. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems (pp. 285-297).

[13] Gromov, B., Abbate, G., Gambardella, L. M., & Giusti, A. (2019, May). Proximity human-robot interaction using pointing gestures and a wrist-mounted IMU. In 2019 International Conference on Robotics and Automation (ICRA) (pp. 8084-8091). IEEE.

[14] Ling, Y., Chen, X., Ruan, Y., Zhang, X., & Chen, X. (2021). Comparative study of gesture recognition based on accelerometer and photoplethysmography sensor for gesture interactions in wearable devices. IEEE Sensors Journal, 21(15), 17107-17117.

[15] Picerno, P., Iosa, M., D’Souza, C., Benedetti, M. G., Paolucci, S., & Morone, G. (2021). Wearable inertial sensors for human movement analysis: a five-year update. Expert review of medical devices, 18(sup1), 79-94.

[16] Hao, M., Chen, K., & Fu, C. (2019). Smoother-based 3-D foot trajectory estimation using inertial sensors. IEEE Transactions on Biomedical engineering, 66(12), 3534-3542.

[17] Calado, A., Lin, B. S., Lee, I. J., & Saggio, G. (2023). Quasi-Static Measurement Performances of Flex Sensor Based and Inertial Measurement Unit Based Sensory Gloves. IEEE Sensors Journal.

[18] Li, G., Wan, B., Su, K., Huo, J., Jiang, C., & Wang, F. (2023). sEMG and IMU Data-based Hand Gesture Recognition Method using Multi-stream CNN with a Fine-tuning Transfer Framework. IEEE Sensors Journal.

[19] Dong, Y., Liu, J., & Yan, W. (2021). Dynamic hand gesture recognition based on signals from specialized data glove and deep learning algorithms. IEEE Transactions on Instrumentation and Measurement, 70, 1-14.

[20] Lee, M., & Bae, J. (2020). Deep learning based real-time recognition of dynamic finger gestures using a data glove. IEEE Access, 8, 219923-219933.

[21] Theodoridou, E., Cinque, L., Mignosi, F., Placidi, G., Polsinelli, M., Tavares, J. M. R., & Spezialetti, M. (2022). Hand tracking and gesture recognition by multiple contactless sensors: A survey. IEEE Transactions on Human-Machine Systems, 53(1), 35-43.

[22] Jin, X. B., Sun, S., Wei, H., & Yang, F. B. (Eds.). (2018). Advances in multi-sensor information fusion: Theory and applications 2017. MDPI.

[23] Pramanik, R., Sikdar, R., & Sarkar, R. (2023). Transformer-based deep reverse attention network for multi-sensory human activity recognition. Engineering Applications of Artificial Intelligence, 122, 106150.

[24] Ryumin, D., Ivanko, D., & Ryumina, E. (2023). Audio-visual speech and gesture recognition by sensors of mobile devices. Sensors, 23(4), 2284.

[25] Qi, W., Ovur, S. E., Li, Z., Marzullo, A., & Song, R. (2021). Multi-sensor guided hand gesture recognition for a teleoperated robot using a recurrent neural network. IEEE Robotics and Automation Letters, 6(3), 6039-6045.

[26] Bai, Y., Yan, B., Zhou, C., Su, T., & Jin, X. (2023). State of art on state estimation: Kalman filter driven by machine learning. Annual Reviews in Control, 56, 100909.

[27] Jin, X. B., Robert Jeremiah, R. J., Su, T. L., Bai, Y. T., & Kong, J. L. (2021). The new trend of state estimation: From model-driven to hybrid-driven methods. Sensors, 21(6), 2085.

[28] Khodabin, M., & Rostami, M. (2015). Mean square numerical solution of stochastic differential equations by fourth order Runge-Kutta method and its application in the electric circuits with noise. Advances in Difference Equations, 2015(1), 62.

[29] Bortolami, S. B., Pierobon, A., DiZio, P., & Lackner, J. R. (2006). Localization of the subjective vertical during roll, pitch, and recumbent yaw body tilt. Experimental brain research, 173, 364-373.

[30] Jin, X. B., Su, T. L., Kong, J. L., Bai, Y. T., Miao, B. B., & Dou, C. (2018). State-of-the-art mobile intelligence: Enabling robots to move like humans by estimating mobility with artificial intelligence. Applied Sciences, 8(3), 379.

[31] Nagy, E., Karl, É., & Molnár, G. (2024). Exploring the Role of Human-Robot Interactions, within the Context of the Effectiveness of a NAO Robot. Acta Polytechnica Hungarica, 21(3).

[32] Mutawa, A. M., Al Mudhahkah, H. M., Al-Huwais, A., Al-Khaldi, N., Al-Otaibi, R., & Al-Ansari, A. (2023). Augmenting Mobile App with NAO Robot for Autism Education. Machines, 11(8), 833.

[33] WANG, C., BAI, Y., CAI, L., HU, M., LIU, L., MA, Y., ... & ZHOU, Z. (2023). High precision electrostatic inertial sensor. Scientia Sinica Physica, Mechanica & Astronomica, 53(5), 250401.

[34] Sameni, R. (2017). Online filtering using piecewisesmoothness priors: Application to normal and abnormal electrocardiogram denoising. Signal Processing, 133, 52-63.


Cite This Article
APA Style
Xie, J., Na, X., & Yi, S. (2024). Enhanced Recognition for Finger Gesture-Based Control in Humanoid Robots Using Inertial Sensors. IECE Transactions on Sensing, Communication, and Control, 1(2), 89–100. https://doi.org/10.62762/TSCC.2024.805710

Article Metrics
Citations:

Crossref

0

Scopus

0

Web of Science

0
Article Access Statistics:
Views: 168
PDF Downloads: 3

Publisher's Note
IECE stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions
IECE or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
IECE Transactions on Sensing, Communication, and Control

IECE Transactions on Sensing, Communication, and Control

ISSN: 3065-7431 (Online) | ISSN: 3065-7423 (Print)

Email: [email protected]

Portico

Portico

All published articles are preserved here permanently:
https://www.portico.org/publishers/iece/

Copyright © 2024 Institute of Emerging and Computer Engineers Inc.