Humanoid robots have much weight in many fields. Their efficient and intuitive control input is critically important and, in many cases, requires remote operation. In this paper, we investigate the potential advantages of inertial sensors as a key element of command signal generation for humanoid robot control systems. The goal is to use inertial sensors to detect precisely when the user is moving which enables precise control commands. The finger gestures are initially captured as signals coming from the inertial sensor. Movement commands are extracted from these signals using filtering and recognition. These commands are subsequently translated into robot movements according to the attitude angle of the inertial sensor. The accuracy and effectiveness of the finger movements using this method are experimentally demonstrated. The implementation of inertial sensors for gesture recognition simplifies the process of sending control inputs, paving the way for more user-friendly and efficient interfaces in humanoid robot operations. This approach not only enhances the precision of control commands but also significantly improves the practicality of deploying humanoid robots in real-world scenarios.
Graphical Abstract
Keywords
inertial sensor
finger gesture
NAO humanoid robot
quaternions
motion capture
Funding
This work was supported without any funding.
Cite This Article
APA Style
Xie, J., Na, X., & Yi, S. (2024). Enhanced Recognition for Finger Gesture-Based Control in Humanoid Robots Using Inertial Sensors. IECE Transactions on Sensing, Communication, and Control, 1(2), 89–100. https://doi.org/10.62762/TSCC.2024.805710
References
Katona, J. (2021). A review of human–computer interaction and virtual reality research fields in cognitive InfoCommunications. Applied Sciences, 11(6), 2646. [Google Scholar]
Bhame, V., Sreemathy, R., & Dhumal, H. (2014, September). Vision based hand gesture recognition using eccentric approach for human computer interaction. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 949-953). IEEE. [Google Scholar]
Chakravarthi, S. S., Rao, B., Challa, N. P., Ranjana, R., & Rai, A. (2023). Gesture Recognition for Enhancing Human Computer Interaction. Journal of Scientific & Industrial Research, 82(04), 438-443. [Google Scholar]
Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015). Hand gesture recognition with 3D convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 1-7). [Google Scholar]
Devineau, G., Moutarde, F., Xi, W., & Yang, J. (2018, May). Deep learning for hand gesture recognition on skeletal data. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) (pp. 106-113). IEEE. [Google Scholar]
Tran, D. S., Ho, N. H., Yang, H. J., Baek, E. T., Kim, S. H., & Lee, G. (2020). Real-time hand gesture spotting and recognition using RGB-D camera and 3D convolutional neural network. Applied Sciences, 10(2), 722. [Google Scholar]
Jaramillo-Yánez, A., Benalcázar, M. E., & Mena-Maldonado, E. (2020). Real-time hand gesture recognition using surface electromyography and machine learning: A systematic literature review. Sensors, 20(9), 2467. [Google Scholar]
Pan, M., Tang, Y., & Li, H. (2023). State-of-the-art in data gloves: A review of hardware, algorithms, and applications. IEEE Transactions on Instrumentation and Measurement, 72, 1-15. [Google Scholar]
Kim, B. K., Jang, M., Kim, J. S., Kang, K., Kim, D. E., & Kim, J. (2022). Investigation of FBG linear/angular acceleration sensor for novel typeinertial measurement. IEEE Transactions on Industrial Electronics, 70(6), 6377-6385. [Google Scholar]
Sonchan, P., Ratchatanantakit, N., O-larnnithipong, N., Adjouadi, M., & Barreto, A. (2023, July). A Self-contained Approach to MEMS MARG Orientation Estimation for Hand Gesture Tracking in Magnetically Distorted Environments. In International Conference on Human-Computer Interaction (pp. 585-602). Cham: Springer Nature Switzerland. [Google Scholar]
Wang, Y., & Zhao, Y. (2023). Handwriting recognition under natural writing habits based on a low-cost inertial sensor. IEEE Sensors Journal. [Google Scholar]
Nguyen, V., Rupavatharam, S., Liu, L., Howard, R., & Gruteser, M. (2019, November). HandSense: capacitive coupling-based dynamic, micro finger gesture recognition. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems (pp. 285-297). [Google Scholar]
Gromov, B., Abbate, G., Gambardella, L. M., & Giusti, A. (2019, May). Proximity human-robot interaction using pointing gestures and a wrist-mounted IMU. In 2019 International Conference on Robotics and Automation (ICRA) (pp. 8084-8091). IEEE. [Google Scholar]
Ling, Y., Chen, X., Ruan, Y., Zhang, X., & Chen, X. (2021). Comparative study of gesture recognition based on accelerometer and photoplethysmography sensor for gesture interactions in wearable devices. IEEE Sensors Journal, 21(15), 17107-17117. [Google Scholar]
Picerno, P., Iosa, M., D’Souza, C., Benedetti, M. G., Paolucci, S., & Morone, G. (2021). Wearable inertial sensors for human movement analysis: a five-year update. Expert review of medical devices, 18(sup1), 79-94. [Google Scholar]
Hao, M., Chen, K., & Fu, C. (2019). Smoother-based 3-D foot trajectory estimation using inertial sensors. IEEE Transactions on Biomedical engineering, 66(12), 3534-3542. [Google Scholar]
Calado, A., Lin, B. S., Lee, I. J., & Saggio, G. (2023). Quasi-Static Measurement Performances of Flex Sensor Based and Inertial Measurement Unit Based Sensory Gloves. IEEE Sensors Journal. [Google Scholar]
Li, G., Wan, B., Su, K., Huo, J., Jiang, C., & Wang, F. (2023). sEMG and IMU Data-based Hand Gesture Recognition Method using Multi-stream CNN with a Fine-tuning Transfer Framework. IEEE Sensors Journal. [Google Scholar]
Dong, Y., Liu, J., & Yan, W. (2021). Dynamic hand gesture recognition based on signals from specialized data glove and deep learning algorithms. IEEE Transactions on Instrumentation and Measurement, 70, 1-14. [Google Scholar]
Lee, M., & Bae, J. (2020). Deep learning based real-time recognition of dynamic finger gestures using a data glove. IEEE Access, 8, 219923-219933. [Google Scholar]
Theodoridou, E., Cinque, L., Mignosi, F., Placidi, G., Polsinelli, M., Tavares, J. M. R., & Spezialetti, M. (2022). Hand tracking and gesture recognition by multiple contactless sensors: A survey. IEEE Transactions on Human-Machine Systems, 53(1), 35-43. [Google Scholar]
Jin, X. B., Sun, S., Wei, H., & Yang, F. B. (Eds.). (2018). Advances in multi-sensor information fusion: Theory and applications 2017. MDPI. [Google Scholar]
Pramanik, R., Sikdar, R., & Sarkar, R. (2023). Transformer-based deep reverse attention network for multi-sensory human activity recognition. Engineering Applications of Artificial Intelligence, 122, 106150. [Google Scholar]
Ryumin, D., Ivanko, D., & Ryumina, E. (2023). Audio-visual speech and gesture recognition by sensors of mobile devices. Sensors, 23(4), 2284. [Google Scholar]
Qi, W., Ovur, S. E., Li, Z., Marzullo, A., & Song, R. (2021). Multi-sensor guided hand gesture recognition for a teleoperated robot using a recurrent neural network. IEEE Robotics and Automation Letters, 6(3), 6039-6045. [Google Scholar]
Bai, Y., Yan, B., Zhou, C., Su, T., & Jin, X. (2023). State of art on state estimation: Kalman filter driven by machine learning. Annual Reviews in Control, 56, 100909. [Google Scholar]
Jin, X. B., Robert Jeremiah, R. J., Su, T. L., Bai, Y. T., & Kong, J. L. (2021). The new trend of state estimation: From model-driven to hybrid-driven methods. Sensors, 21(6), 2085. [Google Scholar]
Khodabin, M., & Rostami, M. (2015). Mean square numerical solution of stochastic differential equations by fourth order Runge-Kutta method and its application in the electric circuits with noise. Advances in Difference Equations, 2015(1), 62. [Google Scholar]
Bortolami, S. B., Pierobon, A., DiZio, P., & Lackner, J. R. (2006). Localization of the subjective vertical during roll, pitch, and recumbent yaw body tilt. Experimental brain research, 173, 364-373. [Google Scholar]
Jin, X. B., Su, T. L., Kong, J. L., Bai, Y. T., Miao, B. B., & Dou, C. (2018). State-of-the-art mobile intelligence: Enabling robots to move like humans by estimating mobility with artificial intelligence. Applied Sciences, 8(3), 379. [Google Scholar]
Nagy, E., Karl, É., & Molnár, G. (2024). Exploring the Role of Human-Robot Interactions, within the Context of the Effectiveness of a NAO Robot. Acta Polytechnica Hungarica, 21(3). [Google Scholar]
Mutawa, A. M., Al Mudhahkah, H. M., Al-Huwais, A., Al-Khaldi, N., Al-Otaibi, R., & Al-Ansari, A. (2023). Augmenting Mobile App with NAO Robot for Autism Education. Machines, 11(8), 833. [Google Scholar]
WANG, C., BAI, Y., CAI, L., HU, M., LIU, L., MA, Y., ... & ZHOU, Z. (2023). High precision electrostatic inertial sensor. Scientia Sinica Physica, Mechanica & Astronomica, 53(5), 250401. [Google Scholar]
Sameni, R. (2017). Online filtering using piecewisesmoothness priors: Application to normal and abnormal electrocardiogram denoising. Signal Processing, 133, 52-63. [Google Scholar]
IECE stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
IECE or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
IECE Transactions on Sensing, Communication, and Control
Our website uses cookies that are essential for its operation and additional cookies to track performance, or to improve and personalize our services. By clicking "Allow all cookies", you agree to the storing of cookies on your device. For more information on how we use cookies, please see our Cookie Policy.
