Human–Robot Interaction via hand gestures: System design and analysis

¹ School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India.
² Centre for Neuro Informatics, School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India.
³ Centre for Healthcare Advancement Innovation and Research, School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India.

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Published online: 29 April 2026

Abstract

This research paper explores a robotic arm that is operated by hand movements so that individuals can work with machines effortlessly. The implementation is composed of computer vision, machine learning and small hardware that converts gestures into certain actions of the robot. The depth-sensing cameras make the information available to convolutional neural networks, and data interpretation is fast. It is very accurate with 97.3 percent accuracy that can respond within less than one hundred milliseconds due to its effective real time processing. Performance does not vary as light conditions vary or with arrival of new users. This could be used in manufacturing, medical assistance and in classrooms. Deep learning, being light weight, can be executed directly using local machines. It does not depend on the external servers that are necessary to run. Although the demonstration is a trial to prove the effectiveness of the set-up in dealing with the complex tasks by simplified actions of the hand, more emphasis is laid on the capability of making machine controls more readily available. The interesting thing is that the work can be employed to proceed with the current methods of moving objects via gestures, and it offers a transparent and straightforward way that do not contradict the manner in which human beings interact with machines in a more natural manner.