Design and Implementation of an Autonomous Line-Following Mobile Robot Using Q-Learning and Computer Vision

¹ Department of Mechanical Engineering, Lendi Institute of Engineering and Technology, Jonnada, Andhra Pradesh 535005, India
² Department of Computer Science and Engineering, Anurag University, Hyderabad, Telangana 500088, India
³ Department of Mechanical Engineering, R.M.K. Engineering College, Kavaraipettai, Tamil Nadu 601206, India
⁴ Department of Information Technology, Prathyusha Engineering College, Tiruvallur, Tamil Nadu 602025, India
⁵ Department of Computer Science - Artifical Intelligence, PSGR Krishnammal College for Women, Coimbatore, Tamil Nadu 641004, India
⁶ Research and Development Cell, Lovely Professional University, Phagwara, Punjab 144411, India

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Published online: 2 March 2026

Abstract

This research paper discusses the creation and application of a fully automated robot that follows a line by means of reinforcement learning techniques, more specifically the Q-learning algorithm. The system proposed comprises a vision-based perception module that uses OpenCV for its development and also includes the simulation environment of CoppeliaSim thus making the navigation process adaptive as well as robust. By interaction with its surroundings at all times, the agent gets trained on the control policy that is optimal in terms of lateral deviation minimization and hence provides stable trajectory tracking. The research comprises a detailed assessment of the Q-table dimensions and hyper-parameters such as the learning rate, discount factor, and exploration rate, systematically determining their impact on learning performance, convergence, and accuracy. According to the experimental findings, a configuration of a 7×7 Q-table strikes the best balance between precision and convergence speed that in turn results in smooth and even path tracking. The method, though quite effective under controlled conditions, does have its drawbacks in terms of state discretization, generalization, and real-time processing, thereby providing directions for the applications of deep reinforcement learning and adaptive perception models in the future.