Android Malware Detection Using an AI-Powered Hybrid XGBoost-RF Model for Enhanced Cybersecurity

Department of Electronics and Communication Engineering. Vels Institute of Science, Technology, and Advanced Studies (VISTAS), Chennai, India

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Published online: 2 March 2026

Abstract

The increased usage of mobile devices emphasizes the critical importance of the devices in day-to-day activities, but its widespread use has generated serious security and privacy issues, especially with applications that access personal information without the user's permission. Android is open-source and it is susceptible to malware rendering it vulnerable to loss of integrity and invasion of data. In this paper, a hybrid AI-based malware detection model is introduced to detect malware in Android with a dataset of 4,465 malware and benign instances of 2,532 malware and 1,930 benign. It provides Kaggle malicious URL dataset with 328 application and URL-based features also added with in it. The multi-layer perceptron, k-nearest neighbors, extreme gradient boosting, random forest, and a hybrid XGBoost-RF model were compared based on the metrics such as accuracy, precision, recall, F1-score and ROC-AUC. The XGBoost-RF hybrid has performed the best and has a high accuracy of 98.12% when compared to the individual models. These outcomes confirm that the effectiveness of the hybrid model is improving the malware detection of Android and which helps to build better cybersecurity protection with the help of advanced machine learning tools.