COMPARATIVE EVALUATION OF EXPLAINABLE MACHINE LEARNING MODELS FOR STUDENT DROPOUT PREDICTION USING LMS ANALYTICS

Abstract

Student dropout in virtual learning environments is a critical challenge for higher education because of its academic, institutional, and socioeconomic implications. This study presents a revised comparative evaluation of machine learning models for student dropout prediction using Learning Management System (LMS) analytics and Explainable Artificial Intelligence (XAI). To address methodological concerns related to data leakage, class imbalance, and the use of post-outcome variables, the experimental workflow was redesigned using a leakage-free pipeline. The train-test split was performed before preprocessing, encoding, scaling, and SMOTE balancing. All transformations were fitted only on training data, and SMOTE was applied exclusively within the training set and cross-validation folds. Two scenarios were evaluated: a diagnostic scenario including final_grade and an early-warning scenario excluding final_grade. In the early-warning scenario, Random Forest achieved the best overall balance in repeated stratified cross-validation, with Accuracy = 0.9586 +/- 0.0207, Precision = 0.8689 +/- 0.1482, F1-score = 0.7403 +/- 0.1205, and ROC-AUC = 0.9742 +/- 0.0237. K-Nearest Neighbors achieved the highest Recall = 0.7933 +/- 0.1867, indicating greater sensitivity for detecting dropout cases. SHAP analysis of the Random Forest early-warning model showed that late_submissions, resource_views, login_frequency, forum_participation, task_submissions, and connection_time were the most influential predictors. The results suggest that explainable machine learning can support transparent and responsible early-warning systems for virtual higher education when models are validated with leakage-free procedures and interpreted under ethical safeguards.