INVESTIGATING EARLY LUNG CANCER DETECTION THROUGH FEATURE SELECTION AND ENSEMBLE MACHINE LEARNING

Abstract

Early detection is very important in improving the survival rate of lung cancer. This work proposes a machine learning framework that embeds feature selection, ensemble learning, and SHAP-based interpretability to predict the risk of lung cancer. Several machine learning models were trained and evaluated, including random forest (RF), XGBoost (XGB), LightGBM (LGBM), and CatBoost (CB). In a dataset with 1,001 records, the essential predictors were selected by the Recursive feature elimination method, hence improving the performance of the model. After feature selection, the ensemble model yielded perfect performance of 100%. Important features selected and identified included "coughing of blood" and "obesity" as the most influential. The results showed that the framework outperformed the accuracy of the state-of-the-art models. Although the results look promising, further external validation of the results and integration with imaging and genomic data is suggested for its enhanced real-world application. This hereby presents a solid approach toward the early detection of lung cancer and management of personal healthcare.