FINFUSION-LLM: A MULTIMODAL HYBRID FRAMEWORK FOR FINANCIAL MARKET PREDICTION USING FINANCIAL NEWS SENTIMENT AND NUMERICAL MARKET INDICATORS

Abstract

Financial market prediction remains a critical challenge due to complex nonlinear dynamics and the abundance of both quantitative and qualitative information. The proposed FinFusion-LLM framework addresses this by fusing transformer-based sentiment analysis of news with numerical market indicators in a hybrid deep learning model. We collect a comprehensive dataset of stock market prices (S&P 500 constituents, 2015–2024) and corresponding financial news headlines (Reuters, Bloomberg) to train and evaluate our model. News articles are processed by a fine-tuned finance-specific language model (FinBERT/LLaMA) to extract context-rich sentiment embeddings, while numerical features (historical prices, technical indicators like RSI, MACD) are fed into temporal layers. These modalities are fused via cross-attention, enabling the model to learn their joint effects on price movements. We compare FinFusion-LLM against baselines (LSTM on price data, sentiment-only models, simple concatenation) using metrics such as accuracy, F1-score, RMSE, and Sharpe ratio. Our results show that FinFusion-LLM significantly outperforms uni-modal models (e.g. ~85% vs 76% accuracy) and yields higher risk-adjusted returns. Key findings include the disproportional impact of negative sentiment on volatility and the superior performance of transformer-based architectures in modeling long-range dependencies. This study’s contributions are: (1) a novel multimodal architecture that combines LLM-derived sentiment and numerical predictors; (2) an empirical evaluation demonstrating enhanced predictive accuracy and trading performance; and (3) insights into how sentiment signals complement technical analysis. Overall, FinFusion-LLM advances AI-driven financial forecasting by leveraging the latest in large language models and data fusion techniques.