MATHEMATICAL MODELING AND COMPARATIVE STUDY OF TRANSFORMER ARCHITECTURES FOR CLINICAL AND BIOMEDICAL SUMMARIZATION

Abstract

Domains in biomedical and clinical studies present enormous volumes of unstructured text, such as PubMed articles, discharge summaries, and radiology reports, that well exceed human processing capacity. This paper presents a mathematically grounded and empirically validated comparative study of transformer architectures for biomedical and clinical summarization, spanning encoder-only or encoderdecoder, and a novel hybrid extractiveabstractive model implemented in this work. The proposed summarization pipeline integrates domain-adaptive tokenization, padding-aware supervision pad→−100, and label alignment that skips BOS tokens-mechanisms shown to be critical for stable training. We propose a hybrid model that fuses sentence- importanceweighted encoder features with a transformer decoder through a gated fusion layer and is trained with a padding-aware cross-entropy objective and outline an optional factuality regularizer for the preservation of biomedical entity consistency. Evaluation employs ROUGE, BERTScore, and an entity-coverage factual consistency proxy, offering a multicriteria perspective on fluency, coherence, and factual reliability. Empirical analysis reveals that encoderdecoder models achieve superior fluency, while encoder-only models maximize factual retention in extractive settings, and the proposed hybrid approach improves entity preservation while maintaining readability-particularly for clinical notes dense with medical entities. Implementation-level contributions include a reproducible Windows-friendly training stack num_workers=0, gradient accumulation for CPU-based execution, and configuration-driven experimental design. Collectively, the mathematical formulation, implemented hybrid architecture, and multi-metric evaluation framework advance clinically oriented summarization research and provide a reproducible foundation for future biomedical NLP studies.