Volume 17, Number 3

Transformer Models for Text Summarization: A Comparative Study of BART, BERT and RoBERTa

  Authors

Daisy Aptovska and Vinayak Elangovan, Penn State University Abington, USA

  Abstract

Text summarization refers to the task of condensing a document into a shorter version while preserving itskey information. Automatic text summarization (ATS), driven by advancements in natural language processing (NLP), has developed rapidly in recent years. ATS methods are commonly categorized by input type (such as single-document or multi-document summarization) and by output type (extractive, abstractive, and hybrid). This article presents a focused review of modern summarization techniques with an emphasis on transformer-based models and large language models (LLMs), specifically BERT, RoBERTa and BART. It examines their architectures, pretraining strategies, and their suitability for extractive and abstractive summarization tasks. The paper also discusses key challenges, including computational requirements, data limitations, and issues such as factual inconsistency in generated summaries, and highlights the strengths and limitations of encoder-only and encoder–decoder models.

  Keywords

Abstractive summarization, extractive summarization, Large Language Models, transformer models, BART, BERT, RoBERTa.

Full Text    Volume 16

 

 

© AIRCC Publishing Corporation