SlideShare a Scribd company logo

Text Summarization Using the T5 Transformer Model

https://www.irjet.net/archives/V10/i8/IRJET-V10I8148.pdf

1 of 3
Download to read offline
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 10 Issue: 08 | Aug 2023 www.irjet.net p-ISSN: 2395-0072
© 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 896
Text Summarization Using the T5 Transformer Model
Ratan Ravichandran1, Sri Bharath Sharma P2, Shriyans Shriniwas Arkal3, Shubhangee Das4,
Prof. Sasikala Nagarajan5
1-5Department of Artificial Intelligence and Machine Learning, Dayananda Sagar University, Bangalore, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - In our information-filled world, it is crucial to
focus on the essential content amidst the overwhelming
volume of information available. Unfortunately, people often
spend a significant amount of time sifting through irrelevant
details, inadvertently overlooking crucial information. To
address this issue, we present a project that utilizes the T5
transformer model in natural language processing to develop
an abstractive text summarization system. By leveraging
advanced language modeling techniques, our project aims to
enhance efficiency, comprehension, and decision-making
processes across various domains.
Key Words: Abstractive summarization,T5transformer
model, Natural language processing.
1.INTRODUCTION
In our information-filled world, focusing on what truly
matters is essential for success. On average, a personspends
a significant amount of their lifetime reading useless
information, often missing out on significant bits by
subconsciously dismissing them. To solve this problem, we
built a text summarizer that condenses lengthy text into
shorter concise summaries, providing a quick overview of
the main information.
Text summarization is a vital tool in today's information-
driven world, allowing us to distil the essence of lengthy
texts into concise summaries. By employing advanced
natural language processing techniques, text summarizers
extract key information, enabling readers to grasp the main
ideas quickly. In this report, we exploretheeffectiveness and
applications of text summarizers, shedding light on their
potential to enhance efficiency, comprehension, and
decision-making processes across various domains.
1.1 The T5 Transformer Model
To achieve this, weuse theT5transformermodelwhichis
a powerful language modelthatcanunderstandandgenerate
human-like text. Constructing a textsummarizerbasedonT5
is beneficial because it allows for concise and accurate
summarization of lengthy documents. T5's ability to capture
contextual relationships and generate coherent summaries
makes it an ideal choice for text summarization tasks,
enabling efficient information extraction and facilitating
quick comprehension of complex texts.
Finally, complete content and organizational editing
before formatting. Please take note of the following items
when proofreading spelling and grammar:
2. Literature Review
Adhika Pramita Widyassari et al.[1] provides an overview of
various techniques and methods used in automatic text
summarization, with a particular focus on the Natural
Language Toolkit (NLTK). The author explores different
approaches, including extractive and abstractive
summarization, and discusses how NLTK can be utilized in
these techniques.
 Preprocessing: NLTK performs essential text
preprocessingtasksliketokenization,stemming,and
stop-word removal, aiding ininformationextraction
by breaking text into words or sentences and
reducing words to their root form.
 Sentence Scoring: NLTK facilitates extractive
summarization by offering tools to calculate
sentence similarity (e.g., cosine similarity) and
assign scores, enabling the selection of relevant
sentences based on their importance.
 Feature Extraction: NLTK's part-of-speech tagging
and named entity recognition assist in identifying
entities and key terms, enhancing summary
accuracy and relevance.
 Language Modeling: In abstractive summarization,
NLTK helps build language models (e.g., n-gram
models) for generating concise and coherent
summaries by predicting probable next words or
phrases.
 Evaluation: NLTK includes evaluation metrics (e.g.,
ROUGE, BLEU) to assess summary quality by
comparing them with reference summaries and
measuring similarity or effectiveness.
Khilji et al. [2] examines Abstractive Text Analysis, described
asa naturallanguageprocessing(NLP)techniquethataimsto
generate a concise and coherent summary of a given text by
understanding its content and generating new sentences.
Abstractive summarizationinvolvescreatingnovelsentences
that capture the key informationandmainideasofthesource
text in a more human-like manner.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 10 Issue: 08 | Aug 2023 www.irjet.net p-ISSN: 2395-0072
© 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 897
3. PROPOSED ARCHITECTURE
Fig -1: System Architecture
Our system employs the cutting-edge T5 transformermodel
for effective text summarization. The process starts with
data preprocessing, including cleaning and organizing.
Tokenization divides data into smaller units for processing.
The T5 model is then trained to understand input and
generate informative summaries.
Once trained, it condenses key information in new input
documents. Evaluation is done using the ROUGE metric,
measuring similarity to human-written summaries. Higher
scores indicate better summarization. This architecture
leverages T5's power to process input, generate concise
summaries, and assess quality, streamlining information
extraction for quicker comprehension and decision-making.
3.1 Architecture Workflow
This code implements a sequence-to-sequence (Seq2Seq)
neural network, leveraging the T5 model, to achieve text
summarization. The process encompasses data preparation,
where libraries are imported, and the "multi_news" dataset
is loaded, split, and organized. Tokenization and
preprocessing are employed to adapt the data, utilizing the
"t5-small" tokenizer and defining a summarization prefix.
The core of the workflow involves model training,wherethe
pre-trained T5 model is fine-tuned for summarization. The
Seq2SeqTrainer facilitates this training, optimizing the
model's capacity to generate accurate and concise
summaries. After training, the model predicts summaries,
and Rouge scores are calculated using the Rouge library to
assess the quality of these summaries.
4. EXPERIMENTATION
4.1 Dataset
This dataset, multi_news found on HuggingFace, consists of
two columns: a feature column containing news text
separated by "|||||," and a targetcolumn withhuman-written
summaries. The target column serves as the reference for
summaries, while summaries in the feature column provide
a condensed overview.
4.2 Model Creation
For model creation, we use a T5 transformer architecture
tailored for sequence-to-sequence language tasks. The
DataCollatorForSeq2Seq ensures proper tokenization and
data collation.TheAutoModelForSeq2SeqLMclassloadspre-
trained T5 weights, to generate coherent sequences, such as
text summarization.
4.1 Model Training
The trainer is configured with the necessary components,
including training arguments, tokenizer, data collator, and
datasets for training and evaluation. By calling the train
function, the training process begins, during which the
model learns to generate concise summaries from the given
input data. Once the training is complete, the trained model
is saved to a specified path for later. Additionally,thetrained
model and a data file are downloaded and copied , enabling
further analysis or storage of the results. The model is
trained for 10 epochs and 25 epochs and the results are
accordingly evaluated.
5. RESULTS AND ANALYSIS
ROUGE, which stands for "Recall-Oriented Understudy for
Gisting Evaluation," is a set of metrics used to evaluate the
quality of summaries or generated text in natural language
processing tasks. It is commonly used in automatic
summarization andmachinetranslation evaluation. Theyare
assessed using ROUGE-1, ROUGE-2, and ROUGE-L. ROUGE
metrics provide a way to quantitatively assess the quality of
summaries or generated text by comparing them to a
reference summary. These metrics are widely used in
research and evaluation of text generation models to
measure their effectiveness in capturingthekeyinformation
or meaning from the source text.
Fig -2: ROUGE Scores for 10 epochs
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 10 Issue: 08 | Aug 2023 www.irjet.net p-ISSN: 2395-0072
© 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 898
Fig -3: ROUGE Scores for 25 epochs
Fig 1 shows the ROUGE scores, it shows that over the span of
10 epochs, the model's scores get better. Notably, thehighest
scores are achieved in the order of ROUGE-L, followed by
ROUGE-2, ROUGE-1, and ROUGE. This pattern indicates the
model'sabilitytocreatecoherentandfluentsummarieswhile
preserving essential information. Despite this progress, the
ROUGE scores remain relatively low, which means there is
room to improve.
Fig 2 shows the model’s progress when it is trained with 25
epochs. Throughout 25 epochs, the model's ROUGE scores
demonstrate progressive enhancement. The highest scores
are consistently observed in the order of ROUGE-L, followed
by ROUGE-2, ROUGE-1, and ROUGE. This pattern highlights
the model's capability to generate summaries that are not
only coherent but also more fluent compared to the original
text, while preserving crucial information.
The model's improvement in ROUGEscorescanbeattributed
to a few key factors. Firstly, longer training exposes the
model to a wider range of information, leading to better
performance. Additionally, extended training duration
enhances the model's grasp of human language, resulting in
improved summaries. Furthermore, as the model learns
more, its accuracy in producing summaries that align with
human-generated content also increases, ensuring factual
correctness.
3. CONCLUSIONS AND FUTURE WORK
Our successful project focused on abstractive text
summarization introduces a system powered by the T5
transformer language model. The project highlights the
utility of abstractive summarization in automating data
extractionand elevatingdecision-makingprocesses.Notably,
a comparative analysis reveals that the abstractive model
outperforms the extractive counterpart, capturing more
comprehensive details.
Looking forward, this technology bears the potential to
revolutionize how humans comprehend and utilize textual
content, enhancing its accessibility and efficacy across
various domains. Future enhancements could include fine-
tuning and domain adaptation to tailor models for specific
industries, enabling more precise and contextually relevant
summaries. Furthermore, addressing the challengeofmulti-
document summarization is crucial for accommodating
scenarios involvingrelateddocuments,requiringmethodsto
generate coherent summaries from multiple sources.
ACKNOWLEDGEMENTS
We are deeply grateful to our guide, Prof. Sasikala Nagarajan
for their support and mentorship throughout the course of
this project.
REFERENCES
[1] Adhika Pramita Widyassari,SupriadiRustad,GuruhFajar
Shidik, Edi Noersasongko,Abdul Syukur,AffandyAffandy,De
Rosal Ignatius Moses Setiadi, “Review of automatic text
summarization techniques & methods", Journal ofKingSaud
University 2022
[2] Khilji, Abdullah & Sinha, Utkarsh & Singh, Pintu & Ali,
Adnan & Pakray,Dr.Partha "Abstractive TextSummarization
Approaches with Analysis of Evaluation Techniques",
Computational IntelligenceinCommunicationsandBusiness
Analytics 2021
[3] Ilya Sutskever, Oriol Vinyals, Quoc V. Le, “Sequence to
Sequence Learning with Neural Networks”, arXiv Cornell
University 2014.
[4 ]Jakob Uszkoreit, “Transformer: A Novel Neural Network
Architecture for Language Understanding”,GoogleResearch
2017
[5] Abigail Rai, Study of Various Methods for Tokenization,
Applications of Internet things pp 193-200 2020.
Ad

Recommended

A Comparative Study of Automatic Text Summarization Methodologies
A Comparative Study of Automatic Text Summarization MethodologiesA Comparative Study of Automatic Text Summarization Methodologies
A Comparative Study of Automatic Text Summarization MethodologiesIRJET Journal
 
IRJET - Text Summarizer.
IRJET -  	  Text Summarizer.IRJET -  	  Text Summarizer.
IRJET - Text Summarizer.IRJET Journal
 
Reviews on swarm intelligence algorithms for text document clustering
Reviews on swarm intelligence algorithms for text document clusteringReviews on swarm intelligence algorithms for text document clustering
Reviews on swarm intelligence algorithms for text document clusteringIRJET Journal
 
AUTOMATIC QUESTION GENERATION USING NATURAL LANGUAGE PROCESSING
AUTOMATIC QUESTION GENERATION USING NATURAL LANGUAGE PROCESSINGAUTOMATIC QUESTION GENERATION USING NATURAL LANGUAGE PROCESSING
AUTOMATIC QUESTION GENERATION USING NATURAL LANGUAGE PROCESSINGIRJET Journal
 
Text Summarization and Conversion of Speech to Text
Text Summarization and Conversion of Speech to TextText Summarization and Conversion of Speech to Text
Text Summarization and Conversion of Speech to TextIRJET Journal
 
Automatic Text Summarization
Automatic Text SummarizationAutomatic Text Summarization
Automatic Text SummarizationIRJET Journal
 
Text Document Classification System
Text Document Classification SystemText Document Classification System
Text Document Classification SystemIRJET Journal
 
Automatic Text Summarization: A Critical Review
Automatic Text Summarization: A Critical ReviewAutomatic Text Summarization: A Critical Review
Automatic Text Summarization: A Critical ReviewIRJET Journal
 

More Related Content

Similar to Text Summarization Using the T5 Transformer Model

Automatic Text Classification Of News Blog using Machine Learning
Automatic Text Classification Of News Blog using Machine LearningAutomatic Text Classification Of News Blog using Machine Learning
Automatic Text Classification Of News Blog using Machine LearningIRJET Journal
 
Named Entity Recognition (NER) Using Automatic Summarization of Resumes
Named Entity Recognition (NER) Using Automatic Summarization of ResumesNamed Entity Recognition (NER) Using Automatic Summarization of Resumes
Named Entity Recognition (NER) Using Automatic Summarization of ResumesIRJET Journal
 
Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...
Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...
Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...IRJET Journal
 
A Survey on Automatic Text Summarization
A Survey on Automatic Text SummarizationA Survey on Automatic Text Summarization
A Survey on Automatic Text SummarizationIRJET Journal
 
IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...
IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...
IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...IRJET Journal
 
IRJET- Automatic Database Schema Generator
IRJET- Automatic Database Schema GeneratorIRJET- Automatic Database Schema Generator
IRJET- Automatic Database Schema GeneratorIRJET Journal
 
IRJET- Diverse Approaches for Document Clustering in Product Development Anal...
IRJET- Diverse Approaches for Document Clustering in Product Development Anal...IRJET- Diverse Approaches for Document Clustering in Product Development Anal...
IRJET- Diverse Approaches for Document Clustering in Product Development Anal...IRJET Journal
 
Automated Context-based Question-Distractor Generation using Extractive Summa...
Automated Context-based Question-Distractor Generation using Extractive Summa...Automated Context-based Question-Distractor Generation using Extractive Summa...
Automated Context-based Question-Distractor Generation using Extractive Summa...IRJET Journal
 
IRJET- Multi-Document Summarization using Fuzzy and Hierarchical Approach
IRJET-  	  Multi-Document Summarization using Fuzzy and Hierarchical ApproachIRJET-  	  Multi-Document Summarization using Fuzzy and Hierarchical Approach
IRJET- Multi-Document Summarization using Fuzzy and Hierarchical ApproachIRJET Journal
 
Review of Topic Modeling and Summarization
Review of Topic Modeling and SummarizationReview of Topic Modeling and Summarization
Review of Topic Modeling and SummarizationIRJET Journal
 
IRJET-Attribute Reduction using Apache Spark
IRJET-Attribute Reduction using Apache SparkIRJET-Attribute Reduction using Apache Spark
IRJET-Attribute Reduction using Apache SparkIRJET Journal
 
Development of Information Extraction for Data Analysis using NLP
Development of Information Extraction for Data Analysis using NLPDevelopment of Information Extraction for Data Analysis using NLP
Development of Information Extraction for Data Analysis using NLPIRJET Journal
 
IRJET - Automated System for Frequently Occurred Exam Questions
IRJET - Automated System for Frequently Occurred Exam QuestionsIRJET - Automated System for Frequently Occurred Exam Questions
IRJET - Automated System for Frequently Occurred Exam QuestionsIRJET Journal
 
IRJET- Deep Web Searching (DWS)
IRJET- Deep Web Searching (DWS)IRJET- Deep Web Searching (DWS)
IRJET- Deep Web Searching (DWS)IRJET Journal
 
Automatic Text Summarization Using Machine Learning
Automatic Text Summarization Using Machine LearningAutomatic Text Summarization Using Machine Learning
Automatic Text Summarization Using Machine LearningIRJET Journal
 
IRJET- Machine Learning Techniques for Code Optimization
IRJET-  	  Machine Learning Techniques for Code OptimizationIRJET-  	  Machine Learning Techniques for Code Optimization
IRJET- Machine Learning Techniques for Code OptimizationIRJET Journal
 
QUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODEL
QUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODELQUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODEL
QUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODELIRJET Journal
 
Document Analyser Using Deep Learning
Document Analyser Using Deep LearningDocument Analyser Using Deep Learning
Document Analyser Using Deep LearningIRJET Journal
 
A Review On Chatbot Design And Implementation Techniques
A Review On Chatbot Design And Implementation TechniquesA Review On Chatbot Design And Implementation Techniques
A Review On Chatbot Design And Implementation TechniquesJoe Osborn
 
IRJET- Determining Document Relevance using Keyword Extraction
IRJET-  	  Determining Document Relevance using Keyword ExtractionIRJET-  	  Determining Document Relevance using Keyword Extraction
IRJET- Determining Document Relevance using Keyword ExtractionIRJET Journal
 

Similar to Text Summarization Using the T5 Transformer Model (20)

Automatic Text Classification Of News Blog using Machine Learning
Automatic Text Classification Of News Blog using Machine LearningAutomatic Text Classification Of News Blog using Machine Learning
Automatic Text Classification Of News Blog using Machine Learning
 
Named Entity Recognition (NER) Using Automatic Summarization of Resumes
Named Entity Recognition (NER) Using Automatic Summarization of ResumesNamed Entity Recognition (NER) Using Automatic Summarization of Resumes
Named Entity Recognition (NER) Using Automatic Summarization of Resumes
 
Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...
Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...
Text Summarization of Food Reviews using AbstractiveSummarization and Recurre...
 
A Survey on Automatic Text Summarization
A Survey on Automatic Text SummarizationA Survey on Automatic Text Summarization
A Survey on Automatic Text Summarization
 
IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...
IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...
IRJET- Towards Efficient Framework for Semantic Query Search Engine in Large-...
 
IRJET- Automatic Database Schema Generator
IRJET- Automatic Database Schema GeneratorIRJET- Automatic Database Schema Generator
IRJET- Automatic Database Schema Generator
 
IRJET- Diverse Approaches for Document Clustering in Product Development Anal...
IRJET- Diverse Approaches for Document Clustering in Product Development Anal...IRJET- Diverse Approaches for Document Clustering in Product Development Anal...
IRJET- Diverse Approaches for Document Clustering in Product Development Anal...
 
Automated Context-based Question-Distractor Generation using Extractive Summa...
Automated Context-based Question-Distractor Generation using Extractive Summa...Automated Context-based Question-Distractor Generation using Extractive Summa...
Automated Context-based Question-Distractor Generation using Extractive Summa...
 
IRJET- Multi-Document Summarization using Fuzzy and Hierarchical Approach
IRJET-  	  Multi-Document Summarization using Fuzzy and Hierarchical ApproachIRJET-  	  Multi-Document Summarization using Fuzzy and Hierarchical Approach
IRJET- Multi-Document Summarization using Fuzzy and Hierarchical Approach
 
Review of Topic Modeling and Summarization
Review of Topic Modeling and SummarizationReview of Topic Modeling and Summarization
Review of Topic Modeling and Summarization
 
IRJET-Attribute Reduction using Apache Spark
IRJET-Attribute Reduction using Apache SparkIRJET-Attribute Reduction using Apache Spark
IRJET-Attribute Reduction using Apache Spark
 
Development of Information Extraction for Data Analysis using NLP
Development of Information Extraction for Data Analysis using NLPDevelopment of Information Extraction for Data Analysis using NLP
Development of Information Extraction for Data Analysis using NLP
 
IRJET - Automated System for Frequently Occurred Exam Questions
IRJET - Automated System for Frequently Occurred Exam QuestionsIRJET - Automated System for Frequently Occurred Exam Questions
IRJET - Automated System for Frequently Occurred Exam Questions
 
IRJET- Deep Web Searching (DWS)
IRJET- Deep Web Searching (DWS)IRJET- Deep Web Searching (DWS)
IRJET- Deep Web Searching (DWS)
 
Automatic Text Summarization Using Machine Learning
Automatic Text Summarization Using Machine LearningAutomatic Text Summarization Using Machine Learning
Automatic Text Summarization Using Machine Learning
 
IRJET- Machine Learning Techniques for Code Optimization
IRJET-  	  Machine Learning Techniques for Code OptimizationIRJET-  	  Machine Learning Techniques for Code Optimization
IRJET- Machine Learning Techniques for Code Optimization
 
QUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODEL
QUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODELQUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODEL
QUICK GLANCE: UNSUPERVISED EXTRACTIVE SUMMARIZATION MODEL
 
Document Analyser Using Deep Learning
Document Analyser Using Deep LearningDocument Analyser Using Deep Learning
Document Analyser Using Deep Learning
 
A Review On Chatbot Design And Implementation Techniques
A Review On Chatbot Design And Implementation TechniquesA Review On Chatbot Design And Implementation Techniques
A Review On Chatbot Design And Implementation Techniques
 
IRJET- Determining Document Relevance using Keyword Extraction
IRJET-  	  Determining Document Relevance using Keyword ExtractionIRJET-  	  Determining Document Relevance using Keyword Extraction
IRJET- Determining Document Relevance using Keyword Extraction
 

More from IRJET Journal

TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...IRJET Journal
 
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURESTUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTUREIRJET Journal
 
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...IRJET Journal
 
Effect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil CharacteristicsEffect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil CharacteristicsIRJET Journal
 
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...IRJET Journal
 
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...IRJET Journal
 
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...IRJET Journal
 
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...IRJET Journal
 
A REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADASA REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADASIRJET Journal
 
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...IRJET Journal
 
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD ProP.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD ProIRJET Journal
 
Survey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare SystemSurvey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare SystemIRJET Journal
 
Review on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridgesReview on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridgesIRJET Journal
 
React based fullstack edtech web application
React based fullstack edtech web applicationReact based fullstack edtech web application
React based fullstack edtech web applicationIRJET Journal
 
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...IRJET Journal
 
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.IRJET Journal
 
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...IRJET Journal
 
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...IRJET Journal
 
Solving Linear Differential Equations with Constant Coefficients
Solving Linear Differential Equations with Constant CoefficientsSolving Linear Differential Equations with Constant Coefficients
Solving Linear Differential Equations with Constant CoefficientsIRJET Journal
 
Assessment of outdoor spaces like corridors and courtyards in a school enviro...
Assessment of outdoor spaces like corridors and courtyards in a school enviro...Assessment of outdoor spaces like corridors and courtyards in a school enviro...
Assessment of outdoor spaces like corridors and courtyards in a school enviro...IRJET Journal
 

More from IRJET Journal (20)

TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
TUNNELING IN HIMALAYAS WITH NATM METHOD: A SPECIAL REFERENCES TO SUNGAL TUNNE...
 
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURESTUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
STUDY THE EFFECT OF RESPONSE REDUCTION FACTOR ON RC FRAMED STRUCTURE
 
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
A COMPARATIVE ANALYSIS OF RCC ELEMENT OF SLAB WITH STARK STEEL (HYSD STEEL) A...
 
Effect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil CharacteristicsEffect of Camber and Angles of Attack on Airfoil Characteristics
Effect of Camber and Angles of Attack on Airfoil Characteristics
 
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
A Review on the Progress and Challenges of Aluminum-Based Metal Matrix Compos...
 
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
Dynamic Urban Transit Optimization: A Graph Neural Network Approach for Real-...
 
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
Structural Analysis and Design of Multi-Storey Symmetric and Asymmetric Shape...
 
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
A Review of “Seismic Response of RC Structures Having Plan and Vertical Irreg...
 
A REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADASA REVIEW ON MACHINE LEARNING IN ADAS
A REVIEW ON MACHINE LEARNING IN ADAS
 
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
Long Term Trend Analysis of Precipitation and Temperature for Asosa district,...
 
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD ProP.E.B. Framed Structure Design and Analysis Using STAAD Pro
P.E.B. Framed Structure Design and Analysis Using STAAD Pro
 
Survey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare SystemSurvey Paper on Cloud-Based Secured Healthcare System
Survey Paper on Cloud-Based Secured Healthcare System
 
Review on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridgesReview on studies and research on widening of existing concrete bridges
Review on studies and research on widening of existing concrete bridges
 
React based fullstack edtech web application
React based fullstack edtech web applicationReact based fullstack edtech web application
React based fullstack edtech web application
 
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
A Comprehensive Review of Integrating IoT and Blockchain Technologies in the ...
 
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
A REVIEW ON THE PERFORMANCE OF COCONUT FIBRE REINFORCED CONCRETE.
 
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
Optimizing Business Management Process Workflows: The Dynamic Influence of Mi...
 
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
Cost Optimization of Construction Using Plastic Waste as a Sustainable Constr...
 
Solving Linear Differential Equations with Constant Coefficients
Solving Linear Differential Equations with Constant CoefficientsSolving Linear Differential Equations with Constant Coefficients
Solving Linear Differential Equations with Constant Coefficients
 
Assessment of outdoor spaces like corridors and courtyards in a school enviro...
Assessment of outdoor spaces like corridors and courtyards in a school enviro...Assessment of outdoor spaces like corridors and courtyards in a school enviro...
Assessment of outdoor spaces like corridors and courtyards in a school enviro...
 

Recently uploaded

S. Kim, NeurIPS 2023, MLILAB, KAISTAI
S. Kim,  NeurIPS 2023,  MLILAB,  KAISTAIS. Kim,  NeurIPS 2023,  MLILAB,  KAISTAI
S. Kim, NeurIPS 2023, MLILAB, KAISTAIMLILAB
 
PM24_Oral_Presentation_Template_Guidelines.pptx
PM24_Oral_Presentation_Template_Guidelines.pptxPM24_Oral_Presentation_Template_Guidelines.pptx
PM24_Oral_Presentation_Template_Guidelines.pptxnissamant
 
GDSC Web Bootcamp - Day - 2 - JavaScript
GDSC Web Bootcamp -  Day - 2   - JavaScriptGDSC Web Bootcamp -  Day - 2   - JavaScript
GDSC Web Bootcamp - Day - 2 - JavaScriptSahithiGurlinka
 
Series of training sessions by our experts for you to provide necessary insig...
Series of training sessions by our experts for you to provide necessary insig...Series of training sessions by our experts for you to provide necessary insig...
Series of training sessions by our experts for you to provide necessary insig...AshishChanchal1
 
SR Globals Profile - Building Vision, Exceeding Expectations.
SR Globals Profile -  Building Vision, Exceeding Expectations.SR Globals Profile -  Building Vision, Exceeding Expectations.
SR Globals Profile - Building Vision, Exceeding Expectations.srglobalsenterprises
 
Presentation of Helmet Detection Using Machine Learning.pptx
Presentation of Helmet Detection Using Machine Learning.pptxPresentation of Helmet Detection Using Machine Learning.pptx
Presentation of Helmet Detection Using Machine Learning.pptxasmitaTele2
 
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...C Sai Kiran
 
Get start with Machine Learning and Vertexai
Get start with Machine Learning and VertexaiGet start with Machine Learning and Vertexai
Get start with Machine Learning and VertexaiAshishChanchal1
 
Metrology Measurements and All units PPT
Metrology Measurements and  All units PPTMetrology Measurements and  All units PPT
Metrology Measurements and All units PPTdinesh babu
 
Center Enamel is the leading fire water tanks manufacturer in China.docx
Center Enamel is the leading fire water tanks manufacturer in China.docxCenter Enamel is the leading fire water tanks manufacturer in China.docx
Center Enamel is the leading fire water tanks manufacturer in China.docxsjzzztc
 
Introduction about Technology roadmap for Industry 4.0
Introduction about Technology roadmap for Industry 4.0Introduction about Technology roadmap for Industry 4.0
Introduction about Technology roadmap for Industry 4.0RaishKhanji
 
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHIINTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHIKiranKandhro1
 
CHAPTER 1_ HTML and CSS Introduction Module
CHAPTER 1_ HTML and CSS Introduction ModuleCHAPTER 1_ HTML and CSS Introduction Module
CHAPTER 1_ HTML and CSS Introduction Modulessusera4f8281
 
Pointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptxPointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptxAnanthi Palanisamy
 
Laser And its Application's - Engineering Physics
Laser And its Application's - Engineering PhysicsLaser And its Application's - Engineering Physics
Laser And its Application's - Engineering PhysicsPurva Nikam
 
nervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesnervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesPhebeLois1
 
Nexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptxNexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptxRohanAgarwal340656
 
Introduction to Binary Tree and Conersion of General tree to Binary Tree
Introduction to Binary Tree  and Conersion of General tree to Binary TreeIntroduction to Binary Tree  and Conersion of General tree to Binary Tree
Introduction to Binary Tree and Conersion of General tree to Binary TreeSwarupaDeshpande4
 
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...GauravBhartie
 
GDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptxGDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptxAnandMenon54
 

Recently uploaded (20)

S. Kim, NeurIPS 2023, MLILAB, KAISTAI
S. Kim,  NeurIPS 2023,  MLILAB,  KAISTAIS. Kim,  NeurIPS 2023,  MLILAB,  KAISTAI
S. Kim, NeurIPS 2023, MLILAB, KAISTAI
 
PM24_Oral_Presentation_Template_Guidelines.pptx
PM24_Oral_Presentation_Template_Guidelines.pptxPM24_Oral_Presentation_Template_Guidelines.pptx
PM24_Oral_Presentation_Template_Guidelines.pptx
 
GDSC Web Bootcamp - Day - 2 - JavaScript
GDSC Web Bootcamp -  Day - 2   - JavaScriptGDSC Web Bootcamp -  Day - 2   - JavaScript
GDSC Web Bootcamp - Day - 2 - JavaScript
 
Series of training sessions by our experts for you to provide necessary insig...
Series of training sessions by our experts for you to provide necessary insig...Series of training sessions by our experts for you to provide necessary insig...
Series of training sessions by our experts for you to provide necessary insig...
 
SR Globals Profile - Building Vision, Exceeding Expectations.
SR Globals Profile -  Building Vision, Exceeding Expectations.SR Globals Profile -  Building Vision, Exceeding Expectations.
SR Globals Profile - Building Vision, Exceeding Expectations.
 
Presentation of Helmet Detection Using Machine Learning.pptx
Presentation of Helmet Detection Using Machine Learning.pptxPresentation of Helmet Detection Using Machine Learning.pptx
Presentation of Helmet Detection Using Machine Learning.pptx
 
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
 
Get start with Machine Learning and Vertexai
Get start with Machine Learning and VertexaiGet start with Machine Learning and Vertexai
Get start with Machine Learning and Vertexai
 
Metrology Measurements and All units PPT
Metrology Measurements and  All units PPTMetrology Measurements and  All units PPT
Metrology Measurements and All units PPT
 
Center Enamel is the leading fire water tanks manufacturer in China.docx
Center Enamel is the leading fire water tanks manufacturer in China.docxCenter Enamel is the leading fire water tanks manufacturer in China.docx
Center Enamel is the leading fire water tanks manufacturer in China.docx
 
Introduction about Technology roadmap for Industry 4.0
Introduction about Technology roadmap for Industry 4.0Introduction about Technology roadmap for Industry 4.0
Introduction about Technology roadmap for Industry 4.0
 
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHIINTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
 
CHAPTER 1_ HTML and CSS Introduction Module
CHAPTER 1_ HTML and CSS Introduction ModuleCHAPTER 1_ HTML and CSS Introduction Module
CHAPTER 1_ HTML and CSS Introduction Module
 
Pointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptxPointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptx
 
Laser And its Application's - Engineering Physics
Laser And its Application's - Engineering PhysicsLaser And its Application's - Engineering Physics
Laser And its Application's - Engineering Physics
 
nervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesnervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nerves
 
Nexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptxNexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptx
 
Introduction to Binary Tree and Conersion of General tree to Binary Tree
Introduction to Binary Tree  and Conersion of General tree to Binary TreeIntroduction to Binary Tree  and Conersion of General tree to Binary Tree
Introduction to Binary Tree and Conersion of General tree to Binary Tree
 
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
 
GDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptxGDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptx
 

Text Summarization Using the T5 Transformer Model

  • 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 10 Issue: 08 | Aug 2023 www.irjet.net p-ISSN: 2395-0072 © 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 896 Text Summarization Using the T5 Transformer Model Ratan Ravichandran1, Sri Bharath Sharma P2, Shriyans Shriniwas Arkal3, Shubhangee Das4, Prof. Sasikala Nagarajan5 1-5Department of Artificial Intelligence and Machine Learning, Dayananda Sagar University, Bangalore, India ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract - In our information-filled world, it is crucial to focus on the essential content amidst the overwhelming volume of information available. Unfortunately, people often spend a significant amount of time sifting through irrelevant details, inadvertently overlooking crucial information. To address this issue, we present a project that utilizes the T5 transformer model in natural language processing to develop an abstractive text summarization system. By leveraging advanced language modeling techniques, our project aims to enhance efficiency, comprehension, and decision-making processes across various domains. Key Words: Abstractive summarization,T5transformer model, Natural language processing. 1.INTRODUCTION In our information-filled world, focusing on what truly matters is essential for success. On average, a personspends a significant amount of their lifetime reading useless information, often missing out on significant bits by subconsciously dismissing them. To solve this problem, we built a text summarizer that condenses lengthy text into shorter concise summaries, providing a quick overview of the main information. Text summarization is a vital tool in today's information- driven world, allowing us to distil the essence of lengthy texts into concise summaries. By employing advanced natural language processing techniques, text summarizers extract key information, enabling readers to grasp the main ideas quickly. In this report, we exploretheeffectiveness and applications of text summarizers, shedding light on their potential to enhance efficiency, comprehension, and decision-making processes across various domains. 1.1 The T5 Transformer Model To achieve this, weuse theT5transformermodelwhichis a powerful language modelthatcanunderstandandgenerate human-like text. Constructing a textsummarizerbasedonT5 is beneficial because it allows for concise and accurate summarization of lengthy documents. T5's ability to capture contextual relationships and generate coherent summaries makes it an ideal choice for text summarization tasks, enabling efficient information extraction and facilitating quick comprehension of complex texts. Finally, complete content and organizational editing before formatting. Please take note of the following items when proofreading spelling and grammar: 2. Literature Review Adhika Pramita Widyassari et al.[1] provides an overview of various techniques and methods used in automatic text summarization, with a particular focus on the Natural Language Toolkit (NLTK). The author explores different approaches, including extractive and abstractive summarization, and discusses how NLTK can be utilized in these techniques.  Preprocessing: NLTK performs essential text preprocessingtasksliketokenization,stemming,and stop-word removal, aiding ininformationextraction by breaking text into words or sentences and reducing words to their root form.  Sentence Scoring: NLTK facilitates extractive summarization by offering tools to calculate sentence similarity (e.g., cosine similarity) and assign scores, enabling the selection of relevant sentences based on their importance.  Feature Extraction: NLTK's part-of-speech tagging and named entity recognition assist in identifying entities and key terms, enhancing summary accuracy and relevance.  Language Modeling: In abstractive summarization, NLTK helps build language models (e.g., n-gram models) for generating concise and coherent summaries by predicting probable next words or phrases.  Evaluation: NLTK includes evaluation metrics (e.g., ROUGE, BLEU) to assess summary quality by comparing them with reference summaries and measuring similarity or effectiveness. Khilji et al. [2] examines Abstractive Text Analysis, described asa naturallanguageprocessing(NLP)techniquethataimsto generate a concise and coherent summary of a given text by understanding its content and generating new sentences. Abstractive summarizationinvolvescreatingnovelsentences that capture the key informationandmainideasofthesource text in a more human-like manner.
  • 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 10 Issue: 08 | Aug 2023 www.irjet.net p-ISSN: 2395-0072 © 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 897 3. PROPOSED ARCHITECTURE Fig -1: System Architecture Our system employs the cutting-edge T5 transformermodel for effective text summarization. The process starts with data preprocessing, including cleaning and organizing. Tokenization divides data into smaller units for processing. The T5 model is then trained to understand input and generate informative summaries. Once trained, it condenses key information in new input documents. Evaluation is done using the ROUGE metric, measuring similarity to human-written summaries. Higher scores indicate better summarization. This architecture leverages T5's power to process input, generate concise summaries, and assess quality, streamlining information extraction for quicker comprehension and decision-making. 3.1 Architecture Workflow This code implements a sequence-to-sequence (Seq2Seq) neural network, leveraging the T5 model, to achieve text summarization. The process encompasses data preparation, where libraries are imported, and the "multi_news" dataset is loaded, split, and organized. Tokenization and preprocessing are employed to adapt the data, utilizing the "t5-small" tokenizer and defining a summarization prefix. The core of the workflow involves model training,wherethe pre-trained T5 model is fine-tuned for summarization. The Seq2SeqTrainer facilitates this training, optimizing the model's capacity to generate accurate and concise summaries. After training, the model predicts summaries, and Rouge scores are calculated using the Rouge library to assess the quality of these summaries. 4. EXPERIMENTATION 4.1 Dataset This dataset, multi_news found on HuggingFace, consists of two columns: a feature column containing news text separated by "|||||," and a targetcolumn withhuman-written summaries. The target column serves as the reference for summaries, while summaries in the feature column provide a condensed overview. 4.2 Model Creation For model creation, we use a T5 transformer architecture tailored for sequence-to-sequence language tasks. The DataCollatorForSeq2Seq ensures proper tokenization and data collation.TheAutoModelForSeq2SeqLMclassloadspre- trained T5 weights, to generate coherent sequences, such as text summarization. 4.1 Model Training The trainer is configured with the necessary components, including training arguments, tokenizer, data collator, and datasets for training and evaluation. By calling the train function, the training process begins, during which the model learns to generate concise summaries from the given input data. Once the training is complete, the trained model is saved to a specified path for later. Additionally,thetrained model and a data file are downloaded and copied , enabling further analysis or storage of the results. The model is trained for 10 epochs and 25 epochs and the results are accordingly evaluated. 5. RESULTS AND ANALYSIS ROUGE, which stands for "Recall-Oriented Understudy for Gisting Evaluation," is a set of metrics used to evaluate the quality of summaries or generated text in natural language processing tasks. It is commonly used in automatic summarization andmachinetranslation evaluation. Theyare assessed using ROUGE-1, ROUGE-2, and ROUGE-L. ROUGE metrics provide a way to quantitatively assess the quality of summaries or generated text by comparing them to a reference summary. These metrics are widely used in research and evaluation of text generation models to measure their effectiveness in capturingthekeyinformation or meaning from the source text. Fig -2: ROUGE Scores for 10 epochs
  • 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 10 Issue: 08 | Aug 2023 www.irjet.net p-ISSN: 2395-0072 © 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 898 Fig -3: ROUGE Scores for 25 epochs Fig 1 shows the ROUGE scores, it shows that over the span of 10 epochs, the model's scores get better. Notably, thehighest scores are achieved in the order of ROUGE-L, followed by ROUGE-2, ROUGE-1, and ROUGE. This pattern indicates the model'sabilitytocreatecoherentandfluentsummarieswhile preserving essential information. Despite this progress, the ROUGE scores remain relatively low, which means there is room to improve. Fig 2 shows the model’s progress when it is trained with 25 epochs. Throughout 25 epochs, the model's ROUGE scores demonstrate progressive enhancement. The highest scores are consistently observed in the order of ROUGE-L, followed by ROUGE-2, ROUGE-1, and ROUGE. This pattern highlights the model's capability to generate summaries that are not only coherent but also more fluent compared to the original text, while preserving crucial information. The model's improvement in ROUGEscorescanbeattributed to a few key factors. Firstly, longer training exposes the model to a wider range of information, leading to better performance. Additionally, extended training duration enhances the model's grasp of human language, resulting in improved summaries. Furthermore, as the model learns more, its accuracy in producing summaries that align with human-generated content also increases, ensuring factual correctness. 3. CONCLUSIONS AND FUTURE WORK Our successful project focused on abstractive text summarization introduces a system powered by the T5 transformer language model. The project highlights the utility of abstractive summarization in automating data extractionand elevatingdecision-makingprocesses.Notably, a comparative analysis reveals that the abstractive model outperforms the extractive counterpart, capturing more comprehensive details. Looking forward, this technology bears the potential to revolutionize how humans comprehend and utilize textual content, enhancing its accessibility and efficacy across various domains. Future enhancements could include fine- tuning and domain adaptation to tailor models for specific industries, enabling more precise and contextually relevant summaries. Furthermore, addressing the challengeofmulti- document summarization is crucial for accommodating scenarios involvingrelateddocuments,requiringmethodsto generate coherent summaries from multiple sources. ACKNOWLEDGEMENTS We are deeply grateful to our guide, Prof. Sasikala Nagarajan for their support and mentorship throughout the course of this project. REFERENCES [1] Adhika Pramita Widyassari,SupriadiRustad,GuruhFajar Shidik, Edi Noersasongko,Abdul Syukur,AffandyAffandy,De Rosal Ignatius Moses Setiadi, “Review of automatic text summarization techniques & methods", Journal ofKingSaud University 2022 [2] Khilji, Abdullah & Sinha, Utkarsh & Singh, Pintu & Ali, Adnan & Pakray,Dr.Partha "Abstractive TextSummarization Approaches with Analysis of Evaluation Techniques", Computational IntelligenceinCommunicationsandBusiness Analytics 2021 [3] Ilya Sutskever, Oriol Vinyals, Quoc V. Le, “Sequence to Sequence Learning with Neural Networks”, arXiv Cornell University 2014. [4 ]Jakob Uszkoreit, “Transformer: A Novel Neural Network Architecture for Language Understanding”,GoogleResearch 2017 [5] Abigail Rai, Study of Various Methods for Tokenization, Applications of Internet things pp 193-200 2020.