The HuggingFace transformers library provides us with a wide range of pretrained models for different tasks such as sentiment analysis, question-answering, and so on.
The library is built on top of the PyTorch and TensorFlow deep learning frameworks, allowing users to easily leverage pretrained models for their NLP tasks. These models have been pretrained on large-scale corpora and have achieved impressive performance on various benchmark datasets.
Let’s see how we can achieve the task of text summarization using these pipelines with HuggingFace. To test the summarization pipeline we are going to use this passage about the benefits of the internet:
The internet has revolutionized nearly every aspect of modern life, offering an array of benefits that have transformed the way we communicate, work, learn, and access information. It has facilitated instant communication worldwide, connecting people regardless of geographical barriers. Through online platforms, individuals can access a wealth of knowledge and educational resources, democratizing learning opportunities. The internet has also streamlined business operations, enabling e-commerce, remote work, and digital entrepreneurship. Additionally, it has empowered activism and social movements, providing a platform for voices to be heard and for communities to mobilize for change. Overall, the internet has significantly enhanced connectivity, efficiency, and access to information, fostering a more interconnected and informed global society.
Let’s test the summarizer variable on the above text:
from transformers import pipelinesummarizer = pipeline(task="summarization")summarizer("The internet has revolutionized nearly every aspect of modern life, offering an array of benefits that have transformed the way we communicate, work, learn, and access information. It has facilitated instant communication worldwide, connecting people regardless of geographical barriers. Through online platforms, individuals can access a wealth of knowledge and educational resources, democratizing learning opportunities. The internet has also streamlined business operations, enabling e-commerce, remote work, and digital entrepreneurship. Additionally, it has empowered activism and social movements, providing a platform for voices to be heard and for communities to mobilize for change. Overall, the internet has significantly enhanced connectivity, efficiency, and access to information, fostering a more interconnected and informed global society.",min_length=10, max_length=40)
import spacy
from transformers import pipeline
nlp = spacy.load("en_core_web_md")
summarizer = pipeline(task="summarization")
summarizer(
"The internet has revolutionized nearly every aspect of modern life, offering an array of benefits that have transformed the way we communicate, work, learn, and access information. It has facilitated instant communication worldwide, connecting people regardless of geographical barriers. Through online platforms, individuals can access a wealth of knowledge and educational resources, democratizing learning opportunities. The internet has also streamlined business operations, enabling e-commerce, remote work, and digital entrepreneurship. Additionally, it has empowered activism and social movements, providing a platform for voices to be heard and for communities to mobilize for change. Overall, the internet has significantly enhanced connectivity, efficiency, and access to information, fostering a more interconnected and informed global society."
)Note: After running the program please open the Jupyter Notebook and run all the cells.
Line 1: We import the necessary libraries
Line 2: We create a variable called summarizer and create a pipeline object inside for the summarization task.
Line 3: We add the text we need to summarize to the summarizer variable. The text will be processed and summarized here.
Free Resources