How to find elements by class using Beautiful Soup

Key takeaways:

  • Install Beautiful Soup by running pip install beautifulsoup4.

  • Import the library using from bs4 import BeautifulSoup and parse HTML with BeautifulSoup(html_content, 'html.parser').

  • Use find, find_all or select to locate elements by class. Pass attrs={'class': 'class_name'} or use class_='class_name' for convenience.

  • Handle the class attribute with class_ to avoid conflicts with Python's reserved keywords.

  • After finding elements, you can extract their text, attributes, or other data using various Beautiful Soup methods.

Beautiful Soup is a Python library used for web scraping and parsing HTML and XML documents. When working with HTML documents, we often use CSS classes to style and structure elements on a webpage. These CSS classes are essential for applying specific styles or grouping elements with similar characteristics. Sometimes, during web scraping or data extraction tasks, we need to target and retrieve elements based on their class attribute.

Step-by-step guide

Follow these steps to find elements by class using Beautiful Soup

1. Installing Beautiful Soup

Before proceeding, ensure that you have Beautiful Soup installed. If not, you can install it using pip:

pip install beautifulsoup4

2. Importing Beautiful Soup

First of all, we need to import the BeautifulSoup in our code. Here is how we can import the BeautifulSoup:

from bs4 import BeautifulSoup

3. Parsing the HTML

To start, we need to parse the HTML document using Beautiful Soup. We can obtain the HTML content from a URL or from a local file. For example, if we have the HTML content in a string called the html_content. We can parse it like this:

soup = BeautifulSoup(html_content, 'html.parser')

4. Finding elements by class name

Here are the three methods of Beautiful Soup that allow selecting elements by their class name:

  • find()

  • find_all()

  • select()

• Using the find() method

The find() method allows us to locate the first element in the HTML document that matches the specified class name. It returns a single element or None if no match is found. We can use the find() to find elements by class name in two ways:

  • Using attrs

  • Using class_

Using attrs

We can find elements by class name by using the attrs parameter provided by the find() method. We will pass a dictionary that contains the 'class' key and the target class name as the value. Here is an example:

main.py
sample.html
from bs4 import BeautifulSoup
# Read the HTML content from the local file
file_path = 'sample.html'
with open(file_path, 'r', encoding='utf-8') as file:
    html_content = file.read()
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')
header_element = soup.find(attrs={'class':'header'})
print("Element with class: header: \n",header_element)
Using class_

We can also directly use the class_ parameter to find elements with that class name. The class_ attribute is appended with an underscore to avoid conflicts with the Python-reserved keyword 'class'. Here's an example of how to use it:

main.py
sample.html
from bs4 import BeautifulSoup
# Read the HTML content from the local file
file_path = 'sample.html'
with open(file_path, 'r', encoding='utf-8') as file:
    html_content = file.read()
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')
header_element = soup.find(class_='header')
print("Element with class: header: \n",header_element)

• Using the find_all() method

The find_all() method allows us to locate all the elements in the HTML document that matches the specified class name. It returns a list of elements or an empty list if no match is found. We can use the same two parameters in the find_all() to find elements by class name:

  • Using attrs

  • Using class_

Using attrs

We can find elements by class name by using the attrs parameter provided by the find_all() method. We will pass a dictionary that contains the 'class' key and the target class name as the value. Here is an example:

main.py
sample.html
from bs4 import BeautifulSoup
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')
header_elements = soup.find_all(attrs={'class':'header'})
print("Elements with class: header:")
for element in header_elements:
    print(element)
Using class_

We can also directly use the class_ parameter to find elements with that class name. Here's an example of how to use it:

main.py
sample.html
from bs4 import BeautifulSoup
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')
header_elements = soup.find_all(class_='header')
print("Elements with class: header:")
for element in header_elements:
    print(element)

• Using the select() method

The select() method allows us to use CSS selectors to find elements, including those with specific class names. The class selector is represented by a dot (.) followed by the class name.

main.py
sample.html
from bs4 import BeautifulSoup
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')
header_elements = soup.select('.header')
print("Elements with class: header:")
for element in header_elements:
    print(element)

select also returns a list of all the elements containing specified class.

5. Accessing the element data

Once we have found the desired elements, we can access their data (e.g., text content, attributes) using various Beautiful Soup methods and attributes. For example:

main.py
sample.html
from bs4 import BeautifulSoup
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(html_content, 'html.parser')
header_elements = soup.select('.header')
print("Elements with class: header:")
for element in header_elements:
    print("Class: ",element["class"])
    print("Text: \n",element.text)

To study more about attributes and methods of Beautiful Soup, check out our Answer on Attributes and methods in BeautifulSoup4.

Ready to master web scraping? 🚀

Unlock the power of web scraping with our course on Mastering Web Scraping Using Python: From Beginner to Advanced! Whether you’re a beginner or looking to enhance your skills, this course will guide you through the essentials to advanced techniques in web scraping.

Conclusion

Beautiful Soup is an excellent tool for extracting data from HTML and XML documents. Using its class name search feature, we can easily locate specific elements within the document based on the assigned class names. This ability makes it a powerful choice for web scraping tasks, data extraction, and analysis.

Frequently asked questions

Haven’t found what you were looking for? Contact Us

How to get text in BeautifulSoup

Use the get_text() method to extract the text content from an element, ignoring HTML tags.

What is innerHTML?

innerHTML is a property in JavaScript that returns or sets the HTML content inside an element, including child tags

How to get all elements with a class name?

Use find_all(class_='class_name') in BeautifulSoup to retrieve all elements with the specified class.

What will document getElementById() return in JavaScript?

The document.getElementById() returns the first element with the specified id attribute, or null if no element is found.

How do you select all elements of a class in CSS?

Use the class selector . followed by the class name (e.g., .class_name).

Free Resources

HowDev By Educative. Copyright ©2025 Educative, Inc. All rights reserved