BeautifulSoup

BeautifulSoup

Used for small projects or quick tests

Links:

• Find All method

• Beautiful Soup: Build a Web Scraper With Python

Packages:

• BeautifulSoup: This module is used for iterating, searching, and modifying the parse tree over the HTML or XML parser. To download it type the below command in the terminal.

• Requests: Requests library is one of the integral part of Python for making HTTP requests to a specified URL. To download it type the below command in the terminal.

pip install beautifulsoup4
pip install requests

Imports:

import requests
from bs4 import BeautifulSoup

Get the parent span from badges, find all top-level spans inside by using find_all() with recursive=False:

from bs4 import BeautifulSoup

page = """<div class="badges">
<span>
    <span title="9 gold badges"><span class="badge1"></span><span class="badgecount">9</span></span>
    <span title="38 silver badges"><span class="badge2"></span><span class="badgecount">38</span></span>
    <span title="56 bronze badges"><span class="badge3"></span><span class="badgecount">56</span></span>
</span>
</div>"""

soup = BeautifulSoup(page)
badges = soup.body.find('div', attrs={'class': 'badges'})
for span in badges.span.find_all('span', recursive=False):
    print span.attrs['title']

prints:

9 gold badges 38 silver badges 56 bronze badges