Beautiful Soup (HTML parser)
Template:Short description Script error: No such module "other uses".
Page Module:Infobox/styles.css has no content.
| Beautiful Soup | |
|---|---|
| [[Programmer|Original authorTemplate:Pluralize from text]] | Leonard Richardson |
| Initial release | Script error: No such module "Date time". |
| Written in | Python |
| Platform | Python |
| Type | HTML parser library, Web scraping |
| License | Page Template:Plainlist/styles.css has no content.
|
| Website | www |
Script error: No such module "Check for conflicting parameters".
Beautiful Soup is a Python package for parsing HTML and XML documents, including those with malformed markup. It creates a parse tree for documents that can be used to extract data from HTML,[2] which is useful for web scraping.[1][3]
History
Beautiful Soup was started in 2004 by Leonard Richardson.[citation needed] It takes its name from the poem Beautiful Soup from Alice's Adventures in Wonderland[citation needed] and is a reference to the term "tag soup" meaning poorly-structured HTML code.[4] Richardson continues to contribute to the project,[5] which is additionally supported by paid open-source maintainers from the company Tidelift.[6]
Versions
Beautiful Soup 3 was the official release line of Beautiful Soup from May 2006 to March 2012. The current release is Beautiful Soup 4.x.
In 2021, Python 2.7 support was retired and the release 4.9.3 was the last to support Python 2.7.[7]
Usage
Beautiful Soup represents parsed data as a tree which can be searched and iterated over with ordinary Python loops.[8]
Code example
The example below uses the Python standard library's urllib[9] to load Wikipedia's main page, then uses Beautiful Soup to parse the document and search for all links within.
#!/usr/bin/env python3
# Anchor extraction from HTML document
from bs4 import BeautifulSoup
from urllib.request import urlopen
with urlopen("https://en.wikipedia.org/wiki/Main_Page") as response:
soup = BeautifulSoup(response, "html.parser")
for anchor in soup.find_all("a"):
print(anchor.get("href", "/"))
Another example is using the Python requests library[10] to get divs on a URL.
import requests
from bs4 import BeautifulSoup
url = "https://wikipedia.org"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
headings = soup.find_all("div")
for heading in headings:
print(heading.text.strip())
See also
References
Page Template:Reflist/styles.css has no content.
- ^ a b Page Module:Citation/CS1/styles.css has no content."Beautiful Soup website". Retrieved 18 April 2012.
Beautiful Soup is licensed under the same terms as Python itself
- ^ Page Module:Citation/CS1/styles.css has no content.Hajba, Gábor László (2018), Hajba, Gábor László (ed.), "Using Beautiful Soup", Website Scraping with Python: Using BeautifulSoup and Scrapy, Apress, pp. 41–96, doi:10.1007/978-1-4842-3925-4_3, ISBN 978-1-4842-3925-4
{{citation}}: CS1 maint: work parameter with ISBN (link) - ^ Page Module:Citation/CS1/styles.css has no content.Python, Real. "Beautiful Soup: Build a Web Scraper With Python – Real Python". realpython.com. Retrieved 2023-06-01.
- ^ Page Module:Citation/CS1/styles.css has no content."Python Web Scraping". Udacity. 2021-02-11. Retrieved 2024-01-24.
- ^ Page Module:Citation/CS1/styles.css has no content."Code : Leonard Richardson". Launchpad. Retrieved 2020-09-19.
- ^ Page Module:Citation/CS1/styles.css has no content.Tidelift. "beautifulsoup4 | pypi via the Tidelift Subscription". tidelift.com. Retrieved 2020-09-19.
- ^ Page Module:Citation/CS1/styles.css has no content.Richardson, Leonard (7 Sep 2021). "Beautiful Soup 4.10.0". beautifulsoup. Google Groups. Retrieved 27 September 2022.
- ^ Page Module:Citation/CS1/styles.css has no content."How To Scrape Web Pages with Beautiful Soup and Python 3 | DigitalOcean". www.digitalocean.com. Retrieved 2023-06-01.
- ^ Page Module:Citation/CS1/styles.css has no content.Python, Real. "Python's urllib.request for HTTP Requests – Real Python". realpython.com. Retrieved 2023-06-01.
- ^ Page Module:Citation/CS1/styles.css has no content.Blog, SerpApi (5 March 2024). "Beautiful Soup: Web Scraping with Python". serpapi.com. Retrieved 2024-06-27.
