Sometimes, we want to use XPath with BeautifulSoup and Python.

In this article, we’ll look at how to use XPath with BeautifulSoup and Python.

How to use XPath with BeautifulSoup and Python?

To use XPath with BeautifulSoup and Python, we can replace BeautifulSoup with lxml.

For instance, we write

from lxml import html
import requests

page = requests.get('http://foo.com')
tree = html.fromstring(page.content)
buyers = tree.xpath('//div[@title="buyer-name"]/text()')
prices = tree.xpath('//span[@class="item-price"]/text()')

print('Buyers: ', buyers)
print('Prices: ', prices)

to call html.fromstring to parse the HTML string into an object.

Then we call xpath with the XPath to get the items by XPath.

Conclusion

To use XPath with BeautifulSoup and Python, we can replace BeautifulSoup with lxml.