Issue with regular expressions while parsing source code [closed] - regex

Question

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
Im trying to get some information from a page source code.
For example, lets take this amazon product.
https://www.amazon.com/gp/product/B07PWCJZJ6?pf_rd_p=2d1ab404-3b11-4c97-b3db-48081e145e35&pf_rd_r=0PF9KX04Y9GAPGCXBDAP
We can check the source code with
view-source:https://www.amazon.com/gp/product/B07PWCJZJ6?pf_rd_p=2d1ab404-3b11-4c97-b3db-48081e145e35&pf_rd_r=0PF9KX04Y9GAPGCXBDAP
My objective is to get some data like the product descriptions (1366x768 LED display for example)
Im basically taking the whole source code and then using regular expressions to get the data I need.
Im doing something like this:
import requests
source = requests.get(someUrl)
data = re.findall(r'<span class=\"a-list-item\">(.*?)<\/span><\/li>', source.content)
Which should give me every product description, but I keep getting the TypeError: cannot use a string pattern on a bytes-like object
I don't know if my regex is wrong or source.content is not giving me the source code

Answer 1

As the diagnostic explains, the regex library wants a string input, not bytes.
The requests documentation is pretty clear:
... access the response body as bytes, for non-text requests:
>>> r.content
Given that you retrieved some HTML text
you will want to decode it,
or let the library decode it for you:
>>> source.content.decode(source.encoding)
or
>>> source.text
Both expressions return a Unicode string,
which would be the perfect input for that regex.
Separate item: make Soup, not Regexes -- bs4 is the more appropriate tool, here.