Not able to scrap content with BeautifulSoap

Vote count:

0

I am web scraping with Python and BeautifulSoap.

I have to scrape this page.

http://ift.tt/1usKVY4

From this page, I have scraped the address of hotel successfully, But I am unable to scrape the User Reviews section

Here is my code

hotel_link = "http://ift.tt/1usKVY4"

hotel_page_html = requests.get(hotel_link,headers = header).text
hotel_page_soup = BeautifulSoup(hotel_page_html)

for hotel_address in hotel_page_soup.select("div#propertyAddressContainer ul#propertyAddress"):
  print("Address: "+hotel_address.select("li")[0].text)

print(hotel_page_soup.select("div.BVRRRatingNormalOutOf"))

As you can see, using the CSS Selector div#propertyAddressContainer ul#propertyAddress, I have got the address but am unable to scrape the User Reviews section.

I have checked the Console while page loads but I dont see anything that User Reviews are loaded by an AJAX call.

So how do I scrape the Reviews section?

asked 39 secs ago

Umair

412

Not able to scrap content with BeautifulSoap