How to Scrape Amazon.com for laptop price and spec via Selenium ?

Purpose

We are trying to search and grab the laptop information in the Amazon website by rendering through the browser via Selenium so that we can easily play around with the data. We originally tried to leverage Scrapy but it does not allow us to go this way.

Tool

• Python package: selenium/ BeautifulSoup/random/ time/ os

Target Website

Amazon.com : laptop

Steps

1. Set up the environment. Download the chromedriver driven by selenium to execute the browser rendering.
2. Go through the structure of the website to locate where the target elements are.

• When you inspect the website, you can pick price of an example and look it up across the source codes. You will find the information located in the tag of <data-component-type=”s-search-results”>.