Amazon Scraper: A command-line tool for scraping Amazon product data

Related tags

Command-line Toolspythonamazonweb-scrapingamazon-scraperamazon-scraping-libraryamazon-scraping
Overview

Amazon Product Scraper: 2021

Description

A command-line tool for scraping Amazon product data to CSV or JSON format(s).

Requirements

  • Python 3
  • pip3

Installation

Using git clone (you'll need git installed for this):

git clone https://github.com/scrapewalrus/amazon-scraper-python-2021

Or download and extract the zip file of the project manually

You'll also need to install requirements for the project to run. Locate amazon-product-scraper folder via terminal and type pip install -r requirements.txt:

Usage

To launch the Amazon scraper locate the amazon-product-scraper folder via terminal and type python amazon_scraper.py -k "your keyword". This will start the program.

NOTE: you must declare either -k or --keyword before entering your keyword. It's a required argument.

Example:

amazon_scraper.py - the name of a scraper file.

-k or --keyword - required argument to pass before entering your keyword.

-p or --proxies - optional argument to enable proxies. To avoid getting blocked I highly recommend using proxies. I'm using Residential Proxies from Oxylabs. For highest success rate, I suggest Residential Proxies over Datacenter as they're almost impossible to detect and have the smallest footprint. If you decide to use different proxy provider services keep in mind that you'll have to make some minor adjustments in get-proxies.py file.

-j or --json - optional argument for storing extracted data in .json format. Default output format is .csv.

Example of product data #1: JSON

[
    {
        "SOURCE_URL": "https://www.amazon.com/s?k=funny+t+shirt+for+women&page=1",
        "PAGE": 1,
        "KEYWORD": "funny t shirt for women",
        "PRODUCT_LINK": "https://www.amazon.com/Mostly-T-Shirt-Womens-Letter-Printed/dp/B07QN2NQ59/ref=sr_1_3?dchild=1&keywords=funny+t+shirt+for+women&qid=1627833682&sr=8-3",
        "PRODUCT_NAME": "I'm Mostly Peace Love and Light Funny T-Shirt Womens Graphic Printed Short Sleeve Tops Tee",
        "PRICE": "$21.99",
        "PRODUCT_RATING": "4.6",
        "NUMBER_OF_RATINGS": "1,637"
    },
    {
        "SOURCE_URL": "https://www.amazon.com/s?k=funny+t+shirt+for+women&page=1",
        "PAGE": 1,
        "KEYWORD": "funny t shirt for women",
        "PRODUCT_LINK": "https://www.amazon.com/YITAN-Women-Graphic-Funny-X-Large/dp/B074QMG4D7/ref=sr_1_4?dchild=1&keywords=funny+t+shirt+for+women&qid=1627833682&sr=8-4",
        "PRODUCT_NAME": "YITAN Women's Cute Juniors Tops Teen Girl Tee Funny T Shirt",
        "PRICE": "$12.99",
        "PRODUCT_RATING": "4.6",
        "NUMBER_OF_RATINGS": "12,281"
    },
    {
        "SOURCE_URL": "https://www.amazon.com/s?k=funny+t+shirt+for+women&page=1",
        "PAGE": 1,
        "KEYWORD": "funny t shirt for women",
        "PRODUCT_LINK": "https://www.amazon.com/DANVOUY-Womens-V-Neck-Doesnt-Definitely/dp/B07V55ZXVS/ref=sr_1_5?dchild=1&keywords=funny+t+shirt+for+women&qid=1627833682&sr=8-5",
        "PRODUCT_NAME": "DANVOUY Womens If My Mouth Doesn't Say It My Face Definitely Will T Shirt",
        "PRICE": "$12.99",
        "PRODUCT_RATING": "4.6",
        "NUMBER_OF_RATINGS": "6,787"
    }]

Example of product data #2: CSV

amazon-csv-product-data-example

Owner
GitHub Repository
Simple tool, to update linux kernel on ubuntu

Kerbswap Simple tool, to update linux kernel on ubuntu Information At the moment, this tool only supports "Ubuntu" distributions, but will be expanded

dword 1 Oct 31, 2021
A command line application, written in Python, for interacting with Spotify.

spotify-py-cli A command line application, written in Python, for interacting with Spotify. The primary purpose behind developing this app was to gain

Drew Loukusa 0 Oct 07, 2021
A simple terminal-based localhost chat application written in python

Chat House A simple terminal-based localhost chat application written in python How to Use? Clone the repo git clone https://github.com/heksadecimal/c

Heks 10 Nov 09, 2021
Command line interface to watch your childhood shows in hindi and english, designed with python

Sweet dreams: Most of your childhood shows Command line interface to watch your

Not Your Surya 3 Feb 13, 2022
Command line tool to keep track of your favorite playlists on YouTube and many other places.

Command line tool to keep track of your favorite playlists on YouTube and many other places.

Wolfgang Popp 144 Jan 05, 2023
Bonjour Software pypahe is a Python Package Helper command-line tool.

pypahe Bonjour Software pypahe is a Python Package Helper command-line tool. Requirements Docker runtime Usage print the latest available version of a

Bonjour Software 0 Aug 10, 2021
A command-line tool to flash python code to Codey Rocky without having to use the online mblock5 IDE.

What? A command-line tool to flash python code to Codey Rocky without having to use the online mblock5 IDE. Description This is a very low-effort proj

1 Dec 29, 2021
Tools crack instagram + fb ayok dicoba keburu premium 😁

FITUR INSTALLASI [1] pkg update && pkg upgrade [2] pkg install git [3] pkg install python [4] pkg install python2 [5] pkg install nano [6]

Jeeck 1 Dec 11, 2021
dsub is a command-line tool that makes it easy to submit and run batch scripts in the cloud.

Open-source command-line tool to run batch computing tasks and workflows on backend services such as Google Cloud.

Data Biosphere 233 Jan 01, 2023
Loading animation; a progress bar

Loading animation; a progress bar. When you know the remaining time or task completion percentage, then you’re able to show an animated progress bar:

Goldy 1 Jan 23, 2022
spade is the next-generation networking command line tool.

spade is the next-generation networking command line tool. Say goodbye to the likes of dig, ping and traceroute with more accessible, more informative and prettier output.

Vivaan Verma 5 Jan 28, 2022
NudeNet wrapper made to provide a simple cli interface to the library

Nudenet Wrapper. Small warpper script for NudeNet Made to provide a small and easy to use cli interface with the library. You can indicate a single im

1 Oct 20, 2021
flora-dev-cli (fd-cli) is command line interface software to interact with flora blockchain.

Install git clone https://github.com/Flora-Network/fd-cli.git cd fd-cli python3 -m venv venv source venv/bin/activate pip install -e . --extra-index-u

14 Sep 11, 2022
Amazon Scraper: A command-line tool for scraping Amazon product data

Amazon Product Scraper: 2021 Description A command-line tool for scraping Amazon product data to CSV or JSON format(s). Requirements Python 3 pip3 Ins

49 Nov 15, 2021
inklayers is a command line program that exports layers from an SVG file.

inklayers is a command line program that exports layers from an SVG file. It can be used to create slide shows by editing a single SVG file.

11 Mar 29, 2022
Colors in Terminal - Python Lang

🎨 Colorate - Python 🎨 About Colorate is an Open Source project that makes it easy to use Python color coding in your projects. After downloading the

0110 Henrique 1 Dec 01, 2021
CLI para o projeto Compilado (Newsletter e Podcast do Código Fonte TV)

Compilado CLI Automatização de tarefas através de linha de comando para a geração de assets para episódios do Compilado, a newsletter e podcast do can

Gabriel Froes 18 Nov 21, 2022
Keybase-cli - Keybase docker container that exposes the keybase CLI and some common commands such as getting files or loading github action secrets

keybase-cli Keybase docker container that exposes the keybase CLI and some commo

Bryce Geiser 4 Aug 04, 2022
Another (unofficial) Qt CLI Installer on multi-platforms

Another Qt installer(aqt) Release: Documentation: Test status: and Coverage: This is a utility alternative to the official graphical Qt installer, for

Hiroshi Miura 528 Jan 02, 2023
A minimal ascii-representation of your local weather.

Ascii-Weather A simple, ascii-based weather visualizer for the terminal. The ascii-art updates to match the current weather and conditions. Uses ipinf

Aaron R. 12 Jan 29, 2022