Python script for crawling ResearchGate.net papers
This code start crawling process by urls in start.txt and give paper details in crawled.json.
First install Python. Then install these libraries:
pip install selenium
pip install webdriver-manager
MAX_FETCH_COUNT: How many pages you want to crawl?
MAX_CACHED_NUM: We renew crawled.json after crawling each MAX_CACHED_NUM papers.
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
🐛 MASS DBLTEK EXPLOIT CHECKER USING SHODAN 🕸 Python script who crawl first shodan page and check DBLTEK vulnerability
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
poolbooru_gelscraper a simple python script for scraping images off gelbooru pools. modules required:requests_html, and os by default saves files with
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins
Aliexpress to telegram post Python script that reads Aliexpress offers urls from a Excel filename (.csv) and post then in a Telegram channel using a b
This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers. The script reads in the d
Auto Invite To The Organization By Star A GitHub Action script to automatically invite everyone to your organization that stars your repository. What
AutoReserve A webdriver-based script for reserving badminton courts. 使用说明 下载 chromedriver 选择当前Chrome对应版本 安装 selenium pip install selenium 更改场次、金额信息dat
wallstreetbets-tracker Using Python and Pushshift.io to Track stocks on the WallStreetBets subreddit.
JDRouterPush 项目简介 本项目调用京东云无线宝API,可每天定时推送积分收益情况,帮助你更好的观察主要信息 更新日志 2021-03-02: 查询绑定的京东账户 通知排版优化 脚本检测更新 支持Server酱Turbo版 2021-02-25: 实现多设备查询 查询今
This is George's Scraping Project To get started cd into the theZoo file and run: chmod +x script.sh then: ./script.sh This will spin up a Postgres co
Proxy Scraper A simple proxy scraper that utilizes the requests module in python. Usage Depending on your python installation your commands may vary.
An introduction to free, automated web scraping with GitHub’s powerful new Actions framework Published at palewi.re/docs/first-github-scraper/ Contrib
Features Scrap from url Todo [+] Search by genre [+] Search by query [+] Scrap from homepage Example # Hentai Scraper from nekopoi import Hent
Requests and Responses¶ Scrapy uses Request and Response objects for crawling web sites. Typically, Request objects are generated in the spiders and p
VG-Scraper VG-Scraper is a convinient program where you can find all the news articles instead of finding one yourself. Installing [Linux] Open a term
You-Get NOTICE: Read this if you are looking for the conventional "Issues" tab. You-Get is a tiny command-line utility to download media contents (vid
薅薅乐 安裝 使用docker docker一键安装: docker run -d --name jd classmatelin/hhl:latest. 使用 进入容器: docker exec -it jd bash 获取JD_COOKIES: python get_jd_cookies.py,
PaperRobot PaperRobot 是一个论文抓取工具，可以快速批量下载大量论文，方便后期进行持续的论文管理与学习。 PaperRobot通过多个接口抓取论文，目前抓取成功率维持在90%以上。通过配置Config文件，可以抓取任意计算机领域相关会议的论文。 Installation Down
序言 github上有很多好项目,但是国内用户连github却非常的慢.每次都要用插件或者其他工具来解决. 这次自己做一个小工具,输入github原地址后,就可以自动替换为代理地址,方便大家更快速的下载. 安装 pip install cit 主要功能与用法 主要功能 change 将目标地址转换为
django-dynamic-scraper Django Dynamic Scraper (DDS) is an app for Django which builds on top of the scraping framework Scrapy and lets you create and
Project A: WebScraper A script that prints out a list of all EXTERNAL references
pastepwn - Paste-Scraping Python Framework Pastebin is a very helpful tool to store or rather share ascii encoded data online. In the world of OSINT,
A simple Discord scraper for discord bots. That includes sending an guild members ids to an file, Mass inviter for joining servers your bot is in and Fetching all the servers of the bot (w/MemberCoun
Python based Web Scraper which can discover javascript files and parse them for juicy information (API keys, IP's, Hidden Paths etc).
grab This is a script that scrapes the longitude and latitude for any restaurant in Manila on food.grab.com, location can be adjusted. Search Result p
LiveSkidorDownload Simple tool to scrape and download cross country ski timings
Raspi-scraper is a configurable python webscraper that checks raspberry pi stocks from verified sellers.
0 Dec 15, 2021
4 Jan 9, 2022
10 Aug 18, 2022
1 Jan 2, 2022
68 Oct 8, 2022
6 Dec 6, 2022
71 Oct 4, 2022
11 Dec 11, 2022
4 Nov 9, 2021
91 Dec 08, 2022
199 Dec 12, 2022
7 Nov 27, 2022
3 Sep 08, 2021
15 Nov 24, 2022
9 Apr 06, 2022
1 Nov 03, 2021
3 Feb 13, 2022
46.4k Jan 03, 2023
575 Dec 28, 2022
47 Nov 23, 2022
35 Aug 29, 2022
1.1k Dec 17, 2022
2 Apr 26, 2022
105 Dec 29, 2022
1 Jan 06, 2022
6 Aug 26, 2022
0 Nov 22, 2021
0 Jan 07, 2022
13 Oct 15, 2022