Web and PDF Scraper Refactoring
This repository contains the example code of the Web and PDF scraper code roast. Here are the links to the videos:
- Part 1: https://youtu.be/MXM6VEtf8SE
- Part 2: (coming soon)
Web and PDF Scraper Refactoring
Overview
Google Scholar Web Scraping
Google Scholar Web Scraping This is a python script that asks for a user to input the url for a google scholar profile, and then it writes publication
1 Dec 12, 2021
A Powerful Spider(Web Crawler) System in Python.
pyspider A Powerful Spider(Web Crawler) System in Python. Write script in Python Powerful WebUI with script editor, task monitor, project manager and
15.7k Jan 04, 2023
An application that on a given url, crowls a web page and gets all words, sorts and counts them.
Web-Scrapping-1 An application that on a given url, crowls a web page and gets all words, sorts and counts them. Installation Using the package manage
1 Jan 16, 2022
A spider for Universal Online Judge(UOJ) system, converting problem pages to PDFs.
Universal Online Judge Spider Introduction This is a spider for Universal Online Judge (UOJ) system (https://uoj.ac/). It also works for all other Onl
1 Dec 07, 2021
A social networking service scraper in Python
snscrape snscrape is a scraper for social networking services (SNS). It scrapes things like user profiles, hashtags, or searches and returns the disco
2.4k Jan 01, 2023
Dex-scrapper - Hobby project for scrapping dex data on VeChain
Folders /zumo_abis # abi extracted from zumo repo /zumo_pools # runtime e
3 Jan 20, 2022
Webservice wrapper for hhursev/recipe-scrapers (python library to scrape recipes from websites)
recipe-scrapers-webservice This is a wrapper for hhursev/recipe-scrapers which provides the api as a webservice, to be consumed as a microservice by o
1 Jul 09, 2022
学习强国 自动化 百分百正确、瞬间答题，分值45分
项目简介 学习强国自动化脚本，解放你的时间！ 使用Selenium、requests、mitmpoxy、百度智能云文字识别开发而成 使用说明 注：Chrome版本 驱动会自动下载 首次使用会生成数据库文件db.db，用于提高文章、视频任务效率。 依赖安装 pip install -r require
359 Dec 30, 2022
Web scraped S&P 500 Data from Wikipedia using Pandas and performed Exploratory Data Analysis on the data.
Web scraped S&P 500 Data from Wikipedia using Pandas and performed Exploratory Data Analysis on the data. Then used Yahoo Finance to get the related stock data and displayed them in the form of chart
3 Sep 09, 2022
WebScrapping Project - G1 Latest News
Web Scrapping com Python Esse projeto consiste em um código para o usuário buscar as últimas nóticias sobre um termo qualquer, no site G1. Para esse p
2 Feb 13, 2022
Web scrapping tool written in python3, using regex, to get CVEs, Source and URLs.
searchcve Web scrapping tool written in python3, using regex, to get CVEs, Source and URLs. Generates a CSV file in the current directory. Uses the NI
32 Oct 10, 2022
Scrap the 42 Intranet's elearning videos in a single click
42intra_scraper Scrap the 42 Intranet's elearning videos in a single click. Why you would want to use it ? Adjust speed at your convenience. (The intr
5 Oct 27, 2022
This tool crawls a list of websites and download all PDF and office documents
This tool crawls a list of websites and download all PDF and office documents. Then it analyses the PDF documents and tries to detect accessibility issues.
7 Sep 30, 2022
A Python module to bypass Cloudflare's anti-bot page.
cloudscraper A simple Python module to bypass Cloudflare's anti-bot page (also known as "I'm Under Attack Mode", or IUAM), implemented with Requests.
2.6k Dec 31, 2022
爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、百度指数、维普万方、Zlibraty、Oalib、小说、招标网、采购网、小红书》
lxSpider 爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、百度指数、维普万方、Zlibraty、Oalib、小说网站、招标采购网》 简介： 时光荏苒，记不清写了多少案例了。
793 Jan 05, 2023
The first public repository that provides free BUBT website scraping API script on Github.
BUBT WEBSITE SCRAPPING SCRIPT I think this is the first public repository that provides free BUBT website scraping API script on github. When I was do
3 Feb 10, 2022
爱奇艺会员,腾讯视频,哔哩哔哩,百度,各类签到
My-Actions 个人收集并适配Github Actions的各类签到大杂烩 不要fork了 ⭐️ star就行 使用方式 新建仓库并同步代码 点击Settings - Secrets - 点击绿色按钮 (如无绿色按钮说明已激活。直接到下一步。) 新增 new secret 并设置 Secr
280 Dec 30, 2022
京东茅台抢购最新优化版本，京东秒杀，添加误差时间调整，优化了茅台抢购进程队列
京东茅台抢购最新优化版本，京东秒杀，添加误差时间调整，优化了茅台抢购进程队列
776 Jul 28, 2021
A scalable frontier for web crawlers
Frontera Overview Frontera is a web crawling framework consisting of crawl frontier, and distribution/scaling primitives, allowing to build a large sc
1.2k Jan 02, 2023
Unja is a fast & light tool for fetching known URLs from Wayback Machine
Unja Fetch Known Urls What's Unja? Unja is a fast & light tool for fetching known URLs from Wayback Machine, Common Crawl, Virus Total & AlienVault's
10 Aug 07, 2022