Web and PDF Scraper Refactoring
This repository contains the example code of the Web and PDF scraper code roast. Here are the links to the videos:
- Part 1: https://youtu.be/MXM6VEtf8SE
- Part 2: (coming soon)
Web and PDF Scraper Refactoring
Overview
a high-performance, lightweight and human friendly serving engine for scrapy
a high-performance, lightweight and human friendly serving engine for scrapy
30 Mar 01, 2022
A Web Scraping Program.
Web Scraping AUTHOR: Saurabh G. MTech Information Security, IIT Jammu. If you find this repository useful. I would appreciate if you Star it and Fork
2 Dec 14, 2022
Complete pipeline for crawling online newspaper article.
Complete pipeline for crawling online newspaper article. The articles are stored to MongoDB. The whole pipeline is dockerized, thus the user does not need to worry about dependencies. Additionally, d
4 May 27, 2022
Web crawling framework based on asyncio.
Web crawling framework for everyone. Written with asyncio, uvloop and aiohttp. Requirements Python3.5+ Installation pip install gain pip install uvloo
2k Jan 05, 2023
Extract gene TSS site form gencode/ensembl/gencode database GTF file and export bed format file.
GetTss python Package extract gene TSS site form gencode/ensembl/gencode database GTF file and export bed format file. Install $ pip install GetTss Us
6 Nov 21, 2022
Auto Join: A GitHub action script to automatically invite everyone to the organization who star your repository.
Auto Invite To The Organization By Star A GitHub Action script to automatically invite everyone to your organization that stars your repository. What
11 Dec 11, 2022
A Python module to bypass Cloudflare's anti-bot page.
cloudflare-scrape A simple Python module to bypass Cloudflare's anti-bot page (also known as "I'm Under Attack Mode", or IUAM), implemented with Reque
3k Jan 04, 2023
Google Maps crawler using Selenium
Google Maps Crawler using Selenium Built as part of the Antifragile Dev Project Selenium crawler that browses Google Maps as a regular user and stores
46 Dec 16, 2022
Quick Project made to help scrape Lexile and Atos(AR) levels from ISBN
Lexile-Atos-Scraper Quick Project made to help scrape Lexile and Atos(AR) levels from ISBN You will need to install the chrome webdriver if you have n
1 Feb 11, 2022
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
Gerapy Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Documentation Documentation
2.9k Jan 03, 2023
Binance Smart Chain Contract Scraper + Contract Evaluator
Pulls Binance Smart Chain feed of newly-verified contracts every 30 seconds, then checks their contract code for links to socials.Returns only those with socials information included, and then submit
14 Dec 09, 2022
This program scrapes information and images for movies and TV shows.
Media-WebScraper This program scrapes information and images for movies and TV shows. Summary For more information on the program, read the WebScrape_
1 Dec 05, 2021
A webdriver-based script for reserving Tsinghua badminton courts.
AutoReserve A webdriver-based script for reserving badminton courts. 使用说明 下载 chromedriver 选择当前Chrome对应版本 安装 selenium pip install selenium 更改场次、金额信息dat
4 Nov 09, 2021
Scrapes mcc-mnc.com and outputs 3 files with the data (JSON, CSV & XLSX)
mcc-mnc.com-webscraper Scrapes mcc-mnc.com and outputs 3 files with the data (JSON, CSV & XLSX) A Python script for web scraping mcc-mnc.com Link: mcc
1 Nov 07, 2021
✂️🕷️ Spider-Cut is a Network Mapper Framework (NMAP Framework)
Spider-Cut is a Network Mapper Framework (NMAP Framework) Installation | Usage | Creators | Donate Installation # Kali Linux | WSL
3 Mar 07, 2022
Python framework to scrape Pastebin pastes and analyze them
pastepwn - Paste-Scraping Python Framework Pastebin is a very helpful tool to store or rather share ascii encoded data online. In the world of OSINT,
105 Dec 29, 2022
中国大学生在线 四史自动答题刷分(现仅支持英雄篇)
中国大学生在线 “四史”学习教育竞答 自动答题 刷分 (现仅支持英雄篇，已更新可用) 若对您有所帮助，记得点个Star 🌟 ！！！ 中国大学生在线 “四史”学习教育竞答 自动答题 刷分 (现仅支持英雄篇，已更新可用) 🥰 🥰 🥰 依赖 本项目依赖的第三方库: requests 在终端执行以下
229 Dec 12, 2022
mlscraper: Scrape data from HTML pages automatically with Machine Learning
🤖 Scrape data from HTML websites automatically with Machine Learning
798 Dec 29, 2022
Google Scholar Web Scraping
Google Scholar Web Scraping This is a python script that asks for a user to input the url for a google scholar profile, and then it writes publication
1 Dec 12, 2021
Async Python 3.6+ web scraping micro-framework based on asyncio
Ruia 🕸️ Async Python 3.6+ web scraping micro-framework based on asyncio. ⚡ Write less, run faster. Overview Ruia is an async web scraping micro-frame
1.6k Jan 01, 2023