Google Maps crawler using Selenium

Overview

Google Maps Crawler using Selenium

antifragile project python version GitHub Code style: black try/except style: tryceratops Open in Visual Studio Code Follow guilatrova

Built as part of the Antifragile Dev Project

Selenium crawler that browses Google Maps as a regular user and stores the data in an object.


Sample

Sample

Extracted data example:

 Place(                                                                                                            │ │
│ │ │   name='Pizza Me Santos',                                                                                    │ │
│ │ │   address='Av. Washington Luis, 565 - loja 05 - Boqueirão, Santos - SP, 11055-001',                          │ │
│ │ │   business_hours={                                                                                           │ │
│ │ │   │   'Wednesday': '6–10:30PM',                                                                              │ │
│ │ │   │   'Thursday': '6–10:30PM',                                                                               │ │
│ │ │   │   'Friday': '6–11PM',                                                                                    │ │
│ │ │   │   'Saturday': '6–11PM',                                                                                  │ │
│ │ │   │   'Sunday': '6–10:30PM',                                                                                 │ │
│ │ │   │   'Monday': '6–10:30PM',                                                                                 │ │
│ │ │   │   'Tuesday': '6–10:30PM'                                                                                 │ │
│ │ │   },                                                                                                         │ │
│ │ │   photo_link='https://lh5.googleusercontent.com/p/AF1QipMyVkKioODaU0A_ogHPXosm_QcMndZN6I6YHIDo=w408-h272-k-no│ │
│ │ │   rate='5.0',                                                                                                │ │
│ │ │   reviews='16 reviews',                                                                                      │ │
│ │ │   extra_attrs={                                                                                              │ │
│ │ │   │   'Menu': 'Menu\npizzame-santos.goomer.app',                                                             │ │
│ │ │   │   'Website: pizzame-santos.goomer.app ': 'pizzame-santos.goomer.app',                                    │ │
│ │ │   │   'Phone: (13) 3385-0059 ': '(13) 3385-0059',                                                            │ │
│ │ │   │   'Plus code: 2MHC+WF Boqueirão, Santos - State of São Paulo': '2MHC+WF Boqueirão, Santos - State of São │ │
│ │ Paulo'                                                                                                         │ │
│ │ │   },                                                                                                         │ │
│ │ │   traits={                                                                                                   │ │
│ │ │   │   'Service options': ['No-contact delivery', 'Delivery', 'Takeaway', 'Dine-in'],                         │ │
│ │ │   │   'Accessibility': ['Wheelchair-accessible entrance'],                                                   │ │
│ │ │   │   'Offerings': ['Organic dishes', 'Vegetarian options'],                                                 │ │
│ │ │   │   'Dining options': ['Dessert'],                                                                         │ │
│ │ │   │   'Amenities': ['Good for kids'],                                                                        │ │
│ │ │   │   'Atmosphere': ['Casual'],                                                                              │ │
│ │ │   │   'Crowd': ['Groups'],                                                                                   │ │
│ │ │   │   'Planning': ['Accepts reservations'],                                                                  │ │
│ │ │   │   'Payments': ['Credit cards']                                                                           │ │
│ │ │   }                                                                                                          │ │
│ │ )
Owner
Guilherme Latrova
Sportist, Creator, Software writer, Coffee appreciator, Lucky husband and God servant :)
Guilherme Latrova
Fundamentus scrapy

Fundamentus_scrapy Baixa informacões que os outros scrapys do fundamentus não realizam. Para iniciar (python main.py), sera criado um arquivo chamado

Guilherme Silva Uchoa 1 Oct 24, 2021
Web Crawlers for Data Labelling of Malicious Domain Detection & IP Reputation Evaluation

Web Crawlers for Data Labelling of Malicious Domain Detection & IP Reputation Evaluation This repository provides two web crawlers to label domain nam

1 Nov 05, 2021
A universal package of scraper scripts for humans

Scrapera is a completely Chromedriver free package that provides access to a variety of scraper scripts for most commonly used machine learning and data science domains.

299 Dec 15, 2022
Demonstration on how to use async python to control multiple playwright browsers for web-scraping

Playwright Browser Pool This example illustrates how it's possible to use a pool of browsers to retrieve page urls in a single asynchronous process. i

Bernardas Ališauskas 8 Oct 27, 2022
Scrapy-based cyber security news finder

Cyber-Security-News-Scraper Scrapy-based cyber security news finder Goal To keep up to date on the constant barrage of information within the field of

2 Nov 01, 2021
A web scraper which checks price of a product regularly and sends price alerts by email if price reduces.

Amazon-Web-Scarper Created a web scraper using simple functions to check price of a product on amazon (can be duplicated to check price at other marke

Swaroop Todankar 1 Jan 17, 2022
PS5 bot to find a console in france for chrismas 🎄🎅🏻 NOT FOR SCALPERS

Une PS5 pour Noël Python + Chrome --headless = une PS5 pour noël MacOS Installer chrome Tweaker le .yaml pour la listes sites a scrap et les criteres

Olivier Giniaux 3 Feb 13, 2022
Amazon scraper using scrapy, a python framework for crawling websites.

#Amazon-web-scraper This is a python program, which use scrapy python framework to crawl all pages of the product and scrap products data. This progra

Akash Das 1 Dec 26, 2021
👁️ Tool for Data Extraction and Web Requests.

httpmapper 👁️ Project • Technologies • Installation • How it works • License Project 🚧 For educational purposes. This is a project that I developed,

15 Dec 05, 2021
Web-Scrapper using Python and Flask

Web-Scrapper "[초급]Python으로 웹 스크래퍼 만들기" 코스 -NomadCoders 기초적인 Python 문법강의부터 시작하여 웹사이트의 html파일에서 원하는 내용을 Scrapping해서 출력, csv 파일로 저장, flask를 이용한 간단한 웹페이지

윤성도 1 Nov 10, 2021
jd_maotai rpa 基于selenium驱动的jd抢购rpa机器人

jd_maotai rpa 基于selenium驱动的jd抢购rpa机器人, 照顾我们这样的马大哈, 不会忘记抢购了, 祝大家过年都能喝上茅台. 特别声明: 本仓库发布的jd_maotai_rpa项目定义为自动化rpa项目, 是用于防止忘记参与jd茅台的活动(由于本人时常忘记), 而不是为了秒杀和抢

35 Nov 18, 2022
Scrape all the media from an OnlyFans account - Updated regularly

Scrape all the media from an OnlyFans account - Updated regularly

CRIMINAL 3.2k Dec 29, 2022
VG-Scraper is a python program using the module called BeautifulSoup which allows anyone to scrape something off an website. This program lets you put in a number trough an input and a number is 1 news article.

VG-Scraper VG-Scraper is a convinient program where you can find all the news articles instead of finding one yourself. Installing [Linux] Open a term

3 Feb 13, 2022
:arrow_double_down: Dumb downloader that scrapes the web

You-Get NOTICE: Read this if you are looking for the conventional "Issues" tab. You-Get is a tiny command-line utility to download media contents (vid

Mort Yao 46.4k Jan 03, 2023
download NCERT books using scrapy

download_ncert_books download NCERT books using scrapy Downloading Books: You can either use the spider by cloning this repo and following the instruc

1 Dec 02, 2022
The open-source web scrapers that feed the Los Angeles Times California coronavirus tracker.

The open-source web scrapers that feed the Los Angeles Times' California coronavirus tracker. Processed data ready for analysis is available at datade

Los Angeles Times Data and Graphics Department 51 Dec 14, 2022
A Web Scraper built with beautiful soup, that fetches udemy course information. Get udemy course information and convert it to json, csv or xml file

Udemy Scraper A Web Scraper built with beautiful soup, that fetches udemy course information. Installation Virtual Environment Firstly, it is recommen

Aditya Gupta 15 May 17, 2022
A spider for Universal Online Judge(UOJ) system, converting problem pages to PDFs.

Universal Online Judge Spider Introduction This is a spider for Universal Online Judge (UOJ) system (https://uoj.ac/). It also works for all other Onl

TriNitroTofu 1 Dec 07, 2021
API to parse tibia.com content into python objects.

Tibia.py An API to parse Tibia.com content into object oriented data. No fetching is done by this module, you must provide the html content. Features:

Allan Galarza 25 Oct 31, 2022
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.

Pattern Pattern is a web mining module for Python. It has tools for: Data Mining: web services (Google, Twitter, Wikipedia), web crawler, HTML DOM par

Computational Linguistics Research Group 8.4k Jan 08, 2023