Download all posts and comments in a subreddit

Overview

subreddit downloader

This subreddit downloader downloads all posts and comments in a subreddit

For a tutorial to use this program please follow this medium article - https://medium.com/@guneet.android/download-subreddit-data-for-nlp-projects-c8fa85c0df87

This script makes use of the pushshift API ( https://github.com/pushshift/api ) to download the entire dump of posts and comments in a specific subreddit

Simply replace the subreddit name in the fourth line of code and run the script

It will download two files one for all the posts and one for all the comments

Patched UTF-8 encoding*

Please note pushshift has rate limitations -

repeated 429 error codes will result in blocklisting of IP

If your data format is in single quotes

Please use this JSON fixer by @mbrzusto

https://gist.github.com/mbrzusto/23fe728966247f25f3ec

Owner
Guneet
Guneet
Download all games from a public Itch.io Game Jam

Itch Jam Downloader Downloads all games from a public Itch.io Game Jam. What you'll need: Python 3.8+ pip install -r requirements.txt For site mirrori

Dragoon Aethis 19 Dec 07, 2022
Python code to crawl computer vision papers from top CV conferences. Currently it supports CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, SIGGRAPH

Python code to crawl computer vision papers from top CV conferences. Currently it supports CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, SIGGRAPH. It leverages selenium, a website testing framework to crawl

Xiaoyang Huang 39 Nov 21, 2022
This script fully automates of downloading tiktok videos, editing them,compiling them and finally uploading them to youtube.

This script fully automates of downloading tiktok videos, editing them,compiling them and finally uploading them to youtube. If you wanted to create a tiktok video compiilation youtubbe channel this

Supriyo Sarkar 32 Dec 16, 2022
GTK4 + Python tutorial with code examples

Taiko's GTK4 Python tutorial Wanna make apps for Linux but not sure how to start with GTK? This guide will hopefully help! The intent is to show you h

190 Jan 08, 2023
QGIS plugin to dwonload DEMs from OpenTopography.org

OpenTopography-DEM-Downloader-qgis-plugin QGIS plugin to dwonload DEMs from OpenTopography.org This plug-in allows you to download DEMs from OpenTopgr

Kyaw Naing Win 7 Sep 20, 2022
Youtube Downloader Telegram Bot ๐Ÿ˜‰

Youtube Dl bot ๐Ÿ˜‰ Prerequisite ffmpeg install dependencies pip3 install -r requirements.txt Setup Bot - Change configuration config.py File - insta

Aryan Vikash 285 Dec 06, 2022
๐Ÿ“ผCommand line tool based on youtube-dl to easily download selected channels from your subscriptions.

youtube-cdl Command line tool based on youtube-dl to easily download selected channels from your subscriptions. This tool is very handy if you want to

Anatoly 64 Dec 25, 2022
Download courses from khanacademy.org

khan-dl A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4.

rand-net 806 Jan 03, 2023
Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included

WV-AMZN-4K-RIPPER Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included For CDM You can Mail :- 11 Dec 23, 2021

Download Youtube videos in mp4 format in a fast, easy, convenient way made with Python!

yt_downloader Download Youtube videos in mp4 format in a fast, easy, convenient way made with Python! Required Modules pytube os time colorama Errors

3 Jul 02, 2022
A cross-platform python based utility to download courses from udemy for personal offline use.

udemy-dl A cross-platform python based utility to download courses from udemy for personal offline use. Warning Udemy has started to encrypt many of t

Nasir Khan 4.6k Dec 31, 2022
Download every approved Obsidian.md community Plugin and Theme

obsidian-repos-downloader Contents What? Why? Setup Requirements Download Run Getting Started Usage - all the arguments Output Directories Flatter Str

Clare Macrae 16 Dec 13, 2022
This simple Python script allows you to download songs on Telegram๐ŸŒธโค๏ธ๐Ÿ˜

SongsDownloaderTgBot ๐Ÿ“บ YouTube Song Downloader Bot For Telegram ๐Ÿ”ฎ 3X Fast Telethon Based Bot โšœ Open Source Bot ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป Demo : ๐—”๐—ป๐—ป๐—ถ๐—ฒ - ๐—˜๐—น๐—ถ๐˜‡?

Sehath Perera 23 Dec 03, 2022
Terminal based YouTube player and downloader

termitube NOTE: THIS REPOSITORY IS A FORK OF mps-youtube as mps-youtube has been unmaintained for almost a year now. Features Search and play audio/vi

Otis/Jacob Root 27 Dec 23, 2022
Application Updater using an download link

Application-Updater This tool will update your app using an storage link

ExtremeDev 1 Dec 20, 2021
This project is helps to download contents from Streamtape by utilizing the API

It scrapes Streamtape api and download contents from the site.

Debiprasad Das 5 Dec 28, 2022
Download India Stocks Historical Data

Kite Helper - Download Stock Market Data ๐ŸŒŽ Website Simple Application to Download any stock market data in .csv format using Kite ๐Ÿƒโ€โ™‚๏ธ Running Serve

Pishang Ujeniya 12 Dec 06, 2022
Downloads and Updates GOG Galaxy 2.0 Plugins/Integrations

GOG Galaxy Plugins Downloader Summary This program downloads GOG Galaxy 2.0 Plugins and installs them to the proper location. You probably do not want

slashbunny 253 Dec 12, 2022
Download any video from YouTube playlists

youtube-dl Download any videos from YouTube playlists. Requirements Python 3 BeautifulSoup4 PyQt PyQtWebEngine pytube pyyoutube python-decouple Usage

Antonio Fortin 1 Oct 26, 2021
Super S3 command line tool

s4cmd Super S3 command line tool Author: Chou-han Yang (@chouhanyang) Current Maintainers: Debodirno Chandra (@debodirno) | Naveen Vardhi (@rozuur) |

Bloomreach 1.2k Dec 27, 2022