Twayback: Downloading deleted Tweets from the Wayback Machine, made easy

Overview

Twayback: Downloading deleted Tweets from the Wayback Machine, made easy

forthebadge made-with-python

windows python

screenshot

Finding and downloading deleted Tweets takes a lot of time. Thankfully, with this tool, it becomes a piece of cake! ๐ŸŽ‚

Twayback is a portmanteau of Twitter and the Wayback Machine. Enter your desired Twitter username, and let Twayback do the rest!

Requirements

  • Python 3
  • waybackpack
    • Install: pip install waybackpack or pip3 install waybackpack
    • Link to repo

Features

  • Can download some or all of a user's archived deleted Tweets.
  • Allows custom time range to narrow search for deleted Tweets archived between two dates.
  • Differentiates between accounts that are active, suspended, or don't/no longer exist.
  • Lets you know if a target handle's archived Tweets have been excluded from the Wayback Machine.

Usage

twayback -u USERNAME [OPTIONS]
Example: twayback -u jack

-u, --username        Specify target user's Twitter handle
-from, --fromdate     Narrow search for deleted Tweets *archived* on and after this date
                      (can be combined with -to)
                      (format YYMMDD)
-to, --todate         Narrow search for deleted Tweets *archived* on and before this date
                      (can be combined with -from)
                      (format YYMMDD)

Installation

For Windows only

  1. Download the latest EXE file.
  2. Launch Command Prompt in the EXE file's directory.
  3. Run the command twayback -u USERNAME (Replace USERNAME with your target handle).

For Windows, Linux, and macOS

  1. Download the latest ZIP file.
  2. Extract ZIP file to a directory of your choice.
  3. Open terminal in that directory.
  4. Run the command pip install -r requirements.txt.
  5. Run the command twayback -u USERNAME (Replace USERNAME with your target handle).

For more information, check out the Usage section above.

Things to keep in mind

  • Quality of the HTML files depends on how the Wayback Machine saved them. Some are better than others.
  • This tool is best for text. You might have some luck with photos. You cannot download videos.
  • By definition, if an account is suspended or no longer exists, all their Tweets would be considered deleted.
  • Custom date range is not about when Tweets were made, but rather when they were archived. For example, a Tweet from 2011 may have been archived today.

Future plans

  • GUI. This is a biggie. I don't know shit about Python, let alone GUI. But I'm hoping I can design one using Tkinter Designer. But I don't know how I can link actions to buttons and shit like that, that stuff is super foreign to me, so any help is appreciated, it would mean so much.

Plenty of thanks to jsvine for his amazing work on waybackpack. Without it, this tool cannot work nearly as well.

I hope you enjoy my little script. Please use it for good. Whatever you are, be a good one.

Comments
  • AttributeError: 'NoneType' object has no attribute 'getText'

    AttributeError: 'NoneType' object has no attribute 'getText'

    I got two errors:

    1. The first error occured after grabbing links from wayback machine UnboundLocalError: local variable 'wayback_id' referenced before assignment

    2. Second, the error occured when I typed 'text' or 'both' AttributeError: 'NoneType' object has no attribute 'getText'

    please help, thanks.

    bug good first issue 
    opened by adrn-mm 9
  • Error in the process

    Error in the process

    Hello,

    Thanks for the app which helps save a lot of time. I have tried with several users and in various ways and it does not work. I pass the screenshots of the errors.

    Thank you very much.

    image image

    opened by barripdmx 8
  • Twayback Partial Re-Write

    Twayback Partial Re-Write

    Hello,

    We recently came across your project, and thought we could contribute by re-writing some parts of the code. An effort was made to keep the logic and structure of the code the same.

    By our metrics, we have achieved a speed-up of around 20-30% for accounts with over 1000 tweets. We don't have any actual tests written for the script however, so we have attached it here for review.

    A substantial change was moving from Selenium to Playwright. Users need to run 'playwright install' to install the playwright browsers before running the script.

    Another smaller, but notable change is that due to the nature of the as_completed function, it is not possible to show any indicator of progress during the gathering of statuses, and the order in which the tweets are returned is scrambled due to the script processing website information based on what web request finishes first.

    If the style & format of the re-write is acceptable, and the changes proposed are not considered to be critical, we will proceed with creating a pull request.

    twayback2.txt

    opened by AccentuSoft 5
  • Parsing fails when encountering text in cyrillic

    Parsing fails when encountering text in cyrillic

    Hi, when I try to search for deleted tweets from an account that uses cyrillic, the process fails with the following exception:

    Parsing text...: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 4/4 [00:07<00:00, 2.00s/it] Traceback (most recent call last): File "twayback.py", line 184, in File "encodings\cp1250.py", line 19, in encode UnicodeEncodeError: 'charmap' codec can't encode character '\u0430' in position 0: character maps to [20476] Failed to execute script 'twayback' due to unhandled exception!

    Anything that could be done on my end? Thanks,

    opened by Traut89now 5
  • KeyError: 'closest' when parsing accounts with larger number of tweets.

    KeyError: 'closest' when parsing accounts with larger number of tweets.

    Hi,

    after the latest update, the tool seems to be working perfectly when scraping accounts with lower number of tweets archived, but when I tried accounts with 1000+ tweets archived, the process failed with: Traceback (most recent call last): File "C:\15tway\twayback B\twayback.py", line 112, in wayback_url = (jsonResponse['archived_snapshots']['closest']['url']) KeyError: 'closest'

    as also seen below on the screenshots.

    image

    opened by Traut89now 4
  • No deleted tweets have been found

    No deleted tweets have been found

    Hi! Does anyone else, after they pull one search of an user, get an "No deleted tweets have been found" when they try to look for another one? I even try with the same one I did first and it pulls the same thing. I even close and open again the terminal but nothing, got the same issue. Was anyone run into this?

    opened by sofiemmc 3
  • Cleaned up code a bit

    Cleaned up code a bit

    No changes to functionality. I went through and added some more comments, renamed variables to be more descriptive and combined some redundancy with list creation

    opened by humandecoded 2
  • Feature proxy

    Feature proxy

    to avoid rate limiting for large groups of tweets added the ability to use proxies with our GET requests. User will need to provide a list of proxy URLs for the script to randomly pull from

    opened by humandecoded 1
  • Twitter Rate Limiting

    Twitter Rate Limiting

    It looks like twitter is throwing up 429 (too many requests) after the first 900 or so hits. According to their API documentation they limit to around 900 hits per 15 minutes. Although we are not using the API, it seems they are putting similar limiting in place per IP.

    I've begun working on a branch that lets users plug in a list of proxies the script will rotate through to avoid 429. This does make the tool less approachable but at this point it's not working on large groups of tweets. Will keep everyone updated

    enhancement 
    opened by humandecoded 1
  • Adding more Detailed information

    Adding more Detailed information

    I am submitting this PR so that newbie cyber people & OSINT Analysts with no python experience can better understand installation and usage.

    • There is no information about using the command git clone in order to clone the repository
    • No information in regards to changing directory | cd
    • No information in regards to the command pip3 install -r requirements.txt
    • No information in regards to the command python3 twayback.py -u username (if we put ourselves in the shoes of anyone without any python knowledge, they will just write in their terminal twayback.py which won't be of any use, it's typically something I would have done in the past as when reading installation & usage instructions I would just copy paste without trying to understand. ๐Ÿคฆ

    Hope this helps a little, awesome tool & thanks for putting it out there !

    opened by C3n7ral051nt4g3ncy 1
  • UPDATE requirements.txt

    UPDATE requirements.txt

    Module aiohttp is required

    Traceback (most recent call last):
      File "/home/gdhindii/twayback/twayback.py", line 18, in <module>
        from aiohttp import ClientSession, TCPConnector
    ModuleNotFoundError: No module named 'aiohttp'
    
    opened by sam5epi0l 1
  • Proxy File List

    Proxy File List

    I am not sure this is an issue or just lack of knowledge from my part, I rather suspect it is the latter, however, I am attempting to recover tweets from an account that has a couple of thousand deleted tweets. I eventually encounter a rate limit error as is to be expected. I saw the way to handle requests like these is to use a proxy file. Now I have a .txt file that I added inside the twayback folder with a list of proxies and formatted according to the guidelines:

    url:port url:port url:port

    and so on. I keep getting error for each proxy server so Im not sure if Im doing this right and what the issue could be. It will stay stuck at 0%, switch between proxies until it eventually just gives up with a "" error.

    My request looks like this:

    python3 twayback.py --proxy-file proxyfiles.txt -u USERNAME

    Am I doing this correctly?

    opened by DirkGaston 2
  • it doesn't work anymore

    it doesn't work anymore

    once I launch the app it closes itself, it was working fine before the last update I tried downloading a pervious version and the same thing happened I tried it on 2 devices and it didn't work on any of them

    opened by unaufhaltbar 4
Releases(10-16-22)
Owner
He/Him. Good-for-nothing GitHubber.
Using Youtube downloader is the fast and easy way to download and save any YouTube video.

Youtube video downloader using Django Using Django as a backend along with pytube module to create Youtbue Video Downloader. https://yt-videos-downloa

Suman Raj Khanal 10 Jun 18, 2022
Download history data from binance and save to dataframe or csv file

Binance history data downloader Download history data from binance and save to dataframe or csv file

10 Dec 02, 2022
Download Photo and Video from Wall of specific user or community

vkontakte-downloader Download Photo and Video from Wall of specific User or Community on https://vk.com Setup Clone the project git clone https://gith

4 Jul 20, 2022
Download courses from khanacademy.org

khan-dl A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4.

rand-net 806 Jan 03, 2023
S3 file download with Python and access with VBA

S3 file download with Python and access with VBA This simple project is using the following stacks: Python AWS S3 VBA/Excel A Bitcoin API With this st

Julio Cesar Scheidt 0 Dec 07, 2021
This simple Python script allows you to download songs on Telegram๐ŸŒธโค๏ธ๐Ÿ˜

SongsDownloaderTgBot ๐Ÿ“บ YouTube Song Downloader Bot For Telegram ๐Ÿ”ฎ 3X Fast Telethon Based Bot โšœ Open Source Bot ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป Demo : ๐—”๐—ป๐—ป๐—ถ๐—ฒ - ๐—˜๐—น๐—ถ๐˜‡?

Sehath Perera 23 Dec 03, 2022
Python youtube playlist downloader

Youtube-Playlist-Downloader-python ๐Ÿ‘ This program is a simple Youtube playlist downloader where you input the playlist link, and then the desired pat

Pepczenko 2 Dec 25, 2021
MMDL (Mega Music Downloader) - A tool to easily download music.

mmdl - Mega Music Downloader What is mmdl โ“ TLDR: MMDL is a cli app which allows you to quickly and efficiently download one or multiple songs from Yo

techboy-coder 30 Dec 13, 2022
Programmers-quest - Programmer's Quest! An open source MMO built on top of the Panda3D game engine and Astron server

Programmer's Quest! Programmer's Quest! The open source Python 3 2D MMORPG showc

Jordan Maxwell 5 Oct 07, 2022
Tool To download 4KHDR DV SDR from AppleTV

# APPLE-TV 4K Downloader Tool To download 4K HDR DV SDR from AppleTV Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the scripts to

5 Dec 25, 2021
Libretrofuzz - Fuzzy Retroarch thumbnail downloader

Fuzzy Retroarch thumbnail downloader In Retroarch, when you use the manual scann

8 Nov 26, 2022
Most versatile Telegram torrent and youtube-dl bot.

TorToolkit Telegram So basically Tortoolkit is aimed to be the most versatile torrent leecher and Youtube-DL bot for telegram. This bot is highly cust

Yash Khadse 541 Dec 22, 2022
Downloads .ksy files and their dependencies straight from the official kaitai-struct format gallery.

ksy-dl Downloads .ksy files and their dependencies straight from the official kaitai-struct format gallery. This tool will: Fetch any of the official

3 Jun 20, 2022
Python module to download all media from a CyberDrop gallery.

CyberDrop Downloader Intro Let's suppose you found out the Eva G (bby_gee) leak on https://cyberdrop.me/a/aWAt4TWY. You wish you could download the en

Quatrecentquatre 1 Dec 12, 2021
Shit-fetch - Shitpost fetcher (downloader)

shit-fetch Download shitpost (random) from https://random-shitpost.com/ Usage ./shitfetch.py --nsfw (true/false) --output ~/Downloads (default : ./)

Pinokaille 1 Jan 02, 2022
Script that allows to download portable installers of different versions Adobe software for macOS

What is this and for what This is a script that allows you to download portable installers of programs from Adobe for macOS with different versions. T

715 Jan 06, 2023
Fully Automated YouTube Channel โ–ถ๏ธwith Added Extra Features.

Fully Automated Youtube Channel โ–’โ–ˆโ–€โ–€โ–ˆ โ–ˆโ–€โ–€โ–ˆ โ–€โ–€โ–ˆโ–€โ–€ โ–€โ–€โ–ˆโ–€โ–€ โ–ˆโ–‘โ–‘โ–ˆ โ–ˆโ–€โ–€โ–„ โ–ˆโ–€โ–€ โ–ˆโ–€โ–€โ–ˆ โ–’โ–ˆโ–€โ–€โ–„ โ–ˆโ–‘โ–‘โ–ˆ โ–‘โ–‘โ–ˆโ–‘โ–‘ โ–‘โ–’โ–ˆโ–‘โ–‘ โ–ˆโ–‘โ–‘โ–ˆ โ–ˆโ–€โ–€โ–„ โ–ˆโ–€โ–€ โ–ˆโ–„โ–„โ–€ โ–’โ–ˆโ–„โ–„โ–ˆ โ–€โ–€โ–€โ–€ โ–‘โ–‘โ–€โ–‘โ–‘ โ–‘โ–’โ–ˆโ–‘โ–‘ โ–‘โ–€โ–€โ–€ โ–€โ–€โ–€โ–‘

sam-sepiol 249 Jan 02, 2023
Music and video downloader, Made with love by Bryan Herrera

Python-Mp3Mp4-Downloader Music and video downloader, Made with love by Bryan Herrera Requirements CHOCOLATELY windows command If your system does not

แˆญแšฑ1แŠ“แ‰ฐแ›ฐ แšปแˆ…แšฅแ‰ฐแƒžแšฑ 104 Dec 27, 2022
Mobile based API for Crunchyroll BETA (and Downloader).

Mobile based API for Crunchyroll BETA (and Downloader). Not restricted on servers and NO CLOUDFLARE

27 Dec 11, 2022
Python module to donwload all Pixiv artworks of a user using it's user ID.

Python module to donwload all Pixiv artworks of a user using it's user ID. You need a PHPSESSID token to export NSFW.

Quatrecentquatre 1 Jan 27, 2022