AI Dungeon Catalog Archive Toolkit

Related tags

DownloaderAIDCAT
Overview

AI Dungeon Content Archive Toolkit (AID CAT)

AID CAT is a command-line utility that will allow you to download JSON backups of:

  1. Your private and published AI Dungeon scenarios, adventures, posts, bookmarks, worlds, friends, followers, and following.
  2. Any user's published scenarios, adventures, and posts, as well as their friends, followers, and following.
  3. Obfuscate all adventures' titles, descriptions, tags, memory, actions, undone actions, and world info with junk data.
    WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!

Things AID CAT will NEVER do:

  1. Download another user's private content.
  2. Share your information with anyone else. All data transferred is strictly between your device and the aidungeon.io GraphQL backend.

Go check out KoboldAI!

Saved AI Dungeon games exported using AID CAT can now be imported and played with KoboldAI! KoboldAI is a browser front-end for playing with multiple local & remote AI models. KoboldAI supports interact with the AI models via Tensorflow and includes a easy-to-use automated install process. Currently supported models are GPT Neo 1.3B/2.7B, GPT-2 Med/Large/XL, Megatron (via InferKit API) as well as the ability to run your fine-tuned custom GPT-Neo (e.g., Neo-horni) and custom GPT-2 (e.g., CloverEdition) models.

Latest Release

2021-06-10: v0.7.0

Added the ability to disable Safe Mode
Hardened the obfuscation script against network instability
Optimized the obfuscation script to run faster
**WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!**

Instructions below copied with edits from original script author's site (referenced files uploaded here for archival purposes):

The wAIfupocalypse is upon us

Let's save your stories!

Recently, Latitude has announced changes that will end AI cooms as we know them today. Going beyond just the explore page, employees may now ban you for content in your unpublished stories. For this reason, I have programmed a script that can automatically download all of the stories and scenarios in your account. Here are the instructions:

  1. Before you can use the script, you must download and install the Python 3 runtime. Make sure to select "Add Python 3.X to PATH" on the first page of the installer.
  2. The script needs your access token so it can access your private stories. The token will only be used for that purpose, and it will not be stored. See how to access it in Firefox or Chrome.
  3. Download the aidcat.py script here. Move it to where you want your stories saved and run it. (usually by double-clicking, or typing python aidcat.py or python3 aidcat.py in the command prompt while in the directory containing the script file).
  4. Enter your login token and press enter. While you wait, think back to the good times you had with your fictional friends.
  5. When the download is complete, a file called stories.json will be created containing all your stories.

I have also created a script that will turn that JSON file into HTML files so it's easier to read. Download it here and run it in the same folder as the JSON file. You might also want to get this stylesheet (put in same folder as html files) to make things look a little nicer.

For mobile users

You can run these scripts using Termux (Android) or Pythonista (iOS). Since it's difficult to access your login token on a mobile browser, here is a script to help you with that: login.py (1.1 KiB)

Termux Instructions

  1. pkg update && pkg install python && curl -O https://raw.githubusercontent.com/CuriousNekomimi/AIDCAT/main/aidcat.py && curl -O https://raw.githubusercontent.com/CuriousNekomimi/AIDCAT/main/login.py
  2. python login.py (To get your x-access-token)
  3. python aidcat.py (To run AIDCAT)

Changelog

2021-05-30: v0.6.9
Added the ability to obfuscate all adventures' title, description, tags, memory, actions, undone actions, and world info with junk data.
**WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!**
Added an info page to sources documenting Latitude.
 
2021-05-11: v0.6.7
HUGE thanks to Eta for making these improvements and for refactoring the program to be object oriented:
- The phrase "any key" was changed to "Enter." (Actually waiting for any keypress is significantly more complicated).
- Option 1 of the auth menu previously said [1] Change your access access token, which was fixed.
- Added a missing error message on "Our Content" page.
- PEP 8 compliance:
  - The header docstring was changed to use """ instead of '''
  - Fixed whitespace around operators in code and method declarations
  - Fixed indentation in a couple places
  - Added whitespace between the # and the start of each comment
  - Added a newline at the end of the file
  - Added two lines of whitespace before and after function definitions
  - (Non-PEP 8) Changed menu choices to all be double quoted strings, for consistency
- Removed default values from actually-mandatory method parameters like save_json's content_type
- Separated the long as heck query strings from the rest of the code and moved them to the bottom of the file, just above if __name__ == '__main__', to improve readability
- Added a User class
  - This class encompasses the operations make_query (new), save_json, get_scenarios/subscenarios/adventures/posts/worlds/social, and get_saves (new, which calls all of the bookmarked operations together)
  - It holds the fields content_cache, and each of the query_s as instance variables (no longer global)
    - This means these can be maintained independently per user, which will help a lot with the batch download feature later on (it allows that to be multithreaded, for example)
- All of the menu code was changed to use User objects representing different users instead of passing around a username parameter
- The menu code was additionally refactored to cut down on duplicated code by a lot

2021-05-10: v0.5.7
Thanks to Eta for these suggestions:
UUID (access token) validation using python's built-in uuid library.
Cleaned up boolean checks.
Better file management. Saving files no longer risks a memory leak if the programmer forgets to call close().

2021-05-09: v0.5.6
Various improvements to the code in terms of efficiency and syntax, thanks Eta!

2021-05-09: v0.5.4
Changed aidcat.py to UTF-8 encoding. If you were seeing weird issues with screens, like corrupted text, this should fix that issue.
Thanks to Eta for the great suggestions for improving efficiency and helping with refactoring overall, including:
Menu headers are now all block quotes.
Reduced the number of escaped characters ( e.g. \' ).
More conventional print() statements.

Future Plans:
Proper GUID validation
HTML format export/converter. (Next priority)
Possibly add the ability to batch download all your friends' and followings' published content.
Ability to get the user's x-access-token from within the application. (Rather than digging around in the browser.)

2021-05-09: v0.5.2 (Completely new program written from scratch)
Command-line user interface with menus.
Download your scenarios, subscenarios, adventures, posts, and worlds (including official purchased worlds).
Download your saved scenarios, adventures, and posts (bookmarks).
Download a list of your friends, following, and followers.
Download other user's published scenarios, subscenarios, and posts.
Download a list of other user's friends, followers, and following.
Saved files tagged with ISO 8601 format times, so previous files aren't accidentally overwritten.
NOTE: Windows (Command prompt/PowerShell) doesn't display UTF-8 encoded characters. If someone has fancy UTF characters in their username, they will show up as empty boxes when pasted in, but the program will still work.

Future plans:
HTML format export/converter. (Next priority)
Possibly add the ability to batch download all your friends' and followings' published content.
Ability to get the user's x-access-token from within the application. (Rather than digging around in the browser.)

2021-05-02:
Added: Ability to save subscenarios (thanks to the original script author)!
Added: Ability to save bookmarks (thanks to the original script author)!
Added: Choice to save scenario bookmarks.
Added: Choice to save adventure bookmarks.

2021-05-02:
Added: Ability to choose whether or not to download scenarios, adventures, or worlds. Useful for users who have thousands of adventures and don't want to download them every time.
Added: More error catching and a reminder to report errors to https://github.com/CuriousNekomimi/AIDungeonRescue/issues.
Note: The original script author has released an update to download scenario options. I'll work on incorporating that code.

2021-05-02:
Added: Ability to save worlds!
Added: Ability to save x-access token for later use (token.txt).
Added: Scenarios, adventures, and worlds, now save to separate json files.
Added: Improved readability of saved scenario, adventure, and world json files with newlines and indents.
Added: Pause for user interaction after running so the user knows the program executed successfully.
Added: Notification that files saved successfully.
Added: Try/Except to main() for several methods.
Refactored: 'stories' to 'adventures'.
Comments
  • Downloading more than 50 actions

    Downloading more than 50 actions

    I ran this almost two weeks ago to export my stories, including one with over 15,000 actions (yes, I know). Everything was exported including that full story, resulting in a 3.88 MB file.

    Since you now have support for importing into KoboldAI, I figured I'd get the latest version of this tool and re-run it to download my AI Dungeon stories. This results in a much smaller file, only 293 KB. I checked and found that it now seems to cap out at the 50 most recent actions of the story.

    I still have the older version of this tool, so I ran it again to see if the change was on your end or AI Dungeon, and it downloaded the full 3.88 MB file again. So that tells me it's a change made to this tool.

    I'm guessing it's to keep things from getting to bloated before importing into KoboldAI or something, only keeping a certain number of relevant actions which would be potentially referenced by it? Is there an option somewhere now to download the full adventures again, or at least can you provide clarification on the reason for this?

    bug 
    opened by DagothRa 3
  • Tweaked and cleaned

    Tweaked and cleaned

    -Used requests sessions instead of urllib. -Put images in their own file. -Separated the UI from the program logic. -Created a module to transform .json to html (I used my tweaked version from Scriptanon's)

    I'm going to start making the automated tests.

    opened by Moist-Cat 2
  • Infinite Loop on Latitude corrupted backwards compatibility

    Infinite Loop on Latitude corrupted backwards compatibility

    Search the term "Latitude " (with the space), which will lead to an error message. It is inside a true while loop with a continue that triggers no advancement nor does it lead to a break, thus causing infinite looping.

    opened by YuliaS11 1
  • Tools to help migrate from AID to KAI

    Tools to help migrate from AID to KAI

    I created some simple tools to help integrate AIDCAT and KAI so people could migrate from AID faster. The scripts I made interpret AIDCAT's output in a "scenario explorer" that lets the person select their scenario, type the placeholders just like in AID and then save the generated story into a file ready to be used in KoboldAI. Same for my own scenarios (it has its own script) and another tool that converts AIDCAT's extracted stories from AID into something readable by KoboldAI.

    I plan to join these three tools into one, but for now they're separated. This might be useful for people using AIDCAT. Here's the link: https://github.com/thaalesalves/aid-scripts/tree/main/other

    opened by thaalesalves 1
  • Saving bookmarked scenarios fails if any sub-scenario fetch returns None.

    Saving bookmarked scenarios fails if any sub-scenario fetch returns None.

    For reasons unknown to me, certain sub-scenarios in my bookmarks return {Scenario: None}. This satisfies the if 'scenario' in result['data'] check but causes an exception when the script tries to progress and halts.

    I was able to do a quick fix locally by changing if 'data' in result and 'scenario' in result['data']: to if 'data' in result and 'scenario' in result['data'] and result['data']['scenario'] != None:

    opened by throwaway47323674 1
  • genhtml_edit.py throws an error when trying to open a file

    genhtml_edit.py throws an error when trying to open a file

    Welcome to the AIDCAT HTML Generator! First we'll ask for your Adventure file, press Enter to continue... Traceback (most recent call last): File "C:\Users\Me\Documents\AIDungeonRescue-main\aidcat\genhtml_edit.py", line 51, in infile = json.load(file) File "C:\Users\Me\AppData\Local\Programs\Python\Python37\lib\json_init_.py", line 293, in load return loads(fp.read(), UnicodeDecodeError: 'cp932' codec can't decode byte 0x9d in position 119195: illegal multibyte sequence

    opened by cjk137 1
  • Improved Token and Error Handling

    Improved Token and Error Handling

    This pull request improves token handling, adding autocorrection for missing hyphens and server validation to check if the token is associated with an account. It will also now display the account name associated with an entered token. Error handling is also improved, and a keyboard interrupt can now properly exit out of the program (without any error text, either).

    opened by Eta0 0
  • Broken with 'Walls Approach' update

    Broken with 'Walls Approach' update

    https://latitude.io/blog/the-walls-approach

    It seems they're now encrypting user stories. When I try to download them with AIDCAT, I get 400 Bad Request. I might have missed my chance, unless this script can be updated to decrypt the stories.

    opened by ImportTaste 16
Releases(v0.7.0)
  • v.0.6.9(May 30, 2021)

    Added the ability to obfuscate all adventures' title, description, tags, memory, actions, undone actions, and world info with junk data. WARNING: Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE! Added an info page to sources documenting Latitude.

    Source code(tar.gz)
    Source code(zip)
  • v0.6.7(May 17, 2021)

  • v0.6.6(May 17, 2021)

    Descriptive exceptions and traceback for HTTP connections errors and file saving. Possible fix for Mac devices not saving files to the current directory.

    Source code(tar.gz)
    Source code(zip)
  • v0.6.5(May 16, 2021)

    Re-refactored actionWindow back to actions . It turns out that actionWindow only returns the last 50 actions. I sincerely apologize for any inconvenience this may have caused!

    Source code(tar.gz)
    Source code(zip)
  • v0.6.2_html_utf8(May 15, 2021)

  • v0.6.2_html(May 14, 2021)

  • v0.6.2(May 13, 2021)

    Eta0: This pull request improves token handling, adding autocorrection for missing hyphens and server validation to check if the token is associated with an account. It will also now display the account name associated with an entered token. Error handling is also improved, and a keyboard interrupt can now properly exit out of the program (without any error text, either).

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(May 11, 2021)

    HUGE thanks to Eta0 for making these improvements and for refactoring the program to be object oriented:

    • The phrase "any key" was changed to "Enter." (Actually waiting for any keypress is significantly more complicated).
    • Option 1 of the auth menu previously said [1] Change your access access token, which was fixed.
    • Added a missing error message on "Our Content" page.
    • PEP 8 compliance:
      • The header docstring was changed to use """ instead of '''
      • Fixed whitespace around operators in code and method declarations
      • Fixed indentation in a couple places
      • Added whitespace between the # and the start of each comment
      • Added a newline at the end of the file
      • Added two lines of whitespace before and after function definitions
      • (Non-PEP 8) Changed menu choices to all be double quoted strings, for consistency
    • Removed default values from actually-mandatory method parameters like save_json's content_type
    • Separated the long as heck query strings from the rest of the code and moved them to the bottom of the file, just above if name == 'main', to improve readability
    • Added a User class
      • This class encompasses the operations make_query (new), save_json, get_scenarios/subscenarios/adventures/posts/worlds/social, and get_saves (new, which calls all of the bookmarked operations together)
      • It holds the fields content_cache, and each of the query_s as instance variables (no longer global)
        • This means these can be maintained independently per user, which will help a lot with the batch download feature later on (it allows that to be multithreaded, for example)
    • All of the menu code was changed to use User objects representing different users instead of passing around a username parameter
    • The menu code was additionally refactored to cut down on duplicated code by a lot
    Source code(tar.gz)
    Source code(zip)
  • v0.5.7(May 10, 2021)

Owner
Mimi
Mimi
TikTok - TikTok Bot to download video or audio from TikTok

TikTok - TikTok Bot to download video or audio from TikTok

JMTHON 51 Mar 04, 2022
📺 YouTube Song Downloader Bot For Telegram 🔮

📺 YouTube Song Downloader Bot For Telegram 🔮 Powerd By TamilBots.

Tamil Bots 146 Dec 31, 2022
A python scripts that downloads doujin from nhentai without having an account

nhentai-downloader a python scripts that downloads doujin from https://nhentai.net without having an account. Usage Needs Python 3^ Linux pip3 install

Earl Sabalo 4 Jun 13, 2022
code for paper"3D reconstruction method based on a generative model in continuous latent space"

PyTorch implementation of 3D-VGT(3D-VAE-GAN-Transformer) This repository contains the source code for the paper "3D reconstruction method based on a g

Tong 5 Apr 25, 2022
squid-dl is a massively parallel yt-dlp-based YouTube downloader.

squid-dl squid-dl is a massively parallel yt-dlp-based YouTube downloader. Installation Run the setup.py, which will install squid-dl and its two depe

tuxlovesyou 51 Jan 05, 2023
Write reproducible code for getting and processing ChEMBL

chembl_downloader Don't worry about downloading/extracting ChEMBL or versioning - just use chembl_downloader to write code that knows how to download

Charles Tapley Hoyt 34 Dec 25, 2022
Persepolis Download Manager is a GUI for aria2.

Persepolis Download Manager Content About FAQ Screenshots Credits About Persepolis is a download manager & a GUI for Aria2. It's written in Python. Pe

Persepolis 5.6k Dec 31, 2022
Python Youtube Video-Playlist Downloader

Youtube-Video-Playlist-Downloader-PyQt5 You can download videos and playlists on YouTube with this script. Script has GUI. Enjoy. Setup git clone http

Yunus Emre Öztürk 2 Jun 06, 2022
Fast TikTok NO Watermark Video Downloader (username or url)

💎 TD [ TikDown v4 ] Star ⭐ if you want more Discord Server * discord.gg/onlp | Waxor#9999 Why not open source anymore ? * BECAUSE PEOPLE SKID, STEA

Tekky 26 Dec 01, 2022
Simple package for Sublime Text 4; download URL's for local viewing and editing

URLDownloader This is a simple example package that allows you to easily download the contents of any web URL to edit locally. Given a URL, the packag

Terence Martin 3 Mar 05, 2022
AkShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

Overview AkShare requires Python(64 bit) 3.7 or greater, aims to make fetch financial data as convenient as possible. Write less, get more! Documentat

Albert King 5.8k Jan 03, 2023
Terminal based YouTube player and downloader

termitube NOTE: THIS REPOSITORY IS A FORK OF mps-youtube as mps-youtube has been unmaintained for almost a year now. Features Search and play audio/vi

Otis/Jacob Root 27 Dec 23, 2022
Python script for downloading audio from YouTube songs/videos.

Python script for downloading audio from YouTube songs/videos. All you have to do is specify the path to your folder and then type song's/video's name and the sound will be downloaded into your folde

Mateusz Polis 0 Oct 05, 2022
Neon: an add-on for making it easier to handle component interactions

Neon Neon is an add-on for Lightbulb making it easier to handle component interactions. Installation pip install git+https://github.com/neonjonn/light

Neon Jonn 9 Apr 29, 2022
Python library to download bulk of images from Bing.com

Python library to download bulk of images form Bing.com. This package uses async url, which makes it very fast while downloading.

Guru Prasad Singh 105 Dec 14, 2022
This package helps you to directly download an APK from Google Play by providing the package id of the app

Apk Downloader About | Features | Technologies | Requirements | Starting | License | Author 🎯 About This package helps you to directly download an AP

Daniel Agyapong 9 Dec 11, 2022
Download h3t4y for later read

h3nt4y_dl Download h3nt4y for later read Tải h3nt4y về đọc thôi nào các bạn ơiiiiiiii! (Tải từ h**taivn nhé) Usage: python get_that_ht4i.py New versio

1 Dec 03, 2021
Script for YouTube creators to share dislike count with their viewers.

Stahování disliků z YouTube - milafon Tento skript slouží jako možnost zobrazit divákům počet disliků u YouTube videí. Vyžaduje implementaci ze strany

4 Sep 28, 2022
A collection of modules I have created to programmatically search for/download imagery from live cam feeds across the state of California.

A collection of modules that I have created to programmatically search for/download imagery from all publicly available live cam feeds across the state of California. In no way am I affiliated with a

Chad Groom 5 Nov 21, 2022
a simple ehentai downloader with jpg 2 pdf

Simple_Ehentai_DownLoader a simple ehentai downloader with jpg 2 pdf 中文介绍 Environment python3.8 How to use before you start,there are some tips. the q

Hibian 6 Dec 11, 2022