A manga download script written in python.

Last update: Nov 28, 2022

Related tags

Overview

manga-dlp

python script to download mangas

Description

A manga download script written in python. It only supports mangadex.org for now. But support for other sites is planned.

Before downloading a new chapter, the script always checks if there is already a chapter with the same name in the download directory. If found the chapter is skipped. So you can run the script on a schedule to only download new chapters without any additional setup.

The default behaiviour is to pack the images to a cbz archvie. If you just want the folder with all the pictures use the flag --nocbz.

Currently Supported sites

Mangadex.org

Usage

With GitHub

git clone https://github.com/olofvndrhr/manga-dlp.git # clone the repository

cd manga-dlp # go in the directory

pip install -r requirements.txt # install required packages

# on windows
python manga-dlp.py <options>
# on unix
python3 manga-dlp.py <options>

With pip (pypi)

(not yet done)

With docker

See the docker README

Options

/downloads" -l LANG, --language LANG Manga language. Defaults to "en" --> english --read READ Path of file with manga links to download. One per line --list List all available chapters. Defaults to false --nocbz Dont pack it to a cbz archive. Defaults to false --forcevol Force naming of volumes. For mangas where chapters reset each volume --wait WAIT Time to wait for each picture to download in seconds(float). Defaults 0.5 --verbose Verbose logging. Defaults to false">

usage: manga-dlp.py [-h] [-u URL] [-c CHAPTERS] [-p PATH] [-l LANG] [--read READ] [--list] [--nocbz] [--forcevol] [--wait WAIT]
[--verbose]

optional arguments:
-h, --help                          Show this help message and exit
-u URL, --url URL                   URL of the manga
-c CHAPTERS, --chapters CHAPTERS    Chapters to download
-p PATH, --path PATH                Download path. Defaults to "
    
     /downloads"
-l LANG, --language LANG            Manga language. Defaults to "en" --> english
--read READ                         Path of file with manga links to download. One per line
--list                              List all available chapters. Defaults to false
--nocbz                             Dont pack it to a cbz archive. Defaults to false
--forcevol                          Force naming of volumes. For mangas where chapters reset each volume
--wait WAIT                         Time to wait for each picture to download in seconds(float). Defaults 0.5
--verbose                           Verbose logging. Defaults to false

Downloads file-structure

.
└── 
   
    /
    └── 
    
     /
        └── 
     
      /

Example:

./downloads/mangatitle/chaptertitle(.cbz)

Read list of links from file

With the option --read you can specify a file with links to multiple mangas. They will be parsed from top to bottom one at a time. Every link will be matched for the right api to use. It is important that you only have one link per line, otherwise they can't be parsed.

Example:

# mangas.txt
link1
link2
link3

python3 manga-dlp.py --read mangas.txt --list

This will list all available chapters for link1, link2 and link3.

Set download path

With the option -p/--path you can specify a path to download the chapters to. The default path is /downloads. Absolute and relative paths are supported.

Example:

python3 manga-dlp.py --path /media/mangas

This will save all mangas/chapters in the path /media/mangas/ /

Contribution / Bugs

For suggestions for improvement, just open a pull request.

If you want to add support for a new site, there is an api template file which you can use.

Otherwise you can open a issue with the name of the site which you want support for. (not guaranteed to be implemented)

If you encounter any bugs, also just open a issue with a description of the problem.

TODO's

~~Make docker container for easy distribution~~ --> Dockerhub
Make pypi package
Add more supported sites

You might also like...

A Python script to download PDB files associated with a Portable Executable (PE)

33 Jan 3, 2023

Simple Python script to download images and videos from public subreddits without using Reddit's API 😎

Subreddit Media Downloader Download images and videos from any public subreddit without using Reddit's API Made with ❤ by Nico 💬 About: This script a

106 Jan 7, 2023

Download your bandcamp collection using this python script.

bandcamp-downloader Download your Bandcamp collection using this python script. It requires you to have a browser with a logged in session of bandcamp

72 Dec 20, 2022

I sure love the mix of newsboat+mpv+youtube-dl to watch videos from my favourite creators directly from my command line. But sometimes I want to download them beforehand and have them sorted into different folders. Here is the script to do exactly that.

newsboat_video_downloader I sure love the mix of newsboat+mpv+youtube-dl to watch videos from my favourite creators directly from my command line. But

16 Dec 12, 2022

Script that allows to download portable installers of different versions Adobe software for macOS

What is this and for what This is a script that allows you to download portable installers of programs from Adobe for macOS with different versions. T

715 Jan 6, 2023

The sole purpose of this script is to download any NFT collection from OpenSea

OpenSea NFT Stealer The sole purpose of this script is to download any NFT collection from OpenSea. Setup Prerequisites: Python 3 Python requests libr

9 Sep 4, 2022

Arxiv2Kindle is a simple script written in python that converts LaTeX source downloaded from Arxiv and recompiles it to better fit a Kindle or other similar reading devices.

Arxiv2Kindle is a simple script written in python that converts LaTeX source downloaded from Arxiv and recompiles it to better fit a read

8 Jul 9, 2022

FireDM is a python open source (Internet Download Manager) with multi-connections, high speed engine, it downloads general files and videos from youtube and tons of other streaming websites .

python open source (Internet Download Manager) with multi-connections, high speed engine, based on python, LibCurl, and youtube_dl https://github.com/firedm/FireDM

1.6k Apr 12, 2022

This is a python based web scraping bot for windows to download all ACCEPTED submissions of any user on Codeforces

CODEFORCES DOWNLOADER This is a python based web scraping bot for windows to download all ACCEPTED submissions of any user on Codeforces Requirements

6 Dec 29, 2022

Comments

Files downloaded have no extension

Hello,

I've just installed manga-dlp in a docker instance and started downloading some manga. The download is working but files are saved without extension. When the program creates the CBZ file the readers I've tried cannot open it, probably because of this reason. (the files are generated correctly as I can unzip them without any issue).

I've noticed that the files in the mangadex server do not have any extension, not sure if this is something recent or not.

I've also tried to generate PDF but the extension is not included in the image so I cannot see if PDF would work anyway.

Docker file

version: "3"
services:
  manga-dlp:
    image: olofvndrhr/manga-dlp:latest
    container_name: manga-dlp
    security_opt:
      - no-new-privileges:true
    networks:
      - proxy
    volumes:
      - ************/Comics:/app/downloads/ # default manga download directory
      - ************/manga-dlp/config/mangas.txt:/app/mangas.txt # default file for manga links to download
      #- ./crontab:/etc/cron.d/mangadlp # path to default crontab
      #- ./schedule:/app/schedules/daily # path to the default schedule which is run daily
    environment:
      - TZ=Europe/************
      - PUID=1000 # custom userid - defaults to 4444
      - PGID=1000 # custom groupid - defaults to 4444
    restart: unless-stopped

networks:
  proxy:
    external: true

Mangas.txt

https://mangadex.org/title/f7888782-0727-49b0-95ec-a3530c70f83b/hajime-no-ippo
[...]

bug enhancement

opened by kn-f 3

Not working with Mangadex API >= 5.4.1

From Mangadex:

⚠️  Deprecated
Deprecate Chapter.data.attributes.{hash, data, dataSaver} fields
They will be accessible only through /at-home/server endpoint after the transition period.
We will keep it present on /chapter based endpoints for 1 week from today after which that information will be entirely gone from any other endpoint.

Since this changes breaks the current script, i need to rewrite it to use the new "at-home" endpoint. I will also overhaul the whole script to remove unnecessary pieces.

Expect the script not to work for a few days until i fix everything.

bug enhancement

opened by olofvndrhr 1

[FEATURE] Volume support for all apis (future proofing)

Is your feature request related to a problem? Please describe. Most manga-dl things do not support making volume folders, due to the fact a lot of sites don't support showing information of what chapters are in what volume.

Describe the solution you'd like I'm going to PR so more sites, your implementation and organization of supporting multiple sites is amazing, to just have a callable class is a great way to do this.

In order to support multiple sites with full meta and sorting, it should have a fallback to the mangadex api to get the missing information.

Describe alternatives you've considered I made a kind of wrapper for mangodl which barely worked and was not automated.

Additional context The api fallback to mangadex may need a login, I haven't looked through everything you've done but I'm glad to find an active, usable project.
feature-request

opened by BitterSweetcandyshop 0

Releases(2.2.16)

2.2.16(Dec 30, 2022)
[2.2.16] - 2022-12-30

Fixed

Log level is now fixed and should not default to 0

Docker schedule should now work again

Changed

Integrate logging logs to loguru via custom sink

Simplify docker shell scripts

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.16-py3-none-any.whl(17.00 KB)
manga_dlp-2.1.16.tar.gz(15.12 KB)
2.2.15(Dec 29, 2022)
[2.2.15] - 2022-12-29

Added

--warn and --loglevel flags

Removed

Remove --lean and --verbose flags and remove custom log levels

Changed

Move from standard library logging to loguru

Move from standard library argparse to click

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.15-py3-none-any.whl(16.80 KB)
manga_dlp-2.1.15.tar.gz(14.92 KB)
2.2.14(Oct 6, 2022)
[2.2.14] - 2022-10-06

Changed

Changed logging format to ISO 8601

Small logging corrections

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.14-py3-none-any.whl(17.48 KB)
manga_dlp-2.1.14.tar.gz(15.18 KB)
2.1.13(Aug 15, 2022)
[2.2.13] - 2022-08-15

Added

Option to run custom hooks before and after each chapter/manga download

Tests for the new hooks

Docs for the new hooks

Tests for mkdocs generation

Changed

Verbose and Debug logging now have a space as a seperator between log level-name and log-level

APIs now have an attribute with their name (for the hooks) - api.api_name

Docs moved to Cloudflare pages (generated with mkdocs)

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.13-py3-none-any.whl(17.45 KB)
manga_dlp-2.1.13.tar.gz(15.15 KB)
2.1.12(Jul 25, 2022)
[2.1.12] - 2022-07-25

Fixed

Image publishing with hatch on pypi should now work again

The schedule fixer for the new .sh schedule should now work correctly

Added

More CI tests: pylint, pylama and autoflake

New function in get_release_notes.sh to get the latest version

Docstrings for MangaDLP and the api module Mangadex

Changed

CI workflow is now faster and runs natively on arm64 (before it was buildx/emulation)

Pylint/pylama code improvements

Version management is now done with hatch (in __about__.py)

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.12-py3-none-any.whl(16.77 KB)
manga_dlp-2.1.12.tar.gz(14.55 KB)
2.1.11(Jul 18, 2022)
[2.1.11] - 2022-07-18

Fixed

The --read option now filters empty lines, so it will not generate an error anymore

An error which was caused by the interactive input method when you did not specify a chapter or to list them

Some typos

Added

Options to configure the default schedule in the docker container via environment variables

Section the the docker README.md for the new environment variables

autoflake test in justfile

Some more things which get logged

Changed

BREAKING: renamed the default schedule from daily to daily.sh. Don't forget to fix your bind-mounts to overwrite the default schedule

Added the .sh suffix to the s6 init scripts for better compatibility

Adjusted the new logging implementation. It shows now more info about the module the log is from, and some other improvements

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.11-py3-none-any.whl(16.25 KB)
manga_dlp-2.1.11.tar.gz(14.25 KB)
2.1.10(Jul 14, 2022)
[2.1.10] - 2022-07-14

Fixed

Removed some unused files

Added

logger.py for all log related settings and functions

Changed

Logging of output. The script now uses the logging library

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.10-py3-none-any.whl(16.20 KB)
manga_dlp-2.1.10.tar.gz(14.18 KB)
2.1.9(Jun 26, 2022)
[2.1.9] - 2022-06-26

Fixed

Timeouts in tests, due to api limitations. Now added a wait time between tests

Pytest path

Added

--lean flag for less output

justfile for setting up a dev environment and testing the code

asdf for version management

Dev requirements in contrib/requirements_dev.txt

README in contrib

Changed

Handling of verbosity and logging. Now there are 4 types of verbosity: normal, lean, verbose and debug

CI/CD pipeline for testing and releases

Coverage testing now also done with tox

Default verbosity of docker container is now --lean

Reorganised pyproject.toml

Source code(tar.gz)
Source code(zip)
manga_dlp-2.1.9-py3-none-any.whl(15.29 KB)
manga_dlp-2.1.9.tar.gz(16.40 KB)
2.1.8(Jun 22, 2022)
[2.1.8] - 2022-06-22

Fixed

Interactive input

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.8.tar.gz(14.05 KB)
2.1.7(Jun 22, 2022)
[2.1.7] - 2022-06-22

Added

tox version testing

New pre-release tests

Build info's with hatch

Pypi build with hatch

Pypi section in README.md

Snyk test results in README.md

Changed

Moved code from manga-dlp.py to input.py for uniformity

The default entrypoint is now mangadlp.input:main

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.7.tar.gz(14.05 KB)
2.1.6(Jun 21, 2022)
[2.1.6] - 2022-06-21

Fixed

Docker labels are now working

Global variables are now fully uppercase

Some errors with static types

Added

bump2version config for releases

More tests with: mypy and isort

New issue templates

Changed

Release workflow now is based on configuration files

Switched from setup.py to pyproject.toml

README.md now has sorted badges

Imports are now sorted with isort

Static types are now checked with mypy

Release note generation is now simplified

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.6.tar.gz(13.61 KB)
2.1.5(Jun 17, 2022)
[2.1.5] - 2022-06-18

Fixed

Image names now have a suffix, as some comic readers have problems with no suffix [fixes issue #2]

Added

--format section in the README

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.5.tar.gz(12.53 KB)
2.1.4(May 29, 2022)
[2.1.4] - 2022-05-29

Fixed

Docker container now works again

Fixed cron in docker container

Changed

Docker container scheduling is now more practical

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.4.tar.gz(12.22 KB)
2.1.3(May 29, 2022)
[2.1.3] - 2022-05-29

Fixed

Error-chapters and skipped-chapters list are now shown again

The Interactive input version now matches --version

Added

Ability to list chapters with interactive input

Changed

Replace exit() with sys.exit()

Renamed class methods to not look like dunder methods

Script execution moved from os.system() to subprocess.call()

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.3.tar.gz(12.18 KB)
2.1.2(May 20, 2022)
[2.1.2] - 2022-05-20

Fixed

List chapters when none were specified

Typos

Added

Ability to download whole volumes

Changed

Moved processing of list with links to input.py

Updated README for volume and chapter selection

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.2.tar.gz(11.96 KB)
2.1.1(May 18, 2022)
[2.1.1] - 2022-05-18

Fixed

Progress bar on verbose output

Sonarqube link for CI

A few typos

Removed unnecessary escapes from file rename regex

Added

API template

Changed

Updated docker baseimage

Rewrote app.py to a class

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.1.tar.gz(11.43 KB)
2.1.0(May 16, 2022)
[2.1.0] - 2022-05-16

Fixed

Detection of files. Now it will skip them again

Added

Ability to save the chapters as pdf (only on amd64/x86)

New output formats: rar, zip

Progress bar to show image download

Interactive input if no command line flags are given

Better KeyboardInterrupt handling

Better error handling

Removed duplicate code

Changed

How the variables are used inside the script

Variables have now the same name as in other scripts (mostly)

Better retrying when a task fails

Source code(tar.gz)
Source code(zip)
manga-dlp-2.1.0.tar.gz(11.37 KB)
2.0.8(May 13, 2022)
[2.0.8] - 2022-05-13

Changed

Rewrote parts of script to be easier to maintain

Moved the input script to the base folder

Moved all arguments to a class

Docker container creation

Source code(tar.gz)
Source code(zip)
manga-dlp-2.0.8.tar.gz(10.51 KB)
2.0.7(May 13, 2022)
[2.0.7] - 2022-05-13

Changed

Changed CI/CD Platform from Drone-CI to Woodpecker-CI

Release title is now only the version

Source code(tar.gz)
Source code(zip)
manga-dlp-2.0.7.tar.gz(9.78 KB)
2.0.6(May 11, 2022)
Fixed

Filenames on windows (ntfs). Removed double quote from file and folder names

Source code(tar.gz)
Source code(zip)
manga-dlp-2.0.6.tar.gz(9.71 KB)
2.0.5(May 11, 2022)
Fixed

Better error handling on "KeyboardInterrupt"

Release notes now fixed

Added

New test cases

Source code(tar.gz)
Source code(zip)
manga-dlp-2.0.5.tar.gz(9.67 KB)
2.0.4(May 10, 2022)
Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Add support for more sites

[2.0.4] - 2022-05-10

Added

New test cases for more coverage

Github release

Updated docker baseimage

[2.0.3] - 2022-05-10

Fixed

Test cases now work again

Sonarqube settings

Added

Coverage report in sonarqube

Gitea release

[2.0.2] - 2022-05-09

Fixed

Restart failed api requests

Added wait time for image gathering, as to stop api rate limiting from mangadex

"--wait" options now works properly again

[2.0.1] - 2022-05-09

Fixed

Regex for removing illegal characters in the filenames now doesn't remove quotes

Updated docker baseimage and fixed the mangadlp tag in it

Update license for 2022

Added

Quick start section in README

Preperation for pypi

[2.0.0] - 2022-05-09

Fixed

Support for new mangadex api

Changed

Code is now formatted with black

Now also supports just the uuid for managex (not a full link)

Source code(tar.gz)
Source code(zip)
manga-dlp-2.0.4.tar.gz(9.57 KB)