This program is to make a video based on Deep Dream

Related tags

VideoDeepDreamAnimV2
Overview

DeepDreamAnimV2

This program is to make a video based on Deep Dream. The program is modified from DeepDreamAnim and DeepDreamVideo with additional functions for bleding two frames based on the optical flows. It also supports the image division to apply the Deep Dream algorithm to a large image.

A sample video created by this script from a 4K-resolution, panoramic video. This sample video is cropped by a view from a head-mounted display.

Use Optical Flow to Adjust Deep Dream Video

The flow option enables the optical flow mode. This allows the optical flow of each frame to be calculated by comparing the difference in the movement of all pixels between the current and previous frame. The hallucinatory patterns on the area where the optical flow was detected is merged with the current (not-yet-hallucinatory) frame based on the weighting provided by the user defined blending ratio (0 = no information, 1 = all information). The blending ratio allows some of the hallucinatory content of the previous frame to be inherited from the previous frame, The Deep Dream algorithm is then applied to this merged frame, instead of Deep Dream starting from scratch for each frame.

  1. The difference in optical flow is calculated between the previous and current frame (before the Deep Dream algorithm is applied).
  2. The hallucinatory patterns within the area of high optical flow in the previous frame are shifted in the direction of the optical flow.
  3. This hallucinatory pattern from the previous frame is merged into the current (not-yet-hallucinatory) frame with a specfied blending ratio.

The blending ratio on the optical flow area and the other areas (background) can be specified separately by using -bm and -bs options. The range of the blending ratio is between 0 and 1. A blending ratio of 1 means that the current frame inherits 100% of the hallucinatory content from the previous frame, and then the deep dream algorithm is applied. A blending ratio of 0 means the previous frame is dicarded, therefore the deep dream algorithm is applied from the scrach.

Dividing the Image

When using a GPU to process the deep dream algorithm, the maximum image size is capped by the video memory of your GPU. In order to process large images such as 4K resolution, it is neccersary to divide the input image into smaller sub-images inorder to apply Deep Dream. -d or --divide option can be used to enable this function.

-d 0 : disable image division

-d 1 : divide the image by MAX_WIDTH and MAX_HEIGHT of image specified by -mw and -mh options

-d 2 : divide the image half when the width of the image is larger than MAX_WIDTH specified by -mw option

The divided image is processed by Deep Dream individually and then merged into one. If using octavescale in the original deep dream algorithm, the image is processed in a smaller size in the begining. In that case, the image division will not be applied to the earlier stage of the itterations (before enlarging the original size).

Usage

usage: dreamer.py [-h] -i INPUT -o OUTPUT -it IMAGE_TYPE [--gpu GPU]
                       [-t MODEL_PATH] [-m MODEL_NAME] [-p PREVIEW]
                       [-oct OCTAVES] [-octs OCTAVESCALE] [-itr ITERATIONS]
                       [-j JITTER] [-z ZOOM] [-s STEPSIZE]
                       [-l LAYERS [LAYERS ...]] [-v VERBOSE]
                       [-g GUIDE_IMAGE] [-flow FLOW] [-flowthresh FLOWTHREASH]
                       [-bm BLEND_FLOW] [-bs BLEND_STATIC] [-d DEVIDE_MODE]
                       [-mw MAX_WIDTH] [-mh MAX_HEIGHT]

Original Arguments:

  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        Input directory where extracted frames are stored
  -o OUTPUT, --output OUTPUT
                        Output directory where processed frames are to be
                        stored
  -it IMAGE_TYPE, --image_type IMAGE_TYPE
                        Specify whether jpg or png
  --gpu GPU             Switch for gpu computation.
  -t MODEL_PATH, --model_path MODEL_PATH
                        Model directory to use
  -m MODEL_NAME, --model_name MODEL_NAME
                        Caffe Model name to use
  -p PREVIEW, --preview PREVIEW
                        Preview image width. Default: 0
  -oct OCTAVES, --octaves OCTAVES
                        Octaves. Default: 4
  -octs OCTAVESCALE, --octavescale OCTAVESCALE
                        Octave Scale. Default: 1.4
  -itr ITERATIONS, --iterations ITERATIONS
                        Iterations. Default: 10
  -j JITTER, --jitter JITTER
                        Jitter. Default: 32
  -z ZOOM, --zoom ZOOM  Zoom in Amount. Default: 1
  -s STEPSIZE, --stepsize STEPSIZE
                        Step Size. Default: 1.5
  -l LAYERS [LAYERS ...], --layers LAYERS [LAYERS ...]
                        Array of Layers to loop through. Default: [customloop]
                        - or choose ie [inception_4c/output] for that single
                        layer
  -v VERBOSE, --verbose VERBOSE
                        verbosity [0-3]
  -g, --guide GUIDE_IMAGE
			A guide image

Additional Arguments:

   -flow, --flow	Optical Flow is taken into accout
   -flowthresh FLOWTHREASH, --flowthresh FLOWTHREASH
			Threshold for detecting a flow
   -bm BLEND_FLOW, --blendflow BLEND_FLOW
			blend ratio for flowing part
   -bs BLEND_STATIC, --blendstatic BLEND_STATIC
			blend ratio for static part

   -d [0-2] --divide [0-2]
			dividing image into sub images [0:disable 1:dividing to maxWidth, maxHeight 2:dividing half if width exceeds maxWidth]
   -mw MAX_WIDTH, --maxWidth MAX_WIDTH
			Maximum width to devide image
   -mh MAX_HEIGHT, --maxHeight MAX_HEIGHT
			Maximum height to devide image (only used for the divide mode 1)

Requirements

  • Python
  • Caffe (and other deepdream dependencies)
  • FFMPEG
  • CV2 (if you use optical flow)

Examples

1-extract.bat:

python dreamer.py -e 1 --input RHI-20Sec.mov --output Input

2-dream.bat:

python dreamer.py --input Input --output Output --octaves 3 --octavescale 1.8 --iterations 16 --jitter 32 --zoom 1 --stepsize 1.5 --flow 1 --flowthresh 6 --blendflow 0.9 --blendstatic 0.1 --layers inception_4d/pool --gpu 1 -d 2 -mw 1500

3-create.bat:

python dreamer.py -c 1 --input Output --output Video.mp4

Owner
Aertist
Aertist
A script to disable steam servers regionwise. [Works on Windows only]

Csgo-server-blocker A script to disable steam servers regionwise. [Works on Windows only] Dependencies python3.x Usage: pip install requirements.txt I

Aditya Bennur 2 Jun 10, 2022
Video processing routines for SciPy

scikit-video Video Processing SciKit BETA Video processing algorithms, including I/O, quality metrics, temporal filtering, motion/object detection, mo

Alex Izvorski 119 Dec 27, 2022
Adblocker for movie subtitles.

SubAdBlock Adblocker for movie subtitles. Usage Place "main.py" and "config.conf" in directory with subtitles and run "main.py". It will automatically

Marko Živić 1 Jan 09, 2022
PyAV is a Pythonic binding for the FFmpeg libraries.

PyAV is a Pythonic binding for the FFmpeg libraries. We aim to provide all of the power and control of the underlying library, but manage the gritty details as much as possible.

PyAV 1.8k Jan 01, 2023
Rune - a video miniplayer made with Python.

Rune - a video miniplayer made with Python.

1 Dec 13, 2021
Become a virtual character with just your webcam!

Become a virtual character with just your webcam!

Rich 300 Jan 03, 2023
A Python library that simplifies working with video from soccer matches.

Match Video This is a Python library that simplifies working with video from soccer matches. It allows match video to be selected intuitively by perio

Grant Wenzinger 2 Jul 21, 2022
Splat a video into a mosaic by sampling a frame at regular intervals

Splat a video into a mosaic by sampling a frame at regular intervals. Useful for seeing the changes over time of an entire video or movie.

Ryan Fox 4 Oct 16, 2022
Stream deck using Arduino and Python

Stream deck using Arduino and Python This is a little project I started due to the fact that I wanted to stream and didn't want to spend lots on a sim

Tal Cherniavsky 2 Feb 11, 2022
A telegram bot for compressing/encoding videos in h264 format.

Video-Encoder-Bot a telegram bot for compressing/encoding videos in h264 format. Configuration Add values in environment variables or add them in conf

Weeb >.< 61 Dec 29, 2022
Code for the manim-generated scenes used in 3blue1brown videos

Code for the manim-generated scenes used in 3blue1brown videos

Grant Sanderson 4.1k Jan 09, 2023
Uncompress DEFLATE streams in pure Python

stream-deflate Uncompress DEFLATE streams in pure Python. Work in progress. This README serves as a rough design spec. Installation pip install stream

Michal Charemza 7 Oct 13, 2022
OpenShot Video Editor is an award-winning free and open-source video editor for Linux, Mac, and Windows, and is dedicated to delivering high quality video editing and animation solutions to the world.

OpenShot Video Editor is an award-winning free and open-source video editor for Linux, Mac, and Windows, and is dedicated to delivering high quality v

OpenShot Studios, LLC 3.1k Jan 01, 2023
Add a "flame" effect on each hand's index onto a video stream.

Add a "flame" effect on each hand's index onto a video stream. recording.webm.mov This script is just a quick hack, it's a bit of glue between mediapi

Paul Willot 7 Sep 15, 2022
This program is to make a video based on Deep Dream

This program is to make a video based on Deep Dream. The program is modified from DeepDreamAnim and DeepDreamVideo with additional functions for bleding two frames based on the optical flows. It also

Aertist 23 Jan 22, 2022
An easy to use GUI based video to image sequence converter (and vice versa).

Vdo & Img Conversion Tools This is a quick conversion tool made with python that can save you a lot of time. With this tool you can extract image sequ

Akash Bora 3 Sep 18, 2022
Add the dislike count back to my YouTube videos via a comment containing that information.

YouTube Dislikes Forrest Knight Python Version 3.0+ Only use if you know what the code actually does. I'm not responsible for your use of this code in

Forrest Knight 155 Dec 19, 2022
A self-hosted streaming platform with Discord authentication, auto-recording and more!

A self-hosted streaming platform with Discord authentication, auto-recording and more!

John Patrick Glattetre 331 Dec 27, 2022
Text2Video's purpose is to help people create videos quickly and easily by simply typing out the video’s script and a description of images to include in the video.

Text2Video Text2Video's purpose is to help people create videos quickly and easily by simply typing out the video’s script and a description of images

Josh Chen 19 Nov 22, 2022
A GUI application for cropping images from videos

v-trimming-gui A GUI application for cropping images from videos. 動画をシークバーで操作しながらスクリーンショットを撮るためのアプリ。 Requirement Python =3.7 opencv-python ^4.5.5 PyS

Menrui 6 Feb 05, 2022