This program is to make a video based on Deep Dream

Related tags

VideoDeepDreamAnimV2
Overview

DeepDreamAnimV2

This program is to make a video based on Deep Dream. The program is modified from DeepDreamAnim and DeepDreamVideo with additional functions for bleding two frames based on the optical flows. It also supports the image division to apply the Deep Dream algorithm to a large image.

A sample video created by this script from a 4K-resolution, panoramic video. This sample video is cropped by a view from a head-mounted display.

Use Optical Flow to Adjust Deep Dream Video

The flow option enables the optical flow mode. This allows the optical flow of each frame to be calculated by comparing the difference in the movement of all pixels between the current and previous frame. The hallucinatory patterns on the area where the optical flow was detected is merged with the current (not-yet-hallucinatory) frame based on the weighting provided by the user defined blending ratio (0 = no information, 1 = all information). The blending ratio allows some of the hallucinatory content of the previous frame to be inherited from the previous frame, The Deep Dream algorithm is then applied to this merged frame, instead of Deep Dream starting from scratch for each frame.

  1. The difference in optical flow is calculated between the previous and current frame (before the Deep Dream algorithm is applied).
  2. The hallucinatory patterns within the area of high optical flow in the previous frame are shifted in the direction of the optical flow.
  3. This hallucinatory pattern from the previous frame is merged into the current (not-yet-hallucinatory) frame with a specfied blending ratio.

The blending ratio on the optical flow area and the other areas (background) can be specified separately by using -bm and -bs options. The range of the blending ratio is between 0 and 1. A blending ratio of 1 means that the current frame inherits 100% of the hallucinatory content from the previous frame, and then the deep dream algorithm is applied. A blending ratio of 0 means the previous frame is dicarded, therefore the deep dream algorithm is applied from the scrach.

Dividing the Image

When using a GPU to process the deep dream algorithm, the maximum image size is capped by the video memory of your GPU. In order to process large images such as 4K resolution, it is neccersary to divide the input image into smaller sub-images inorder to apply Deep Dream. -d or --divide option can be used to enable this function.

-d 0 : disable image division

-d 1 : divide the image by MAX_WIDTH and MAX_HEIGHT of image specified by -mw and -mh options

-d 2 : divide the image half when the width of the image is larger than MAX_WIDTH specified by -mw option

The divided image is processed by Deep Dream individually and then merged into one. If using octavescale in the original deep dream algorithm, the image is processed in a smaller size in the begining. In that case, the image division will not be applied to the earlier stage of the itterations (before enlarging the original size).

Usage

usage: dreamer.py [-h] -i INPUT -o OUTPUT -it IMAGE_TYPE [--gpu GPU]
                       [-t MODEL_PATH] [-m MODEL_NAME] [-p PREVIEW]
                       [-oct OCTAVES] [-octs OCTAVESCALE] [-itr ITERATIONS]
                       [-j JITTER] [-z ZOOM] [-s STEPSIZE]
                       [-l LAYERS [LAYERS ...]] [-v VERBOSE]
                       [-g GUIDE_IMAGE] [-flow FLOW] [-flowthresh FLOWTHREASH]
                       [-bm BLEND_FLOW] [-bs BLEND_STATIC] [-d DEVIDE_MODE]
                       [-mw MAX_WIDTH] [-mh MAX_HEIGHT]

Original Arguments:

  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        Input directory where extracted frames are stored
  -o OUTPUT, --output OUTPUT
                        Output directory where processed frames are to be
                        stored
  -it IMAGE_TYPE, --image_type IMAGE_TYPE
                        Specify whether jpg or png
  --gpu GPU             Switch for gpu computation.
  -t MODEL_PATH, --model_path MODEL_PATH
                        Model directory to use
  -m MODEL_NAME, --model_name MODEL_NAME
                        Caffe Model name to use
  -p PREVIEW, --preview PREVIEW
                        Preview image width. Default: 0
  -oct OCTAVES, --octaves OCTAVES
                        Octaves. Default: 4
  -octs OCTAVESCALE, --octavescale OCTAVESCALE
                        Octave Scale. Default: 1.4
  -itr ITERATIONS, --iterations ITERATIONS
                        Iterations. Default: 10
  -j JITTER, --jitter JITTER
                        Jitter. Default: 32
  -z ZOOM, --zoom ZOOM  Zoom in Amount. Default: 1
  -s STEPSIZE, --stepsize STEPSIZE
                        Step Size. Default: 1.5
  -l LAYERS [LAYERS ...], --layers LAYERS [LAYERS ...]
                        Array of Layers to loop through. Default: [customloop]
                        - or choose ie [inception_4c/output] for that single
                        layer
  -v VERBOSE, --verbose VERBOSE
                        verbosity [0-3]
  -g, --guide GUIDE_IMAGE
			A guide image

Additional Arguments:

   -flow, --flow	Optical Flow is taken into accout
   -flowthresh FLOWTHREASH, --flowthresh FLOWTHREASH
			Threshold for detecting a flow
   -bm BLEND_FLOW, --blendflow BLEND_FLOW
			blend ratio for flowing part
   -bs BLEND_STATIC, --blendstatic BLEND_STATIC
			blend ratio for static part

   -d [0-2] --divide [0-2]
			dividing image into sub images [0:disable 1:dividing to maxWidth, maxHeight 2:dividing half if width exceeds maxWidth]
   -mw MAX_WIDTH, --maxWidth MAX_WIDTH
			Maximum width to devide image
   -mh MAX_HEIGHT, --maxHeight MAX_HEIGHT
			Maximum height to devide image (only used for the divide mode 1)

Requirements

  • Python
  • Caffe (and other deepdream dependencies)
  • FFMPEG
  • CV2 (if you use optical flow)

Examples

1-extract.bat:

python dreamer.py -e 1 --input RHI-20Sec.mov --output Input

2-dream.bat:

python dreamer.py --input Input --output Output --octaves 3 --octavescale 1.8 --iterations 16 --jitter 32 --zoom 1 --stepsize 1.5 --flow 1 --flowthresh 6 --blendflow 0.9 --blendstatic 0.1 --layers inception_4d/pool --gpu 1 -d 2 -mw 1500

3-create.bat:

python dreamer.py -c 1 --input Output --output Video.mp4

Owner
Aertist
Aertist
Video-to-GIF-Converter - A small code snippet that can be used to convert any video to a gif

Video to GIF Converter Project Description: This is a small code snippet that ca

Hassan Shahzad 3 Jun 22, 2022
Tautulli - A Python based monitoring and tracking tool for Plex Media Server.

Tautulli A python based web application for monitoring, analytics and notifications for Plex Media Server. This project is based on code from Headphon

Tautulli 4.7k Jan 07, 2023
Program for converting video to GIF

video-to-gif Program for converting video to GIF Install the lib: pip install moviepy Usage: Specify the path to the video file. clip = VideoFileClip(

Artem Mokin 0 Dec 13, 2021
KonomiTV: Kind and Optimized Next brOadcast watching systeM Infrastructure for TV

備考・注意事項 現在 α 版で、まだ実験的なプロダクトです。通常利用には耐えないでしょうし、サポートもできません。 安定しているとは到底言いがたい品質ですが、それでも構わない方のみ導入してください。 使い方などの説明も用意できていないため、自力でトラブルに対処できるエンジニアの方以外には現状おすすめ

tsukumi 244 Dec 31, 2022
A Python media index

pyvideo https://pyvideo.org is simply an index of Python-related media records. The raw data being used here comes out of the pyvideo/data repo. Befor

pyvideo 235 Dec 24, 2022
Vigia-youtube - The YouTube Watch bot is able to monitor channels on Google's video platform

Vigia do YouTube O bot Vigia do YouTube é capaz de monitorar canais na plataform

Alessandro Feitosa Jr 10 Oct 03, 2022
A wrapper around ffmpeg to make it work in a concurrent and memory-buffered fashion.

Media Fixer Have you ever had a film or TV show that your TV wasn't able to play its audio? Well this program is for you. Media Fixer is a program whi

Halit Şimşek 3 May 04, 2022
A Advanced Anime Theme VC Video Player created for playing vidio in the voice chats of Telegram Groups

Yui Vidio Player A Advanced Anime Theme VC Video Player created for playing vidio in the voice chats of Telegram Groups Demo Setting up Add this Bot t

Achu biju 32 Sep 16, 2021
Lightweight, zero-dependency proxy and storage RTSP server

python-rtsp-server Python-rtsp-server is a lightweight, zero-dependency proxy and storage server for several IP-cameras and multiple clients. Features

Vlad 39 Nov 23, 2022
This is a tool for making a every day video if you take a picture of you everyday

Face-Everyday-Maker-Studio Description This project is a tool for making a everyday video, which is timelapse video or slides video, of images but for

John A Betancourt G 9 Sep 06, 2022
BlogBot - a Python script that create blogs from YouTube videos.

BlogBot - Convert Youtube Videos To Blogs BlogBot is a Python script that create blogs from YouTube videos.

Nikhil Bhamere 4 Apr 22, 2022
Synchronize Two Cameras in Real Time using Multiprocessing

Synchronize Two Cameras in Real Time using Multiprocessing In progress ... 📁 Project Structure 📚 Install Libraries for this Project (requirements.tx

Eduardo Carvalho Nunes 2 Oct 31, 2021
Python Simple Mass Video Clipper (PSMVC)

Python Simple Mass Video Clipper (PSMVC) PSMVC é um gerador de cortes via terminal construído em python. Uso Basta abrir o arquivo start.py Dependenci

Bruno 2 Oct 16, 2021
Telegram Video Stream

Video Stream An Advanced VC Video Player created for playing video in the voice chats of Telegram Groups And Channel Configs TOKEN - Get bot token fro

mr_lokaman 46 Dec 25, 2022
Python and OpenCV-based scene cut/transition detection program & library.

Video Scene Cut Detection and Analysis Tool Latest Release: v0.5.6.1 (October 11, 2021) Main Webpage: py.scenedetect.com Documentation: manual.scenede

Brandon Castellano 1.8k Jan 02, 2023
A GUI application for cropping images from videos

v-trimming-gui A GUI application for cropping images from videos. 動画をシークバーで操作しながらスクリーンショットを撮るためのアプリ。 Requirement Python =3.7 opencv-python ^4.5.5 PyS

Menrui 6 Feb 05, 2022
Autocut the Twitch VODs based on Marker

Markut Given the VOD of the stream and the markers that are exported as a CSV file, generate a video using ffmpeg that cuts out part of the VOD accord

Tsoding 18 Dec 19, 2022
Stream music with ffmpeg and python

youtube-stream Stream music with ffmpeg and python original Usage set the KEY in stream.sh run server.py run stream.sh (You can use Git bash or WSL in

Giyoung Ryu 14 Nov 17, 2021
A script to disable steam servers regionwise. [Works on Windows only]

Csgo-server-blocker A script to disable steam servers regionwise. [Works on Windows only] Dependencies python3.x Usage: pip install requirements.txt I

Aditya Bennur 2 Jun 10, 2022
Uncompress DEFLATE streams in pure Python

stream-deflate Uncompress DEFLATE streams in pure Python. Work in progress. This README serves as a rough design spec. Installation pip install stream

Michal Charemza 7 Oct 13, 2022