Honours project, on creating a depth estimation map from two stereo images of featureless regions

Overview

image-processing

This module generates depth maps for shape-blocked-out images

Install

If working with anaconda, then from the root directory:

conda env create --file environment.yml
conda activate image-processing

Otherwise, if python 3 is installed, pip can be used to ensure the required packages are available. From the root directory, run

pip install -r requirements.txt

Files

The core functional files are collection.py, image.py, shape.py, edge.py, segment.py. They each contain a class of the same name. They logically follow this order and encapsulate each other, so collection creates three image objects for the left, center and right images. Each image object creates a number of shape objects. Shape objects create edge objects. Edge objects create segment objects. helper.py contains assisting functions used by these various classes.

This design aids in splitting up all the information and processes necessary to perform the desired function and logically groups it to ease comprehension. Each ought to be well-commented enough to generally understand what each part is doing.

The only one intended to be accessed to retrieve depth maps is collection.py as it orchestrates the entire process.

Usage

Both main.py and auto_gen.py are designed to access collection and to have it create depth maps. They require the initial images to be stored within a directory in assets/ , and each with three further subdirectories, cameraLeft/, cameraCenter/, and cameraRight/ . They save their results to saves/ with the generated images being stored in saves/generated/ . All .img files are object-files generated during this process to reduce the workload needed the next time the same process is executed.

Main

main.py is for individual depth map generation. There are four arguments able to be passed to specify details to the execution.

  1. The directory name desired from within assets/ .
  2. The numerical index (starting at 0) of the specific image desired within the innermost subdirectories
  3. The number representing which image should the depth map visual be based on (0 for left, 1 for center, 2 for right)
  4. Should the resulting depth image be saved
  5. Should the resulting depth image be displayed

While it can take up to these four arguments, no arguments is also possible. Then, the directory within assets/ is randomly selected, as is the index of the image set, and which image is used to generate the depth map visual. It will save and display the results. Partial arguments is also fine, so long as order is maintained.

Example: To display on the left image but not save occluded_road's first image set

python main.py occluded_road 0 0 False True

note: the last argument, True, is redundant in this case
 
Example: Any road_no_occlusion image set, any image used to create the depth map visual (automatically will save and display)

python main.py road_no_occlusion

 

Example: Anything (automatically will save and display)

python main.py

 

The value of having it execute a certain image when its depth image has already been generated is that it will quickly pull it up in the viewer and unlike the static image one can view the individual pixel values the mouse hovers over in the top-right corner.

 

Auto_gen

Alternatively, auto_gen.py is intended for the automated creation of all depth map images.

python auto_gen.py

By simply executing it, it will determine the depth map image all image sets and save them all. The terminal output is saved to a txt file stored in saves/logs. It does not display the results, as that would greatly heed the process of creating all of the results.

Alternatively, it can take two arguments.

  1. Specifies a directory within assets/ to use rather than executing for all of them, similar to the first argument for main.py
  2. Specifies the image to be used as the basis for the depth map visual, similar to the third argument for main.py (0-2 for left, center, and right)

Example: All depth images

python auto_gen.py

 
Example: All Shape_based_stereoPairs depth images using the right image

python auto_gen.py Shape_based_stereoPairs 2

 

For both, if an existing depth map exists, it will not be redone even if the image expected to be used is different. To do so, remove both the .jpg and .img and re-run.

How it works

 

Initialization

Upon creation of an instance of collection, it first intantiates the left image's Image intance. The shape colours are determined and then each shape is instantiated. The bounding box of the given shape is determined as well as its left and right edges, and their segments.

Collection uses the colours determined by the left Image to speed up the other two image's instantiations.

After everything has been created, the segments of each edge, of each shape, in each image must be assigned. First this process requires determining the displacement of edges, which is then used to determine which shape owns and doesn't own which segment.

Generally at this stage all but a few stragglers are assigned. The remaining are due to shapes having few edges, and the only one it could own is shared with the ground or sky shape, and thus difficult to tell which owns it. Using additional information about the shapes ownership is assigned. Finally, it checks to see if any shapes are the ground or sky, as their depths are not calculated.

At this stage, the image objects are saved.

Depth calculation

Then, using this information about the edges of a shape, its depth can be more accurately calculated. Only edges it owns are used to determine its depth. So if it only has its right side, only the right edge is used. Alternatively if both are owned, the midpoint is used.

However, if the shape is determined to have a varying depth, then its depth can alternatively be calculated using the change of slope between the images.

Finally, once all depth values are found, a modified version of the original image is created with its shape colours replaced with their determined depth values, the sky is replaced with pure black, and the ground with pure white. This image is then possibly saved and possibly displayed. Which image is used to re-colour for the depth map depends on either a given argument or random selection.

A curated list of awesome deep long-tailed learning resources.

A curated list of awesome deep long-tailed learning resources.

vanint 210 Dec 25, 2022
Official Pytorch implementation of "Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes", CVPR 2022

Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes / 3DCrowdNet News 💪 3DCrowdNet achieves the state-of-the-art accuracy on 3D

Hongsuk Choi 113 Dec 21, 2022
Run Effective Large Batch Contrastive Learning on Limited Memory GPU

Gradient Cache Gradient Cache is a simple technique for unlimitedly scaling contrastive learning batch far beyond GPU memory constraint. This means tr

Luyu Gao 198 Dec 29, 2022
Codebase for "ProtoAttend: Attention-Based Prototypical Learning."

Codebase for "ProtoAttend: Attention-Based Prototypical Learning." Authors: Sercan O. Arik and Tomas Pfister Paper: Sercan O. Arik and Tomas Pfister,

47 2 May 17, 2022
Automated Hyperparameter Optimization Competition

QQ浏览器2021AI算法大赛 - 自动超参数优化竞赛 ACM CIKM 2021 AnalyticCup 在信息流推荐业务场景中普遍存在模型或策略效果依赖于“超参数”的问题,而“超参数"的设定往往依赖人工经验调参,不仅效率低下维护成本高,而且难以实现更优效果。因此,本次赛题以超参数优化为主题,从真

20 Dec 09, 2021
A Moonraker plug-in for real-time compensation of frame thermal expansion

Frame Expansion Compensation A Moonraker plug-in for real-time compensation of frame thermal expansion. Installation Credit to protoloft, from whom I

58 Jan 02, 2023
Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"

Memory Compressed Attention Implementation of the Self-Attention layer of the proposed Memory-Compressed Attention, in Pytorch. This repository offers

Phil Wang 47 Dec 23, 2022
MolRep: A Deep Representation Learning Library for Molecular Property Prediction

MolRep: A Deep Representation Learning Library for Molecular Property Prediction Summary MolRep is a Python package for fairly measuring algorithmic p

AI-Health @NSCC-gz 83 Dec 24, 2022
Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI

Hourglass Transformer - Pytorch (wip) Implementation of Hourglass Transformer, in Pytorch. It will also contain some of my own ideas about how to make

Phil Wang 61 Dec 25, 2022
Log4j JNDI inj. vuln scanner

Log-4-JAM - Log 4 Just Another Mess Log4j JNDI inj. vuln scanner Requirements pip3 install requests_toolbelt Usage # make sure target list has http/ht

Ashish Kunwar 66 Nov 09, 2022
Implementation detail for paper "Multi-level colonoscopy malignant tissue detection with adversarial CAC-UNet"

Multi-level-colonoscopy-malignant-tissue-detection-with-adversarial-CAC-UNet Implementation detail for our paper "Multi-level colonoscopy malignant ti

CVSM Group - email: <a href=[email protected]"> 84 Nov 22, 2022
RLHive: a framework designed to facilitate research in reinforcement learning.

RLHive is a framework designed to facilitate research in reinforcement learning. It provides the components necessary to run a full RL experiment, for both single agent and multi agent environments.

88 Jan 05, 2023
Honours project, on creating a depth estimation map from two stereo images of featureless regions

image-processing This module generates depth maps for shape-blocked-out images Install If working with anaconda, then from the root directory: conda e

2 Oct 17, 2022
Solver for Large-Scale Rank-One Semidefinite Relaxations

STRIDE: spectrahedral proximal gradient descent along vertices A Solver for Large-Scale Rank-One Semidefinite Relaxations About STRIDE is designed for

48 Dec 20, 2022
StellarGraph - Machine Learning on Graphs

StellarGraph Machine Learning Library StellarGraph is a Python library for machine learning on graphs and networks. Table of Contents Introduction Get

S T E L L A R 2.6k Jan 05, 2023
Locally cache assets that are normally streamed in POPULATION: ONE

Population One Localizer This is no longer needed as of the build shipped on 03/03/22, thank you bigbox :) Locally cache assets that are normally stre

Ahman Woods 2 Mar 04, 2022
The official implementation of the research paper "DAG Amendment for Inverse Control of Parametric Shapes"

DAG Amendment for Inverse Control of Parametric Shapes This repository is the official Blender implementation of the paper "DAG Amendment for Inverse

Elie Michel 157 Dec 26, 2022
Easily pull telemetry data and create beautiful visualizations for analysis.

This repository is a work in progress. Anything and everything is subject to change. Porpo Table of Contents Porpo Table of Contents General Informati

Ryan Dawes 33 Nov 30, 2022
Sentinel-1 vessel detection model used in the xView3 challenge

sar_vessel_detect Code for the AI2 Skylight team's submission in the xView3 competition (https://iuu.xview.us) for vessel detection in Sentinel-1 SAR

AI2 6 Sep 10, 2022
This is an official source code for implementation on Extensive Deep Temporal Point Process

Extensive Deep Temporal Point Process This is an official source code for implementation on Extensive Deep Temporal Point Process, which is composed o

Haitao Lin 8 Aug 15, 2022