Bacon - Band-limited Coordinate Networks for Multiscale Scene Representation

Related tags

Geolocationbacon
Overview

BACON: Band-limited Coordinate Networks for Multiscale Scene Representation

Project Page | Video | Paper

Official PyTorch implementation of BACON.
BACON: Band-limited Coordinate Networks for Multiscale Scene Representation
David B. Lindell*, Dave Van Veen, Jeong Joon Park, Gordon Wetzstein
Stanford University

Quickstart

To setup a conda environment use these commands

conda env create -f environment.yml
conda activate bacon

# download all datasets
python download_datasets.py

Now you can train networks to fit a 1D function, images, signed distance fields, or neural radiance fields with the following commands.

cd experiments
python train_1d.py --config ./config/1d/bacon_freq1.ini  # train 1D function
python train_img.py --config ./config/img/bacon.ini  # train image
python train_sdf.py --config ./config/sdf/bacon_armadillo.ini  # train SDF
python train_radiance_field.py --config ./config/nerf/bacon_lr.ini  # train NeRF

To visualize outputs in Tensorboard, run the following.

tensorboard --logdir=../logs --port=6006

Band-limited Coordinate Networks

Band-limited coordinate networks have an analytical Fourier spectrum and interpretible behavior. We demonstrate using these networks for fitting simple 1D signals, images, 3D shapes via signed distance functions and neural radiance fields.

Datasets

Datasets can be downloaded using the download_datasets.py script. This script

Training

We provide scripts for training and configuration files to reproduce the results in the paper.

1D Examples

To run the 1D examples, use the experiments/train_1d.py script with any of the config files in experiments/config/1d. These scripts allow training models with BACON, Fourier Features, or SIREN. For example, to train a BACON model you can run

python train_1d.py --config ./config/1d/bacon_freq1.ini

To change the bandwidth of BACON, adjust the maximum frequency with the --max_freq flag. This sets network-equivalent sampling rate used to represent the signal. For example, if the signal you wish to represent has a maximum frequency of 5 cycles per unit interval, this value should be set to at least the Nyquist rate of 2 samples per cycle or 10 samples per unit interval. By default, the frequencies represented by BACON are quantized to intervals of 2*pi; thus, the network is periodic over an interval from -0.5 to 0.5. That is, the output of the network will repeat for input coordinates that exceed an absolute value of 0.5.

Image Fitting

Image fitting can be performed using the config files in experiments/config/img and the train_img.py script. We support training BACON, Fourier Features, SIREN, and networks with the positional encoding from Mip-NeRF.

SDF Fitting

Config files for SDF fitting are in experiments/config/sdf and can be used with the train_sdf.py script. Be sure to download the example datasets before running this script.

We also provide a rendering script to extract meshes from the trained models. The render_sdf.py program extracts a mesh using marching cubes and, optionally, our proposed multiscale adaptive SDF evaluation procedure.

NeRF Reconstruction

Use the config files in experiments/config/nerf with the train_radiance_field.py script to train neural radiance fields. Note that training the full resolution model can takes a while (a few days) so it may be easier to train a low-resolution model to get started. We provide a low-resolution config file in experiments/config/nerf/bacon_lr.ini.

To render output images from a trained model, use the render_nerf.py script. Note that the Blender synthetic datasets should be downloaded and the multiscale dataset generated before running this script.

Initialization Scheme

Finally, we also show a visualization of our initialization scheme in experiments/plot_activation_distributions.py. As shown in the paper, our initialization scheme prevents the distribution of activations from becoming vanishingly small, even for deep networks.

Pretrained models

For convenience, we include pretrained models for the SDF fitting and NeRF reconstruction tasks in the pretrained_models directory. The outputs of these models can be rendered directly using the experiments/render_sdf.py and experiments/render_nerf.py scripts.

Citation

@article{lindell2021bacon,
author = {Lindell, David B. and Van Veen, Dave and Park, Jeong Joon and Wetzstein, Gordon},
title = {BACON: Band-limited coordinate networks for multiscale scene representation},
journal = {arXiv preprint arXiv:2112.04645},
year={2021}
}

Acknowledgments

This project was supported in part by a PECASE by the ARO and NSF award 1839974.

Owner
Stanford Computational Imaging Lab
Next-generation computational imaging and display systems.
Stanford Computational Imaging Lab
Python script that can be used to generate latitude/longitude coordinates for GOES-16 full-disk extent.

goes-latlon Python script that can be used to generate latitude/longitude coordinates for GOES-16 full-disk extent. ๐ŸŒŽ ๐Ÿ›ฐ๏ธ The grid files can be acces

Douglas Uba 3 Apr 06, 2022
PySAL: Python Spatial Analysis Library Meta-Package

Python Spatial Analysis Library PySAL, the Python spatial analysis library, is an open source cross-platform library for geospatial data science with

Python Spatial Analysis Library 1.1k Dec 18, 2022
Simple, concise geographical visualization in Python

Geographic visualizations for HoloViews. Build Status Coverage Latest dev release Latest release Docs What is it? GeoViews is a Python library that ma

HoloViz 445 Jan 02, 2023
Python interface to PROJ (cartographic projections and coordinate transformations library)

pyproj Python interface to PROJ (cartographic projections and coordinate transformations library). Documentation Stable: http://pyproj4.github.io/pypr

832 Dec 31, 2022
A multi-page streamlit app for the geospatial community.

A multi-page streamlit app for the geospatial community.

Qiusheng Wu 522 Dec 30, 2022
User friendly Rasterio plugin to read raster datasets.

rio-tiler User friendly Rasterio plugin to read raster datasets. Documentation: https://cogeotiff.github.io/rio-tiler/ Source Code: https://github.com

372 Dec 23, 2022
Create Siege configuration files from Cloud Optimized GeoTIFF.

cogeo-siege Documentation: Source Code: https://github.com/developmentseed/cogeo-siege Description Create siege configuration files from Cloud Optimiz

Development Seed 3 Dec 01, 2022
Wraps GEOS geometry functions in numpy ufuncs.

PyGEOS PyGEOS is a C/Python library with vectorized geometry functions. The geometry operations are done in the open-source geometry library GEOS. PyG

362 Dec 23, 2022
Streamlit Component for rendering Folium maps

streamlit-folium This Streamlit Component is a work-in-progress to determine what functionality is desirable for a Folium and Streamlit integration. C

Randy Zwitch 224 Dec 30, 2022
a Geolocator made in python

Geolocator A Geolocator made in python โœจ Features locates ur location using ur ip thats it! ๐Ÿ’โ€โ™€๏ธ How to use first download the locator.py file instal

Portgas D Ace 1 Oct 27, 2021
LicenseLocation - License Location With Python

LicenseLocation Hi,everyone! โค ๐Ÿงก ๐Ÿ’› ๐Ÿ’š ๐Ÿ’™ ๐Ÿ’œ This is my first project! โœ” Actual

The Bin 1 Jan 25, 2022
Python Data. Leaflet.js Maps.

folium Python Data, Leaflet.js Maps folium builds on the data wrangling strengths of the Python ecosystem and the mapping strengths of the Leaflet.js

6k Jan 02, 2023
WebGL2 powered geospatial visualization layers

deck.gl | Website WebGL2-powered, highly performant large-scale data visualization deck.gl is designed to simplify high-performance, WebGL-based visua

Vis.gl 10.5k Jan 08, 2023
The geospatial toolkit for redistricting data.

maup maup is the geospatial toolkit for redistricting data. The package streamlines the basic workflows that arise when working with blocks, precincts

Metric Geometry and Gerrymandering Group 60 Dec 05, 2022
Documentation and samples for ArcGIS API for Python

ArcGIS API for Python ArcGIS API for Python is a Python library for working with maps and geospatial data, powered by web GIS. It provides simple and

Esri 1.4k Dec 30, 2022
Daily social mapping project in November 2021. Maps made using PyGMT whenever possible.

Daily social mapping project in November 2021. Maps made using PyGMT whenever possible.

Wei Ji 20 Nov 24, 2022
A light-weight, versatile XYZ tile server, built with Flask and Rasterio :earth_africa:

Terracotta is a pure Python tile server that runs as a WSGI app on a dedicated webserver or as a serverless app on AWS Lambda. It is built on a modern

DHI GRAS 531 Dec 28, 2022
This app displays interesting statistical weather records and trends which can be used in climate related research including study of global warming.

This app displays interesting statistical weather records and trends which can be used in climate related research including study of global warming.

0 Dec 27, 2021
gjf: A tool for fixing invalid GeoJSON objects

gjf: A tool for fixing invalid GeoJSON objects The goal of this tool is to make it as easy as possible to fix invalid GeoJSON objects through Python o

Yazeed Almuqwishi 91 Dec 06, 2022
Advanced raster and geometry manipulations

buzzard In a nutshell, the buzzard library provides powerful abstractions to manipulate together images and geometries that come from different kind o

Earthcube Lab 30 Jun 20, 2022