Bacon - Band-limited Coordinate Networks for Multiscale Scene Representation

Related tags

Geolocationbacon
Overview

BACON: Band-limited Coordinate Networks for Multiscale Scene Representation

Project Page | Video | Paper

Official PyTorch implementation of BACON.
BACON: Band-limited Coordinate Networks for Multiscale Scene Representation
David B. Lindell*, Dave Van Veen, Jeong Joon Park, Gordon Wetzstein
Stanford University

Quickstart

To setup a conda environment use these commands

conda env create -f environment.yml
conda activate bacon

# download all datasets
python download_datasets.py

Now you can train networks to fit a 1D function, images, signed distance fields, or neural radiance fields with the following commands.

cd experiments
python train_1d.py --config ./config/1d/bacon_freq1.ini  # train 1D function
python train_img.py --config ./config/img/bacon.ini  # train image
python train_sdf.py --config ./config/sdf/bacon_armadillo.ini  # train SDF
python train_radiance_field.py --config ./config/nerf/bacon_lr.ini  # train NeRF

To visualize outputs in Tensorboard, run the following.

tensorboard --logdir=../logs --port=6006

Band-limited Coordinate Networks

Band-limited coordinate networks have an analytical Fourier spectrum and interpretible behavior. We demonstrate using these networks for fitting simple 1D signals, images, 3D shapes via signed distance functions and neural radiance fields.

Datasets

Datasets can be downloaded using the download_datasets.py script. This script

Training

We provide scripts for training and configuration files to reproduce the results in the paper.

1D Examples

To run the 1D examples, use the experiments/train_1d.py script with any of the config files in experiments/config/1d. These scripts allow training models with BACON, Fourier Features, or SIREN. For example, to train a BACON model you can run

python train_1d.py --config ./config/1d/bacon_freq1.ini

To change the bandwidth of BACON, adjust the maximum frequency with the --max_freq flag. This sets network-equivalent sampling rate used to represent the signal. For example, if the signal you wish to represent has a maximum frequency of 5 cycles per unit interval, this value should be set to at least the Nyquist rate of 2 samples per cycle or 10 samples per unit interval. By default, the frequencies represented by BACON are quantized to intervals of 2*pi; thus, the network is periodic over an interval from -0.5 to 0.5. That is, the output of the network will repeat for input coordinates that exceed an absolute value of 0.5.

Image Fitting

Image fitting can be performed using the config files in experiments/config/img and the train_img.py script. We support training BACON, Fourier Features, SIREN, and networks with the positional encoding from Mip-NeRF.

SDF Fitting

Config files for SDF fitting are in experiments/config/sdf and can be used with the train_sdf.py script. Be sure to download the example datasets before running this script.

We also provide a rendering script to extract meshes from the trained models. The render_sdf.py program extracts a mesh using marching cubes and, optionally, our proposed multiscale adaptive SDF evaluation procedure.

NeRF Reconstruction

Use the config files in experiments/config/nerf with the train_radiance_field.py script to train neural radiance fields. Note that training the full resolution model can takes a while (a few days) so it may be easier to train a low-resolution model to get started. We provide a low-resolution config file in experiments/config/nerf/bacon_lr.ini.

To render output images from a trained model, use the render_nerf.py script. Note that the Blender synthetic datasets should be downloaded and the multiscale dataset generated before running this script.

Initialization Scheme

Finally, we also show a visualization of our initialization scheme in experiments/plot_activation_distributions.py. As shown in the paper, our initialization scheme prevents the distribution of activations from becoming vanishingly small, even for deep networks.

Pretrained models

For convenience, we include pretrained models for the SDF fitting and NeRF reconstruction tasks in the pretrained_models directory. The outputs of these models can be rendered directly using the experiments/render_sdf.py and experiments/render_nerf.py scripts.

Citation

@article{lindell2021bacon,
author = {Lindell, David B. and Van Veen, Dave and Park, Jeong Joon and Wetzstein, Gordon},
title = {BACON: Band-limited coordinate networks for multiscale scene representation},
journal = {arXiv preprint arXiv:2112.04645},
year={2021}
}

Acknowledgments

This project was supported in part by a PECASE by the ARO and NSF award 1839974.

Owner
Stanford Computational Imaging Lab
Next-generation computational imaging and display systems.
Stanford Computational Imaging Lab
Tool to suck data from ArcGIS Server and spit it into PostgreSQL

chupaESRI About ChupaESRI is a Python module/command line tool to extract features from ArcGIS Server map services. Name? Think "chupacabra" or "Chupa

John Reiser 34 Dec 04, 2022
glTF to 3d Tiles Converter. Convert glTF model to Glb, b3dm or 3d tiles format.

gltf-to-3d-tiles glTF to 3d Tiles Converter. Convert glTF model to Glb, b3dm or 3d tiles format. Usage ฮป python main.py --help Usage: main.py [OPTION

58 Dec 27, 2022
Calculate the area inside of any GeoJSON geometry. This is a port of Mapbox's geojson-area for Python

geojson-area Calculate the area inside of any GeoJSON geometry. This is a port of Mapbox's geojson-area for Python. Installation $ pip install area U

Alireza 87 Dec 14, 2022
Spatial Interpolation Toolbox is a Python-based GUI that is able to interpolate spatial data in vector format.

Spatial Interpolation Toolbox This is the home to Spatial Interpolation Toolbox, a graphical user interface (GUI) for interpolating geographic vector

Michael Ward 2 Nov 01, 2021
Program that shows all the details of the given IP address. Build with Python and ipinfo.io API

ip-details This is a program that shows all the details of the given IP address. Build with Python and ipinfo.io API Usage To use this program, run th

4 Mar 01, 2022
A Jupyter - Leaflet.js bridge

ipyleaflet A Jupyter / Leaflet bridge enabling interactive maps in the Jupyter notebook. Usage Selecting a basemap for a leaflet map: Loading a geojso

Jupyter Widgets 1.3k Dec 27, 2022
A ready-to-use curated list of Spectral Indices for Remote Sensing applications.

A ready-to-use curated list of Spectral Indices for Remote Sensing applications. GitHub: https://github.com/davemlz/awesome-ee-spectral-indices Docume

David Montero Loaiza 488 Jan 03, 2023
iNaturalist observations along hiking trails

This tool reads the route of a hike and generates a table of iNaturalist observations along the trails. It also shows the observations and the route of the hike on a map. Moreover, it saves waypoints

7 Nov 11, 2022
Code and coordinates for Matt's 2021 xmas tree

xmastree2021 Code and coordinates for Matt's 2021 xmas tree This repository contains the code and coordinates used for Matt's 2021 Christmas tree, as

Stand-up Maths 117 Jan 01, 2023
Python project to generate Kerala's distrcit level panchayath map.

Kerala-Panchayath-Maps Python project to generate Kerala's distrcit level panchayath map. As of now, geojson files of Kollam and Kozhikode are added t

Athul R T 2 Jan 10, 2022
h3-js provides a JavaScript version of H3, a hexagon-based geospatial indexing system.

h3-js The h3-js library provides a pure-JavaScript version of the H3 Core Library, a hexagon-based geographic grid system. It can be used either in No

Uber Open Source 648 Jan 07, 2023
This is the antenna performance plotted from tinyGS reception data.

tinyGS-antenna-map This is the antenna performance plotted from tinyGS reception data. See their repository. The code produces a plot that provides Az

Martin J. Levy 14 Aug 21, 2022
leafmap - A Python package for geospatial analysis and interactive mapping in a Jupyter environment.

A Python package for geospatial analysis and interactive mapping with minimal coding in a Jupyter environment

Qiusheng Wu 1.4k Jan 02, 2023
LicenseLocation - License Location With Python

LicenseLocation Hi,everyone! โค ๐Ÿงก ๐Ÿ’› ๐Ÿ’š ๐Ÿ’™ ๐Ÿ’œ This is my first project! โœ” Actual

The Bin 1 Jan 25, 2022
Geocode rows in a SQLite database table

Geocode rows in a SQLite database table

Chris Amico 225 Dec 08, 2022
gpdvega is a bridge between GeoPandas and Altair that allows to seamlessly chart geospatial data

gpdvega gpdvega is a bridge between GeoPandas a geospatial extension of Pandas and the declarative statistical visualization library Altair, which all

Ilia Timofeev 49 Jul 25, 2022
Enable geospatial data mining through Google Earth Engine in Grasshopper 3D, via its most recent Hops component.

AALU_Geo Mining This repository is produced for a masterclass at the Architectural Association Landscape Urbanism programme. Requirements Rhinoceros (

4 Nov 16, 2022
OSMnx: Python for street networks. Retrieve, model, analyze, and visualize street networks and other spatial data from OpenStreetMap.

OSMnx OSMnx is a Python package that lets you download geospatial data from OpenStreetMap and model, project, visualize, and analyze real-world street

Geoff Boeing 4k Jan 08, 2023
Rasterio reads and writes geospatial raster datasets

Rasterio Rasterio reads and writes geospatial raster data. Geographic information systems use GeoTIFF and other formats to organize and store gridded,

Mapbox 1.9k Jan 07, 2023
Record railway train route profile with GNSS tools

Train route profile recording with GNSS technology based on ARDUINO platform Project target Develop GNSS recording tools based on the ARDUINO platform

tomcom 1 Jan 01, 2022