Differentiable rasterization applied to 3D model simplification tasks

Related tags

Deep Learningnvdiffmodeling
Overview

nvdiffmodeling

Teaser image

Differentiable rasterization applied to 3D model simplification tasks, as described in the paper:

Appearance-Driven Automatic 3D Model Simplification
Jon Hasselgren, Jacob Munkberg, Jaakko Lehtinen, Miika Aittala and Samuli Laine
https://research.nvidia.com/publication/2021-04_Appearance-Driven-Automatic-3D
https://arxiv.org/abs/2104.03989

License

Copyright © 2021, NVIDIA Corporation. All rights reserved.

This work is made available under the Nvidia Source Code License.

For business inquiries, please contact [email protected]

Installation

Requires VS2019+, Cuda 10.2+ and PyTorch 1.6+

Tested in Anaconda3 with Python 3.6 and PyTorch 1.8

One time setup (Windows)

Install the Cuda toolkit (required to build the PyTorch extensions). We support Cuda 10.2 and above. Pick the appropriate version of PyTorch compatible with the installed Cuda toolkit. Below is an example with Cuda 11.1

conda create -n dmodel python=3.6
activate dmodel
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge
conda install imageio
pip install PyOpenGL glfw

Install nvdiffrast: https://github.com/NVlabs/nvdiffrast in the dmodel conda env.

Every new command prompt

activate dmodel

Examples

Sphere to cow example:

python train.py --config configs/spot.json

The results will be stored in the out folder. The Spot model was created and released into the public domain by Keenan Crane.

Additional assets can be downloaded here [205MB]. Unzip and place the subfolders in the project data folder, e.g., data\skull. All assets are copyright of their respective authors, see included license files for further details.

Included examples

  • skull.json - Joint normal map and shape optimization on a skull
  • ewer.json - Ewer model from a reduced mesh as initial guess
  • gardenina.json - Aggregate geometry example
  • hibiscus.json - Aggregate geometry example
  • figure_brushed_gold_64.json - LOD example, trained against a supersampled reference
  • figure_displacement.json - Joint shape, normal map, and displacement map example

The json files that end in _paper.json are configs with the settings used for the results in the paper. They take longer and require a GPU with sufficient memory.

Server usage (through Docker)

  • Build docker image (run the command from the code root folder). docker build -f docker/Dockerfile -t diffmod:v1 . Requires a driver that supports Cuda 10.1 or newer.

  • Start an interactive docker container: docker run --gpus device=0 -it --rm -v /raid:/raid -it diffmod:v1 bash

  • Detached docker: docker run --gpus device=1 -d -v /raid:/raid -w=[path to the code] diffmod:v1 python train.py --config configs/spot.json

Comments
  • Proposition for more comprehensive Readme

    Proposition for more comprehensive Readme

    First of all, thank you for making this freely available for everyone! This is really a great move!

    I wanted to suggest a few things for the readme: 1.) Add "Microsoft Visual C++" to the requirements. It should be clear that you have to tick the C++ development environment while installing VS2019.

    2.) Install Git via "conda install git" (This makes it easier to install nvdiffrast and clone the repo)

    3.) Change "Install NVDiffRast" to something more descriptive: pip install -e git+https://github.com/NVlabs/[email protected]#egg=nvdiffrast

    4.) Add another step to the installation, detailing to move to a desired folder via the Anaconda Prompt and cloning this repo: cd YOUR/PATH/HERE git clone https://github.com/NVlabs/nvdiffmodeling.git and afterwards to move into the folder cd nvdiffmodeling

    5.) Installing CUDA via Anaconda only deploys the bare essential files. The CUDA path is not set when doing it this way. So I downloaded the CUDA toolkit from nvidia and installed it afterwards, to fix that.

    I am sure that all of these things are obvious for professionals, but it took me a while to figure all of this out, so I would like to save other people's time.

    opened by morph3us-net 7
  • About the raymarch fitting case

    About the raymarch fitting case

    Thanks for the marvelous work! I have some questions related to the shadertoy fitting case: It seems figure 22 & figure 23 use some special uv un-wrap methods, can you elaborate the algorithm under the hood? image

    opened by outlawever 4
  • RuntimeError: Error building extension 'renderutils_plugin'

    RuntimeError: Error building extension 'renderutils_plugin'

    I am trying to get this to work to test the workflow but ran into the error I have installed "Cuda" 11.1 and everything else as in the tutorial Graphics accelerator 3080t

    After trying to run a test command, it gives me the following text Do you have any ideas how to fix this? I am more of a technical artist and a cursory search did not return much

    
(dmodel) I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main>python train.py --config configs/spot.json
Using C:\Users\alexp\AppData\Local\torch_extensions\torch_extensions\Cache as PyTorch extensions root...
C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for cl: 'utf-8' codec can't decode byte 0x8e in position 0: invalid start byte
  warnings.warn(f'Error checking compiler version for {compiler}: {error}')
Detected CUDA files, patching ldflags
Emitting ninja build file C:\Users\alexp\AppData\Local\torch_extensions\torch_extensions\Cache\renderutils_plugin\build.ninja...
Building extension module renderutils_plugin...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/7] cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\common.cpp /Focommon.o
FAILED: common.o
cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\common.cpp /Focommon.o
Оптимизирующий компилятор Microsoft (R) C/C++ версии 19.25.28611 для x64
(C) Корпорация Майкрософт (Microsoft Corporation).  Все права защищены.

Примечание: включение файла:  C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\cuda_runtime.h
Примечание: включение файла:   C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_config.h
Примечание: включение файла:   C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\builtin_types.h
Примечание: включение файла:    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\device_types.h
Примечание: включение файла:     C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_defines.h
Примечание: включение файла:    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\driver_types.h
Примечание: включение файла:     C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_defines.h
Примечание: включение файла:     C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\vector_types.h
Примечание: включение файла:      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_defines.h
Примечание: включение файла:     C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\limits.h
Примечание: включение файла:      C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vcruntime.h
Примечание: включение файла:       C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\sal.h
Примечание: включение файла:        C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\concurrencysal.h
Примечание: включение файла:       C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vadefs.h
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\driver_types.h(78): fatal error C1083: Не удается открыть файл включение: stddef.h: No such file or directory,
[2/7] cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\torch_bindings.cpp /Fotorch_bindings.o
FAILED: torch_bindings.o
cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\torch_bindings.cpp /Fotorch_bindings.o
Оптимизирующий компилятор Microsoft (R) C/C++ версии 19.25.28611 для x64
(C) Корпорация Майкрософт (Microsoft Corporation).  Все права защищены.

Примечание: включение файла:  C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch/extension.h
Примечание: включение файла:   C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include\torch/all.h
Примечание: включение файла:    C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include\torch/cuda.h
Примечание: включение файла:     C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch/csrc/WindowsTorchApiMacro.h
Примечание: включение файла:      C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\c10/macros/Export.h
Примечание: включение файла:       C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\c10/macros/cmake_macros.h
Примечание: включение файла:     C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\cstdint
Примечание: включение файла:      C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\yvals.h
Примечание: включение файла:       C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\yvals_core.h
Примечание: включение файла:        C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vcruntime.h
Примечание: включение файла:         C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\sal.h
Примечание: включение файла:          C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\concurrencysal.h
Примечание: включение файла:         C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vadefs.h
Примечание: включение файла:        C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\xkeycheck.h
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\yvals.h(12): fatal error C1083: Не удается открыть файл включение: crtdbg.h: No such file or directory,
[3/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output mesh.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\mesh.cu -o mesh.cuda.o
FAILED: mesh.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output mesh.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\mesh.cu -o mesh.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
mesh.cu
[4/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output loss.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\loss.cu -o loss.cuda.o
FAILED: loss.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output loss.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\loss.cu -o loss.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
loss.cu
[5/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bsdf.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\bsdf.cu -o bsdf.cuda.o
FAILED: bsdf.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bsdf.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\bsdf.cu -o bsdf.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
bsdf.cu
[6/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output normal.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\normal.cu -o normal.cuda.o
FAILED: normal.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output normal.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\normal.cu -o normal.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
normal.cu
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1672, in _run_ninja_build
    env=env)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train.py", line 20, in <module>
    import src.renderutils as ru
  File "I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\__init__.py", line 9, in <module>
    from .ops import xfm_points, xfm_vectors, image_loss, prepare_shading_normal, lambert, pbr_specular, pbr_bsdf, _fresnel_shlick, _ndf_ggx, _lambda_ggx, _masking_smith
  File "I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\ops.py", line 61, in <module>
    torch.utils.cpp_extension.load(name='renderutils_plugin', sources=source_paths, extra_ldflags=ldflags, with_cuda=True, verbose=True)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1092, in load
    keep_intermediates=keep_intermediates)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1303, in _jit_compile
    is_standalone=is_standalone)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
    error_prefix=f"Error building extension '{name}'")
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1682, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'renderutils_plugin'
    opened by TheAlexPG 4
  • about effect Fig.22 and Fig 23

    about effect Fig.22 and Fig 23

    I read your paper and video in detail，about LEARNING MESH AND MATERIALS FROM IMPLICIT SURFACES,How to improve the effect？ Can you give me some suggestions on setting super parameters? I used my own model as the target, started from the sphere you provided, and carried out experiments. I found that the feet and face were always unsatisfactory? There is a big gap with the rendering of snails and elephants mentioned in your paper. can you give me some adivce. image

    opened by Lucklycat 2
  • Texture maps have a lot of noise

    Texture maps have a lot of noise

    Thank you very much for your work！ When I tried the ewer example, I found the following textures： image When you zoom in, you will find many red, yellow and green noises image This is still the case when I import the output obj into blender, but the contrast image rendered from nvdiffast does not show this situation. What is the reason? image

    opened by Lucklycat 2
  • How to improve the optimization effect

    How to improve the optimization effect

    Your work is great. Thank you very much for your openness. I tried to use rendering on my own model, but I encountered the following problems and would like to consult: 1.In the process of rendering, I found that the face of the character, especially the eyes, was not poorly optimized, and the color could not be reflected. Are there any parameters that need to be adjusted? image 2.During rendering, I encountered that. mtl contains a map_ The texture of d cannot be represented in the renderer. How can I solve this problem? image

    Thank you for your reply

    opened by Lucklycat 2
  • Lower training resolution => heavier texture pixelation. But why?

    Lower training resolution => heavier texture pixelation. But why?

    Hi, and thanks for great paper and comprehensive code! Could you please help me understand one thing:

    I'm running configs/dancer_displacement.json. Despite "texture_res" being 2048, these 2048 x 2048 textures "learn" huge pixels at lower "train_res":

    image

    And this isn't just poor interpolation in Blender, but these are actual large solid blocks (of different scale!) in normal_map_opt and kd_map_opt! Here is a part of texture_n.png from the top left experiment, 1:1 scale:

    image

    Why do these maps get more pixelated instead of getting more blurry? I've spent couple days searching the code for the source of this behaviour, and I believe there are only three relevant lines (for now let's consider texture only):

    Interpolation doesn't use 'nearest' method so it shouldn't be the source of big pixels. And because views are always different, rasterization should be a problem too. But probably I'm just missing some rendering subtleties?

    Thanks again!

    opened by shrubb 2
  • msvcrt.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config

    msvcrt.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config

    hi, Thanks for your project. When I try to run sphere to cow demo, I will report this error: msvcrt.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config renderutils_plugin.pyd : fatal error LNK1120: Unresolved external command ninja: build stopped: subcommand failed.

    Is it because of my Nanjia version? my current version is: nanjia:1.10.2 pytorch:1.8.1 torchvision:0.9.1 cudnn:7.6.5 cuda:10.2

    opened by myshiop 2
  • matrix gradients does not flow back when using renderutils xfm_points

    matrix gradients does not flow back when using renderutils xfm_points

    hi, when using renderutils.xfm_points (e.g. in render.py v_pos_clip = ru.xfm_points(mesh.v_pos[None, ...], mtx_in) - and trying to optimize camera pose in mtx_in - gradients does not flow back. Only when using use_python=True it worked.

    opened by maorp 1
  • using camera calibration matrices

    using camera calibration matrices

    Hi, first thanks for sharing this work! I'm trying to solve inverse rendering problem using a photogrammetry dataset which comes with accurate camera calibration mats. i have started from the spot example and i need to convert my intrinsic\extrinsic params to be used in the rastering and cant find an example or a guide to do it. the only hint is that I know it should be converted to openGL and NDC according to nvdiffrast, ill be happy to get an advice, thanks.

    opened by maorp 1
  • some question about the complex material, such as that ray marching snail

    some question about the complex material, such as that ray marching snail

    Hi, Thank you very much for your research. I'm trying to use your framework to approximate complex materials, but the blender complex material dataset I made with my own is not very good, because I encountered many problems with camera and light settings in my own rendering, Is the ray marching snail used in this paper directly rendered with shadertoy, or is the ray marching process written in Python?

    opened by myshiop 1
  • where can i find the Animation and Skinning demo

    where can i find the Animation and Skinning demo

    hi, i am read the papaer--Appearance-Driven Automatic 3D Model Simplification, think it's great. But in the demo,i can not find the Animation and Skinning demo, where can i find it ? I want to study how to do it. please help ! thanks

    opened by Lucklycat 2
  • CUDA out of memory.

    CUDA out of memory.

    RuntimeError: CUDA out of memory. Tried to allocate 130.00 MiB (GPU 0; 8.00 GiB total capacity; 7.11 GiB already allocated; 0 bytes free; 7.30 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

    opened by d-sharafeldeen 2
  • How to test on custom .obj file?

    How to test on custom .obj file?

    I am trying to test model simplification on my own .obj. I have created a .mtl file for the .obj but I got an error. How to export .obj file to train the model?

    opened by sanbuddhacharyas 0
  • Unable to execute the program, error during cpp_extension.py running at start

    Unable to execute the program, error during cpp_extension.py running at start 

    (dmodel) D:\NVIDIATools\nvdiffmodeling-main>python train.py --config configs/spot.json
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5'
Using C:\Users\redacted\AppData\Local\torch_extensions\torch_extensions\Cache\py36_cpu as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file C:\Users\redacted\AppData\Local\torch_extensions\torch_extensions\Cache\py36_cpu\renderutils_plugin\build.ninja...
Traceback (most recent call last):
  File "train.py", line 20, in <module>
    import src.renderutils as ru
  File "D:\NVIDIATools\nvdiffmodeling-main\src\renderutils\__init__.py", line 9, in <module>
    from .ops import xfm_points, xfm_vectors, image_loss, prepare_shading_normal, lambert, pbr_specular, pbr_bsdf, _fresnel_shlick, _ndf_ggx, _lambda_ggx, _masking_smith
  File "D:\NVIDIATools\nvdiffmodeling-main\src\renderutils\ops.py", line 61, in <module>
    torch.utils.cpp_extension.load(name='renderutils_plugin', sources=source_paths, extra_ldflags=ldflags, with_cuda=True, verbose=True)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1136, in load
    keep_intermediates=keep_intermediates)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1347, in _jit_compile
    is_standalone=is_standalone)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1445, in _write_ninja_file_and_build_library
    is_standalone=is_standalone)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1834, in _write_ninja_file_to_build_library
    cuda_flags = common_cflags + COMMON_NVCC_FLAGS + _get_cuda_arch_flags()
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1606, in _get_cuda_arch_flags
    arch_list[-1] += '+PTX'
IndexError: list index out of range

(dmodel) D:\NVIDIATools\nvdiffmodeling-main>

    After installing the prerequisites, only difference to the instruction is CUDA 11.5 (and i changed the version to the pip install line to that same version) Getting the above when attempting to run the example spot cow training.

    Something i'm doing wrong, incompatibility with versions or something else entirely?

    opened by RiversJohn 3
Releases(v1.0)
Owner
NVIDIA Research Projects
NVIDIA Research Projects
GitHub Repository
Gapmm2: gapped alignment using minimap2 (align transcripts to genome)

gapmm2: gapped alignment using minimap2 This tool is a wrapper for minimap2 to r

Jon Palmer 2 Jan 27, 2022
The best solution of the Weather Prediction track in the Yandex Shifts challenge

yandex-shifts-weather The repository contains information about my solution for the Weather Prediction track in the Yandex Shifts challenge https://re

Ivan Yu. Bondarenko 15 Dec 18, 2022
retweet 4 satoshi ⚡️

rt4sat retweet 4 satoshi This bot is the codebase for https://twitter.com/rt4sat please feel free to create an issue if you saw any bugs basically thi

6 Sep 30, 2022
Powerful and efficient Computer Vision Annotation Tool (CVAT)

Computer Vision Annotation Tool (CVAT) CVAT is free, online, interactive video and image annotation tool for computer vision. It is being used by our

OpenVINO Toolkit 8.6k Jan 01, 2023
Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit

CNTK Chat Windows build status Linux build status The Microsoft Cognitive Toolkit (https://cntk.ai) is a unified deep learning toolkit that describes

Microsoft 17.3k Dec 29, 2022
Post-Training Quantization for Vision transformers.

PTQ4ViT Post-Training Quantization Framework for Vision Transformers. We use the twin uniform quantization method to reduce the quantization error on

Zhihang Yuan 61 Dec 28, 2022
Dense matching library based on PyTorch

Dense Matching A general dense matching library based on PyTorch. For any questions, issues or recommendations, please contact Prune at

Prune Truong 399 Dec 28, 2022
Attention-guided gan for synthesizing IR images

SI-AGAN Attention-guided gan for synthesizing IR images This repository contains the Tensorflow code for "Pedestrian Gender Recognition by Style Trans

1 Oct 25, 2021
Deep Halftoning with Reversible Binary Pattern

Deep Halftoning with Reversible Binary Pattern ICCV Paper | Project Website | BibTex Overview Existing halftoning algorithms usually drop colors and f

Menghan Xia 17 Nov 22, 2022
Few-shot Learning of GPT-3

Few-shot Learning With Language Models This is a codebase to perform few-shot "in-context" learning using language models similar to the GPT-3 paper.

Tony Z. Zhao 224 Dec 28, 2022
ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectives

Status: Under development (expect bug fixes and huge updates) ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectiv

37 Dec 28, 2022
Code for technical report "An Improved Baseline for Sentence-level Relation Extraction".

RE_improved_baseline Code for technical report "An Improved Baseline for Sentence-level Relation Extraction". Requirements torch = 1.8.1 transformers

Wenxuan Zhou 74 Nov 29, 2022
Banglore House Prediction Using Flask Server (Python)

Banglore House Prediction Using Flask Server (Python) 🌐 Links 🌐 📂 Repo In this repository, I've implemented a Machine Learning-based Bangalore Hous

Dhyan Shah 1 Jan 24, 2022
[NeurIPS'21 Spotlight] PyTorch code for our paper "Aligned Structured Sparsity Learning for Efficient Image Super-Resolution"

ASSL This repository is for a new network pruning method (Aligned Structured Sparsity Learning, ASSL) for efficient single image super-resolution (SR)

Huan Wang 47 Nov 28, 2022
Python implementation of Lightning-rod Agent, the Stack4Things board-side probe

Iotronic Lightning-rod Agent Python implementation of Lightning-rod Agent, the Stack4Things board-side probe. Free software: Apache 2.0 license Websit

2 May 19, 2022
PyoMyo - Python Opensource Myo library

PyoMyo Python module for the Thalmic Labs Myo armband. Cross platform and multithreaded and works without the Myo SDK. pip install pyomyo Documentati

PerlinWarp 81 Jan 08, 2023
A high-performance distributed deep learning system targeting large-scale and automated distributed training.

HETU Documentation | Examples Hetu is a high-performance distributed deep learning system targeting trillions of parameters DL model training, develop

DAIR Lab 150 Dec 21, 2022
All supplementary material used by me while TA-ing CS3244: Machine Learning

CS3244-Tutorial-Material All supplementary material used by me while TA-ing CS3244: Machine Learning at NUS School of Computing. What is this? I teach

Rishabh Anand 18 Sep 23, 2022
PyTorch implementation of MSBG hearing loss model and MBSTOI intelligibility metric

PyTorch implementation of MSBG hearing loss model and MBSTOI intelligibility metric This repository contains the implementation of MSBG hearing loss m

BUT <a href=[email protected]"> 9 Nov 08, 2022
code for `Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation`

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation (CVPR 2021) Introduction PBR is a conceptually simple yet effective

H.Chen 143 Jan 05, 2023