WarpRNNT loss ported in Numba CPU/CUDA for Pytorch

Last update: Oct 22, 2022

Related tags

Overview

RNNT loss in Pytorch - Numba JIT compiled (warprnnt_numba)

Warp RNN Transducer Loss for ASR in Pytorch, ported from HawkAaron/warp-transducer and a replica of the stable version in NVIDIA Neural Module repository (NVIDIA NeMo).

NOTE: The code here will have experimental extensions and may be potentially unstable, use the version in NeMo for long term supported loss version of RNNT for PyTorch.

Supported Features

Currently supports :

WarpRNNT loss in pytorch for CPU / CUDA (jit compiled)
FastEmit
Gradient Clipping (from Torch Audio)

Installation

You will need PyTorch (usually the latest version should be used), plus installation of Numba in a Conda environment (pip only environment is untested but may work).

# Follow installation instructions to install pytorch from website (with cuda if required)
conda install -c conda-force numba or conda update -c conda-forge numba (to get latest version)

# Then install this library
pip install --upgrade git+https://github.com/titu1994/warprnnt_numba.git

Usage

Import warprnnt_numba and use RNNTLossNumba. If attempting to use CUDA version of loss, it is advisable to test that your installed CUDA version is compatible with numba version using numba_utils.

There is also included a very slow numpy/pytorch explicit-loop based loss implementation for verification of exact correct results.

import torch
import numpy as np
import warprnnt_numba

# Define the loss function
fastemit_lambda = 0.001  # any float >= 0.0
loss_pt = warprnnt_numba.RNNTLossNumba(blank=4, reduction='sum', fastemit_lambda=fastemit_lambda)

# --------------
# Example usage

device = "cuda"
torch.random.manual_seed(0)

# Assume Batchsize=2, Acoustic Timesteps = 8, Label Timesteps = 5 (including BLANK=BOS token),
# and Vocabulary size of 5 tokens (including RNNT BLANK)
acts = torch.randn(2, 8, 5, 5, device=device, requires_grad=True)
sequence_length = torch.tensor([5, 8], dtype=torch.int32,
                               device=device)  # acoustic sequence length. One element must be == acts.shape[1].

# Let 0 be MASK/PAD value, 1-3 be token ids, and 4 represent RNNT BLANK token
# The BLANK token is overloaded for BOS token as well here, but can be different token.
# Let first sample be padded with 0 (actual length = 3). Loss is computed according to supplied `label_lengths`.
# and gradients for the 4th index onwards (0 based indexing).
labels = torch.tensor([[4, 1, 1, 3, 0], [4, 2, 2, 3, 1]], dtype=torch.int32, device=device)
label_lengths = torch.tensor([3, 4], dtype=torch.int32,
                             device=device)  # Lengths here must be WITHOUT the BOS token.

# If on CUDA, log_softmax is computed internally efficiently (preserving memory and speed)
# Compute it explicitly for CPU, this is done automatically for you inside forward() of the loss.
# -1-th vocab index is RNNT blank token here.
loss_func = warprnnt_numba.RNNTLossNumba(blank=4, reduction='none',
                                         fastemit_lambda=0.0, clamp=0.0)
loss = loss_func(acts, labels, sequence_length, label_lengths)
print("Loss :", loss)
loss.sum().backward()

# When parsing the gradients, look at grads[0] -
# Since it was padded in T (sequence_length=5 < T=8), there are gradients only for grads[0, :5, :, :].
# Since it was padded in U (label_lengths=3+1 < U=5), there are gradeints only for grads[0, :5, :3+1, :].
grads = acts.grad
print("Gradients of activations :")
print(grads)

Tests

Tests will perform CPU only checks if there are no GPUs. If GPUs are present, will run all tests once for cuda:0 as well.

pytest tests/

Requirements

pytorch >= 1.10. Older versions might work, not tested.
numba - Minimum required version is 0.53.0, preferred is 0.54+.

Comments

GPU under utilization due to low occupancy.

Thank you for the warprnnt_numba, I got the warnning (show blow) when I use this loss in my code. Is this known issue? How can it be debugged and solved?

Thank you!

opened by jiay7 2
Fix runtime speed
Improve runtime speed of numba loss

Fix issue with data movement of costs tensor from llForward to pytorch data view in numba

This alone costs a linear loop (scaling with batch size) that is roughly 10x the kernel costs themselves.

Fix by writing a small kernel to copy the data and update the costs.
opened by titu1994 0

Releases(v0.4.0)

v0.4.0(Jan 30, 2022)
Supports

Simple RNNT loss with Atomic Locks implementation

Improvements

Improve runtime speed of numba loss

Fix issue with data movement of costs tensor from llForward to pytorch data view in numba

This alone costs a linear loop (scaling with batch size) that is roughly 10x the kernel costs themselves.

Fix by writing a small kernel to copy the data and update the costs.

Source code(tar.gz)
Source code(zip)
v0.2.2(Jan 24, 2022)
Initial release of Warp RNNT loss with Numba JIT compile (CPU/CUDA)

Supports:

Pytorch RNNT loss (CPU and JIT compiled CUDA)

FastEmit

Gradient clipping

Source code(tar.gz)
Source code(zip)

WarpRNNT loss ported in Numba CPU/CUDA for Pytorch

Related tags

Overview

RNNT loss in Pytorch - Numba JIT compiled (warprnnt_numba)

Supported Features

Installation

Usage

Tests

Requirements

You might also like...

This Repo is the official CUDA implementation of ICCV 2019 Oral paper for CARAFE: Content-Aware ReAssembly of FEatures

Example repository for custom C++/CUDA operators for TorchScript

Convert Python 3 code to CUDA code.

This demo showcase the use of onnxruntime-rs with a GPU on CUDA 11 to run Bert in a data pipeline with Rust.

LightSeq is a high performance training and inference library for sequence processing and generation implemented in CUDA

CUDA Python Low-level Bindings

A dead simple python wrapper for darknet that works with OpenCV 4.1, CUDA 10.1

Prevent `CUDA error: out of memory` in just 1 line of code.

An addernet CUDA version

Comments

GPU under utilization due to low occupancy.

Fix runtime speed

Improve runtime speed of numba loss

Releases(v0.4.0)

v0.4.0(Jan 30, 2022)

Supports

Improvements

v0.2.2(Jan 24, 2022)

Owner

Somshubra Majumdar

Sequential GCN for Active Learning

Spearmint Bayesian optimization codebase

AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation

Este conversor criará a medida exata para sua receita de capuccino gelado da grandiosa Rafaella Ballerini!

RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

An open-source project for applying deep learning to medical scenarios

Open source implementation of "A Self-Supervised Descriptor for Image Copy Detection" (SSCD).

Official pytorch implementation of paper Dual-Level Collaborative Transformer for Image Captioning (AAAI 2021).

Language Used: Python . Made in Jupyter(Anaconda) notebook.

Model Zoo for MindSpore

Pytorch implementation of "A simple neural network module for relational reasoning" (Relational Networks)

RL-driven agent playing tic-tac-toe on starknet against challengers.

Dilated RNNs in pytorch

Implementation of "A MLP-like Architecture for Dense Prediction"

Weight initialization schemes for PyTorch nn.Modules

💊 A 3D Generative Model for Structure-Based Drug Design (NeurIPS 2021)

Contextual Attention Localization for Offline Handwritten Text Recognition

TumorInsight is a Brain Tumor Detection and Classification model built using RESNET50 architecture.

A Fast Knowledge Distillation Framework for Visual Recognition

Code from PropMix, accepted at BMVC'21