Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Last update: Dec 06, 2022

Related tags

Overview

Fastformer-PyTorch

Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need.

Usage :

import torch
import Fastformer

model = Fastformer(dim = 3, decode_dim = 8)
x = torch.randn(4, 6, 3)
mask = torch.ones(1, 8).bool()
result = model(x, mask)
print(result.size())

Citation :

@misc{wu2021fastformer,
    title={Fastformer: Additive Attention Can Be All You Need},
    author={Chuhan Wu, Fangzhao Wu, Tao Qi and Yongfeng Huang},
    year={2021},
    eprint={2108.09084v2},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

If this implement have any problem please let me know, thank you.

Owner

Hong-Jia Chen

Master student at National Chung Cheng University, Taiwan. Interested in Deep Learning and Computer Vision.

GitHub Repository

A new codebase for Group Activity Recognition. It contains codes for ICCV 2021 paper: Spatio-Temporal Dynamic Inference Network for Group Activity Recognition and some other methods.

Spatio-Temporal Dynamic Inference Network for Group Activity Recognition The source codes for ICCV2021 Paper: Spatio-Temporal Dynamic Inference Networ

40 Dec 12, 2022

Hierarchical Attentive Recurrent Tracking

Hierarchical Attentive Recurrent Tracking This is an official Tensorflow implementation of single object tracking in videos by using hierarchical atte

147 Aug 07, 2021

A PyTorch library for Vision Transformers

VFormer A PyTorch library for Vision Transformers Getting Started Read the contributing guidelines in CONTRIBUTING.rst to learn how to start contribut

142 Nov 28, 2022

Help you understand Manual and w/ Clutch point while driving.

简体中文 forza_auto_gear forza_auto_gear is a tool for Forza Horizon 5. It will help us understand the best gear shift point using Manual or w/ Clutch in

15 Oct 08, 2022

Code for "Unsupervised State Representation Learning in Atari"

Unsupervised State Representation Learning in Atari Ankesh Anand*, Evan Racah*, Sherjil Ozair*, Yoshua Bengio, Marc-Alexandre Côté, R Devon Hjelm This

217 Jan 03, 2023

Python-kafka-reset-consumergroup-offset-example - Python Kafka reset consumergroup offset example

Python Kafka reset consumergroup offset example This is a simple example of how

1 Feb 16, 2022

v objective diffusion inference code for PyTorch.

v-diffusion-pytorch v objective diffusion inference code for PyTorch, by Katherine Crowson (@RiversHaveWings) and Chainbreakers AI (@jd_pressman). The

635 Dec 30, 2022

Pervasive Attention: 2D Convolutional Networks for Sequence-to-Sequence Prediction

This is a fork of Fairseq(-py) with implementations of the following models: Pervasive Attention - 2D Convolutional Neural Networks for Sequence-to-Se

490 Dec 15, 2022

FLVIS: Feedback Loop Based Visual Initial SLAM

FLVIS Feedback Loop Based Visual Inertial SLAM 1-Video EuRoC DataSet MH_05 Handheld Test in Lab FlVIS on UAV Platform 2-Relevent Publication: Under Re

182 Dec 04, 2022

🔥 Cannlytics-powered artificial intelligence 🤖

Cannlytics AI 🔥 Cannlytics-powered artificial intelligence 🤖 🏗️ Installation 🏃‍♀️ Quickstart 🧱 Development 🦾 Automation 💸 Support 🏛️ License ?

3 Nov 11, 2022

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto

145 Dec 30, 2022

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Related tags

Overview

Fastformer-PyTorch

Usage :

Citation :

If this implement have any problem please let me know, thank you.

Owner

Hong-Jia Chen

A new codebase for Group Activity Recognition. It contains codes for ICCV 2021 paper: Spatio-Temporal Dynamic Inference Network for Group Activity Recognition and some other methods.

Hierarchical Attentive Recurrent Tracking

A PyTorch library for Vision Transformers

Help you understand Manual and w/ Clutch point while driving.

Code for "Unsupervised State Representation Learning in Atari"

Python-kafka-reset-consumergroup-offset-example - Python Kafka reset consumergroup offset example

v objective diffusion inference code for PyTorch.

Pervasive Attention: 2D Convolutional Networks for Sequence-to-Sequence Prediction

FLVIS: Feedback Loop Based Visual Initial SLAM

🔥 Cannlytics-powered artificial intelligence 🤖

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

CIFAR-10_train-test - training and testing codes for dataset CIFAR-10

A simple, clean TensorFlow implementation of Generative Adversarial Networks with a focus on modeling illustrations.

Official implementation for "Image Quality Assessment using Contrastive Learning"

iBOT: Image BERT Pre-Training with Online Tokenizer

Official implementation of paper "Query2Label: A Simple Transformer Way to Multi-Label Classification".

Official implementation of FCL-taco2: Fast, Controllable and Lightweight version of Tacotron2 @ ICASSP 2021

This is the official implementation of VaxNeRF (Voxel-Accelearated NeRF).

Camview - A CLI-tool used to stream CCTV online footage based on URL params

Fast image augmentation library and an easy-to-use wrapper around other libraries