Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

Overview

Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

Authors:

*: Equal Contribution

Introduction

This repo contains implementation of the group VSA and binary HDC model with random Fourier feature (RFF) encoding, described in the paper Understanding Hyperdimensional Computing for Parallel Single-Pass Learning.

Our RFF method and group VSA can outperform the state-of-the-art HDC model while maintaining hardware efficiency. For example, on MNIST,

Model 1-Epoch Accuracy 10-Epoch Accuracy Circuit-Depth Complexity
Percep. 94.3 % 94.3 % 1299
SOTA HDC NA 89.0 % 295
RFF HDC 95.4 % 95.4 % 295
RFF G(2^3)-VSA 96.3 % 95.7 % 405

Dependencies and Data

Numpy and PyTorch>=1.0.0 are required to run the implementation. Supported datasets include MNIST, Fashion-MNIST, CIFAR-10, ISOLET and UCI-HAR. We provide the ISOLET and UCI-HAR data in dataset folder.

Usage

Please create the ./encoded_data folder before running the following code.

$ python main.py [-h] [-lr LR] [-gamma GAMMA] [-epoch EPOCH] [-gorder GORDER] [-dim DIM] 
[-data_dir DATA_DIR] [-model MODEL]
optional arguments:
  -h, --help            show this help message and exit
  -lr LR                learning rate for optimizing class representative
  -gamma GAMMA          kernel parameter for computing covariance
  -epoch EPOCH          epochs of training
  -gorder GORDER        order of the cyclic group required for G-VSA
  -dim DIM              dimension of hypervectors
  -resume               resume from existing encoded hypervectors
  -data_dir DATA_DIR    Directory used to save encoded data (hypervectors)
  -dataset {mnist,fmnist,cifar,isolet,ucihar}
                        dataset (mnist | fmnist | cifar | isolet | ucihar)
  -raw_data_dir RAW_DATA_DIR
                        Raw data directory to the dataset
  -model {rff-hdc,linear-hdc,rff-gvsa}
                        feature and model to use: (rff-hdc | linear-hdc | rff-gvsa)

For example,

$ python main.py -gamma 0.3 -epoch 10 -gorder 8 -dim 10000 -dataset mnist -model rff-gvsa

Citation

If you find this repo useful, please cite:


Owner
Cornell RelaxML
Chris De Sa's Research Group
Cornell RelaxML
PyTorch implementation of the paper: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features

Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features Estimate the noise transition matrix with f-mutual information. This co

<a href=[email protected]"> 1 Jun 05, 2022
Simple-System-Convert--C--F - Simple System Convert With Python

Simple-System-Convert--C--F REQUIREMENTS Python version : 3 HOW TO USE Run the c

Jonathan Santos 2 Feb 16, 2022
A general-purpose programming language, focused on simplicity, safety and stability.

The Rivet programming language A general-purpose programming language, focused on simplicity, safety and stability. Rivet's goal is to be a very power

The Rivet programming language 17 Dec 29, 2022
Flask101 - FullStack Web Development with Python & JS - From TAQWA

Task: Create a CLI Calculator Step 0: Creating Virtual Environment $ python -m

Hossain Foysal 1 May 31, 2022
Corruption Invariant Learning for Re-identification

Corruption Invariant Learning for Re-identification The official repository for Benchmarks for Corruption Invariant Person Re-identification (NeurIPS

Minghui Chen 73 Dec 08, 2022
RefineMask (CVPR 2021)

RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features (CVPR 2021) This repo is the official implementation of RefineMask:

Gang Zhang 191 Jan 07, 2023
FluxTraining.jl gives you an endlessly extensible training loop for deep learning

A flexible neural net training library inspired by fast.ai

86 Dec 31, 2022
A Novel Incremental Learning Driven Instance Segmentation Framework to Recognize Highly Cluttered Instances of the Contraband Items

A Novel Incremental Learning Driven Instance Segmentation Framework to Recognize Highly Cluttered Instances of the Contraband Items This repository co

Taimur Hassan 3 Mar 16, 2022
Pgn2tex - Scripts to convert pgn files to latex document. Useful to build books or pdf from pgn studies

Pgn2Latex (WIP) A simple script to make pdf from pgn files and studies. It's sti

12 Jul 23, 2022
A pre-trained model with multi-exit transformer architecture.

ElasticBERT This repository contains finetuning code and checkpoints for ElasticBERT. Towards Efficient NLP: A Standard Evaluation and A Strong Baseli

fastNLP 48 Dec 14, 2022
Source code for the ACL-IJCNLP 2021 paper entitled "T-DNA: Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation" by Shizhe Diao et al.

T-DNA Source code for the ACL-IJCNLP 2021 paper entitled Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adapta

shizhediao 17 Dec 22, 2022
Code and dataset for AAAI 2021 paper FixMyPose: Pose Correctional Describing and Retrieval Hyounghun Kim, Abhay Zala, Graham Burri, Mohit Bansal.

FixMyPose / फिक्समाइपोज़ Code and dataset for AAAI 2021 paper "FixMyPose: Pose Correctional Describing and Retrieval" Hyounghun Kim*, Abhay Zala*, Grah

4 Sep 19, 2022
Implementation of CaiT models in TensorFlow and ImageNet-1k checkpoints. Includes code for inference and fine-tuning.

CaiT-TF (Going deeper with Image Transformers) This repository provides TensorFlow / Keras implementations of different CaiT [1] variants from Touvron

Sayak Paul 9 Jun 26, 2022
The official code of "SCROLLS: Standardized CompaRison Over Long Language Sequences".

SCROLLS This repository contains the official code of the paper: "SCROLLS: Standardized CompaRison Over Long Language Sequences". Links Official Websi

TAU NLP Group 39 Dec 23, 2022
Train Scene Graph Generation for Visual Genome and GQA in PyTorch >= 1.2 with improved zero and few-shot generalization.

Scene Graph Generation Object Detections Ground truth Scene Graph Generated Scene Graph In this visualization, woman sitting on rock is a zero-shot tr

Boris Knyazev 93 Dec 28, 2022
Neural Network to colorize grayscale images

#colornet Neural Network to colorize grayscale images Results Grayscale Prediction Ground Truth Eiji K used colornet for anime colorization Sources Au

Pavel Hanchar 3.6k Dec 24, 2022
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022
Cross View SLAM

Cross View SLAM This is the associated code and dataset repository for our paper I. D. Miller et al., "Any Way You Look at It: Semantic Crossview Loca

Ian D. Miller 99 Dec 09, 2022
Official code release for ICCV 2021 paper SNARF: Differentiable Forward Skinning for Animating Non-rigid Neural Implicit Shapes.

Official code release for ICCV 2021 paper SNARF: Differentiable Forward Skinning for Animating Non-rigid Neural Implicit Shapes.

235 Dec 26, 2022
EvoJAX is a scalable, general purpose, hardware-accelerated neuroevolution toolkit

EvoJAX: Hardware-Accelerated Neuroevolution EvoJAX is a scalable, general purpose, hardware-accelerated neuroevolution toolkit. Built on top of the JA

Google 598 Jan 07, 2023