Self-Supervised Learning

Last update: Dec 14, 2021

Overview

Self-Supervised Learning

Features

self_supervised offers features like

modular framework
support for multi-gpu training using PyTorch Lightning
easy to use and written in a PyTorch like style
supports custom backbone models for self-supervised pre-training

Supported Models

Supported Loss Function

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Model	Epochs	Batch Size	Test Accuracy
MoCo	800	256	0.827
SimCLR	800	256	0.847
SimSiam	800	256	0.827
BarlowTwins	800	256	0.801
BYOL	800	256	0.851

Cifar10

Model	Epochs	Batch Size	Test Accuracy
MoCo	200	128	0.83
SimCLR	200	128	0.78
SimSiam	200	128	0.73
BarlowTwins	200	128	0.84
BYOL	200	128	0.85
MoCo	200	512	0.85
SimCLR	200	512	0.83
SimSiam	200	512	0.81
BarlowTwins	200	512	0.78
BYOL	200	512	0.84
MoCo	800	128	0.89
SimCLR	800	128	0.87
SimSiam	800	128	0.80
MoCo	800	512	0.90
SimCLR	800	512	0.89
SimSiam	800	512	0.91

Tutorials

Want to jump to the tutorials and see lightly in action?

Self-Supervised Learning

Related tags

Overview

Self-Supervised Learning

Features

Supported Models

Supported Loss Function

Benchmarks

ImageNette

Cifar10

Tutorials

Further Reading

Owner

Robin

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation

Code for a seq2seq architecture with Bahdanau attention designed to map stereotactic EEG data from human brains to spectrograms, using the PyTorch Lightning.

Code of Puregaze: Purifying gaze feature for generalizable gaze estimation, AAAI 2022.

End-to-End Speech Processing Toolkit

Official repository of "Investigating Tradeoffs in Real-World Video Super-Resolution"

A program to recognize fruits on pictures or videos using yolov5

A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.

Breaking the Dilemma of Medical Image-to-image Translation

Official implementation of "Membership Inference Attacks Against Self-supervised Speech Models"

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection

meProp: Sparsified Back Propagation for Accelerated Deep Learning

Unofficial PyTorch implementation of Neural Additive Models (NAM) by Agarwal, et al.

Deploy recommendation engines with Edge Computing

Open source code for the paper of Neural Sparse Voxel Fields.

SIEM Logstash parsing for more than hundred technologies

Pytorch for Segmentation

PyTorch implementation of the Quasi-Recurrent Neural Network - up to 16 times faster than NVIDIA's cuDNN LSTM

PyTorch reimplementation of hand-biomechanical-constraints (ECCV2020)

🕹️ Official Implementation of Conditional Motion In-betweening (CMIB) 🏃

Official codebase for Decision Transformer: Reinforcement Learning via Sequence Modeling.