Self-Supervised Learning

Last update: Dec 14, 2021

Overview

Self-Supervised Learning

Features

self_supervised offers features like

modular framework
support for multi-gpu training using PyTorch Lightning
easy to use and written in a PyTorch like style
supports custom backbone models for self-supervised pre-training

Supported Models

Supported Loss Function

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Model	Epochs	Batch Size	Test Accuracy
MoCo	800	256	0.827
SimCLR	800	256	0.847
SimSiam	800	256	0.827
BarlowTwins	800	256	0.801
BYOL	800	256	0.851

Cifar10

Model	Epochs	Batch Size	Test Accuracy
MoCo	200	128	0.83
SimCLR	200	128	0.78
SimSiam	200	128	0.73
BarlowTwins	200	128	0.84
BYOL	200	128	0.85
MoCo	200	512	0.85
SimCLR	200	512	0.83
SimSiam	200	512	0.81
BarlowTwins	200	512	0.78
BYOL	200	512	0.84
MoCo	800	128	0.89
SimCLR	800	128	0.87
SimSiam	800	128	0.80
MoCo	800	512	0.90
SimCLR	800	512	0.89
SimSiam	800	512	0.91

Tutorials

Want to jump to the tutorials and see lightly in action?

Self-Supervised Learning

Related tags

Overview

Self-Supervised Learning

Features

Supported Models

Supported Loss Function

Benchmarks

ImageNette

Cifar10

Tutorials

Further Reading

Owner

Robin

PyTorch implementation for "HyperSPNs: Compact and Expressive Probabilistic Circuits", NeurIPS 2021

This is the repo for the paper "Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement".

Pytorch implementation of our paper under review -- 1xN Pattern for Pruning Convolutional Neural Networks

Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data - Official PyTorch Implementation (CVPR 2022)

Training a deep learning model on the noisy CIFAR dataset

Neighborhood Reconstructing Autoencoders

[CVPR'20] TTSR: Learning Texture Transformer Network for Image Super-Resolution

Distributionally robust neural networks for group shifts

This project is used for the paper Differentiable Programming of Isometric Tensor Network

Sentiment analysis translations of the Bhagavad Gita

Code for a real-time distributed cooperative slam(RDC-SLAM) system for ROS compatible platforms.

Tutorial for the PERFECTING FACTORY 5.0 WITH EDGE-POWERED AI workshop

For visualizing the dair-v2x-i dataset

A high-performance distributed deep learning system targeting large-scale and automated distributed training.

[AAAI 2022] Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding

This is the official repository for our paper: ''Pruning Self-attentions into Convolutional Layers in Single Path''.

Deep ViT Features as Dense Visual Descriptors

Implementation of Nalbach et al. 2017 paper.

Method for facial emotion recognition compitition of Xunfei and Datawhale .

Face Alignment using python