Self-Supervised Learning

Last update: Dec 14, 2021

Overview

Self-Supervised Learning

Features

self_supervised offers features like

modular framework
support for multi-gpu training using PyTorch Lightning
easy to use and written in a PyTorch like style
supports custom backbone models for self-supervised pre-training

Supported Models

Supported Loss Function

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Model	Epochs	Batch Size	Test Accuracy
MoCo	800	256	0.827
SimCLR	800	256	0.847
SimSiam	800	256	0.827
BarlowTwins	800	256	0.801
BYOL	800	256	0.851

Cifar10

Model	Epochs	Batch Size	Test Accuracy
MoCo	200	128	0.83
SimCLR	200	128	0.78
SimSiam	200	128	0.73
BarlowTwins	200	128	0.84
BYOL	200	128	0.85
MoCo	200	512	0.85
SimCLR	200	512	0.83
SimSiam	200	512	0.81
BarlowTwins	200	512	0.78
BYOL	200	512	0.84
MoCo	800	128	0.89
SimCLR	800	128	0.87
SimSiam	800	128	0.80
MoCo	800	512	0.90
SimCLR	800	512	0.89
SimSiam	800	512	0.91

Tutorials

Want to jump to the tutorials and see lightly in action?

Self-Supervised Learning

Related tags

Overview

Self-Supervised Learning

Features

Supported Models

Supported Loss Function

Benchmarks

ImageNette

Cifar10

Tutorials

Further Reading

Owner

Robin

This program presents convolutional kernel density estimation, a method used to detect intercritical epilpetic spikes (IEDs)

Exploring Classification Equilibrium in Long-Tailed Object Detection, ICCV2021

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

An implementation of Deep Forest 2021.2.1.

An implementation of the methods presented in Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data.

Deep learning for Engineers - Physics Informed Deep Learning

This project aims at providing a concise, easy-to-use, modifiable reference implementation for semantic segmentation models using PyTorch.

DA2Lite is an automated model compression toolkit for PyTorch.

Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Features"

This repository provides code for "On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness".

Paddle pit - Rethinking Spatial Dimensions of Vision Transformers

CLASP - Contrastive Language-Aminoacid Sequence Pretraining

Developing your First ML Workflow of the AWS Machine Learning Engineer Nanodegree Program

An Exact Solver for Semi-supervised Minimum Sum-of-Squares Clustering

2021 CCF BDCI 全国信息检索挑战杯（CCIR-Cup）智能人机交互自然语言理解赛道第二名参赛解决方案

All materials of Cassandra Event, Udyam'22

Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT

Code for "Modeling Indirect Illumination for Inverse Rendering", CVPR 2022

A Collection of Papers and Codes for ICCV2021 Low Level Vision and Image Generation

PyTorch DepthNet Training on Still Box dataset