Self-Supervised Learning

Last update: Dec 14, 2021

Overview

Self-Supervised Learning

Features

self_supervised offers features like

modular framework
support for multi-gpu training using PyTorch Lightning
easy to use and written in a PyTorch like style
supports custom backbone models for self-supervised pre-training

Supported Models

Supported Loss Function

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Model	Epochs	Batch Size	Test Accuracy
MoCo	800	256	0.827
SimCLR	800	256	0.847
SimSiam	800	256	0.827
BarlowTwins	800	256	0.801
BYOL	800	256	0.851

Cifar10

Model	Epochs	Batch Size	Test Accuracy
MoCo	200	128	0.83
SimCLR	200	128	0.78
SimSiam	200	128	0.73
BarlowTwins	200	128	0.84
BYOL	200	128	0.85
MoCo	200	512	0.85
SimCLR	200	512	0.83
SimSiam	200	512	0.81
BarlowTwins	200	512	0.78
BYOL	200	512	0.84
MoCo	800	128	0.89
SimCLR	800	128	0.87
SimSiam	800	128	0.80
MoCo	800	512	0.90
SimCLR	800	512	0.89
SimSiam	800	512	0.91

Tutorials

Want to jump to the tutorials and see lightly in action?

Self-Supervised Learning

Related tags

Overview

Self-Supervised Learning

Features

Supported Models

Supported Loss Function

Benchmarks

ImageNette

Cifar10

Tutorials

Further Reading

Owner

Robin

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

Text-Based Ideal Points

Streamlit component for TensorBoard, TensorFlow's visualization toolkit

Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX

Integrated physics-based and ligand-based modeling.

This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

CC-GENERATOR - A python script for generating CC

Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Gauge equivariant mesh cnn

Optimizing DR with hard negatives and achieving SOTA first-stage retrieval performance on TREC DL Track (SIGIR 2021 Full Paper).

Robbing the FED: Directly Obtaining Private Data in Federated Learning with Modified Models

Semi-supervised Implicit Scene Completion from Sparse LiDAR

[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias

This repository includes the code of the sequence-to-sequence model for discontinuous constituent parsing described in paper Discontinuous Grammar as a Foreign Language.

This repo provides a demo for the CVPR 2021 paper "A Fourier-based Framework for Domain Generalization" on the PACS dataset.

The official implementation of "Rethink Dilated Convolution for Real-time Semantic Segmentation"

This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

[CoRL 2021] A robotics benchmark for cross-embodiment imitation.

Distributed Evolutionary Algorithms in Python

Machine learning framework for both deep learning and traditional algorithms