Tensorflow 2 implementations of the C-SimCLR and C-BYOL self-supervised visual representation methods from "Compressive Visual Representations" (NeurIPS 2021)

Overview

Compressive Visual Representations

This repository contains the source code for our paper, Compressive Visual Representations. We developed information-compressed versions of the SimCLR and BYOL self-supervised learning algorithms, which we call C-SimCLR and C-BYOL, using the Conditional Entropy Bottleneck, and achieved significant improvements in accuracy and robustness, yielding linear evaluation performance competitive with fully supervised models.

cvr_perf

We include implementations of the C-SimCLR and C-BYOL algorithms developed in our paper, as well as SimCLR and BYOL baselines.

Getting Started

Install the necessary dependencies with pip install -r requirements.txt. We recommend creating a new virtual environment.

To train a model with C-SimCLR on ImageNet run bash scripts/csimclr.sh. And to train a model with C-BYOL, run bash scripts/cbyol.sh.

Refer to the scripts for further configuration options, and also to train the corresponding SimCLR and BYOL baselines.

These command lines use the hyperparameters used to train the models in our paper. In particular, we used a batch size of 4096 using 32 Cloud TPUs. Using different accelerators will require reducing the batch size. To get started with Google Cloud TPUs, we recommend following this tutorial.

Checkpoints

The following table contains pretrained checkpoints for C-SimCLR, C-BYOL and also their respective baselines, SimCLR and BYOL. All models are trained on ImageNet. The Top-1 accuracy is obtained by training a linear classifier on top of a ``frozen'' backbone whilst performing self-supervised training of the network.

Algorithm Backbone Training epochs ImageNet Top-1 Checkpoint
SimCLR ResNet 50 1000 71.1 link
SimCLR ResNet 50 2x 1000 74.6 link
C-SimCLR ResNet 50 1000 71.8 link
C-SimCLR ResNet 50 2x 1000 74.7 link
BYOL ResNet 50 1000 74.4 link
BYOL ResNet 50 2x 1000 77.3 link
C-BYOL ResNet 50 1000 75.9 link
C-BYOL ResNet 50 2x 1000 79.1 link
C-BYOL ResNet 101 1000 78.0 link
C-BYOL ResNet 152 1000 78.8 link
C-BYOL ResNet 50 1500 76.0 link

Reference

If you use C-SimCLR or C-BYOL, please use the following BibTeX entry.

@InProceedings{lee2021compressive,
  title={Compressive Visual Representations},
  author={Lee, Kuang-Huei and Arnab, Anurag and Guadarrama, Sergio and Canny, John and Fischer, Ian},
  booktitle={NeurIPS},
  year={2021}
}

Credits

This repository is based on SimCLR. We also match our BYOL implementation in Tensorflow 2 to the original implementation of BYOL in JAX.

Disclaimer: This is not an official Google product.

Owner
Google Research
Google Research
RATCHET is a Medical Transformer for Chest X-ray Diagnosis and Reporting

RATCHET: RAdiological Text Captioning for Human Examined Thoraxes RATCHET is a Medical Transformer for Chest X-ray Diagnosis and Reporting. Based on t

26 Nov 14, 2022
More than a hundred strange attractors

dysts Analyze more than a hundred chaotic systems. Basic Usage Import a model and run a simulation with default initial conditions and parameter value

William Gilpin 185 Dec 23, 2022
Pytorch implementation of Implicit Behavior Cloning.

Implicit Behavior Cloning - PyTorch (wip) Pytorch implementation of Implicit Behavior Cloning. Install conda create -n ibc python=3.8 pip install -r r

Kevin Zakka 49 Dec 25, 2022
Fuzzy Overclustering (FOC)

Fuzzy Overclustering (FOC) In real-world datasets, we need consistent annotations between annotators to give a certain ground-truth label. However, in

2 Nov 08, 2022
Official implementation of the paper Vision Transformer with Progressive Sampling, ICCV 2021.

Vision Transformer with Progressive Sampling This is the official implementation of the paper Vision Transformer with Progressive Sampling, ICCV 2021.

yuexy 123 Jan 01, 2023
Using pretrained GROVER to extract the atomic fingerprints from molecule

Extracting atomic fingerprints from molecules using pretrained Graph Neural Network models (GROVER).

Xuan Vu Nguyen 1 Jan 28, 2022
Pytorch code for ICRA'21 paper: "Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation"

Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation This repository is the pytorch implementation of our paper: Hierarchical Cr

43 Nov 21, 2022
Official source code of paper 'IterMVS: Iterative Probability Estimation for Efficient Multi-View Stereo'

IterMVS official source code of paper 'IterMVS: Iterative Probability Estimation for Efficient Multi-View Stereo' Introduction IterMVS is a novel lear

Fangjinhua Wang 127 Jan 04, 2023
MQBench Quantization Aware Training with PyTorch

MQBench Quantization Aware Training with PyTorch I am using MQBench(Model Quantization Benchmark)(http://mqbench.tech/) to quantize the model for depl

Ling Zhang 29 Nov 18, 2022
Makes patches from huge resolution .svs slide files using openslide

openslide_patcher Makes patches from huge resolution .svs slide files using openslide Example collage I made from outputs:

2 Dec 23, 2021
Official PyTorch implementation of the paper Image-Based CLIP-Guided Essence Transfer.

TargetCLIP- official pytorch implementation of the paper Image-Based CLIP-Guided Essence Transfer This repository finds a global direction in StyleGAN

Hila Chefer 221 Dec 13, 2022
Embeddinghub is a database built for machine learning embeddings.

Embeddinghub is a database built for machine learning embeddings.

Featureform 1.2k Jan 01, 2023
Supervised Contrastive Learning for Downstream Optimized Sequence Representations

SupCL-Seq 📖 Supervised Contrastive Learning for Downstream Optimized Sequence representations (SupCS-Seq) accepted to be published in EMNLP 2021, ext

Hooman Sedghamiz 18 Oct 21, 2022
Predicts an answer in yes or no.

Oui-ou-non-prediction Predicts an answer in 'yes' or 'no'. It is based on the game 'effeuiller la marguerite' in which the person plucks flower petals

Ananya Gupta 1 Jan 15, 2022
SoGCN: Second-Order Graph Convolutional Networks

SoGCN: Second-Order Graph Convolutional Networks This is the authors' implementation of paper "SoGCN: Second-Order Graph Convolutional Networks" in Py

Yuehao 7 Aug 16, 2022
TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition

TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition Xue, Wenyuan, et al. "TGRNet: A Table Graph Reconstruction Network for Ta

Wenyuan 68 Jan 04, 2023
A novel benchmark dataset for Monocular Layout prediction

AutoLay AutoLay: Benchmarking Monocular Layout Estimation Kaustubh Mani, N. Sai Shankar, J. Krishna Murthy, and K. Madhava Krishna Abstract In this pa

Kaustubh Mani 39 Apr 26, 2022
Instance Semantic Segmentation List

Instance Semantic Segmentation List This repository contains lists of state-or-art instance semantic segmentation works. Papers and resources are list

bighead 87 Mar 06, 2022
The sixth place winning solution (6/220) in 2021 Gaofen Challenge.

SwinTransformer + OBBDet The sixth place winning solution (6/220) in the track of Fine-grained Object Recognition in High-Resolution Optical Images, 2

ming71 46 Dec 02, 2022
Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

tf-fsvd TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions Cite If you f

Sami Abu-El-Haija 14 Nov 25, 2021