Bootstrapped Representation Learning on Graphs

Related tags

Deep Learningbgrl
Overview

Bootstrapped Representation Learning on Graphs

Overview of BGRL

This is the PyTorch implementation of BGRL Bootstrapped Representation Learning on Graphs

The main scripts are train_transductive.py and train_ppi.py used for training on the transductive task datasets and the PPI dataset respectively.

For linear evaluation, using the checkpoints we provide

Setup

To set up a Python virtual environment with the required dependencies, run:

python3 -m venv bgrl_env
source bgrl_env/bin/activate
pip install --upgrade pip

Follow instructions to install PyTorch 1.9.1 and PyG:

pip install torch==1.9.1+cu111 -f https://download.pytorch.org/whl/torch_stable.html
pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.9.0+cu111.html
pip install absl-py==0.12.0 tensorboard==2.6.0 ogb

The code uses PyG (PyTorch Geometric). All datasets are available through this package.

Experiments on transductive tasks

Train model from scratch

To run BGRL on a dataset from the transductive setting, use train_transductive.py and one of the configuration files that can be found in config/.

For example, to train on the Coauthor-CS dataset, use the following command:

python3 train_transductive.py --flagfile=config/coauthor-cs.cfg

Flags can be overwritten:

python3 train_transductive.py --flagfile=config/coauthor-cs.cfg\
                              --logdir=./runs/coauthor-cs-256\
                              --predictor_hidden_size=256

Evaluation is performed periodically during training. We fit a logistic regression model on top of the representation to assess its performance throughout training. Evaluation is triggered every eval_epochsand will not back-propagate any gradient to the encoder.

Test accuracies under linear evaluation are reported on TensorBoard. To start the tensorboard server run the following command:

tensorboard --logdir=./runs

Perform linear evaluation using the provided model weights

The configuration files we provide allow to reproduce the results in the paper, summarized in the table below. We also provide weights of the BGRL-trained encoders for each dataset.

WikiCS Amazon Computers Amazon Photos CoauthorCS CoauthorPhy
BGRL 79.98 ± 0.10
(weights)
90.34 ± 0.19
(weights)
93.17 ± 0.30
(weights)
93.31 ± 0.13
(weights)
95.73 ± 0.05
(weights)

To run linear evaluation, using the provided weights, run the following command for any of the datasets:

python3 linear_eval_transductive.py --flagfile=config-eval/coauthor-cs.cfg

Note that the dataset is split randomly between train/val/test, so the reported accuracy might be slightly different with each run. In our reported table, we average across multiple splits, as well as multiple randomly initialized network weights.

Experiments on inductive task with multiple graphs

To train on the PPI dataset, use train_ppi.py:

python3 train_ppi.py --flagfile=config/ppi.cfg

The evaluation for PPI is different due to the size of the dataset, we evaluate by training a linear layer on top of the representations via gradient descent for 100 steps.

The configuration files for the different architectures can be found in config/. We provide weights of the BGRL-trained encoder as well.

PPI
BGRL 69.41 ± 0.15 (weights)

To run linear evaluation, using the provided weights, run the following command:

python3 linear_eval_ppi.py --flagfile=config-eval/ppi.cfg

Note that our reported score is based on an average over multiple runs.

Citation

If you find the code useful for your research, please consider citing our work:

@misc{thakoor2021bootstrapped,
     title={Large-Scale Representation Learning on Graphs via Bootstrapping}, 
     author={Shantanu Thakoor and Corentin Tallec and Mohammad Gheshlaghi Azar and Mehdi Azabou and Eva L. Dyer and Rémi Munos and Petar Veličković and Michal Valko},
     year={2021},
     eprint={2102.06514},
     archivePrefix={arXiv},
     primaryClass={cs.LG}}
Owner
NerDS Lab :: Neural Data Science Lab
machine learning and neuroscience
NerDS Lab :: Neural Data Science Lab
End-to-end machine learning project for rices detection

Basmatinet Welcome to this project folks ! Whether you like it or not this project is all about riiiiice or riz in french. It is also about Deep Learn

Béranger 47 Jun 18, 2022
LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

LWCC: A LightWeight Crowd Counting library for Python LWCC is a lightweight crowd counting framework for Python. It wraps four state-of-the-art models

Matija Teršek 39 Dec 28, 2022
The implementation our EMNLP 2021 paper "Enhanced Language Representation with Label Knowledge for Span Extraction".

LEAR The implementation our EMNLP 2021 paper "Enhanced Language Representation with Label Knowledge for Span Extraction". See below for an overview of

杨攀 93 Jan 07, 2023
Good Semi-Supervised Learning That Requires a Bad GAN

Good Semi-Supervised Learning that Requires a Bad GAN This is the code we used in our paper Good Semi-supervised Learning that Requires a Bad GAN Ziha

Zhilin Yang 177 Dec 12, 2022
python library for invisible image watermark (blind image watermark)

invisible-watermark invisible-watermark is a python library and command line tool for creating invisible watermark over image.(aka. blink image waterm

Shield Mountain 572 Jan 07, 2023
Code for the paper "JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design"

JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design This repository contains code for the paper: JA

Aspuru-Guzik group repo 55 Nov 29, 2022
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research

MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research

Facebook Research 338 Dec 29, 2022
A Pytorch Implementation of Domain adaptation of object detector using scissor-like networks

A Pytorch Implementation of Domain adaptation of object detector using scissor-like networks Please follow Faster R-CNN and DAF to complete the enviro

2 Oct 07, 2022
POT : Python Optimal Transport

POT: Python Optimal Transport This open source Python library provide several solvers for optimization problems related to Optimal Transport for signa

Python Optimal Transport 1.7k Dec 31, 2022
A variational Bayesian method for similarity learning in non-rigid image registration (CVPR 2022)

A variational Bayesian method for similarity learning in non-rigid image registration We provide the source code and the trained models used in the re

daniel grzech 14 Nov 21, 2022
Awesome Monocular 3D detection

Awesome Monocular 3D detection Paper list of 3D detetction, keep updating! Contents Paper List 2022 2021 2020 2019 2018 2017 2016 KITTI Results Paper

Zhikang Zou 184 Jan 04, 2023
[WWW 2022] Zero-Shot Stance Detection via Contrastive Learning

PT-HCL for Zero-Shot Stance Detection The code of this repository is constantly being updated... Please look forward to it! Introduction This reposito

Akuchi 12 Dec 21, 2022
source code of “Visual Saliency Transformer” (ICCV2021)

Visual Saliency Transformer (VST) source code for our ICCV 2021 paper “Visual Saliency Transformer” by Nian Liu, Ni Zhang, Kaiyuan Wan, Junwei Han, an

89 Dec 21, 2022
CoANet: Connectivity Attention Network for Road Extraction From Satellite Imagery

CoANet: Connectivity Attention Network for Road Extraction From Satellite Imagery This paper (CoANet) has been published in IEEE TIP 2021. This code i

Jie Mei 53 Dec 03, 2022
Efficient Training of Audio Transformers with Patchout

PaSST: Efficient Training of Audio Transformers with Patchout This is the implementation for Efficient Training of Audio Transformers with Patchout Pa

165 Dec 26, 2022
The reference baseline of final exam for XMU machine learning course

Mini-NICO Baseline The baseline is a reference method for the final exam of machine learning course. Requirements Installation we use /python3.7 /torc

JoaquinChou 3 Dec 29, 2021
Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

This repo is the official implementation of "Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework". @inproceedings{zhou2021insta

34 Dec 31, 2022
KIDA: Knowledge Inheritance in Data Aggregation

KIDA: Knowledge Inheritance in Data Aggregation This project releases our 1st place solution on NeurIPS2021 ML4CO Dual Task. Slide and model weights a

24 Sep 08, 2022
Face Mask Detector by live camera using tensorflow-keras, openCV and Python

Face Mask Detector 😷 by Live Camera Detecting masked or unmasked faces by live camera with percentange of mask occupation About Project: This an Arti

Karan Shingde 2 Apr 04, 2022
List some popular DeepFake models e.g. DeepFake, FaceSwap-MarekKowal, IPGAN, FaceShifter, FaceSwap-Nirkin, FSGAN, SimSwap, CihaNet, etc.

deepfake-models List some popular DeepFake models e.g. DeepFake, CihaNet, SimSwap, FaceSwap-MarekKowal, IPGAN, FaceShifter, FaceSwap-Nirkin, FSGAN, Si

Mingcan Xiang 100 Dec 17, 2022