Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

Related tags

Deep Learningtf-fsvd
Overview

tf-fsvd

TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions

Cite

If you find our code useful, you may cite us as:

@inproceedings{haija2021fsvd,
  title={Fast Graph Learning with Unique Optimal Solutions},
  author={Sami Abu-El-Haija AND Valentino Crespi AND Greg Ver Steeg AND Aram Galstyan},
  year={2021},
  booktitle={arxiv:2102.08530},
}

Introduction

This codebase contains TensorFlow implementation of Functional SVD, an SVD routine that accepts objects with 3 attributes: dot, T, and shape. The object must be able to exactly multiply an (implicit) matrix M by any other matrix. Specifically, it should implement:

  1. dot(M1): should return M @ M1
  2. T: property should return another object that (implicitly) contains transpose of M.
  3. shape: property should return the shape of the (implicit) matrix M.

In most practical cases, M is implicit i.e. need not to be exactly computed. For consistency, such objects could inherit the abstract class ProductFn.

Simple Usage Example

Suppose you have an explicit sparse matrix mat

import scipy.sparse
import tf_fsvd

m = scipy.sparse.csr_mat( ... )
fn = tf_fsvd.SparseMatrixPF(m)

u, s, v = tf_fsvd.fsvd(fn, k=20)  # Rank 20 decomposition

The intent of this utility is for implicit matrices. For which, you may implement your own ProductFn class. You can take a look at BlockWisePF or WYSDeepWalkPF.

File Structure / Documentation

  • File tf_fsvd.py contains the main logic for TensorFlow implementation of Functional SVD (function fsvd), as well as a few classes for constructing implicit matrices.
    • SparseMatrixPF: when implicit matrix is a pre-computed sparse matrix. Using this class, you can now enjoy the equivalent of tf.linalg.svd on sparse tensors :-).
    • BlockWisePF: when implicit matrix is is column-wise concatenation of other implicit matrices. The concatenation is computed by suppling a list of ProductFn's
  • Directory implementations: contains implementations of simple methods employing fsvd.
  • Directory baselines: source code adapting competitive methods to produce metrics we report in our paper (time and accuracy).
  • Directory experiments: Shell scripts for running baselines and our implementations.
  • Directory results: Output directory containing results.

Running Experiments

ROC-AUC Link Prediction over AsymProj/WYS datasets

The AsymProj datasets are located in directory datasets/asymproj.

You can run the script for training on AsympProj datasets and measuring test ROC-AUC as:

python3 implementations/linkpred_asymproj.py

You can append flag --help to above to see which flags you can set for changing the dataset or the SVD rank.

You can run sweep on svd rank, for each of those datasets, by invoking:

# Sweep fSVD rank (k) on 4 link pred datasets. Make 3 runs per (dataset, k)
# Time is dominated by statement `import tensorflow as tf`
python3 experiments/fsvd_linkpred_k_sweep.py | bash  # You may remove "| bash" if you want to hand-pick commands.

# Summarize results onto CSV
python3 experiments/summarize_svdf_linkpred_sweep.py > results/linkpred_d_sweep/fsvd.csv

# Plot the sweep curve
python3 experiments/plot_sweep_k_linkpred.py

and running all printed commands. Alternatively, you can pipe the output of above to bash. This should populate directory results/linkpred_d_sweep/fsvd/.

Baselines

  • You can run the Watch Your Step baseline as:

     bash experiments/baselines/run_wys.sh
    

    which runs only once for every link prediction dataset. Watch Your Step spends some time computing the transition matrix powers (T^2, .., T^5).

  • You can run NetMF baselines (both approximate and exact) as:

    bash experiments/baselines/run_netmf.sh
    
  • You can run node2vec baseline as:

    experiments/baselines/run_n2v.sh
    

Classification Experiments over Planetoid Citation datasets

These datasets are from the planetoid paper. To obtain them, you should clone their repo:

mkdir -p ~/data
cd ~/data
git clone [email protected]:kimiyoung/planetoid.git

You can run the script for training and testing on planetoid datasets as:

python3 implementations/node_ssc_planetoid.py

You can append flag --help to above to see which flags you can set for changing the dataset or the number of layers.

You can sweep the number of layers running:

# Directly invokes python many times
LAYERS=`python3 -c "print(','.join(map(str, range(17))))"`
python3 experiments/planetoid_hp_search.py --wys_windows=1 --wys_neg_coefs=1 --layers=${LAYERS}

The script experiments/planetoid_hp_search.py directly invokes implementations/node_ssc_planetoid.py. You can visualize the accuracy VS depth curve by running:

python3 experiments/plot_sweep_depth_planetoid.py

Link Prediction for measuring [email protected] for Drug-Drug Interactions Network

You can run our method like:

python3 implementations/linkpred_ddi.py

This averages 10 runs (by default) and prints mean and standard deviation of validation and test metric ([email protected])

Owner
Sami Abu-El-Haija
Sami Abu-El-Haija
Fang Zhonghao 13 Nov 19, 2022
Building Ellee — A GPT-3 and Computer Vision Powered Talking Robotic Teddy Bear With Human Level Conversation Intelligence

Using an object detection and facial recognition system built on MobileNetSSDV2 and Dlib and running on an NVIDIA Jetson Nano, a GPT-3 model, Google Speech Recognition, Amazon Polly and servo motors,

24 Oct 26, 2022
TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification

TransPrompt This code is implement for our EMNLP 2021's paper 《TransPrompt:Towards an Automatic Transferable Prompting Framework for Few-shot Text Cla

WangJianing 23 Dec 21, 2022
Turi Create simplifies the development of custom machine learning models.

Quick Links: Installation | Documentation | WWDC 2019 | WWDC 2018 Turi Create Check out our talks at WWDC 2019 and at WWDC 2018! Turi Create simplifie

Apple 10.9k Jan 01, 2023
Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning, NeurIPS 2021 (Spotlight)

Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning, NeurIPS 2021 (Spotlight) Abstract Due to the limited and even imbalanced dat

Hanzhe Hu 99 Dec 12, 2022
Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022)

Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022) By Shilong Zhang*, Zhuoran Yu*, Liyang Liu*, Xinjiang Wang, Aojun Zhou,

Shilong Zhang 129 Dec 24, 2022
Supervised forecasting of sequential data in Python.

Supervised forecasting of sequential data in Python. Intro Supervised forecasting is the machine learning task of making predictions for sequential da

The Alan Turing Institute 54 Nov 15, 2022
Rank 3 : Source code for OPPO 6G Data Generation Challenge

OPPO 6G Data Generation with an E2E Framework Homepage of OPPO 6G Data Generation Challenge Datasets H1_32T4R.mat H2_32T4R.mat Please put the original

Sen Pei 97 Jan 07, 2023
curl-impersonate: A special compilation of curl that makes it impersonate Chrome & Firefox

curl-impersonate A special compilation of curl that makes it impersonate real browsers. It can impersonate the four major browsers: Chrome, Edge, Safa

lwthiker 1.9k Jan 03, 2023
Alleviating Over-segmentation Errors by Detecting Action Boundaries

Alleviating Over-segmentation Errors by Detecting Action Boundaries Forked from ASRF offical code. This repo is the a implementation of replacing orig

13 Dec 12, 2022
HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

HiFiGAN Denoiser This is a Unofficial Pytorch implementation of the paper HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep F

Rishikesh (ऋषिकेश) 134 Dec 27, 2022
Official implementation of Pixel-Level Bijective Matching for Video Object Segmentation

BMVOS This is the official implementation of Pixel-Level Bijective Matching for Video Object Segmentation, to appear in WACV 2022. @article{cho2021pix

Suhwan Cho 13 Dec 14, 2022
PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

Libo Qin 25 Sep 06, 2022
Aligning Latent and Image Spaces to Connect the Unconnectable

About This repo contains the official implementation of the Aligning Latent and Image Spaces to Connect the Unconnectable paper. It is a GAN model whi

Ivan Skorokhodov 203 Jan 03, 2023
zeus is a Python implementation of the Ensemble Slice Sampling method.

zeus is a Python implementation of the Ensemble Slice Sampling method. Fast & Robust Bayesian Inference, Efficient Markov Chain Monte Carlo (MCMC), Bl

Minas Karamanis 197 Dec 04, 2022
Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation

TVT Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation Datasets: Digit: MNIST, SVHN, USPS Object: Office, Office-Home, Vi

37 Dec 15, 2022
Computer Vision and Pattern Recognition, NUS CS4243, 2022

CS4243_2022 Computer Vision and Pattern Recognition, NUS CS4243, 2022 Cloud Machine #1 : Google Colab (Free GPU) Follow this Notebook installation : h

Xavier Bresson 142 Dec 15, 2022
Black box hyperparameter optimization made easy.

BBopt BBopt aims to provide the easiest hyperparameter optimization you'll ever do. Think of BBopt like Keras (back when Theano was still a thing) for

Evan Hubinger 70 Nov 03, 2022
Location-Sensitive Visual Recognition with Cross-IOU Loss

The trained models are temporarily unavailable, but you can train the code using reasonable computational resource. Location-Sensitive Visual Recognit

Kaiwen Duan 146 Dec 25, 2022
DECAF: Deep Extreme Classification with Label Features

DECAF DECAF: Deep Extreme Classification with Label Features @InProceedings{Mittal21, author = "Mittal, A. and Dahiya, K. and Agrawal, S. and Sain

46 Nov 06, 2022