An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

Last update: Dec 31, 2022

Related tags

Overview

PYPARSVD

This implementation allows for a singular value decomposition which is:

Distributed using MPI4Py
Streaming - data can be shown in batches to update the left singular vectors
Randomized for further acceleration of any serial components of the overall algorithm.

The streaming algorithm used in this implementation is available in: "Sequential Karhunen–Loeve Basis Extraction and its Application to Images" by Avraham Levy and Michael Lindenbaum. IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 9, NO. 8, AUGUST 2000. This algorithm is implemented in Online_SVD_Serial.py.

The distributed computation of the SVD follows the implementation in "Approximate partitioned method of snapshots for POD." by Wang, Zhu, Brian McBee, and Traian Iliescu. Journal of Computational and Applied Mathematics 307 (2016): 374-384. This algorithm is validated in APMOS_Validation/.

The parallel QR algorithm (the TSQR method) required for the streaming feature may be found in "Direct QR factorizations for tall-and-skinny matrices in MapReduce architectures." by Benson, Austin R., David F. Gleich, and James Demmel. 2013 IEEE international conference on big data. IEEE, 2013. This algorithm is validated in Parallel_QR.

The randomized algorithm used to accelerate the computation of the serial SVD in partitioned method of snapshots may be found in "Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions." by Halko, Nathan, Per-Gunnar Martinsson, and Joel A. Tropp. SIAM review 53.2 (2011): 217-288.

To enable this feature set low_rank=True for initializing the online_svd_calculator class object in online_svd_parallel.py

To reproduce results on a shared memory platform (needs atleast 6 available ranks): export OPENBLAS_NUM_THREADS=1 to ensure numpy does not multithread for this experiment.

Run python data_splitter.py to generate exemplar data etc.
Run python online_svd_serial.py for serial deployment of streaming algorithm.
Run mpirun -np 6 python online_svd_parallel.py for parallel/streaming deployment.

Caution: Due to differences in the parallel and serial versions of the algorithm, singular vectors may be "flipped". An orthogonality check is also deployed for an additional sanity check.

Example extractions of left singular vectors and singular values

Even the simple problem demonstrated here (8192 spatial points and 800 snapshots) achieves a dramatic acceleration in time to solution from serial to parallelized-streaming implementations (~25X). Note that the key advantage of the parallelized version is the lack of a data-transfer requirement in case this routine is being called from a simulation.

You might also like...

Streaming over lightweight data transformations

Description Data augmentation libarary for Deep Learning, which supports images, segmentation masks, labels and keypoints. Furthermore, SOLT is fast a

Research Unit of Medical Imaging, Physics and Technology

256 Jan 8, 2023

Music library streaming app written in Flask & VueJS

djtaytay This is a little toy app made to explore Vue, brush up on my Python, and make a remote music collection accessable through a web interface. I

6 May 27, 2022

Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

scikit-event-correlation Event Correlation and Changing Detection Algorithm Theo

5 Oct 30, 2022

Securetar - A streaming wrapper around python tarfile and allow secure handling files and support encryption

Secure Tar Secure Tarfile library It's a streaming wrapper around python tarfile

2 Dec 9, 2022

Real-time Object Detection for Streaming Perception, CVPR 2022

StreamYOLO Real-time Object Detection for Streaming Perception Jinrong Yang, Songtao Liu, Zeming Li, Xiaoping Li, Sun Jian Real-time Object Detection

237 Dec 27, 2022

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

English | 简体中文 Welcome to the PaddlePaddle GitHub. PaddlePaddle, as the only independent R&D deep learning platform in China, has been officially open

19.4k Jan 4, 2023

Releases(v1.0)

v1.0(Feb 25, 2021)

A Parallelized, streaming, and randomized implementation of the SVD for Python using mpi4py.

Contact [email protected] (or create issue) for details.

Romit Maulik
Source code(tar.gz)
Source code(zip)

An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

Related tags

Overview

PYPARSVD

You might also like...

Streaming over lightweight data transformations

Music library streaming app written in Flask & VueJS

Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

Securetar - A streaming wrapper around python tarfile and allow secure handling files and support encryption

Real-time Object Detection for Streaming Perception, CVPR 2022

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

Model parallel transformers in Jax and Haiku

Code and data for ACL2021 paper Cross-Lingual Abstractive Summarization with Limited Parallel Resources.

Symbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis in JAX

Releases(v1.0)

v1.0(Feb 25, 2021)

Owner

Romit Maulik

TensorFlow, PyTorch and Numpy layers for generating Orthogonal Polynomials

Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

The AWS Certified SysOps Administrator

Recurrent Conditional Query Learning

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

Build upon neural radiance fields to create a scene-specific implicit 3D semantic representation, Semantic-NeRF

code for paper"A High-precision Semantic Segmentation Method Combining Adversarial Learning and Attention Mechanism"

A tutorial showing how to train, convert, and run TensorFlow Lite object detection models on Android devices, the Raspberry Pi, and more!

Fully Convolutional Networks for Semantic Segmentation by Jonathan Long, Evan Shelhamer, and Trevor Darrell. CVPR 2015 and PAMI 2016.

Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift (ICCV 2021)

Proto-RL: Reinforcement Learning with Prototypical Representations

Systematic generalisation with group invariant predictions

Point cloud processing tool library.

Distance correlation and related E-statistics in Python

Supervised Contrastive Learning for Product Matching

Deep Learning segmentation suite designed for 2D microscopy image segmentation

Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

A disassembler for the RP2040 Programmable I/O State-machine!

MAVE: : A Product Dataset for Multi-source Attribute Value Extraction

An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

Related tags

Overview

PYPARSVD

You might also like...

Streaming over lightweight data transformations

Music library streaming app written in Flask & VueJS

Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

Securetar - A streaming wrapper around python tarfile and allow secure handling files and support encryption

Real-time Object Detection for Streaming Perception, CVPR 2022

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

Model parallel transformers in Jax and Haiku

Code and data for ACL2021 paper Cross-Lingual Abstractive Summarization with Limited Parallel Resources.

Symbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis in JAX

Releases(v1.0)

v1.0(Feb 25, 2021)

Owner

Romit Maulik

TensorFlow, PyTorch and Numpy layers for generating Orthogonal Polynomials

Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

The AWS Certified SysOps Administrator

Recurrent Conditional Query Learning

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

Build upon neural radiance fields to create a scene-specific implicit 3D semantic representation, Semantic-NeRF

code for paper"A High-precision Semantic Segmentation Method Combining Adversarial Learning and Attention Mechanism"

A tutorial showing how to train, convert, and run TensorFlow Lite object detection models on Android devices, the Raspberry Pi, and more!

Fully Convolutional Networks for Semantic Segmentation by Jonathan Long*, Evan Shelhamer*, and Trevor Darrell. CVPR 2015 and PAMI 2016.

Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift (ICCV 2021)

Proto-RL: Reinforcement Learning with Prototypical Representations

Systematic generalisation with group invariant predictions

Point cloud processing tool library.

Distance correlation and related E-statistics in Python

Supervised Contrastive Learning for Product Matching

Deep Learning segmentation suite designed for 2D microscopy image segmentation

Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

A disassembler for the RP2040 Programmable I/O State-machine!

MAVE: : A Product Dataset for Multi-source Attribute Value Extraction

Fully Convolutional Networks for Semantic Segmentation by Jonathan Long, Evan Shelhamer, and Trevor Darrell. CVPR 2015 and PAMI 2016.