Aquarius - Enabling Fast, Scalable, Data-Driven Virtual Network Functions

Last update: Jun 02, 2022

Related tags

Overview

Aquarius

Aquarius - Enabling Fast, Scalable, Data-Driven Virtual Network Functions

NOTE: We are currently going through the open-source process required by our institution. The content will soon be available. The steps that need to be completed are listed below:

Introduction

This repository implements a data-collection and data-exploitation mechanism Aquarius as a load balancer plugin in VPP. For the sake of reproducibility, software and data artifacts for performance evaluation are maintained in this repository.

Directory Roadmap

- config                    // configuration files in json format        
- sc-author-kit-log         // artifacts description of testbed hardware, required by sc21 committee
- src                       // source code
    + client/server         // scripts that run on client/server VMs
    + lb                    // scripts that run on lb VMs
        * dev               // dev version (for offline feature collection)
        * deploy            // deploy version (for online policy evaluation)
    + utils                 // utility scripts that help to run the testbed
    + vpp                   // vpp plugin
        * dev               // dev version (for offline feature collection)
        * deploy            // deploy version (for online policy evaluation)
    + test                  // unit test codes
- data                      
    + trace                 // network traces replayed on the testbed
    + results (omitted)     // This is where all the datasets are dumped (will be automatically created once we run experiments)
    + img                   // VM image files (omitted here because of file size, server configurations are documented in README)
    + vpp_deb               // stores deb files for installing VPP on VMs
        * dev               // dev version (for offline feature collection)
        * deploy            // deploy version (for online policy evaluation)

Get Started

Pre-Configuration

Run python3 setup.py, which does the following things:

update the root directory in config/global_config.json to the directory of the cloned aquarius repository (replace the /home/yzy/aquarius);
clone the VPP repository in src/vpp/base;
update the physical_server_ip in config/global_config.json to the IP addresses of the actual server IP addresses in use;
update the vlan_if as the last network interface on the local machine
update the physical_server_ip in config/cluster/unittest-1.json to the local hostname

VM images

To prepare a VM original image, refer to the README file in data. To run all the experiments without issues, create a ssh-key on the host servers and copy the public key to the VMs so that commands can be executed from the host using ssh -t -t.

Run example

A simple example is created using a small network topology (1 client, 1 edge router, 1 load balancer, and 4 application servers) on a single machine. Simply follow the jupyter notebook in notebook/unittest. Make sure the configurations are well adapted to your own host machine. Also make sure that the host machine has at least 20 CPUs. Otherwise, the configuration can be modified in config/cluster/unittest-1.json. To reduce the amount of CPUs required, change the number of vcpu of each node in the json file.

Reproducibility

To reproduce the results in Aquarius paper, three notebooks are presented in notebook/reproduce. The dataset that are generated from the experiments are stored in data/reproduce. To run these experiments, 4 physical machines with 12 physcial cores (48 CPUs) each are required. MACROs in the notebook should be well adapted. For instance, VLAN should be configured across the actual inerfacesin use. An example of network topology is depicted below.

Notes

Running the scripts, e.g. src/utils/testbed_utils.py, requires root access.

Aquarius

Aquarius - Enabling Fast, Scalable, Data-Driven Virtual Network Functions

You might also like...

Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"

M3D-VTON: A Monocular-to-3D Virtual Try-On Network Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network" Paper | Suppl

109 Dec 29, 2022

Official PyTorch implementation of "RMGN: A Regional Mask Guided Network for Parser-free Virtual Try-on" (IJCAI-ECAI 2022)

RMGN-VITON RMGN: A Regional Mask Guided Network for Parser-free Virtual Try-on In IJCAI-ECAI 2022(short oral). [Paper] [Supplementary Material] Abstra

27 Dec 1, 2022

Code of paper "CDFI: Compression-Driven Network Design for Frame Interpolation", CVPR 2021

CDFI (Compression-Driven-Frame-Interpolation) [Paper] (Coming soon...) | [arXiv] Tianyu Ding*, Luming Liang*, Zhihui Zhu, Ilya Zharkov IEEE Conference

95 Dec 4, 2022

InDuDoNet+: A Model-Driven Interpretable Dual Domain Network for Metal Artifact Reduction in CT Images

InDuDoNet+: A Model-Driven Interpretable Dual Domain Network for Metal Artifact Reduction in CT Images Hong Wang, Yuexiang Li, Haimiao Zhang, Deyu Men

4 Dec 27, 2022

AI-based, context-driven network device ranking

Batea A batea is a large shallow pan of wood or iron traditionally used by gold prospectors for washing sand and gravel to recover gold nuggets. Batea

269 Nov 26, 2022

RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020)

RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020) Hong Wang, Qi Xie, Qian Zhao, and Deyu Meng [PDF] [Supplementary M

6 Sep 27, 2022

A PyTorch Implementation of "SINE: Scalable Incomplete Network Embedding" (ICDM 2018).

Scalable Incomplete Network Embedding ⠀⠀ A PyTorch implementation of Scalable Incomplete Network Embedding (ICDM 2018). Abstract Attributed network em

69 Sep 22, 2022

The AugNet Python module contains functions for the fast computation of image similarity.

AugNet AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation arxiv link In our work, we propose AugNet, a new deep le

74 Dec 28, 2022

A data-driven approach to quantify the value of classifiers in a machine learning ensemble.

Documentation | External Resources | Research Paper Shapley is a Python library for evaluating binary classifiers in a machine learning ensemble. The

188 Dec 29, 2022

Releases(sc22-v1.0-alpha)

sc22-v1.0-alpha(Jun 11, 2022)

ALPHA version of Aquarius release for SC22.

This release aims at demonstrating the basic workflow of the artifacts of Aquarius. Besides the jupyter notebooks which documents the actual procedure of producing all the experimental results in the paper, a unittest is provided to guide you through the basic workflow of the artifact.

Please refer to the latest main branch of the Github repo to reproduce the core results presented in the paper: https://github.com/ZhiyuanYaoJ/Aquarius
Source code(tar.gz)
Source code(zip)

Owner

Zhiyuan YAO

PhD student at L'Ecole Polytechnique.

GitHub Repository

Open source repository for the code accompanying the paper 'PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations'.

PatchNets This is the official repository for the project "PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations". For details,

16 May 22, 2022

An implementation of a discriminant function over a normal distribution to help classify datasets.

CS4044D Machine Learning Assignment 1 By Dev Sony, B180297CS The question, report and source code can be found here. Github Repo Solution 1 Based on t

6 Nov 09, 2021

I created My own Virtual Artificial Intelligence named genesis, He can assist with my Tasks and also perform some analysis,,

Virtual-Artificial-Intelligence-genesis- I created My own Virtual Artificial Intelligence named genesis, He can assist with my Tasks and also perform

1 Nov 05, 2021

This is a virtual picture dragging application. Users may virtually slide photos across the screen. The distance between the index and middle fingers determines the movement. Smaller distances indicate click and motion, whereas bigger distances indicate only hand movement.

Virtual_Image_Dragger This is a virtual picture dragging application. Users may virtually slide photos across the screen. The distance between the ind

17 Dec 17, 2022

Command-line tool for downloading and extending the RedCaps dataset.

RedCaps Downloader This repository provides the official command-line tool for downloading and extending the RedCaps dataset. Users can seamlessly dow

33 Dec 14, 2022

Python module providing a framework to trace individual edges in an image using Gaussian process regression.

Edge Tracing using Gaussian Process Regression Repository storing python module which implements a framework to trace individual edges in an image usi

7 Dec 27, 2022

Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation".

FPS-Net Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation", accepted by ISPRS journal of Photogrammetry

15 Nov 30, 2022

Deep Learning Theory

Deep Learning Theory 整理了一些深度学习的理论相关内容，持续更新。 Overview Recent advances in deep learning theory 总结了目前深度学习理论研究的六个方向的一些结果，概述型，没做深入探讨(2021)。 1.1 complexity

103 Jan 04, 2023

Final project code: Implementing MAE with downscaled encoders and datasets, for ESE546 FA21 at University of Pennsylvania

546 Final Project: Masked Autoencoder Haoran Tang, Qirui Wu 1. Training To train the network, please run mae_pretraining.py. Please modify folder path

0 Apr 22, 2022

Aquarius - Enabling Fast, Scalable, Data-Driven Virtual Network Functions

Related tags

Overview

Aquarius

Introduction

Directory Roadmap

Get Started

Pre-Configuration

VM images

Run example

Reproducibility

Notes

Aquarius

You might also like...

Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"

Official PyTorch implementation of "RMGN: A Regional Mask Guided Network for Parser-free Virtual Try-on" (IJCAI-ECAI 2022)

Code of paper "CDFI: Compression-Driven Network Design for Frame Interpolation", CVPR 2021

InDuDoNet+: A Model-Driven Interpretable Dual Domain Network for Metal Artifact Reduction in CT Images

AI-based, context-driven network device ranking

RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020)

A PyTorch Implementation of "SINE: Scalable Incomplete Network Embedding" (ICDM 2018).

The AugNet Python module contains functions for the fast computation of image similarity.

A data-driven approach to quantify the value of classifiers in a machine learning ensemble.

Releases(sc22-v1.0-alpha)

sc22-v1.0-alpha(Jun 11, 2022)

Owner

Zhiyuan YAO

Open source repository for the code accompanying the paper 'PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations'.

An implementation of a discriminant function over a normal distribution to help classify datasets.

I created My own Virtual Artificial Intelligence named genesis, He can assist with my Tasks and also perform some analysis,,

This is a virtual picture dragging application. Users may virtually slide photos across the screen. The distance between the index and middle fingers determines the movement. Smaller distances indicate click and motion, whereas bigger distances indicate only hand movement.

Command-line tool for downloading and extending the RedCaps dataset.

Python module providing a framework to trace individual edges in an image using Gaussian process regression.

Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation".

Deep Learning Theory

Final project code: Implementing MAE with downscaled encoders and datasets, for ESE546 FA21 at University of Pennsylvania

This python-based package offers a way of creating a parametric OpenMC plasma source from plasma parameters.

Python interface for the DIGIT tactile sensor

RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Implementation for "Manga Filling Style Conversion with Screentone Variational Autoencoder" (SIGGRAPH ASIA 2020 issue)

Code for "Multi-Compound Transformer for Accurate Biomedical Image Segmentation"

Multilingual Image Captioning

CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches

这是一个yolox-pytorch的源码，可以用于训练自己的模型。

Decentralized Reinforcment Learning: Global Decision-Making via Local Economic Transactions (ICML 2020)

Official code of the paper "ReDet: A Rotation-equivariant Detector for Aerial Object Detection" (CVPR 2021)

Notification Triggers for Python