DeLag: Detecting Latency Degradation Patterns in Service-based Systems

Overview

DeLag: Detecting Latency Degradation Patterns in Service-based Systems

Replication package of the work "DeLag: Detecting Latency Degradation Patterns in Service-based Systems".

Requirements

  • Python 3.6
  • Java 8
  • Apache Spark 2.3.1 (set $SPARK_HOME env variable with the folder path))
  • Elasticsearch for Spark 2.X 7.6.0 (set $ES_SPARK env variable with the jar path)
  • Maven 3.6.0 (only for datasets generation)
  • Docker 18.03 (only for datasets generation)

Use the following command to install Python dependencies

pip install --upgrade pip
pip install -r requirements.txt

The generation of datasets and the experimentation of techniques were performed on a dual Intel Xeon CPU E5-2650 v3 at 2.30GHz, totaling 40 cores and 80GB of RAM. We recommend to run the scripts of this replication package on a machine with similar specs.

Datasets

The datasets folder contains the datasets of traces used in the evaluation (in parquet format). Each row of each dataset represents a request and contains:

  • traceId: the ID of the request:
  • [requestLatency]: the overall latency of the request. It is represented by the column ts-travel-service_queryInfo in the Train-Ticket case study and by the column HomeControllerHome in the E-Shopper case study.
  • experiment: if equal to 0 (resp. 1) the request is affected by the ADC (resp. ) otherwise is not affected by any ADCs.
  • [RPC]: the cumulative execution time of [RPC] within the request.

Datasets generation

The datasets-generation folder contains the bash scripts used to generate the datasets used in the evaluation.

Techniques

The techniques folder contains the implementations of DeLag, CoTr, KrSa and DeCaf. In the following you can find the main Python classes used to implement each technique:

  • DeLag: class GeneticRangeAnalysis
  • CoTr: classes RangeAnalysis and GA
  • KrSa: classes RangeAnalysis and BranchAndBound
  • DeCaf: class DeCaf.

Experiments

The experiments folder contains the Python scripts used to execute DeLag and baselines techniques on the generated datasets.

Results

The results folder contains the results of our experimentation. Each row of each csv file represents a run of a particural technique on a dataset and contains:

  • exp: the dataset ID.
  • algo: the technique experimented. The notation used to indicate each techique is described below:
    • gra: DeLag - DeLag: Detecting Latency Degradation Patterns in Service-based Systems
    • bnb: KrSa - Understanding Latency Variations of Black Box Services (WWW 2013)
    • ga: CoTr - Detecting Latency Degradation Patterns in Service-based Systems (ICPE 2020)
    • decaf DeCaf - DeCaf: Diagnosing and Triaging Performance Issues in Large-Scale Cloud Services (ICSE 2020)
    • kmeans: K-means
    • hierarchical: HC - Hierachical clustering
  • trial: the ID of the run (techniques may be repeated multiple times on a dataset to mitigate result variabilility)
  • precision: effectiveness measure - Precision ()
  • recall: effectiveness measure - Recall ()
  • fmeasure: effectiveness measure - F1-score ()
  • time: execution time in seconds

Scripts

The scripts folder contains the Python scripts used to generate the figures and tables of the paper.

Systems

The systems folder contains the two case study systems.

You might also like...
McGill Physics Hackathon 2021: Reaction-Diffusion Models for the Generation of Biological Patterns
McGill Physics Hackathon 2021: Reaction-Diffusion Models for the Generation of Biological Patterns

DiffuseAnimals: Reaction-Diffusion Models for the Generation of Biological Patterns Introduction Reaction-diffusion equations can be utilized in order

Official PyTorch implementation of "Synthesis of Screentone Patterns of Manga Characters"

Manga Character Screentone Synthesis Official PyTorch implementation of "Synthesis of Screentone Patterns of Manga Characters" presented in IEEE ISM 2

DeepHawkeye is a library to detect unusual patterns in images using features from pretrained neural networks
DeepHawkeye is a library to detect unusual patterns in images using features from pretrained neural networks

English | 简体中文 Introduction DeepHawkeye is a library to detect unusual patterns in images using features from pretrained neural networks Reference Pat

Architecture Patterns with Python (TDD, DDD, EDM)

architecture-traning Architecture Patterns with Python (TDD, DDD, EDM) Chapter 5. 높은 기어비와 낮은 기어비의 TDD 5.2 도메인 계층 테스트를 서비스 계층으로 옮겨야 하는가? 도메인 계층 테스트 def

A DeepStack custom model for detecting common objects in dark/night images and videos.
A DeepStack custom model for detecting common objects in dark/night images and videos.

DeepStack_ExDark This repository provides a custom DeepStack model that has been trained and can be used for creating a new object detection API for d

A custom DeepStack model for detecting 16 human actions.
A custom DeepStack model for detecting 16 human actions.

DeepStack_ActionNET This repository provides a custom DeepStack model that has been trained and can be used for creating a new object detection API fo

Python Tensorflow 2 scripts for detecting objects of any class in an image without knowing their label.
Python Tensorflow 2 scripts for detecting objects of any class in an image without knowing their label.

Tensorflow-Mobile-Generic-Object-Localizer Python Tensorflow 2 scripts for detecting objects of any class in an image without knowing their label. Ori

Python TFLite scripts for detecting objects of any class in an image without knowing their label.
Python TFLite scripts for detecting objects of any class in an image without knowing their label.

Python TFLite scripts for detecting objects of any class in an image without knowing their label.

Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples / ICLR 2018

Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples This project is for the paper "Training Confidence-Calibrated Clas

Releases(v1.1)
  • v1.1(Dec 22, 2022)

    Replication package of the work "DeLag: Using Multi-Objective Optimization to Enhance the Detection of Latency Degradation Patterns in Service-based Systems"

    Source code(tar.gz)
    Source code(zip)
Owner
SEALABQualityGroup @ University of L'Aquila
SEALABQualityGroup @ University of L'Aquila
This is the official pytorch implementation of AutoDebias, an automatic debiasing method for recommendation.

AutoDebias This is the official pytorch implementation of AutoDebias, a debiasing method for recommendation system. AutoDebias is proposed in the pape

Dong Hande 77 Nov 25, 2022
Object-Centric Learning with Slot Attention

Slot Attention This is a re-implementation of "Object-Centric Learning with Slot Attention" in PyTorch (https://arxiv.org/abs/2006.15055). Requirement

Untitled AI 72 Jan 02, 2023
Official PyTorch Implementation of "AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting".

AgentFormer This repo contains the official implementation of our paper: AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecast

Ye Yuan 161 Dec 23, 2022
DI-smartcross - Decision Intelligence Platform for Traffic Crossing Signal Control

DI-smartcross DI-smartcross - Decision Intelligence Platform for Traffic Crossin

OpenDILab 213 Jan 02, 2023
Controlling a game using mediapipe hand tracking

These scripts use the Google mediapipe hand tracking solution in combination with a webcam in order to send game instructions to a racing game. It features 2 methods of control

3 May 17, 2022
This is 2nd term discrete maths project done by UCU students that uses backtracking to solve various problems.

Backtracking Project Sponsors This is a project made by UCU students: Olha Liuba - crossword solver implementation Hanna Yershova - sudoku solver impl

Dasha 4 Oct 17, 2021
This is the repository for The Machine Learning Workshops, published by AI DOJO

This is the repository for The Machine Learning Workshops, published by AI DOJO. It contains all the workshop's code with supporting project files necessary to work through the code.

AI Dojo 12 May 06, 2022
Toolbox of models, callbacks, and datasets for AI/ML researchers.

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch Website • Installation • Main

Pytorch Lightning 1.4k Dec 30, 2022
This is the official code for the paper "Ad2Attack: Adaptive Adversarial Attack for Real-Time UAV Tracking".

Ad^2Attack:Adaptive Adversarial Attack on Real-Time UAV Tracking Demo video 📹 Our video on bilibili demonstrates the test results of Ad^2Attack on se

Intelligent Vision for Robotics in Complex Environment 10 Nov 07, 2022
(Personalized) Page-Rank computation using PyTorch

torch-ppr This package allows calculating page-rank and personalized page-rank via power iteration with PyTorch, which also supports calculation on GP

Max Berrendorf 69 Dec 03, 2022
Patch-Based Deep Autoencoder for Point Cloud Geometry Compression

Patch-Based Deep Autoencoder for Point Cloud Geometry Compression Overview The ever-increasing 3D application makes the point cloud compression unprec

17 Dec 05, 2022
DeepStruc is a Conditional Variational Autoencoder which can predict the mono-metallic nanoparticle from a Pair Distribution Function.

ChemRxiv | [Paper] XXX DeepStruc Welcome to DeepStruc, a Deep Generative Model (DGM) that learns the relation between PDF and atomic structure and the

Emil Thyge Skaaning Kjær 13 Aug 01, 2022
Self-Supervised Image Denoising via Iterative Data Refinement

Self-Supervised Image Denoising via Iterative Data Refinement Yi Zhang1, Dasong Li1, Ka Lung Law2, Xiaogang Wang1, Hongwei Qin2, Hongsheng Li1 1CUHK-S

Zhang Yi 72 Jan 01, 2023
Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Machine Learning Sleep Schedule Tracker What is it? Convolutional neural network web app trained to track our infant’s sleep schedule using our Google

g-parki 7 Jul 15, 2022
Code for our paper at ECCV 2020: Post-Training Piecewise Linear Quantization for Deep Neural Networks

PWLQ Updates 2020/07/16 - We are working on getting permission from our institution to release our source code. We will release it once we are granted

54 Dec 15, 2022
Weight initialization schemes for PyTorch nn.Modules

nninit Weight initialization schemes for PyTorch nn.Modules. This is a port of the popular nninit for Torch7 by @kaixhin. ##Update This repo has been

Alykhan Tejani 69 Jan 26, 2021
Back to Event Basics: SSL of Image Reconstruction for Event Cameras

Back to Event Basics: SSL of Image Reconstruction for Event Cameras Minimal code for Back to Event Basics: Self-Supervised Learning of Image Reconstru

TU Delft 42 Dec 26, 2022
DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

52 Dec 30, 2022
learned_optimization: Training and evaluating learned optimizers in JAX

learned_optimization: Training and evaluating learned optimizers in JAX learned_optimization is a research codebase for training learned optimizers. I

Google 533 Dec 30, 2022
Implementation of the famous Image Manipulation\Forgery Detector "ManTraNet" in Pytorch

Who has never met a forged picture on the web ? No one ! Everyday we are constantly facing fake pictures touched up in Photoshop but it is not always

Rony Abecidan 77 Dec 16, 2022