CRISP: Critical Path Analysis of Microservice Traces

This repo contains code to compute and present critical path summary from Jaeger microservice traces. To use first collect the microservice traces of a specific endpoint in a directory (say traces). Let the traces be for OP operation and SVC service (these are Jaeger termonologies). python3 process.py --operationName OP --serviceName SVC -t <path to trace> -o . --parallelism 8 will produce the critical path summary using 8 concurrent processes. The summary will be output in the current directory as an HTML file with a heatmap, flamegraph, and summary text in criticalPaths.html. It will also produce three flamegraphs flame-graph-*.svg for three different percentile values.

The script accepts the following options:

python3 process.py --help
usage: process.py [-h] -a OPERATIONNAME -s SERVICENAME [-t TRACEDIR] [--file FILE] -o OUTPUTDIR
                  [--parallelism PARALLELISM] [--topN TOPN] [--numTrace NUMTRACE] [--numOperation NUMOPERATION]

optional arguments:
  -h, --help            show this help message and exit
  -a OPERATIONNAME, --operationName OPERATIONNAME
                        operation name
  -s SERVICENAME, --serviceName SERVICENAME
                        name of the service
  -t TRACEDIR, --traceDir TRACEDIR
                        path of the trace directory (mutually exclusive with --file)
  --file FILE           input path of the trace file (mutually exclusivbe with --traceDir)
  -o OUTPUTDIR, --outputDir OUTPUTDIR
                        directory where output will be produced
  --parallelism PARALLELISM
                        number of concurrent python processes.
  --topN TOPN           number of services to show in the summary
  --numTrace NUMTRACE   number of traces to show in the heatmap
  --numOperation NUMOPERATION
                        number of operations to show in the heatmap

CRISP: Critical Path Analysis of Microservice Traces

Related tags

Overview

CRISP: Critical Path Analysis of Microservice Traces

Owner

Uber Research

ELFXtract is an automated analysis tool used for enumerating ELF binaries

Template for a Dataflow Flex Template in Python

A lightweight, hub-and-spoke dashboard for multi-account Data Science projects

Monitor the stability of a pandas or spark dataframe ⚙︎

A data analysis using python and pandas to showcase trends in school performance.

CaterApp is a cross platform, remotely data sharing tool created for sharing files in a quick and secured manner.

Validation and inference over LinkML instance data using souffle

We're Team Arson and we're using the power of predictive modeling to combat wildfires.

Vaex library for Big Data Analytics of an Airline dataset

ETL pipeline on movie data using Python and postgreSQL

Approximate Nearest Neighbor Search for Sparse Data in Python!

ETL flow framework based on Yaml configs in Python

Pypeln is a simple yet powerful Python library for creating concurrent data pipelines.

Mortgage-loan-prediction - Show how to perform advanced Analytics and Machine Learning in Python using a full complement of PyData utilities

Extract data from a wide range of Internet sources into a pandas DataFrame.

Datashader is a data rasterization pipeline for automating the process of creating meaningful representations of large amounts of data.

Very basic but functional Kakuro solver written in Python.

Data pipelines built with polars

Data cleaning tools for Business analysis

PyPSA: Python for Power System Analysis