web application for flight log analysis & review

Overview

Flight Review

Build Status

This is a web application for flight log analysis. It allows users to upload ULog flight logs, and analyze them through the browser.

It uses the bokeh library for plotting and the Tornado Web Server.

Flight Review is deployed at https://review.px4.io.

Plot View

3D View

3D View

Installation and Setup

Requirements

Ubuntu

sudo apt-get install sqlite3 fftw3 libfftw3-dev

Note: Under some Ubuntu and Debian environments you might have to install ATLAS

sudo apt-get install libatlas3-base

macOS

macOS already provides SQLite3. Use Homebrew to install fftw:

brew install fftw

Installation

# After git clone, enter the directory
git clone --recursive https://github.com/PX4/flight_review.git
cd flight_review/app
pip install -r requirements.txt
# Note: preferably use a virtualenv

Setup

  • By default the app will load config_default.ini configuration file
  • You can override any setting from config_default.ini with a user config file config_user.ini (untracked)
  • Any setting on config_user.ini has priority over config_default.ini
  • Run setup_db.py to initialize the database.

Note: setup_db.py can also be used to upgrade the database tables, for instance when new entries are added (it automatically detects that).

Usage

For local usage, the server can be started directly with a log file name, without having to upload it first:

cd app
./serve.py -f <file.ulg>

To start the whole web application:

cd app
./serve.py --show

The plot_app directory contains a bokeh server application for plotting. It can be run stand-alone with bokeh serve --show plot_app (or with cd plot_app; bokeh serve --show main.py, to start without the html template).

The whole web application is run with the serve.py script. Run ./serve.py -h for further details.

Interactive Usage

The plotting can also be used interative using a Jupyter Notebook. It requires python knowledge, but provides full control over what and how to plot with immediate feedback.

  • Start the notebook
  • Locate and open the test notebook file testing_notebook.ipynb.
# Launch jupyter notebook
jupyter notebook testing_notebook.ipynb

Implementation

The web site is structured around a bokeh application in app/plot_app (app/plot_app/configured_plots.py contains all the configured plots). This application also handles the statistics page, as it contains bokeh plots as well. The other pages (upload, browse, ...) are implemented as tornado handlers in app/tornado_handlers/.

plot_app/helper.py additionally contains a list of log topics that the plot application can subscribe to. A topic must live in this list in order to be plotted.

Tornado uses a single-threaded event loop. This means all operations should be non-blocking (see also http://www.tornadoweb.org/en/stable/guide/async.html). (This is currently not the case for sending emails).

Reading ULog files is expensive and thus should be avoided if not really necessary. There are two mechanisms helping with that:

  • Loaded ULog files are kept in RAM using an LRU cache with configurable size (when using the helper method). This works from different requests and sessions and from all source contexts.
  • There's a LogsGenerated DB table, which contains extracted data from ULog for faster access.

Caching

In addition to in-memory caching there is also some on-disk caching: KML files are stored on disk. Also the parameters and airframes are cached and downloaded every 24 hours. It is safe to delete these files (but not the cache directory).

Notes about python imports

Bokeh uses dynamic code loading and the plot_app/main.py gets loaded on each session (page load) to isolate requests. This also means we cannot use relative imports. We have to use sys.path.append to include modules in plot_app from the root directory (Eg tornado_handlers.py). Then to make sure the same module is only loaded once, we use import xy instead of import plot_app.xy. It's useful to look at print('\n'.join(sys.modules.keys())) to check this.

Docker usage

This section explains how to work with docker.

Arguments

Edit the .env file according to your setup:

  • PORT - The number of port, what listen service in docker, default 5006
  • USE_PROXY - The set his, if you use reverse proxy (Nginx, ...)
  • DOMAIN - The address domain name for origin, default = *
  • CERT_PATH - The SSL certificate volume path
  • EMAIL - Email for challenging Let's Encrypt DNS

Paths

  • /opt/service/config_user.ini - Path for config
  • /opt/service/data - Folder where stored database
  • .env - Environment variables for nginx and app docker container

Build Docker Image

cd app
docker build -t px4flightreview -f Dockerfile .

Work with docker-compose

Run the following command to start docker container. Please modify the .env and add app/config_user.ini with respective stages.

Uncomment the BOKEH_ALLOW_WS_ORIGIN with your local IP Address when developing, this is for the bokeh application's websocket to work.

Development

docker-compose -f docker-compose.dev.yml up

Test Locally

Test locally with nginx:

docker-compose up

Remember to Change NGINX_CONF to use default_ssl.conf and add the EMAIL for production.

Production

htpasswd -c ./nginx/.htpasswd username
# here to create a .htpasswd for nginx basic authentication
chmod u+x init-letsencrypt.sh
./init-letsencrypt.sh

Contributing

Contributions are welcome! Just open a pull request with detailed description why the changes are needed, or open an issue for bugs, feature requests, etc...

Owner
PX4 Drone Autopilot
Professional Open Source Autopilot Stack
PX4 Drone Autopilot
Gesture controlled media player

Media Player Gesture Control Gesture controller for media player with MediaPipe, VLC and OpenCV. Contents About Setup About A tool for using gestures

Atharva Joshi 2 Dec 22, 2021
Chem: collection of mostly python code for molecular visualization, QM/MM, FEP, etc

chem: collection of mostly python code for molecular visualization, QM/MM, FEP,

5 Sep 02, 2022
A Bokeh project developed for learning and teaching Bokeh interactive plotting!

Bokeh-Python-Visualization A Bokeh project developed for learning and teaching Bokeh interactive plotting! See my medium blog posts about making bokeh

Will Koehrsen 350 Dec 05, 2022
This is a small repository for me to implement my simply Data Visualisation skills through Python.

Data Visualisations This is a small repository for me to implement my simply Data Visualisation skills through Python. Steam Population Chart from 10/

9 Dec 31, 2021
A streamlit component for bi-directional communication with bokeh plots.

Streamlit Bokeh Events A streamlit component for bi-directional communication with bokeh plots. Its just a workaround till streamlit team releases sup

Ashish Shukla 123 Dec 25, 2022
A simple python tool for explore your object detection dataset

A simple tool for explore your object detection dataset. The goal of this library is to provide simple and intuitive visualizations from your dataset and automatically find the best parameters for ge

GRADIANT - Centro Tecnolóxico de Telecomunicacións de Galicia 142 Dec 25, 2022
A python visualization of the A* path finding algorithm

A python visualization of the A* path finding algorithm. It allows you to pick your start, end location and make obstacles and then view the process of finding the shortest path. You can also choose

Kimeon 4 Aug 02, 2022
Learn Data Science with focus on adding value with the most efficient tech stack.

DataScienceWithPython Get started with Data Science with Python An engaging journey to become a Data Scientist with Python TL;DR Download all Jupyter

Learn Python with Rune 110 Dec 22, 2022
A grammar of graphics for Python

plotnine Latest Release License DOI Build Status Coverage Documentation plotnine is an implementation of a grammar of graphics in Python, it is based

Hassan Kibirige 3.3k Jan 01, 2023
A Simple Flask-Plotly Example for NTU 110-1 DSSI Class

A Simple Flask-Plotly Example for NTU 110-1 DSSI Class Live Demo Prerequisites We will use Flask and Ploty to build a Flask application. If you haven'

Ting Ni Wu 1 Dec 11, 2021
A concise grammar of interactive graphics, built on Vega.

Vega-Lite Vega-Lite provides a higher-level grammar for visual analysis that generates complete Vega specifications. You can find more details, docume

Vega 4k Jan 08, 2023
demir.ai Dataset Operations

demir.ai Dataset Operations With this application, you can have the empty values (nan/null) deleted or filled before giving your dataset to machine le

Ahmet Furkan DEMIR 8 Nov 01, 2022
Pretty Confusion Matrix

Pretty Confusion Matrix Why pretty confusion matrix? We can make confusion matrix by using matplotlib. However it is not so pretty. I want to make con

Junseo Ko 5 Nov 22, 2022
Blender addon that creates a temporary window of any type from the 3D View.

CreateTempWindow2.8 Blender addon that creates a temporary window of any type from the 3D View. Features Can the following window types: 3D View Graph

3 Nov 27, 2022
This component provides a wrapper to display SHAP plots in Streamlit.

streamlit-shap This component provides a wrapper to display SHAP plots in Streamlit.

Snehan Kekre 30 Dec 10, 2022
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 01, 2023
:art: Diagram as Code for prototyping cloud system architectures

Diagrams Diagram as Code. Diagrams lets you draw the cloud system architecture in Python code. It was born for prototyping a new system architecture d

MinJae Kwon 27.5k Dec 30, 2022
Customizing Visual Styles in Plotly

Customizing Visual Styles in Plotly Code for a workshop originally developed for an Unconference session during the Outlier Conference hosted by Data

Data Design Dimension 9 Aug 03, 2022
A Python toolbox for gaining geometric insights into high-dimensional data

"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and say 'fourteen' very loudly. Everyone does it." - Geoff Hinton Overview

Contextual Dynamics Laboratory 1.8k Dec 29, 2022
Render Jupyter notebook in the terminal

jut - JUpyter notebook Terminal viewer. The command line tool view the IPython/Jupyter notebook in the terminal. Install pip install jut Usage $jut --

Kracekumar 169 Dec 27, 2022