A Python library for reading, writing and visualizing the OMEGA Format

Overview

OMEGA Format - Python Library

This module is developed by ika - RWTH Aachen as a contribution to the VVM project which aims to develop test procedures and to provide frameworks and methods for the safety verification of automated vehicles. VVM is working on the use case of Urban Intersections and focuses on driving functions up to full automation of vehicles (SAE Level 4 and 5).

As part of the project a data format for storing reference and perception data from pilotings, test drives and simulation in urban traffic is developed. This module enables the creation, reading and visualization of data conforming to this data format. Additionally, it can check files for conformance and perform basic sanity checks on the data.

Data Format

The base of both, the reference data format and the perception data format is the HDF5 file format. This library utilizes h5py to interact with those.

Reference Data

The OMEGA Format reference recording format is used to store data that represents the 'true' state of road users, infrastructure information, weather and more during a piloting, testing or simulation. The representation is on an object list basis. The following diagram shows an overview of the hierarchy in the OMEGA Format reference recording format. A more detailed description can be found in the specification document and the signal list.

Perception Data

The PerceptionRecording format is used to store data that represents what a vehicle under test, sensor under test or similar perceives from its surroundings. It is designed to be compared against the ReferenceRecording format. The following diagram shows an overview of the hierarchy in the PerceptionRecording format. A more detailed description is coming soon.

Installation

The dependencies are managed with conda environments. Conda can be installed following the conda installation instructions.

To create a new conda environment omega_env and install the module run the following in your console:

conda env create -n omega_env -f environment_visualization.yml
conda activate omega_env

If you want an editable install (modifications to the files in the directory are immediately used by the module) run:

pip install -e .[visualization]

To update the existing environment upon addition of new dependencies:

conda env update -n omega_env -f environment_visualization.yml

Usage

Validate your data

To check if your file conforms to the specification run.

omega_format verify --reference <FILENAME>

or

omega_format verify --perception <FILENAME>

When using the library and creating objects or rading from an hdf5 file, by default, sanity checks are performed. To circumvent those pass validate=False to the from_hdf5 function or use or use cls.construct instead of cls as the constructor of an object. In the backend pydantic is used for the sanity checks.

Visualize your data

To visualize a data file execute the following in your conda environment:

omega_format visualize --snip --max-snippets=2 <FILENAME>

or in Python:

import omega_format
from omega_format.visualization import Visualizer, SnippetContainer
reference_recording = omega_format.ReferenceRecording.from_hdf5('path/to/the/reference_recording_file.hdf5')
visualizer = Visualizer(SnippetContainer.create_list(references=reference_recording))
visualizer.start_gui_and_visualization()

A window will open that lets you interact with and inspect your data.

Extending the visualizer

By subclassing omega_format.visualization.VisualizationModule and adding an instance of your subclass to the visualizers list of the Visualizer you can extend the functionality of the visualizer. The subclass has to implement at least one of the functions visualize_static and visualize_dynamics, returning a list of pyqt widgets to plot. For more details take a look at the omega_format.vis.VisualizationModule or the other modules defined in the visualization.modules directory.

Create a reference data

This module maps the reference and perception data file specifications to a hierarchy of python classes. The root classes are ReferenceRecording and PerceptionRecording respectively. First initializing an object from that class and fill its properties with the objects of the classes in question (e.g. Weather RoadUser, Lane). After adding all your data, call to_hdf5 on the ReferenceRecording or PerceptionRecording and a format compliant hdf5 file will be created for you.

import numpy as np
import omega_format
from datetime import datetime

rr = omega_format.ReferenceRecording(meta_data=omega_format.MetaData(recorder_number=1,
                                                                     recording_number=1,
                                                                     daytime=datetime.now()
                                                                     reference_point_lat=50.786687,
                                                                     reference_point_lon=6.046312),
                               timestamps=omega_format.Timestamps(val=np.array([0])),
                               )
rr.weather = omega_foramt.Weather()
rr.road_users[0] = omega_format.RoadUser(type=omega_format.ReferenceTypes.RoadUser.Type.CAR, sub_type=omega_format.ReferenceTypes.RoadUser.SubType.General.REGULAR,
                                    birth=0, bb=omega_format.BoundingBox(np.array([2,3,0])),
                                    tr=omega_format.Trajectory(pos_x=np.array([0]),pos_y=np.array([0]),pos_z=np.array([0]),
                                                          roll=np.array([0]),pitch=np.array([0]),heading=np.array([0])))
rr.roads[0] = omega_format.Road(location=omega_format.ReferenceTypes.RoadLocation.URBAN)
rr.to_hdf5('test.hdf5')

Further Help

Standalone viewer of hdf5 files

There are plenty of tools, e.g.

Documentation

You can create a documentation with pdoc3. To do this first install pdoc3 with pip install pdoc3 and then run pdoc3 --http localhost:8889 --template-dir .\doc\templates\ .\omega_format from the root of this repo to view the documentation in your web browser.

License

The library is published under the MIT license specified in LICENSE. An overview over the licenses of the dependencies in this library is listed in LICENSES_OF_REQUIREMENTS.md.

Contact

In case of questions regarding the format, this repository or otherwise related feel free to raise an issue or contact Michael Schuldes ([email protected]).

Acknowledgement

The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Energy within the project “Verifikations- und Validierungsmethoden automatisierter Fahrzeuge im urbanen Umfeld". The authors would like to thank the consortium for the successful cooperation.

bmwi_logo

You might also like...
Script to generate a massive volume of data in sql, csv, json or xml format

DataGenerator Made with Python Open for pull requests 1. Dependencies To install required dependencies run pip install -r requirements.txt 2. Executi

Export watched content from Tautulli to the Letterboxd CSV Import Format

Export watched content from Tautulli to the Letterboxd CSV Import Format

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type. It provides a command line utility, Python library and plugins for various editors to quickly sort all your imports.

ticktock is a minimalist library to view Python time performance of Python code.

ticktock is a minimalist library to view Python time performance of Python code.

RapidFuzz is a fast string matching library for Python and C++

RapidFuzz is a fast string matching library for Python and C++, which is using the string similarity calculations from FuzzyWuzzy

pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.
pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.

pydsinternals - Directory Services Internals Library A Python native library containing necessary classes, functions and structures to interact with W

Library for processing molecules and reactions in python way

Chython [ˈkʌɪθ(ə)n] Library for processing molecules and reactions in python way. Features: Read/write/convert formats: MDL .RDF (.RXN) and .SDF (.MOL

A simple gpsd client and python library.

gpsdclient A small and simple gpsd client and library Installation Needs Python 3 (no other dependencies). If you want to use the library, use pip: pi

Comments
  • ISO 8601

    ISO 8601

    Metadata daytime timestamp should be ISO 8601 compliant to reduce ambiguity:

    https://github.com/ika-rwth-aachen/omega_format/blob/745f67d774d2da04201de9fe24fa24468a8b191b/omega_format/meta_data.py#L81 ->

    daytime=datetime.strptime(cls.assure_string(dt), '%Y-%m-%dT%H:%M:%S.%f%z') if dt is not None else None,
    
    opened by kai-storms 0
  • Typo in FlatMarkingType enum

    Typo in FlatMarkingType enum

    FlatMarkingType.Plain should probably be FlatMarkingType.Plane. This is clear from the documentation anyhow, but just to avoid confusion ;-)

    https://github.com/ika-rwth-aachen/omega_format/blob/4bd733044128ea0008bd495cfb077d831600a4c2/omega_format/enums/reference_types.py#L217

    opened by lu-w 0
Releases(v4.0)
Owner
Institut für Kraftfahrzeuge, RWTH Aachen, ika
Institut für Kraftfahrzeuge, RWTH Aachen, ika
BOLT12 Lightning Address Format

BOLT12 Address Support (DRAFT!) Inspired by the awesome lightningaddress.com, except for BOLT12: Supports BOLT12 Allows BOLT12 vendor string authentic

Rusty Russell 28 Sep 14, 2022
password generator

Password generator technologies used What is? It is Password generator How to Download? Download on releases Clone repo git clone https://github.com/m

1 Dec 16, 2021
Software to help automate collecting crowdsourced annotations using Mechanical Turk.

Video Crowdsourcing Software to help automate collecting crowdsourced annotations using Mechanical Turk. The goal of this project is to enable crowdso

Mike Peven 1 Oct 25, 2021
✨ Un code pour voir les disponibilités des vaccins contre le covid totalement fait en Python par moi, et en français.

Vaccine Notifier ❗ Un chois aléatoire d'un article sur Wikipedia totalement fait en Python par moi, et en français. 🔮 Grâce a une requète API, on peu

MrGabin 3 Jun 06, 2021
Playing with python imports and inducing those pesky errors.

super-duper-python-imports In this repository we are playing with python imports and inducing those pesky ImportErrors. File Organization project │

James Kelsey 2 Oct 14, 2021
Create powerful passwords easily and with many options with this program

Password_Generator About the Program: You can create powerful passwords with this program with many options easily! Features: You can copy the generat

Sina.f 0 Jul 14, 2022
Make some improvements in the Pizza class and pizzashop file by refactoring.

Make some improvements in the Pizza class and pizzashop file by refactoring.

James Brucker 1 Oct 18, 2021
A script copies movie and TV files to your GD drive, or create Hard Link in a seperate dir, in Emby-happy struct.

torcp A script copies movie and TV files to your GD drive, or create Hard Link in a seperate dir, in Emby-happy struct. Usage: python3 torcp.py -h Exa

ccf2012 105 Dec 22, 2022
pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.

pydsinternals - Directory Services Internals Library A Python native library containing necessary classes, functions and structures to interact with W

Podalirius 36 Dec 14, 2022
Simple RGB to HEX game made in python

Simple RGB to HEX game made in python

5 Aug 26, 2022
Aurin - A quick AUR installer for Arch Linux. Install packages from AUR website in a click.

Aurin - A quick AUR installer for Arch Linux. Install packages from AUR website in a click.

Suleman 51 Nov 04, 2022
Cleaning-utils - a collection of small Python functions and classes which make cleaning pipelines shorter and easier

cleaning-utils [] [] [] cleaning-utils is a collection of small Python functions

4 Aug 31, 2022
Creating low-level foundations and abstractions for asynchronous programming in Python.

DIY Async I/O Creating low-level foundations and abstractions for asynchronous programming in Python (i.e., implementing concurrency without using thr

Doc Jones 4 Dec 11, 2021
Homebase Name Changer for Fortnite: Save the World.

Homebase Name Changer This program allows you to change the Homebase name in Fortnite: Save the World. How to use it? After starting the HomebaseNameC

PRO100KatYT 7 May 21, 2022
Python type-checker written in Rust

pravda Python type-checker written in Rust Features Fully typed with annotations and checked with mypy, PEP561 compatible Add yours! Installation pip

wemake.services 31 Oct 21, 2022
Shypan, a simple, easy to use, full-featured library written in Python.

Shypan, a simple, easy to use, full-featured library written in Python.

ShypanLib 4 Dec 08, 2021
ZX Spectrum Utilities: (zx-spectrum-utils)

Here are a few utility programs that can be used with the zx spectrum. The ZX Spectrum is one of the first home computers from the early 1980s.

Graham Oakes 4 Mar 07, 2022
Easy compression and extraction for any compression or archival format.

Tzar: Tar, Zip, Anything Really Easy compression and extraction for any compression or archival format. Usage/Examples tzar compress large-dir compres

DanielVZ 37 Nov 02, 2022
Create password - Generate Random Password with Passphrase

Generate Random Password with Passphrase This is a python code to generate stron

1 Jan 18, 2022
Finger is a function symbol recognition engine for binary programs

Finger is a function symbol recognition engine for binary programs

332 Jan 01, 2023