A model to predict steering torque fully end-to-end

Overview

torque_model

The torque model is a spiritual successor to op-smart-torque, which was a project to train a neural network to control a car's steering fully end to end.

The input is the current wheel angle and future wheel angle (among other things), and the net's output is what torque the human was applying at the time to reach that future state smoothly and confidently. This bypasses the need to manually tune a PID, LQR, or INDI controller, while gaining human-like control over the steering wheel.

Needs to be cloned into an openpilot repo to take advantage of its tools.

The problem

As talked about in great detail and with a simple thought experiment in comma.ai's blog post here about end to end lateral planning, the same concept of behavioral cloning not being able to recover from disturbances applies here.

Behavior cloning and lack of perturbations

The way we generate automatically-labeled training data for a model that predicts how to control a steering wheel is rather simple; any time a human is driving we just take the current (t0s) and future (t0.3s) steering wheel angles and then just have the model predict whatever torque the human was applying at t0s to get us there.

This seems to work great, and the validation loss also seems to be really low! However, when you actually try to drive on this model or put it in a simulator, you can quickly see that any small disturbances (like wind, road camber, etc) quickly lead to a feedback loop or just plain inability to correct back to our desired steering angle.

This is due to the automatically-generated training and validation data containing only samples where the current and future (desired during runtime) steering wheel angles are very close together (just a couple degrees), as a symptom of only using data where the future angle is just fractions of a second away.

To fully realize the problem, think about what would happen if you wanted this model to predict what a human would actuate if the steering wheel is centered, but our desired angle is something like 90 degrees. As the model has never seen a difference of angles higher than just a couple of degrees, it either outputs a very small torque value, or just nonsense, as this input is vastly outside of its training distribution.

The solution

The solution talked about in the blog post above is to use a very simple simulator to warp the input video to be offset left or right, and then tell the model what path the human actually drove. A similar approach can also be taken here, where we generate random samples with an arbitrary steering wheel angle error, and then use a simple model of steering wheel torque, like a PF (proportional-feedforward) controller as the output to predict.

For the example above where we start at 0 degrees and want to reach 90 degrees, we can inject samples into the training data where we have that exact situation and then have the output be what a simple PF controller would output. Then during runtime in the car, when the model corrects for this arbitrary high angle error situation, the current and desired steering wheel angles become much closer together, and the model can then use its knowledge of how humans control under these circumstances.

The future

The current model described and implememted here is non-temporal, meaning the model has no knowledge of the past, where the steering wheel was, and inferring where it's heading. While the input data includes the steering angle rate, there's a lot of information missing it could use to improve its predictions, as well as a model bug where including the angle rate during runtime causes very smoothed and laggy predictions (probably due to the generated synthetic samples not taking any angle rate into account).

Ideally the model has some knowledge of the past, however this means we need an accurate simulator to train the model with perturbations added, so it can correct for disturbances in the real world.

Owner
Shane Smiskol
I mess around with self driving cars, neural networks, and real world data!
Shane Smiskol
scikit-learn is a python module for machine learning built on top of numpy / scipy

About scikit-learn is a python module for machine learning built on top of numpy / scipy. The purpose of the scikit-learn-tutorial subproject is to le

Gael Varoquaux 122 Dec 12, 2022
jaxfg - Factor graph-based nonlinear optimization library for JAX.

Factor graphs + nonlinear optimization in JAX

Brent Yi 134 Dec 21, 2022
Convoys is a simple library that fits a few statistical model useful for modeling time-lagged conversions.

Convoys is a simple library that fits a few statistical model useful for modeling time-lagged conversions. There is a lot more info if you head over to the documentation. You can also take a look at

Better 240 Dec 26, 2022
JMP is a Mixed Precision library for JAX.

Mixed precision training [0] is a technique that mixes the use of full and half precision floating point numbers during training to reduce the memory bandwidth requirements and improve the computatio

DeepMind 108 Dec 31, 2022
A Time Series Library for Apache Spark

Flint: A Time Series Library for Apache Spark The ability to analyze time series data at scale is critical for the success of finance and IoT applicat

Two Sigma 970 Jan 04, 2023
TensorFlow implementation of an arbitrary order Factorization Machine

This is a TensorFlow implementation of an arbitrary order (=2) Factorization Machine based on paper Factorization Machines with libFM. It supports: d

Mikhail Trofimov 785 Dec 21, 2022
Responsible AI Workshop: a series of tutorials & walkthroughs to illustrate how put responsible AI into practice

Responsible AI Workshop Responsible innovation is top of mind. As such, the tech industry as well as a growing number of organizations of all kinds in

Microsoft 9 Sep 14, 2022
stability-selection - A scikit-learn compatible implementation of stability selection

stability-selection - A scikit-learn compatible implementation of stability selection stability-selection is a Python implementation of the stability

185 Dec 03, 2022
Time-series momentum for momentum investing strategy

Time-series-momentum Time-series momentum strategy. You can use the data_analysis.py file to find out the best trigger and window for a given asset an

Victor Caldeira 3 Jun 18, 2022
TensorFlow Decision Forests (TF-DF) is a collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models.

TensorFlow Decision Forests (TF-DF) is a collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models. The library is a collection of Keras models

538 Jan 01, 2023
Exemplary lightweight and ready-to-deploy machine learning project

Exemplary lightweight and ready-to-deploy machine learning project

snapADDY GmbH 6 Dec 20, 2022
Scikit-Learn useful pre-defined Pipelines Hub

Scikit-Pipes Scikit-Learn useful pre-defined Pipelines Hub Usage: Install scikit-pipes It's advised to install sklearn-genetic using a virtual env, in

Rodrigo Arenas 1 Apr 26, 2022
Iris-Heroku - Putting a Machine Learning Model into Production with Flask and Heroku

Puesta en Producción de un modelo de aprendizaje automático con Flask y Heroku L

Jesùs Guillen 1 Jun 03, 2022
Tutorials, examples, collections, and everything else that falls into the categories: pattern classification, machine learning, and data mining

**Tutorials, examples, collections, and everything else that falls into the categories: pattern classification, machine learning, and data mining.** S

Sebastian Raschka 4k Dec 30, 2022
BentoML is a flexible, high-performance framework for serving, managing, and deploying machine learning models.

Model Serving Made Easy BentoML is a flexible, high-performance framework for serving, managing, and deploying machine learning models. Supports multi

BentoML 4.4k Jan 04, 2023
Responsible Machine Learning with Python

Examples of techniques for training interpretable ML models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.

ph_ 624 Jan 06, 2023
Automated Time Series Forecasting

AutoTS AutoTS is a time series package for Python designed for rapidly deploying high-accuracy forecasts at scale. There are dozens of forecasting mod

Colin Catlin 652 Jan 03, 2023
Anytime Learning At Macroscale

On Anytime Learning At Macroscale Learning from sequential data dumps (key) Requirements Python 3.7 Pytorch 1.9.0 Hydra 1.1.0 (pip install hydra-core

Meta Research 8 Mar 29, 2022
A Python package to preprocess time series

Disclaimer: This package is WIP. Do not take any APIs for granted. tspreprocess Time series can contain noise, may be sampled under a non fitting rate

Maximilian Christ 57 Dec 17, 2022
Neural Machine Translation (NMT) tutorial with OpenNMT-py

Neural Machine Translation (NMT) tutorial with OpenNMT-py. Data preprocessing, model training, evaluation, and deployment.

Yasmin Moslem 29 Jan 09, 2023