A Simple Example for Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env

Last update: Nov 23, 2022

Overview

Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env

This repository implements a simple algorithm for imitation learning: DAGGER. In this example, the agent only learns to control the steer [-1, 1], the speed is computed automatically in gym_torcs.TorcsEnv.

Requirements

Ubuntu (I only test on this)
Python 3
TensorLayer and TensorFlow
Gym-Torcs

Setting Up

It is a little bit boring to set up the environment, but any incorrect configurations will lead to FAILURE. After installing Gym-Torcs, please follow the instructions to confirm everything work well:

Open a terminal:
- Run sudo torcs -vision to start a game
- Race --> Practice --> Configure Race: set the driver to scr_server 1 instead of player
- Open Torcs server by selecting Race --> Practice --> New Race: This should result that Torcs keeps a blue screen with several text information.
Open another terminal:
- Run python snakeoil3_gym.py on another terminal, it will shows how the fake AI control the car.
- Press F2 to see the driver view.
Set image size to 64x64x3:
- The model is trained on 64x64 RGB observation.
- Run sudo torcs -vision to start a game
- Options --> Display --> select 64x64 --> Apply

Usage

Make sure everything above work well and then run:

python dagger.py

It will start a Torcs server at the beginning of every episode, and terminate the server when the car crashs or the speed is too low. Note that, the self-contained gym_torcs.py is modified from Gym-Torcs, you can try different settings (like default speed, terminated speed) by modifying it.

Results

After Episode 1, the car crashes after 315 steps.

After Episode 3, the car does not crash anymore !!!

The number of steps and episodes might vary depending on the parameters initialization.

ENJOY !

You might also like...

PyTorch implementation of SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching

SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching This is the official PyTorch implementation of SMODICE: Versatile Offline I

14 Aug 30, 2022

Neon-erc20-example - Example of creating SPL token and wrapping it with ERC20 interface in Neon EVM

Example of wrapping SPL token by ERC2-20 interface in Neon Requirements Install

7 Mar 28, 2022

Example-custom-ml-block-keras - Custom Keras ML block example for Edge Impulse

Custom Keras ML block example for Edge Impulse This repository is an example on

8 Nov 2, 2022

Python-kafka-reset-consumergroup-offset-example - Python Kafka reset consumergroup offset example

Python Kafka reset consumergroup offset example This is a simple example of how

1 Feb 16, 2022

Pytorch code for "State-only Imitation with Transition Dynamics Mismatch" (ICLR 2020)

This repo contains code for our paper State-only Imitation with Transition Dynamics Mismatch published at ICLR 2020. The code heavily uses the RL mach

20 Sep 8, 2022

[CVPR 2022] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision (Oral)

PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision Kehong Gong*, Bingbing Li*, Jianfeng Zhang*, Ta

256 Dec 28, 2022

Learning to Estimate Hidden Motions with Global Motion Aggregation

Learning to Estimate Hidden Motions with Global Motion Aggregation (GMA) This repository contains the source code for our paper: Learning to Estimate

221 Dec 18, 2022

Official repository for the CVPR 2021 paper "Learning Feature Aggregation for Deep 3D Morphable Models"

Deep3DMM Official repository for the CVPR 2021 paper Learning Feature Aggregation for Deep 3D Morphable Models. Requirements This code is tested on Py

38 Dec 27, 2022

A pytorch reproduction of { Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation }.

A PyTorch Reproduction of HCN Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation. Ch

210 Dec 31, 2022

Comments

About the convergence and overfit

Hi, thanks for your job and I rewrite it using Keras in the attitude of learning. And I use your recommended hyper-parameters but when I run my program it's apt to overfit. Later on, I change the hyper-parameters , add BN and explicit initialization function of each layer. But it's still overfitting and the car runs 700 steps at the best time but still can't go through the all track. I have spent more than two weeks to tune it. I'm so confused of the tuning, why the same hyper-parameters can't achieve the same result? Why the network is so apt to overfit? For convenience, I update my programmer imitationLearning.py Can you give me some idea? Than you in advance.

opened by marooncn 0

A Simple Example for Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env

Related tags

Overview

Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env

Requirements

Setting Up

Usage

Results

You might also like...

PyTorch implementation of SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching

Neon-erc20-example - Example of creating SPL token and wrapping it with ERC20 interface in Neon EVM

Example-custom-ml-block-keras - Custom Keras ML block example for Edge Impulse

Python-kafka-reset-consumergroup-offset-example - Python Kafka reset consumergroup offset example

Pytorch code for "State-only Imitation with Transition Dynamics Mismatch" (ICLR 2020)

[CVPR 2022] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision (Oral)

Learning to Estimate Hidden Motions with Global Motion Aggregation

Official repository for the CVPR 2021 paper "Learning Feature Aggregation for Deep 3D Morphable Models"

A pytorch reproduction of { Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation }.

Comments

About the convergence and overfit

Releases(0.1)

0.1(Aug 10, 2017)

Owner

Hao

Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models (published in ICLR2018)

Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

[arXiv] What-If Motion Prediction for Autonomous Driving ❓🚗💨

Reimplementation of NeurIPS'19: "Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting" by Shu et al.

Pytorch implementation of AREL

NaturalProofs: Mathematical Theorem Proving in Natural Language

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

Pytorch implementation for the Temporal and Object Quantification Networks (TOQ-Nets).

Implementation of TimeSformer, a pure attention-based solution for video classification

Codes of paper "Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling"

这是一个利用facenet和retinaface实现人脸识别的库，可以进行在线的人脸识别。

No-reference Image Quality Assessment(NIQA) Algorithms (BRISQUE, NIQE, PIQE, RankIQA, MetaIQA)

Multi-Modal Machine Learning toolkit based on PyTorch.

This repository includes the official project for the paper: TransMix: Attend to Mix for Vision Transformers.

Generating Fractals on Starknet with Cairo

This repository is a basic Machine Learning train & validation Template (Using PyTorch)

GANimation: Anatomically-aware Facial Animation from a Single Image (ECCV'18 Oral) [PyTorch]

This repository contains the source code for the paper Tutorial on amortized optimization for learning to optimize over continuous domains by Brandon Amos

In this project we combine techniques from neural voice cloning and musical instrument synthesis to achieve good results from as little as 16 seconds of target data.

Hardware accelerated, batchable and differentiable optimizers in JAX.