2021 Real Robot Challenge Phase2 attemp

Overview

Real_Robot_Challenge_Phase2_AE_attemp

We(team name:thriftysnipe) are the first place winner of Phase1 in 2021 Real Robot Challenge.
Please see this page for more details: https://real-robot-challenge.com/leaderboard
To see more details about out Phase1 works: https://github.com/wq13552463699/Real_Robot_challenge
We were granted the access to Phase 2.

I am sorry, the project is too complex with too much large files, It is too hard to upload them all on Github. I just attached a part of the core code here for you to take a quick lreview. If you think my attempts is approriate, you can go to this Google Drive to download the full project file(all codes, results, trained models, environmental files,.etc):
https://drive.google.com/file/d/14vjCrWU6vzMdXxVSR2FeskMvuQpgqWqM/view?usp=sharing

RRC phase2 task description:

Randomly place 25 dices with the size of 0.01x0.01x0.01m in the environment. Use own controller to drive the three-finger robot to rearrange the dice to a specific pattern. Unfortunately, due to the set task is too difficult, no team could complete the task on the actual robot, so all teams with record are awarded third place in this phase. But I think our attempt has a reference value, if later scholars conduct related research, our method may be useful.

Our considerations:

We consider using a reinforcement learning algorithm as the controller in this phase. However, in this phase, information that can play as observations, such as coordinates and orientation of the dices, cannot be obtained from the environment directly but they are crucial for RL to run.
The alternative observations we can use are the images of the three cameras set in 3 different angles in the environment and their segmentation masks. We picked segmentation masks rather than the raw images since the attendance of noise and redundancy in the raw images were too much. Please see the following segmentation mask example(RGB's 3 channels represent segmentation masks from 3 different angles).

The segmentation masks have the dimension of 270x270x3, if directly passing it to the RL agent, which would lead to computational explosion and hard to converge. Hence, we planned to use some means to extract the principal components that can play as observations from it. In addition, the observation value also includes readable read-robot data(joint angle of the robot arm, end effector position, end effector speed, etc.).

Segmentation mask dimensionality reduction

This is the most important part of this task. We tried different methods, such as GAN, VAE, AE, to extract the principal conponents from the images. The quality of data dimensionality reduction can be easily seem from the discripency of reconstructed and oringinal images or the loss curves. After many trials(adjusting hyperparameters, network structure, depth, etc.), we got different trained VAE, GAN and AE models. We conducted offline tests on the obtained model and compared the results, we were surprised to find that the AE performed the best. When the latent of AE is 384, the quality of the reconstructed image is the best. The result is shown in the figure below.

The loss function also converges to an acceptable range:

Build up observation and trian RL agent.

We use the best AE encoder to deal with the segmentation masks to generate the observation and stitch with the readable data. The structure of the overall obervation is shown as follow:
We fed the above observations to several current cutting-edge model based and model free reinforcement learning algorithms, including DDPG+HER, PPO, SLAC, PlaNet and Dreamer. We thought it would work and enable the agent to learn for somewhat anyway. But it is a pity that after many attempts, the model still didn't have any trend to converge. Due to time limited, our attempts were over here.

Some reasons might lead to fail

  1. We used AE as the observation model. Although the AE's dimensionality reduction capability were the best, the latent space of AE were disordered and didn't make sense to RL agent. The observations passed to the RL must be fixed and orderly. Continuous delivery of unfixed data caused a dimensional disaster. For example, the third number in the observation vector passed at t1 represents 'infos of the 1st dice', and the number on the same position at t2 represents the 'infos of the 3rd dice'. This disorderly change with time makes RL very confused.
  2. The extracted latent space from segmentation mask dominates the observations, making RL ignore the existence of robots. The latent space size is 384, but which for the robot data is 27. The two are far apart, and there is a big data bias.
  3. Robot arm blocked the dices, segmentation masks can only represent a part of the dice. This problem cannot be avoided and can only be solved by more powerful image processing technology. This is also a major challenge in the current Image-based RL industry

Contribution

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Please make sure to update tests as appropriate.

Owner
Qiang Wang
PhD at UCD. Research interest: Reinforcement Learning; Computer vision&Touch; Representation learning
Qiang Wang
This is the remake of the program PYOBD. It works on Python3 and all new libraries. It was tested on Linux, Windows, and it should work on MAC too.

This is the remake of the program PYOBD. It works on Python3 and all new libraries. It was tested on Linux, Windows, and it should work on MAC too. You just need an ELM327 USB or bluetooth device and

127 Jan 06, 2023
MPY tool - manage files on devices running MicroPython

mpytool MPY tool - manage files on devices running MicroPython It is an alternative to ampy Target of this project is to make more clean code, faster,

Pavel Revak 5 Aug 17, 2022
Plug and Play on Internet of Things with LoRa wireless modulation.

IoT-PnP Plug and Play on Internet of Things with LoRa wireless modulation. Device Side In the '505_PnP' folder has a modified ardunino template code s

Lambert Yang 1 May 19, 2022
Controlling fireworks with micropython

Controlling-fireworks-with-micropython How the code works line 1-4 from machine

Montso Mokake 1 Jan 08, 2022
Modi2-firmware-updater - MODI+ Firmware Updater With Python

MODI+ Firmware Updater 실행 준비 python3(파이썬3.9 혹은 그 이상의 버전)를 컴퓨터에 설치 python3 -m pip

LUXROBO 1 Feb 04, 2022
Homeautomation system created with Raspberry Pi 3 and Firebase.

Homeautomation System - Raspberry Pi 3 Desenvolvido com Python, Flask com AJAX e Firebase permite o controle local e remoto Itens necessários Raspberr

Joselino Santos 0 Mar 09, 2022
Simple Weather Check base on Hefeng api, Work on raspberry Pi

Simple Weather Check base on Hefeng api, Work on raspberry Pi

Retr0mous 28 Sep 17, 2022
从零开始打造一个智能家居系统

SweetHome 从零开始打造智能家居系统的尝试,主要的实现有 可以扫码添加设备并控制设备的Android App 可以控制亮灭的灯,并可以设置在Android App连接到指定Wifi后自动亮起 可以控制开关的窗帘,机械结构部分自己设计并3D打印出来 树莓派主控,实现Http请求接口和ZigBe

金榜 5 May 01, 2022
Parametric open source reconstructions of Voron printed parts

The Parametric Voron This repository contains Fusion 360 reconstructions of various printed parts from the Voron printers

Matthew Lloyd 26 Dec 19, 2022
ROS2 nodes for Waveshare Alphabot2-Pi mobile robot.

ROS2 for Waveshare Alphabot2-Pi This repo contains ROS2 packages for the Waveshare Alphabot2-Pi mobile robot: alphabot2: it contains the nodes used to

Michele Rizzo 2 Oct 11, 2022
The software that powers the sPot: a 4th generation

This code is meant to accompany this project in which a Spotify client is built into an iPod "Classic" from 2004. Everything is meant to run on a Raspberry Pi Zero W.

Guy Dupont 683 Dec 28, 2022
A set of postprocessing scripts and macro to accelerate the gyroid infill print speed with Klipper

A set of postprocessing scripts and macro to accelerate the gyroid infill print speed with Klipper

Jérôme W. 75 Jan 07, 2023
A simple small scale electric car was build which can be driven by remote control and features a fully autonomous parking procedure.

personal-autonomous-parking-car-raspberry A simple electric car model was build using Raspbery pi. The car has remote control and autonomous operation

Kostas Ziovas 2 Jan 26, 2022
Create (templateable) cameras that display qr codes in homeassistant

QRCam This custom component creates cameras displaying qrcodes. The QRCodes can be static or generated from templates. If you use a template as conten

Jannes Müller 5 Oct 06, 2022
Example for Calculating Robot Dynamics Using Pinocchio Library

A Example for Calculating Robot Dynamics Using Pinocchio Library Developed by: Xinyang Tian. Platform: Linux + Pinocchio. In this work, i use Pinocchi

Rot_Tianers 33 Dec 28, 2022
Custom component for interacting with Octopus Energy

Home Assistant Octopus Energy ** WARNING: This component is currently a work in progress ** Custom component built from the ground up to bring your Oc

David Kendall 116 Jan 02, 2023
Andreas Frisch 1 Jan 10, 2022
Home Assistant integration for energy consumption data from UK SMETS (Smart) meters using the Hildebrand Glow API.

Hildebrand Glow (DCC) Integration Home Assistant integration for energy consumption data from UK SMETS (Smart) meters using the Hildebrand Glow API. T

Aniket 153 Dec 30, 2022
🐱🖨Cat printer is a portable thermal printer sold on AliExpress for around $20.

Cat printer is a portable thermal printer sold on AliExpress for around $20. This repository contains Python code for talking to the cat printer over

671 Jan 05, 2023
Vvim - Keyboardless Vim interactions

This is done via a hardware glove that the user wears. The glove detects the finger's positions and translates them into key presses. It's currently a work in progress.

Boyd Kane 8 Nov 17, 2022