Garbage classification using structure data.

Overview

垃圾分类模型使用说明

1.包含以下数据文件

文件 描述
data/MaterialMapping.csv 物体以及其归类的信息
data/TestRecords 光谱原始测试数据 CSV 文件
data/TestRecordDesc.zip CSV 文件描述文件
data/Boundaries.csv 物体轮廓信息

2.包含以下模型文件

文件夹 描述
output/Category/ 包含预测大类别的分类模型
output/Material/ 包含预测大类别(4类)的分类模型
output/Backgroud/ 包含预测小类别(50类)的分类模型

3.环境配置

  进入garbage路径,在anaconda命令行运行pip install -r requirements.txt

4.数据预处理

  在anaconda命令行运行python data_preprocess.py,即可在data文件夹中生成AllEmbracingDataset.csv。若将来更新数据,按照和原来相同的格式和路径保存在data文件夹中,即可用data_preprocess.py生成更新后的数据集

  • 运行数据预处理Python脚本,将上述数据的信息集合到一个数据文件中
python code/data_preprocess.py -data_dir D:/datasets/garbage \
                        -test \
                        -groupbyObjID

运行脚本生成的数据文件 datasets/AllEmbracingDataset.csv 数据集

5.模型训练Python脚本

python code/train_gbdt_lr.py -data_dir D:/datasets/garbage/ \
                    -use_groupbyID True \
                    -output_dir output/ \
                    -skip_data_preprocess

其他 Python脚本说明:

  • feature_engineering.py 特征工程代码
  • ref.py 数据处理和模型推理所需的配置文件
  • utils.py 数据处理所需的一些函数
  • gbdt_feature.py 用gbdt模型生成特征

6.模型推理Python脚本

python code/predict_gbdt_lr.py -data_dir D:/datasets/garbage/ \
                    -use_groupbyID True \
                    -output_dir output/ \
                    -skip_data_preprocess \
                    -save_dir output/ 

  注1:只要同一个ObjID的多条数据的预测结果有一个不是背景零,最终预测结果就不是背景零。

  注2:预测出的Material只会是在训练数据中出现过的唯一标记号。这次数据中不同的唯一标记号共有148个,具体可参见output/log/log.txt中的LabelEncoder.classes

  • 预测结果文件(predictions.csv)说明:对每个物体(即每个ObjID,通常对应多条测试记录)给出多个预测结果汇总后的预测结果。
# 域名 意义
1 ObjID 被测物体唯一标记。同一物体会对应多条测试记录
2 Category 物体分类,从训练数据中获取
3 Material 物体对应的唯一标识号,从训练数据中获取
4 pred_Category 模型所预测出的物体分类
5 pred_Material 模型所预测出的物体唯一标识号
6 pred_background 模型预测的背景和物体 (背景标记为 0,物体标记为 1)
7 pred_Category_final 模型所预测出的物体分类
8 pred_Material_final 模型所预测出的物体材料分类

7. 模型精度

  对于Category、Material和Background三种场景的预测,我们均使用GBDT+LR模型。尝试过SVM、XGBoost、LightGBM和GBDT+LR模型,对比之下,GBDT+LR模型表现最好。   在测试集上的Accuracy如下:

场景 Accuracy
Category 0.7583130575831306
Material 0.6042173560421735
Background 0.996044825313118
Owner
wenqi
Learning is all you need!
wenqi
A community run, 5-day PyTorch Deep Learning Bootcamp

Deep Learning Winter School, November 2107. Tel Aviv Deep Learning Bootcamp : http://deep-ml.com. About Tel-Aviv Deep Learning Bootcamp is an intensiv

Shlomo Kashani. 1.3k Sep 04, 2021
Deep Learning Models for Causal Inference

Extensive tutorials for learning how to build deep learning models for causal inference using selection on observables in Tensorflow 2.

Bernard J Koch 151 Dec 31, 2022
code for CVPR paper Zero-shot Instance Segmentation

Code for CVPR2021 paper Zero-shot Instance Segmentation Code requirements python: python3.7 nvidia GPU pytorch1.1.0 GCC =5.4 NCCL 2 the other python

zhengye 86 Dec 13, 2022
Official implementation of "Can You Spot the Chameleon? Adversarially Camouflaging Images from Co-Salient Object Detection" in CVPR 2022.

Jadena Official implementation of "Can You Spot the Chameleon? Adversarially Camouflaging Images from Co-Salient Object Detection" in CVPR 2022. arXiv

Qing Guo 13 Nov 29, 2022
Multi-Agent Reinforcement Learning (MARL) method to learn scalable control polices for multi-agent target tracking.

scalableMARL Scalable Reinforcement Learning Policies for Multi-Agent Control CD. Hsu, H. Jeong, GJ. Pappas, P. Chaudhari. "Scalable Reinforcement Lea

Christopher Hsu 17 Nov 17, 2022
Optimus: the first large-scale pre-trained VAE language model

Optimus: the first pre-trained Big VAE language model This repository contains source code necessary to reproduce the results presented in the EMNLP 2

314 Dec 19, 2022
Deep Q-network learning to play flappybird.

AI Plays Flappy Bird I've trained a DQN that learns to play flappy bird on it's own. Try the pre-trained model First install the pip requirements and

Anish Shrestha 3 Mar 01, 2022
An Approach to Explore Logistic Regression Models

User-centered Regression An Approach to Explore Logistic Regression Models This tool applies the potential of Attribute-RadViz in identifying correlat

0 Nov 12, 2021
Codes and models of NeurIPS2021 paper - DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks

DominoSearch This is repository for codes and models of NeurIPS2021 paper - DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense n

11 Sep 10, 2022
CRF-RNN for Semantic Image Segmentation - PyTorch version

This repository contains the official PyTorch implementation of the "CRF-RNN" semantic image segmentation method, published in the ICCV 2015

Sadeep Jayasumana 170 Dec 13, 2022
PyTorch implementation of the TTC algorithm

Trust-the-Critics This repository is a PyTorch implementation of the TTC algorithm and the WGAN misalignment experiments presented in Trust the Critic

0 Nov 29, 2021
Memory-efficient optimum einsum using opt_einsum planning and PyTorch kernels.

opt-einsum-torch There have been many implementations of Einstein's summation. numpy's numpy.einsum is the least efficient one as it only runs in sing

Haoyan Huo 9 Nov 18, 2022
Pytorch implementation for the paper: Contrastive Learning for Cold-start Recommendation

Contrastive Learning for Cold-start Recommendation This is our Pytorch implementation for the paper: Yinwei Wei, Xiang Wang, Qi Li, Liqiang Nie, Yan L

45 Dec 13, 2022
Object tracking implemented with YOLOv4, DeepSort, and TensorFlow.

Object tracking implemented with YOLOv4, DeepSort, and TensorFlow. YOLOv4 is a state of the art algorithm that uses deep convolutional neural networks to perform object detections. We can take the ou

The AI Guy 1.1k Dec 29, 2022
PyTorch implementation of the NIPS-17 paper "Poincaré Embeddings for Learning Hierarchical Representations"

Poincaré Embeddings for Learning Hierarchical Representations PyTorch implementation of Poincaré Embeddings for Learning Hierarchical Representations

Facebook Research 1.6k Dec 25, 2022
Official implementation of "An Image is Worth 16x16 Words, What is a Video Worth?" (2021 paper)

An Image is Worth 16x16 Words, What is a Video Worth? paper Official PyTorch Implementation Gilad Sharir, Asaf Noy, Lihi Zelnik-Manor DAMO Academy, Al

213 Nov 12, 2022
Computer Vision Script to recognize first person motion, developed as final project for the course "Machine Learning and Deep Learning"

Overview of The Code BaseColab/MLDL_FPAR.pdf: it contains the full explanation of our work Base Colab: it contains the base colab used to perform all

Simone Papicchio 4 Jul 16, 2022
Bottleneck Transformers for Visual Recognition

Bottleneck Transformers for Visual Recognition Experiments Model Params (M) Acc (%) ResNet50 baseline (ref) 23.5M 93.62 BoTNet-50 18.8M 95.11% BoTNet-

Myeongjun Kim 236 Jan 03, 2023
Official code for 'Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentationon Complex Urban Driving Scenes'

PEBAL This repo contains the Pytorch implementation of our paper: Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentationon Complex Urba

Yu Tian 115 Dec 29, 2022
Source code release of the paper: Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation.

GNet-pose Project Page: http://guanghan.info/projects/guided-fractal/ UPDATE 9/27/2018: Prototxts and model that achieved 93.9Pck on LSP dataset. http

Guanghan Ning 83 Nov 21, 2022