An official PyTorch implementation of the paper "Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences", ICCV 2021.

Last update: Nov 05, 2022

Related tags

Overview

PyTorch implementation of Learning by Aligning (ICCV 2021)

This is an official PyTorch implementation of the paper "Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences", ICCV 2021.

For more details, visit our project site or see our paper.

Requirements

Python 3.8
PyTorch 1.7.1
GPU memory >= 11GB

Getting started

First, clone our git repository.

git clone https://github.com/cvlab-yonsei/LbA.git
cd LbA

Docker

You can use docker pull sanghslee/ps:1.7.1-cuda11.0-cudnn8-runtime

Prepare datasets

SYSU-MM01: download from this link.
- For SYSU-MM01, you need to preprocess the .jpg files into .npy files by running:
  - python utils/pre_preprocess_sysu.py --data_dir /path/to/SYSU-MM01
- Modify the dataset directory below accordingly.
  - L63 of train.py
  - L54 of test.py

Train

run python train.py --method full
Important:
- Performances reported during training does not reflect exact performances of your model. This is due to 1) evaluation protocols of the datasets and 2) random seed configurations.
- Make sure you seperately run test.py to obtain correct results to be reported in your paper.

Test

run python test.py --method full
The results should be around:

dataset	method	mAP	rank-1
SYSU-MM01	baseline	49.54	50.43
SYSU-MM01	full	54.14	55.41

Pretrained weights

Download [SYSU-MM01]
The results should be:

dataset	method	mAP	rank-1
SYSU-MM01	full	55.22	56.31

Bibtex

@article{park2021learning,
  title={Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences},
  author={Park, Hyunjong and Lee, Sanghoon and Lee, Junghyup and Ham, Bumsub},
  journal={arXiv preprint arXiv:2108.07422},
  year={2021}
}

Credits

Our implementation is based on Mang Ye's code here.

Comments

something about run this code

thanks for your code, there is something wrong when i run you code,in this line: loss = torch.mean(comask_pos * self.criterion(feat, feat_recon_pos, feat_recon_neg)) the wrong is:RuntimeError: The size of tensor a (9) must match the size of tensor b (18) at non-singleton dimension 3 could you give me some help?

opened by zhuchuanleiqq 12
When running "train. Py", there is a problem on line 132 of the "model. Py" file:

When running "train. Py", there is a problem on line（loss = torch.mean(comask_pos * self.criterion(feat, feat_recon_pos, feat_recon_neg))） 132 of the "model. Py" file: Traceback：RuntimeError: The size of tensor a (9) must match the size of tensor b (18) at non-singleton dimension 3

opened by redsoup 1
Question about the training speed

Thanks for your work.

When I tried to reproduce your results with an Nvidia 2080Ti (as recommended by the paper), however, the training speed seemed very slow. It nearly took 20 minutes for each epoch on SYSU-MM01, which mismatched with the reported 8 hours training time.

I have already used cuda for acceleration. Thus, I wonder how did this happen. Thank you.

opened by hansonchen1996 1
Problems about the performance

I have run your source code on both SYSU and RegDB datasets, but I didn't get the performance of your paper. So I want to know how to set the hyper-parameter to get the performance of your paper?

opened by Mrkkew 1
Visualization problem

Hello， Thanks for your great work, I am wondering about the visualization part, use mask and comask matrix in SYSU-MM01 dataset. Can I get some details about the steps of your visualization method? Thank you very much.

opened by sunset233 0

Releases(v1.0)

v1.0(Aug 22, 2021)

Source code(tar.gz)
Source code(zip)
sysu_pretrained.t(273.10 MB)

Owner

CV Lab @ Yonsei University

GitHub Repository

A Python wrapper for the tesseract-ocr API

tesserocr A simple, Pillow-friendly, wrapper around the tesseract-ocr API for Optical Character Recognition (OCR). tesserocr integrates directly with

1.7k Dec 31, 2022

Code for the paper STN-OCR: A single Neural Network for Text Detection and Text Recognition

STN-OCR: A single Neural Network for Text Detection and Text Recognition This repository contains the code for the paper: STN-OCR: A single Neural Net

496 Jan 05, 2023

Here use convulation with sobel filter from scratch in opencv python .

2 Nov 11, 2021

Bu uygulamada Python ve Opencv kullanarak bilgisayar kamerasından yüz tespiti yapıyoruz.

opencv_yuz_bulma Bu uygulamada Python ve Opencv kullanarak bilgisayar kamerasından yüz tespiti yapıyoruz. Bilgisarın kendi kamerasını kullanmak için;

6 Apr 16, 2022

OCR, Scene-Text-Understanding, Text Recognition

Scene-Text-Understanding Survey [2015-PAMI] Text Detection and Recognition in Imagery: A Survey paper [2014-Front.Comput.Sci] Scene Text Detection and

354 Dec 12, 2022

WACV 2022 Paper - Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching

Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching Code based on our WACV 2022 Accepted Paper: https://arxiv.org/pdf/

13 Dec 17, 2022

deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications

Automatic Weapon Detection Deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications. Loved the pro

4 Mar 04, 2022

利用Paddle框架复现CRAFT

CRAFT-Paddle 利用Paddle框架复现CRAFT CRAFT 本项目基于paddlepaddle框架复现CRAFT，并参加百度第三届论文复现赛，将在2021年5月15日比赛完后提供AIStudio链接～敬请期待参考项目： CRAFT: Character-Region Awarenes

2 Mar 07, 2022

Sign Language Recognition service utilizing a deep learning model with Long Short-Term Memory to perform sign language recognition.

Sign Language Recognition Service This is a Sign Language Recognition service utilizing a deep learning model with Long Short-Term Memory to perform s

1 Jan 08, 2022

Detect the mathematical formula from the given picture and the same formula is extracted and converted into the latex code

Mathematical formulae extractor The goal of this project is to create a learning based system that takes an image of a math formula and returns corres

6 May 22, 2022

An official PyTorch implementation of the paper "Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences", ICCV 2021.

Related tags

Overview

PyTorch implementation of Learning by Aligning (ICCV 2021)

Requirements

Getting started

Docker

Prepare datasets

Train

Test

Pretrained weights

Bibtex

Credits

Comments

something about run this code

When running "train. Py", there is a problem on line 132 of the "model. Py" file:

Question about the training speed

Problems about the performance

Visualization problem

Releases(v1.0)

v1.0(Aug 22, 2021)

Owner

CV Lab @ Yonsei University

A Python wrapper for the tesseract-ocr API

Code for the paper STN-OCR: A single Neural Network for Text Detection and Text Recognition

Here use convulation with sobel filter from scratch in opencv python .

Bu uygulamada Python ve Opencv kullanarak bilgisayar kamerasından yüz tespiti yapıyoruz.

OCR, Scene-Text-Understanding, Text Recognition

WACV 2022 Paper - Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching

deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications

利用Paddle框架复现CRAFT

Sign Language Recognition service utilizing a deep learning model with Long Short-Term Memory to perform sign language recognition.

Detect the mathematical formula from the given picture and the same formula is extracted and converted into the latex code

A set of workflows for corpus building through OCR, post-correction and normalisation

A synthetic data generator for text recognition

A curated list of papers and resources for scene text detection and recognition

Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

Convert scans of handwritten notes to beautiful, compact PDFs

Fully-automated scripts for collecting AI-related papers

零样本学习测评基准，中文版

Introduction to image processing, most used and popular functions of OpenCV

textspotter - An End-to-End TextSpotter with Explicit Alignment and Attention

Web interface for browsing arXiv papers