Codebase for BMVC 2021 paper "Text Based Person Search with Limited Data"

Last update: Nov 24, 2022

Related tags

Deep Learning TextReID

Overview

Text Based Person Search with Limited Data

This is the codebase for our BMVC 2021 paper.

Please bear with me refactoring this codebase after CVPR deadline 😅

Abstract

Text-based person search (TBPS) aims at retrieving a target person from an image gallery with a descriptive text query. Solving such a fine-grained cross-modal retrieval task is challenging, which is further hampered by the lack of large-scale datasets. In this paper, we present a framework with two novel components to handle the problems brought by limited data. Firstly, to fully utilize the existing small-scale benchmarking datasets for more discriminative feature learning, we introduce a cross-modal momentum contrastive learning framework to enrich the training data for a given mini-batch. Secondly, we propose to transfer knowledge learned from existing coarse-grained large-scale datasets containing image-text pairs from drastically different problem domains to compensate for the lack of TBPS training data. A transfer learning method is designed so that useful information can be transferred despite the large domain gap. Armed with these components, our method achieves new state of the art on the CUHK-PEDES dataset with significant improvements over the prior art in terms of Rank-1 and mAP.

Comments

Research prepared to obtain a diploma degree in computer and Automation Engineering.

Hello!

My research focuses on Person search using Visual-Textual Attributes. Having said that, I would like to use your model to assist me in my project, but I have some issues when I finish train and test the model. My problem is trying to write code to run the model to get the same response as the photo. so Can you help me please!

opened by ram7772 6
Cannot find test_query and train_query folders
Hi @BrandonHanx

In the ReadMe file, it is mentioned to setup the datasets dir as follows:

└── cuhkpedes ├── annotations │ ├── test.json │ ├── train.json │ └── val.json ├── clip_vocab_vit.npy └── imgs ├── cam_a ├── cam_b ├── CUHK01 ├── CUHK03 ├── Market ├── test_query └── train_query

After downloading the cuhkpedes data set, we get only the imgs folder, containing cam_a, cam_b and CUHK01 folders. there is no test_query and train_query folders. Also, these folders are not in the repository. Could you provide more information regarding on these folders, more exactly, what kind of information they contain and how they must be set up?

Also, there are few more folders that are not part of the cuhkpedes, such as CUHK03 and Market. Do we need these data sets to reproduce the results?

Best regards, liviust
opened by liviust 5
some problem in training and testing

Hello

I have some problem. first: I don't find test_query and train_query file when I get images from [Dr. Shuang Li] second: I have this problem for testing and training.

opened by ram7772 4
Problem about the clip_vocab_vit.npy

Hi :) I have a question about the pre-processing document clip_vocab_vit.npy. My understanding is that it contains the tensor of the CLIP-Text-Encoder output corresponding to each word (total 9408). My question is, the output dimension of CLIP-TEXT-ENCODER is 1024, but the tensor dimension of each word in clip_vocab_vit.npy is 512. Is there some other operation in it? Thanks

opened by Frost-Yang-99 2
There is only caption_all.json in the dataset CUHK-PEDES, what are the train.json and test.json in the dataset part
Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

Go to '...'

Click on '....'

Scroll down to '....'

See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]

Browser [e.g. chrome, safari]

Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]

OS: [e.g. iOS8.1]

Browser [e.g. stock browser, safari]

Version [e.g. 22]

Additional context Add any other context about the problem here.
opened by SwimKY 1

Releases(v0.1.1)

v0.1.1(Dec 10, 2021)

Full Changelog: https://github.com/BrandonHanx/TextReID/compare/v0.1.0...v0.1.1
Source code(tar.gz)
Source code(zip)

Owner

Xiao Han

Ph.D. student @ UoSurrey CVSSP, B.Eng. @ ZJU ISEE

GitHub Repository

Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms

FNet: Mixing Tokens with Fourier Transforms Pytorch implementation of Fnet : Mixing Tokens with Fourier Transforms. Citation: @misc{leethorp2021fnet,

218 Jan 05, 2023

This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.

Multimodal Deep Learning 🎆 🎆 🎆 Announcing the multimodal deep learning repository that contains implementation of various deep learning-based model

398 Dec 30, 2022

Official Python implementation of the FuzionCoin protocol

PyFuzc Official Python implementation of the FuzionCoin protocol WARNING: Under construction. Use at your own risk. Some functions may not work. Setup

3 Jul 07, 2022

Code for Boundary-Aware Segmentation Network for Mobile and Web Applications

BASNet Boundary-Aware Segmentation Network for Mobile and Web Applications This repository contain implementation of BASNet in tensorflow/keras. comme

8 Nov 24, 2022

Numerical Methods with Python, Numpy and Matplotlib

Numerical Bric-a-Brac Collections of numerical techniques with Python and standard computational packages (Numpy, SciPy, Numba, Matplotlib ...). Diffe

10 Dec 20, 2021

MiniSom is a minimalistic implementation of the Self Organizing Maps

MiniSom Self Organizing Maps MiniSom is a minimalistic and Numpy based implementation of the Self Organizing Maps (SOM). SOM is a type of Artificial N

1.2k Jan 03, 2023

PyTorch implementation of "Optimization Planning for 3D ConvNets"

Optimization-Planning-for-3D-ConvNets Code for the ICML 2021 paper: Optimization Planning for 3D ConvNets. Authors: Zhaofan Qiu, Ting Yao, Chong-Wah N

2 Jan 12, 2022

Face and other object detection using OpenCV and ML Yolo

Object-and-Face-Detection-Using-Yolo- Opencv and YOLO object and face detection is implemented. You only look once (YOLO) is a state-of-the-art, real-

3 Feb 15, 2022

Implementation of TimeSformer, a pure attention-based solution for video classification

TimeSformer - Pytorch Implementation of TimeSformer, a pure and simple attention-based solution for reaching SOTA on video classification.

602 Jan 03, 2023

Graph Regularized Residual Subspace Clustering Network for hyperspectral image clustering

5 Jul 18, 2022

A new test set for ImageNet

ImageNetV2 The ImageNetV2 dataset contains new test data for the ImageNet benchmark. This repository provides associated code for assembling and worki

186 Dec 18, 2022

Code for "ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on", accepted at WACV 2021 Generation of Human Behavior Workshop.

ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on [ Paper ] [ Project Page ] This repository contains the code fo

97 Dec 13, 2022

Codebase for BMVC 2021 paper "Text Based Person Search with Limited Data"

Related tags

Overview

Text Based Person Search with Limited Data

Abstract

Comments

Research prepared to obtain a diploma degree in computer and Automation Engineering.

Cannot find test_query and train_query folders

some problem in training and testing

Problem about the clip_vocab_vit.npy

There is only caption_all.json in the dataset CUHK-PEDES, what are the train.json and test.json in the dataset part

Releases(v0.1.1)

v0.1.1(Dec 10, 2021)

Owner

Xiao Han

Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms

This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.

Official Python implementation of the FuzionCoin protocol

Code for Boundary-Aware Segmentation Network for Mobile and Web Applications

Numerical Methods with Python, Numpy and Matplotlib

MiniSom is a minimalistic implementation of the Self Organizing Maps

PyTorch implementation of "Optimization Planning for 3D ConvNets"

Face and other object detection using OpenCV and ML Yolo

Implementation of TimeSformer, a pure attention-based solution for video classification

Graph Regularized Residual Subspace Clustering Network for hyperspectral image clustering

A new test set for ImageNet

Code for "ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on", accepted at WACV 2021 Generation of Human Behavior Workshop.

Full-featured Decision Trees and Random Forests learner.

Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection

load .txt to train YOLOX, same as Yolo others

Simulation environments for the CrazyFlie quadrotor: Used for Reinforcement Learning and Sim-to-Real Transfer

Pyramid Scene Parsing Network, CVPR2017.

Automate issue discovery for your projects against Lightning nightly and releases.

Code of Classification Saliency-Based Rule for Visible and Infrared Image Fusion

LightNet++: Boosted Light-weighted Networks for Real-time Semantic Segmentation