Image Similarity Challenge

Goal of the Competition

Competitors built models to help detect whether a given query image is derived from any of the images in a large reference set.

Content tracing is a crucial component on all social media platforms today, used for such tasks as flagging misinformation and manipulative advertising, preventing uploads of graphic violence, and enforcing copyright protections. But when dealing with the billions of new images generated every day on sites like Facebook, manual content moderation just doesn't scale. They depend on algorithms to help automatically flag or remove bad content.

This competition allowed participants to test their skills in building a key part of that content tracing system, and in so doing contribute to making social media more trustworthy and safe for the people who use it.

A reference image is manipulated to produce new images.
In this challenge competitors built models to detect whether a given query image is derived from a reference set.

There were two tracks to this challenge:

For the Matching Track, competitors created models that directly detect whether a query image is derived from one of the images in a large corpus of reference images.
For the Descriptor Track, competitors generated useful vector representations of images (up to 256 dimensions). These descriptors are compared with Euclidean distance to detect whether a query image is derived from one of the images in a large corpus of reference images.

Winning Submissions

See below for links to winning submissions' arXiv papers and code.

Matching Track

Place	Team or User	Code	Paper	Score	Summary of Model
1	VisionForce	GitHub repository	D2LV: A Data-Driven and Local-Verification Approach for Image Copy Detection	0.8329	A "data-driven and local-verification (D^2LV)" approach using pre-training on a set of basic and advanced image augmentations, and a global-local and local-global matching strategy for testing.
2	separate	GitHub repository	2nd Place Solution to Facebook AI Image Similarity Challenge Matching Track	0.8291	A Vision Transformer approach that uses concatenated query and reference images to learn the relationship between query and reference images directly.
3	imgFp	GitHub repository	3rd Place: A Global and Local Dual Retrieval Solution to Facebook AI Image Similarity Challenge	0.7682	A global+local recall approach with EsViT for global recall and SIFT point features for local recall.

Descriptor Track

Place	Team or User	Code	Paper	Score	Summary of Model
1	lyakaap	GitHub repository	Contrastive Learning with Large Memory Bank and Negative Embedding Subtraction for Accurate Copy Detection	0.6354	Uses an EfficientNet backbone trained with contrastive loss and cross-batch memory, and a training neighbor subtraction step in post-processing.
2	S-square	GitHub repository	Producing augmentation-invariant embeddings from real-life imagery	0.5905	Ensembles EfficientNet and NFNet backbones using an ArcFace loss function, and applies a sample normalization step in post-processing.
3	VisionForce	GitHub repository	Bag of Tricks and A Strong baseline for Image Copy Detection	0.5788	Uses a pretrained Barlow Twins model, yolov5 model to detect overlays, and a descriptor stretching step in post-processing.

Winners of the Facebook Image Similarity Challenge

Related tags

Overview

Image Similarity Challenge

Goal of the Competition

Winning Submissions

Matching Track

Descriptor Track

Owner

DrivenData

Raindrop strategy for Irregular time series

This repository comes with the paper "On the Robustness of Counterfactual Explanations to Adverse Perturbations"

TransReID: Transformer-based Object Re-Identification

Cancer-and-Tumor-Detection-Using-Inception-model - In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks, specifically here the Inception model by google.

Vision-Language Pre-training for Image Captioning and Question Answering

Lazy, a tool for running things in idle time

An executor that performs image segmentation on fashion items

Finding all things on-prem Microsoft for password spraying and enumeration.

PaddleRobotics is an open-source algorithm library for robots based on Paddle, including open-source parts such as human-robot interaction, complex motion control, environment perception, SLAM positioning, and navigation.

PyTorch implementation of Pay Attention to MLPs

An implementation of an abstract algebra for music tones (pitches).

Duke Machine Learning Winter School: Computer Vision 2022

《Truly shift-invariant convolutional neural networks》(2021)

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

Repository for the NeurIPS 2021 paper: "Exploiting Domain-Specific Features to Enhance Domain Generalization".

The repository is for safe reinforcement learning baselines.

Federated Deep Reinforcement Learning for the Distributed Control of NextG Wireless Networks.

Build an Amazon SageMaker Pipeline to Transform Raw Texts to A Knowledge Graph

This is the code for the paper "Jinkai Zheng, Xinchen Liu, Wu Liu, Lingxiao He, Chenggang Yan, Tao Mei: Gait Recognition in the Wild with Dense 3D Representations and A Benchmark. (CVPR 2022)"

Hierarchical probabilistic 3D U-Net, with attention mechanisms (—𝘈𝘵𝘵𝘦𝘯𝘵𝘪𝘰𝘯 𝘜-𝘕𝘦𝘵, 𝘚𝘌𝘙𝘦𝘴𝘕𝘦𝘵) and a nested decoder structure with deep supervision (—𝘜𝘕𝘦𝘵++).