Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

Last update: Jan 03, 2023

Related tags

Deep Learning zero-shot-image-to-text

Overview

Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

[Paper] [Colab is coming soon]

Approach

Example

Usage

To run captioning on a single image:

$ python run.py 
--reset_context_delta
--caption_img_path "example_images/captions/COCO_val2014_000000097017.jpg"

To run model on visual arithmetic:

$ python run.py 
--reset_context_delta
--end_factor 1.06
--fusion_factor 0.95
--grad_norm_factor 0.95
--run_type arithmetics
--arithmetics_imgs "example_images/arithmetics/woman2.jpg" "example_images/arithmetics/king2.jpg" "example_images/arithmetics/man2.jpg"
--arithmetics_weights 1 1 -1

To run model on real world knowledge:

$ python run.py
--reset_context_delta --cond_text "Image of" 
--end_factor 1.04 
--caption_img_path "example_images/real_world/simpsons.jpg"

To run model on OCR:

$ python run.py
--reset_context_delta --cond_text "Image of text that says" 
--end_factor 1.04 
--caption_img_path "example_images/OCR/welcome_sign.jpg"

Owner

GitHub Repository

Adaptive Graph Convolution for Point Cloud Analysis

Adaptive Graph Convolution for Point Cloud Analysis This repository contains the implementation of AdaptConv for point cloud analysis. Adaptive Graph

64 Dec 21, 2022

pytorch, hand(object) detect ,yolo v5，手检测

YOLO V5 物体检测，包括手部检测。项目介绍手部检测手部检测示例如下：视频示例：项目配置作者开发环境： Python 3.7 PyTorch = 1.5.1 数据集手部检测数据集该项目数据集采用 TV-Hand 和 COCO-Hand (COCO-Hand-Big 部分) 进

11 Dec 20, 2022

Fully Convlutional Neural Networks for state-of-the-art time series classification

Deep Learning for Time Series Classification As the simplest type of time series data, univariate time series provides a reasonably good starting poin

572 Dec 23, 2022

Code for reproducing key results in the paper "InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets"

Status: Archive (code is provided as-is, no updates expected) InfoGAN Code for reproducing key results in the paper InfoGAN: Interpretable Representat

1k Dec 19, 2022

x-transformers-paddle 2.x version

x-transformers-paddle x-transformers-paddle 2.x version paddle 2.x版本 https://github.com/lucidrains/x-transformers 。 requirements paddlepaddle-gpu==2.2

7 Dec 08, 2022

This repository is an open-source implementation of the ICRA 2021 paper: Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order Pooling.

Locus This repository is an open-source implementation of the ICRA 2021 paper: Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order

96 Dec 15, 2022

fastgradio is a python library to quickly build and share gradio interfaces of your trained fastai models.

34 Jan 05, 2023

A PyTorch implementation of the Relational Graph Convolutional Network (RGCN).

Torch-RGCN Torch-RGCN is a PyTorch implementation of the RGCN, originally proposed by Schlichtkrull et al. in Modeling Relational Data with Graph Conv

66 Nov 30, 2022

A 2D Visual Localization Framework based on Essential Matrices [ICRA2020]

A 2D Visual Localization Framework based on Essential Matrices This repository provides implementation of our paper accepted at ICRA: To Learn or Not

27 Nov 07, 2022

Model parallel transformers in Jax and Haiku

Mesh Transformer Jax A haiku library using the new(ly documented) xmap operator in Jax for model parallelism of transformers. See enwik8_example.py fo

4.8k Jan 01, 2023

An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi

MetaICL: Learning to Learn In Context This includes an original implementation of "MetaICL: Learning to Learn In Context" by Sewon Min, Mike Lewis, Lu

141 Jan 07, 2023

Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

Related tags

Overview

Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

Approach

Example

Usage

To run captioning on a single image:

To run model on visual arithmetic:

To run model on real world knowledge:

To run model on OCR:

Owner

Adaptive Graph Convolution for Point Cloud Analysis

pytorch, hand(object) detect ,yolo v5，手检测

Fully Convlutional Neural Networks for state-of-the-art time series classification

Code for reproducing key results in the paper "InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets"

x-transformers-paddle 2.x version

This repository is an open-source implementation of the ICRA 2021 paper: Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order Pooling.

fastgradio is a python library to quickly build and share gradio interfaces of your trained fastai models.

A PyTorch implementation of the Relational Graph Convolutional Network (RGCN).

A 2D Visual Localization Framework based on Essential Matrices [ICRA2020]

Model parallel transformers in Jax and Haiku

An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi

Research code of ICCV 2021 paper "Mesh Graphormer"

Official Pytorch implementation of MixMo framework

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

Kaggle | 9th place single model solution for TGS Salt Identification Challenge

Implementation of Axial attention - attending to multi-dimensional data efficiently

Extension to fastai for volumetric medical data

PyTorch implementation of Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose

(ICCV 2021) ProHMR - Probabilistic Modeling for Human Mesh Recovery

Improving Non-autoregressive Generation with Mixup Training