Official implementation of SynthTIGER (Synthetic Text Image GEneratoR) ICDAR 2021

Overview

🐯 SynthTIGER: Synthetic Text Image GEneratoR

Official implementation of SynthTIGER | Paper | Datasets

Moonbin Yim1, Yoonsik Kim1, Han-cheol Cho1, Sungrae Park2

1 Clova AI Research, NAVER Corp.

2 Upstage AI Research





Contents

Updates

Datasets

SynthTIGER is available for download at google drive.

synthtiger_v1.0.zip (36G) (md5: 5b5365f4fe15de24e403a9256079be70)

  • Original paper version.

synthtiger_v1.1.zip (38G) (md5: b2757a7e2b5040b14ed64c473533b592)

  • Used MJ/ST lexicon instead of MJ/ST label.
  • Fixed a bug that applies transformation twice on curved text.
  • Fixed a bug that incorrectly converts grayscale to RGB.
Version IIIT5k SVT IC03 IC13 IC15 SVTP CUTE80 Total
1.0 93.2 87.3 90.5 92.9 72.1 77.7 80.6 85.9
1.1 93.4 87.6 91.4 93.2 73.9 77.8 80.6 86.6

Structure

The structure of the dataset is as follows. The dataset contains 10M images.

gt.txt
images/
    0/
        0.jpg
        1.jpg
        ...
        9998.jpg
        9999.jpg
    1/
    ...
    998/
    999/

The format of gt.txt is as follows. Image path and label are separated by tab. (<image_path>\t<label>)

images/0/0.jpg	10
images/0/1.jpg	date:
...
images/999/9999998.jpg	STUFFIER
images/999/9999999.jpg	Re:

Usage

# for macOS
$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

# install python packages
$ pip install -r requirements.txt

$ python gen.py --template TEMPLATE
                --config CONFIG
                --output OUTPUT
                [--count COUNT]
                [--worker WORKER]

Requirements

  • python >= 3.6
  • libraqm

Parameters

Name Type Default Description
template string Template module path
config string Config file path
output string Folder path to save data
count integer 100 Number of data
worker integer 1 Number of workers

Examples

Default text images

# horizontal
python gen.py --template templates/default.py --config templates/default_horizontal.yaml --output results --worker 4

# vertical
python gen.py --template templates/default.py --config templates/default_vertical.yaml --output results --worker 4

Multiline text images

python gen.py --template templates/multiline.py --config templates/multiline.yaml --output results --worker 4

Advanced Usage

Non-Latin language data generation

  1. Prepare corpus and fonts

    corpus - txt file, line by line (example)

    font - ttf/otf file (example)

  2. Extract renderable charsets

    python tools/extract_font_charset.py --input fonts --worker 4

    This script extracts renderable charsets for all font files. (example)

    Text files are generated in the input path with the same names as the fonts.

  3. Edit corpus path and font path in config file

  4. Run gen.py

Colormap customization

  1. Prepare images

    image - jpg/jpeg/png/bmp file

  2. Create colormaps

    python tools/create_colormap.py --input images --output colormap.txt --worker 4

    This script creates colormaps for all image files. (example)

  3. Edit colormap path in config file

  4. Run gen.py

Citation

@article{yim2021synthtiger,
  title={SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models},
  author={Yim, Moonbin and Kim, Yoonsik and Cho, Han-Cheol and Park, Sungrae},
  journal={arXiv preprint arXiv:2107.09313},
  year={2021}
}

License

SynthTIGER
Copyright (c) 2021-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

The following directories and their subdirectories are licensed the same as their origins. Please refer to NOTICE

docs/
docsrc/
resources/font/
Comments
  • How to get character bbox annotation?

    How to get character bbox annotation?

    this project is very helpful for generating synth text for scene text recognition, and it seem to generate text image by combine several character images, but the outputs doesn't contain information of each character, is it possible to get character annotation, for example, each character and its location?

    opened by GuokunWang 5
  • Question about background image

    Question about background image

    Hi, @moonbings Thank you for sharing nice work.

    I wonder where's the code that load background image such as below?

    image

    And I also wonder when the Synthdog will be released

    opened by yellowjs0304 3
  • training configuration of STR model

    training configuration of STR model

    Hi, thank you for open-sourcing your work. I have a question about the training configuration of STR model used in your paper. How did you set the sensitive character mode and data_filtering_off option in BEST model?

    opened by tzm-tora 2
  • gen.py throws munmap_chunk(): invalid pointer and stops generating images

    gen.py throws munmap_chunk(): invalid pointer and stops generating images

    On running gen.py with the command posted in the readme, the process throws

    munmap_chunk(): invalid pointer

    and is stuck. It stops generating images. Each worker generates a maximum of 1 image before this error is thrown.

    opened by arundprabhu 2
  • libraqm dependency error occurs

    libraqm dependency error occurs

    Thanks for the nice package! When i followed your descriptions and used it, i met some error. KeyError: 'setting text direction, language or font features is not supported without libraqm' I have installed python package 'synthtiger' and dependency following the shell script How can i solve this issue? Thanks,

    opened by mandal4 1
  • How to generate Non-Broken bengali synthetic data for text recognition?

    How to generate Non-Broken bengali synthetic data for text recognition?

    thank you for your awesome sharing. i tried to generate text recognition data using your library for bangla printed documents,i made all the necessary changes to make this synthesis engine working for bengali text recognition data generation, however for bangla it is breaking text during text to image conversion just like this trdg issue of mine : https://github.com/Belval/TextRecognitionDataGenerator/issues/253 (where the authors of that repo is not responsive at all)

    here is a sample that i got while using this synthesis engine.

    label : নির্লজ্জ

    image : https://i.ibb.co/0Y0W5GG/image.jpg ( নির্লজ্জ word got broken )

    do you know how to solve this issue? thanks a lot in advance for your great work.

    opened by mobassir94 1
  • Other latin language

    Other latin language

    Hi, great work ! What exactly should one modify to change language. I found this: https://github.com/clovaai/synthtiger/tree/master/resources/charset but not sure if I need to generate something from that.

    thanks in advance :)

    opened by Globolik 1
  • Fix seed bug and update test code

    Fix seed bug and update test code

    Description

    • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
    • Added retry parameter in generator function to make testing easier.
      • If retry is False, when an error occurs during data generation, it does not retry and return data of None value.
    • Updated test code.

    Changes in this PR

    • Updated main.py, gen.py.
    • Updated test code.

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update seed feature

    Update seed feature

    Description

    • Random seed is also applied to main process.
    • Added get_global_random_states, set_global_random_states, set_global_random_seed functions. This functions can be used in templates.

    Changes in this PR

    • Updated main.py, gen.py.
    • Added functions (get_global_random_states, set_global_random_states, set_global_random_seed).

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Add seed option

    Add seed option

    Description

    This engine cannot reproduce same data. So I added random seed option to solve this issue.

    I also changed multi processing structure. When the new engine puts tasks in the task queue, each worker gets and runs it. And each task is assigned an index. Note that the incoming indexes are random when storing data because the task completion times are different.

    Changes in this PR

    • Added random seed option. (-s or --seed)
    • Refactored main.py, gen.py
    • Updated README.md
    • Updated .pylintrc

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update ubuntu depends

    Update ubuntu depends

    Description

    Updated shell script to get latest package list before installing dependencies in ubuntu.

    Changes in this PR

    • Updated install_ubuntu_depends.sh.

    How has this been tested?

    Check CI test.

    opened by moonbings 0
  • Issue with synthtiger_v1.1.zip

    Issue with synthtiger_v1.1.zip

    Hi All,

    I have downloaded all parts of synthtiger_v1.1.zip. When I combined into a single zip file, the resulting zip file is not valid.

    Anyone faces similar problem ?

    Appreciate your advices.

    opened by rinabuoy 1
  • Improving documentation

    Improving documentation

    Hi team,

    Is there any chance you could improve the documentation for this library to describe what the functions and classes actually do? I'm having trouble figuring out how to utilize this library for different kinds of unstructured & semi-structured text images.

    opened by haydenedelson 0
  • Images are very distorted

    Images are very distorted

    Hello @moonbings During synthetic dataset generation, some image are very distorted and i don't have any idea how to fix it. I played with some parameters but it didnt work for me. Any solution?

    Samples image are not clear 23 33 83

    opened by khawar-islam 1
  • RuntimeError: Text is not visible

    RuntimeError: Text is not visible

    Hello @moonbings While generating Korean dataset, I have added all given requirements like fonts and everything. I am using below command to generate dataset but getting an error between data generation process.

    python -m synthtiger -o results -w 4 -v examples/synthtiger/template.py SynthTiger examples/synthtiger/config_vertical.yaml

    Generated 56 data
    Generated 57 data
    Generated 58 data
    Generated 59 data
    Generated 60 data
    Traceback (most recent call last):
      File "/media/cvpr/CM_22/synthtiger/synthtiger/gen.py", line 71, in _generate
        data = template.generate()
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 117, in generate
        image = _blend_images(fg_image, bg_image, self.visibility_check)
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 246, in _blend_images
        raise RuntimeError("Text is not visible")
    RuntimeError: Text is not visible
    
    Generated 61 data
    Generated 62 data
    Generated 63 data
    Generated 64 data
    Generated 65 data
    Generated 66 data
    
    opened by khawar-islam 1
Releases(1.2.1)
  • 1.2.1(Nov 11, 2022)

    • Fix seed bug and update test code (https://github.com/clovaai/synthtiger/pull/46)
      • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
      • Added retry parameter in generator function to make testing easier.
      • Updated test code.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Nov 10, 2022)

    • Updated seed feature (https://github.com/clovaai/synthtiger/pull/44)
      • Random seed is also applied to main process.
      • Added get_global_random_states, set_global_random_states, set_global_random_seed functions.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Nov 9, 2022)

    • Added character bbox and mask outputs (https://github.com/clovaai/synthtiger/pull/33, https://github.com/clovaai/synthtiger/pull/39)
    • Added seed option (https://github.com/clovaai/synthtiger/pull/42)
    • Changed multi processing structure (https://github.com/clovaai/synthtiger/pull/42)
      • The incoming indexes are random when storing data because the task completion times are different.
    • Updated ubuntu depends (https://github.com/clovaai/synthtiger/pull/41)
    • Updated some style and add some information in README.md (https://github.com/clovaai/synthtiger/pull/40)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(May 8, 2022)

  • 1.0.1(Feb 17, 2022)

  • 1.0.0(Feb 15, 2022)

Owner
Clova AI Research
Open source repository of Clova AI Research, NAVER & LINE
Clova AI Research
Code for "Long Range Probabilistic Forecasting in Time-Series using High Order Statistics"

Long Range Probabilistic Forecasting in Time-Series using High Order Statistics This is the code produced as part of the paper Long Range Probabilisti

16 Dec 06, 2022
An open-source, low-cost, image-based weed detection device for fallow scenarios.

Welcome to the OpenWeedLocator (OWL) project, an opensource hardware and software green-on-brown weed detector that uses entirely off-the-shelf compon

Guy Coleman 145 Jan 05, 2023
Meta graph convolutional neural network-assisted resilient swarm communications

Resilient UAV Swarm Communications with Graph Convolutional Neural Network This repository contains the source codes of Resilient UAV Swarm Communicat

62 Dec 06, 2022
FS2KToolbox FS2K Dataset Towards the translation between Face

FS2KToolbox FS2K Dataset Towards the translation between Face -- Sketch. Download (photo+sketch+annotation): Google-drive, Baidu-disk, pw: FS2K. For

Deng-Ping Fan 5 Jan 03, 2023
Official Code for AdvRush: Searching for Adversarially Robust Neural Architectures (ICCV '21)

AdvRush Official Code for AdvRush: Searching for Adversarially Robust Neural Architectures (ICCV '21) Environmental Set-up Python == 3.6.12, PyTorch =

11 Dec 10, 2022
Simple improvement of VQVAE that allow to generate x2 sized images compared to baseline

vqvae_dwt_distiller.pytorch Simple improvement of VQVAE that allow to generate x2 sized images compared to baseline. It allows to generate 512x512 ima

Sergei Belousov 25 Jul 19, 2022
Image Completion with Deep Learning in TensorFlow

Image Completion with Deep Learning in TensorFlow See my blog post for more details and usage instructions. This repository implements Raymond Yeh and

Brandon Amos 1.3k Dec 23, 2022
Unofficial implementation of "Coordinate Attention for Efficient Mobile Network Design"

Unofficial implementation of "Coordinate Attention for Efficient Mobile Network Design". CoordAttention tensorflow slim

Billy 9 Aug 22, 2022
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

DingDing 143 Jan 01, 2023
2021:"Bridging Global Context Interactions for High-Fidelity Image Completion"

TFill arXiv | Project This repository implements the training, testing and editing tools for "Bridging Global Context Interactions for High-Fidelity I

Chuanxia Zheng 111 Jan 08, 2023
GNN-based Recommendation Benchmark

GRecX A Fair Benchmark for GNN-based Recommendation Homepage and Documentation Homepage: Documentation: Paper: GRecX: An Efficient and Unified Benchma

73 Oct 17, 2022
Instant Real-Time Example-Based Style Transfer to Facial Videos

FaceBlit: Instant Real-Time Example-Based Style Transfer to Facial Videos The official implementation of FaceBlit: Instant Real-Time Example-Based Sty

Aneta Texler 131 Dec 19, 2022
LF-YOLO (Lighter and Faster YOLO) is used to detect defect of X-ray weld image.

This project is based on ultralytics/yolov3. LF-YOLO (Lighter and Faster YOLO) is used to detect defect of X-ray weld image. Download $ git clone http

26 Dec 13, 2022
NAVER BoostCamp Final Project

CV 14조 final project Super Resolution and Deblur module Inference code & Pretrained weight Repo SwinIR Deblur 실행 방법 streamlit run WebServer/Server_SRD

JiSeong Kim 5 Sep 06, 2022
Viewmaker Networks: Learning Views for Unsupervised Representation Learning

Viewmaker Networks: Learning Views for Unsupervised Representation Learning Alex Tamkin, Mike Wu, and Noah Goodman Paper link: https://arxiv.org/abs/2

Alex Tamkin 31 Dec 01, 2022
Implementation for our ICCV2021 paper: Internal Video Inpainting by Implicit Long-range Propagation

Implicit Internal Video Inpainting Implementation for our ICCV2021 paper: Internal Video Inpainting by Implicit Long-range Propagation paper | project

202 Dec 30, 2022
A disassembler for the RP2040 Programmable I/O State-machine!

piodisasm A disassembler for the RP2040 Programmable I/O State-machine! Usage Just run piodisasm.py on a file that contains the PIO code as hex! (Such

Ghidra Ninja 29 Dec 06, 2022
Efficient 6-DoF Grasp Generation in Cluttered Scenes

Contact-GraspNet Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, Dieter

NVIDIA Research Projects 148 Dec 28, 2022
Semantic Scholar's Author Disambiguation Algorithm & Evaluation Suite

S2AND This repository provides access to the S2AND dataset and S2AND reference model described in the paper S2AND: A Benchmark and Evaluation System f

AI2 54 Nov 28, 2022
Azua - build AI algorithms to aid efficient decision-making with minimum data requirements.

Project Azua 0. Overview Many modern AI algorithms are known to be data-hungry, whereas human decision-making is much more efficient. The human can re

Microsoft 197 Jan 06, 2023