TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition

Related tags

Deep LearningTGRNet
Overview

TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition

Xue, Wenyuan, et al. "TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition." arXiv preprint arXiv:2106.10598 (2021).

This work has been accepted for presentation at ICCV2021. The preview version has released at arXiv.org (https://arxiv.org/abs/2106.10598).

Abstract

A table arranging data in rows and columns is a very effective data structure, which has been widely used in business and scientific research. Considering large-scale tabular data in online and offline documents, automatic table recognition has attracted increasing attention from the document analysis community. Though human can easily understand the structure of tables, it remains a challenge for machines to understand that, especially due to a variety of different table layouts and styles. Existing methods usually model a table as either the markup sequence or the adjacency matrix between different table cells, failing to address the importance of the logical location of table cells, e.g., a cell is located in the first row and the second column of the table. In this paper, we reformulate the problem of table structure recognition as the table graph reconstruction, and propose an end-to-end trainable table graph reconstruction network (TGRNet) for table structure recognition. Specifically, the proposed method has two main branches, a cell detection branch and a cell logical location branch, to jointly predict the spatial location and the logical location of different cells. Experimental results on three popular table recognition datasets and a new dataset with table graph annotations (TableGraph-350K) demonstrate the effectiveness of the proposed TGRNet for table structure recognition.

Getting Started

Requirements

Create the environment from the environment.yml file conda env create --file environment.yml or install the software needed in your environment independently. If you meet some problems when installing PyTorch Geometric, please follow the official installation indroduction (https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html).

dependencies:
  - python==3.7.0
  - pip==20.2.4
  - pip:
    - dominate==2.5.1
    - imageio==2.8.0
    - networkx==2.3
    - numpy==1.18.2
    - opencv-python==4.4.0.46
    - pandas==1.0.3
    - pillow==7.1.1
    - torchfile==0.1.0
    - tqdm==4.45.0
    - visdom==0.1.8.9
    - Polygon3==3.0.8

PyTorch Installation

# CUDA 10.2
pip install torch==1.5.0 torchvision==0.6.0
# CUDA 10.1
pip install torch==1.5.0+CU101 torchvision==0.6.0+CU101 -f https://download.pytorch.org/whl/torch_stable.html
# CUDA 9.2
pip install torch==1.5.0+CU92 torchvision==0.6.0+CU92 -f https://download.pytorch.org/whl/torch_stable.html

PyTorch Geometric Installation

pip install torch-scatter==2.0.4 -f https://pytorch-geometric.com/whl/torch-1.5.0+${CUDA}.html
pip install torch-sparse==0.6.3 -f https://pytorch-geometric.com/whl/torch-1.5.0+${CUDA}.html
pip install torch-cluster==1.5.4 -f https://pytorch-geometric.com/whl/torch-1.5.0+${CUDA}.html
pip install torch-spline-conv==1.2.0 -f https://pytorch-geometric.com/whl/torch-1.5.0+${CUDA}.html
pip install torch-geometric

where ${CUDA} should be replaced by your specific CUDA version (cu92, cu101, cu102).

Datasets Preparation

cd ./datasets
tar -zxvf datasets.tar.gz
## The './datasets/' folder should look like:
- datasets/
  - cmdd/
  - icdar13table/
  - icdar19_ctdar/
  - tablegraph24k/

Pretrained Models Preparation

IMPORTANT Acoording to feedbacks from users (I also tested by myself), the pretrained models may not work for some enviroments. I have tested the following enviroment that can work as expected.

  - CUDA 9.2
  - torch 1.7.0+torchvision 0.8.0
  - torch-cluster 1.5.9
  - torch-geometric 1.6.3
  - torch-scatter 2.0.6
  - torch-sparse 0.6.9
  - torch-spline-conv 1.2.1
  • Download pretrained models from Google Dive or Alibaba Cloud.
  • Put checkpoints.tar.gz in "./checkpoints/" and extract it.
cd ./checkpoints
tar -zxvf checkpoints.tar.gz
## The './checkpoints/' folder should look like:
- checkpoints/
  - cmdd_overall/
  - icdar13table_overall/
  - icdar19_lloc/
  - tablegraph24k_overall/

Test

We have prepared scripts for test and you can just run them.

- test_cmdd.sh
- test_icdar13table.sh
- test_tablegraph-24k.sh
- test_icdar19ctdar.sh

Train

Todo

Owner
Wenyuan
Beijing Jiaotong University
Wenyuan
Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods

Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods Introduction Graph Neural Networks (GNNs) have demonstrated

37 Dec 15, 2022
Anonymize BLM Protest Images

Anonymize BLM Protest Images This repository automates @BLMPrivacyBot, a Twitter bot that shows the anonymized images to help keep protesters safe. Us

Stanford Machine Learning Group 40 Oct 13, 2022
Dados coletados e programas desenvolvidos no processo de iniciação científica

Iniciacao_cientifica_FAPESP_2020-14845-6 Dados coletados e programas desenvolvidos no processo de iniciação científica Os arquivos .py são os programa

1 Jan 10, 2022
Migration of Edge-based Distributed Federated Learning

FedFly: Towards Migration in Edge-based Distributed Federated Learning About the research Due to mobility, a device participating in Federated Learnin

qub-blesson 11 Nov 13, 2022
A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset.

A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset. This repo contains scripts to train RL agents to navigate the closed world and collect vi

MUGEN 11 Oct 22, 2022
Dark Finix: All in one hacking framework with almost 100 tools

Dark Finix - Hacking Framework. Dark Finix is a all in one hacking framework wit

Md. Nur habib 2 Feb 18, 2022
A boosting-based Multiple Instance Learning (MIL) package that includes MIL-Boost and MCIL-Boost

A boosting-based Multiple Instance Learning (MIL) package that includes MIL-Boost and MCIL-Boost

Jun-Yan Zhu 27 Aug 08, 2022
基于Pytorch实现优秀的自然图像分割框架!(包括FCN、U-Net和Deeplab)

语义分割学习实验-基于VOC数据集 usage: 下载VOC数据集,将JPEGImages SegmentationClass两个文件夹放入到data文件夹下。 终端切换到目标目录,运行python train.py -h查看训练 (torch) Li Xiang 28 Dec 21, 2022

Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

ACTION-Net Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21). Getting Started EgoGesture data folder struct

V-Sense 171 Dec 26, 2022
Repo for paper "Dynamic Placement of Rapidly Deployable Mobile Sensor Robots Using Machine Learning and Expected Value of Information"

Repo for paper "Dynamic Placement of Rapidly Deployable Mobile Sensor Robots Using Machine Learning and Expected Value of Information" Notes I probabl

Berkeley Expert System Technologies Lab 0 Jul 01, 2021
Multi-agent reinforcement learning algorithm and environment

Multi-agent reinforcement learning algorithm and environment [en/cn] Pytorch implements multi-agent reinforcement learning algorithms including IQL, Q

万鲲鹏 7 Sep 20, 2022
E2VID_ROS - E2VID_ROS: E2VID to a real-time system

E2VID_ROS Introduce We extend E2VID to a real-time system. Because Python ROS ca

Robin Shaun 7 Apr 17, 2022
Perturb-and-max-product: Sampling and learning in discrete energy-based models

Perturb-and-max-product: Sampling and learning in discrete energy-based models This repo contains code for reproducing the results in the paper Pertur

Vicarious 2 Mar 14, 2022
Relative Uncertainty Learning for Facial Expression Recognition

Relative Uncertainty Learning for Facial Expression Recognition The official implementation of the following paper at NeurIPS2021: Title: Relative Unc

35 Dec 28, 2022
DISTIL: Deep dIverSified inTeractIve Learning.

DISTIL: Deep dIverSified inTeractIve Learning. An active/inter-active learning library built on py-torch for reducing labeling costs.

decile-team 110 Dec 06, 2022
Megaverse is a new 3D simulation platform for reinforcement learning and embodied AI research

Megaverse Megaverse is a new 3D simulation platform for reinforcement learning and embodied AI research. The efficient design of the engine enables ph

Aleksei Petrenko 191 Dec 23, 2022
Image super-resolution through deep learning

srez Image super-resolution through deep learning. This project uses deep learning to upscale 16x16 images by a 4x factor. The resulting 64x64 images

David Garcia 5.3k Dec 28, 2022
Malware Bypass Research using Reinforcement Learning

Malware Bypass Research using Reinforcement Learning

Bobby Filar 76 Dec 26, 2022
An introduction to bioimage analysis - http://bioimagebook.github.io

Introduction to Bioimage Analysis This book tries explain the main ideas of image analysis in a practical and engaging way. It's written primarily for

Bioimage Book 20 Nov 28, 2022
A model that attempts to learn and benefit from data collected on card counting.

A model that attempts to learn and benefit from data collected on card counting. A decision tree like model is built to win more often than loose and increase the bet of the player appropriately to c

1 Dec 17, 2021