VISNOTATE: An Opensource tool for Gaze-based Annotation of WSI Data

Overview

VISNOTATE: An Opensource tool for Gaze-based Annotation of WSI Data

Introduction

This repo contains the source code for 'Visnotate' which is a tool that can be used to track gaze patterns on Whole Slide Images (WSI) in the svs format. Visnotate was used to evaluate the efficacy of gaze-based labeling of histopathology data. The details of our research on gaze-based annotation can be found in the following paper:

  • Komal Mariam, Osama Mohammed Afzal, Wajahat Hussain, Muhammad Umar Javed, Amber Kiyani, Nasir Rajpoot, Syed Ali Khurram and Hassan Aqeel Khan, "On Smart Gaze based Annotation of Histopathology Images for Training of Deep Convolutional Neural Networks", submitted to IEEE Journal of Biomedical and Health Informatics.

blockDiagram

Requirements

  • Openslide
  • Python 3.7

Installation and Setup

  1. Install openslide. This process is different depending on the operating system.

    Windows

    1. Download 64-bit Windows Binaries from the openslide download page. Direct link to download the latest version at the time of writing.
    2. Extract the zip archive.
    3. Copy all .dll files from bin to C:/Windows/System32.

    Debian/Ubuntu

    # apt-get install openslide-tools

    Arch Linux

    $ git clone https://aur.archlinux.org/openslide.git
    $ cd openslide
    $ makepkg -si

    macOS

    $ brew install openslide
  2. For some operating systems, tkinter needs to be installed as well.

    Debian/Ubuntu

    # apt-get install python3-tk

    Arch Linux

    # pacman -S tk
  3. (Optional) If recording gaze points using a tracker, install the necessary software from its website.

  4. Clone this repository.

    git clone https://github.com/UmarJ/lsiv-python3.git visnotate
    cd visnotate
    
  5. Create and activate a new python virtual environment if needed. Then install required python modules.

    python -m pip install -r requirements.txt
    
  6. (Optional) Start gaze tracking software in the background if tracking gaze points.

  7. Run interface_recorder.py.

    python interface_recorder.py
    

Supported Hardware and Software

At this time visinotate supports the GazePoint GP3, tracking hardware. WSI's are read using openslide software and we support only the .svs file format. We do have plans to add support for other gaze tracking hardware and image formats later.

Screenshots

The Visnotate Interface

Interface Screenshot

Collected Gazepoints

Gazepoints Screenshot

Generated Heatmap

Heatmap Screenshot

Reference

This repo was used to generate the results for the following paper on Gaze-based labelling of Pathology data.

  • Komal Mariam, Osama Mohammed Afzal, Wajahat Hussain, Muhammad Umar Javed, Amber Kiyani, Nasir Rajpoot, Syed Ali Khurram and Hassan Aqeel Khan, "On Smart Gaze based Annotation of Histopathology Images for Training of Deep Convolutional Neural Networks", submitted to IEEE Journal of Biomedical and Health Informatics.

BibTex Reference: Available after acceptance.

Owner
SigmaLab
This github account belongs to the Sigma Lab. Which is Dr. Hassan Aqeel Khan's research group.
SigmaLab
Official Implementation of "Designing an Encoder for StyleGAN Image Manipulation"

Designing an Encoder for StyleGAN Image Manipulation (SIGGRAPH 2021) Recently, there has been a surge of diverse methods for performing image editing

749 Jan 09, 2023
Robust Consistent Video Depth Estimation

[CVPR 2021] Robust Consistent Video Depth Estimation This repository contains Python and C++ implementation of Robust Consistent Video Depth, as descr

Facebook Research 213 Dec 17, 2022
Generating Images with Recurrent Adversarial Networks

Generating Images with Recurrent Adversarial Networks Python (Theano) implementation of Generating Images with Recurrent Adversarial Networks code pro

Daniel Jiwoong Im 121 Sep 08, 2022
[SIGIR22] Official PyTorch implementation for "CORE: Simple and Effective Session-based Recommendation within Consistent Representation Space".

CORE This is the official PyTorch implementation for the paper: Yupeng Hou, Binbin Hu, Zhiqiang Zhang, Wayne Xin Zhao. CORE: Simple and Effective Sess

RUCAIBox 26 Dec 19, 2022
Official PyTorch code for Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021)

Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021) This repository is the official PyTorc

Jingyun Liang 139 Dec 29, 2022
A tensorflow implementation of an HMM layer

tensorflow_hmm Tensorflow and numpy implementations of the HMM viterbi and forward/backward algorithms. See Keras example for an example of how to use

Zach Dwiel 283 Oct 19, 2022
Joint project of the duo Hacker Ninjas

Project Smoothie Společný projekt dua Hacker Ninjas. První pokus o hříčku po třech týdnech učení se programování. Jakub Kolář e:\

Jakub Kolář 2 Jan 07, 2022
Object detection, 3D detection, and pose estimation using center point detection:

Objects as Points Object detection, 3D detection, and pose estimation using center point detection: Objects as Points, Xingyi Zhou, Dequan Wang, Phili

Xingyi Zhou 6.7k Jan 03, 2023
BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition 2022)

BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition

Rui Qian 17 Dec 12, 2022
g9.py - Torch interactive graphics

g9.py - Torch interactive graphics A Torch toy in the browser. Demo at https://srush.github.io/g9py/ This is a shameless copy of g9.js, written in Pyt

Sasha Rush 13 Nov 16, 2022
UPSNet: A Unified Panoptic Segmentation Network

UPSNet: A Unified Panoptic Segmentation Network Introduction UPSNet is initially described in a CVPR 2019 oral paper. Disclaimer This repository is te

Uber Research 622 Dec 26, 2022
Testing and Estimation of structural breaks in Stata

xtbreak estimating and testing for many known and unknown structural breaks in time series and panel data. For an overview of xtbreak test see xtbreak

Jan Ditzen 13 Jun 19, 2022
Source code for "UniRE: A Unified Label Space for Entity Relation Extraction.", ACL2021.

UniRE Source code for "UniRE: A Unified Label Space for Entity Relation Extraction.", ACL2021. Requirements python: 3.7.6 pytorch: 1.8.1 transformers:

Wang Yijun 109 Nov 29, 2022
[ICCV2021] Official Pytorch implementation for SDGZSL (Semantics Disentangling for Generalized Zero-Shot Learning)

Semantics Disentangling for Generalized Zero-shot Learning This is the official implementation for paper Zhi Chen, Yadan Luo, Ruihong Qiu, Zi Huang, J

25 Dec 06, 2022
[TPDS'21] COSCO: Container Orchestration using Co-Simulation and Gradient Based Optimization for Fog Computing Environments

COSCO Framework COSCO is an AI based coupled-simulation and container orchestration framework for integrated Edge, Fog and Cloud Computing Environment

imperial-qore 39 Dec 25, 2022
An index of recommendation algorithms that are based on Graph Neural Networks.

An index of recommendation algorithms that are based on Graph Neural Networks.

FIB LAB, Tsinghua University 564 Jan 07, 2023
A python tutorial on bayesian modeling techniques (PyMC3)

Bayesian Modelling in Python Welcome to "Bayesian Modelling in Python" - a tutorial for those interested in learning how to apply bayesian modelling t

Mark Regan 2.4k Jan 06, 2023
This repo is customed for VisDrone.

Object Detection for VisDrone(无人机航拍图像目标检测) My environment 1、Windows10 (Linux available) 2、tensorflow = 1.12.0 3、python3.6 (anaconda) 4、cv2 5、ensemble

53 Jul 17, 2022
RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation

Multipath RefineNet A MATLAB based framework for semantic image segmentation and general dense prediction tasks on images. This is the source code for

Guosheng Lin 575 Dec 06, 2022
Computational inteligence project on faces in the wild dataset

Table of Contents The general idea How these scripts work? Loading data Needed modules and global variables Parsing the arrays in dataset Extracting a

tooraj taraz 4 Oct 21, 2022