DRIFT is a tool for Diachronic Analysis of Scientific Literature.

Overview

Logo

GitHub license GitHub stars GitHub forks GitHub issues

Open in Streamlit arXiv


About

DRIFT is a tool for Diachronic Analysis of Scientific Literature. The application offers user-friendly and customizable utilities for two modes: Training and Analysis. Currently, the application supports customizable training of diachronic word embeddings with the TWEC model. The application supports a variety of analysis methods to monitor trends and patterns of development in scientific literature:

  1. Word Cloud
  2. Productivity/Frequency Plot
  3. Acceleration Plot
  4. Semantic Drift
  5. Tracking Clusters
  6. Acceleration Heatmap
  7. Track Trends with Similarity
  8. Keyword Visualisation
  9. LDA Topic Modelling

NOTE: The online demo is hosted using Streamlit sharing. This is a single-instance single-process deployment, accessible to all the visitors publicly (avoid sharing sensitive information on the demo). Hence, it is highly recommended to use your own independent local deployment of the application for a seamless and private experience. One can alternatively fork this repository and host it using Streamlit sharing.

We would love to know about any issues found on this repository. Please submit an issue for any query, or contact us here. If you use this application in your work, you can cite this repository and the paper here.


Setup

Clone the repository:

git clone https://github.com/rajaswa/DRIFT.git
cd DRIFT

Install the requirements:

make install_req

Data

The dataset we have used for our demo, and the analysis in the paper was scraped using the arXiv API (see script). We scraped papers from the cs.CL subject. This dataset is available here.

The user can upload their own dataset to the DRIFT application. The unprocessed dataset should be present in the following format (as a JSON file):

{
   <year_1>:[
      <paper_1>,
      <paper_2>,
      ...
   ],
   <year_2>:[
      <paper_1>,
      <paper_2>,
      ...
   ],
   ...,
   <year_m>:[
      <paper_1>,
      <paper_2>,
      ...
   ],
}

where year_x is a string (e.g., "1998"), and paper_x is a dictionary. An example is given below:

{
   "url":"http://arxiv.org/abs/cs/9809020v1",
   "date":"1998-09-15 23:49:32+00:00",
   "title":"Linear Segmentation and Segment Significance",
   "authors":[
      "Min-Yen Kan",
      "Judith L. Klavans",
      "Kathleen R. McKeown"
   ],
   "abstract":"We present a new method for discovering a segmental discourse structure of a\ndocument while categorizing segment function. We demonstrate how retrieval of\nnoun phrases and pronominal forms, along with a zero-sum weighting scheme,\ndetermines topicalized segmentation. Futhermore, we use term distribution to\naid in identifying the role that the segment performs in the document. Finally,\nwe present results of evaluation in terms of precision and recall which surpass\nearlier approaches.",
   "journal ref":"Proceedings of 6th International Workshop of Very Large Corpora\n  (WVLC-6), Montreal, Quebec, Canada: Aug. 1998. pp. 197-205",
   "category":"cs.CL"
}

The only important key is "abstract", which has the raw text. The user can name this key differently. See the Training section below for more details.


Usage

Launch the app

To launch the app, run the following command from the terminal:

streamlit run app.py

Train Mode

Preprocesing

The preprocessing stage takes a JSON file structured as shown in the Data section. They key for raw text is provided on which preprocessing takes place. During the preprocessing of the text, year-wise text files are created in a desired directory. During the preprocessing:

  • All html tags are removed from the text.
  • Contractions are replaced (e.g. 'don't' is converted to 'do not')
  • Punctuations, non-ascii characters, stopwords are removed.
  • All verbs are lemmatized.

After this, each processed text is stored in the respective year file separated by a new-line, along with all the data in a single file as compass.txt

Training

word_cloud_usage

The training mode uses the path where the processed text files are stored, and trains the TWEC model on the given text. The TWEC model trains a Word2Vec model on compass.txt and then the respective time-slices are trained on this model to get corresponding word vectors. In the sidebar, we provide several options like - whether to use Skipgram over CBOW, number of dynamic iterations for training, number of static iterations for training, negative sampling, etc. After training, we store the models at the specified path, which are used later in the analysis.

Analysis Mode

Word Cloud

word_cloud_usage

A word cloud, or tag cloud, is a textual data visualization which allows anyone to see in a single glance the words which have the highest frequency within a given body of text. Word clouds are typically used as a tool for processing, analyzing and disseminating qualitative sentiment data.

References:

Productivity/Frequency Plot

prod_freq_usage

Our main reference for this method is this paper. In short, this paper uses normalized term frequency and term producitvity as their measures.

  • Term Frequency: This is the normalized frequency of a given term in a given year.
  • Term Productivity: This is a measure of the ability of the concept to produce new multi-word terms. In our case we use bigrams. For each year y and single-word term t, and associated n multi-word terms m, the productivity is given by the entropy:

prod_plot_1

prod_plot_2

Based on these two measures, they hypothesize three kinds of terms:

  • Growing Terms: Those which have increasing frequency and productivity in the recent years.
  • Consolidated Terms: Those that are growing in frequency, but not in productivity.
  • Terms in Decline: Those which have reached an upper bound of productivity and are being used less in terms of frequency.

Then, they perform clustering of the terms based on their frequency and productivity curves over the years to test their hypothesis. They find that the clusters formed show similar trends as expected.

NOTE: They also evaluate quality of their clusters using pseudo-labels, but we do not use any automated labels here. They also try with and without double-counting multi-word terms, but we stick to double-counting. They suggest it is more explanable.

Acceleration Plot

acceleration_plot_usage

This plot is based on the word-pair acceleration over time. Our inspiration for this method is this paper. Acceleration is a metric which calculates how quickly the word embeddings for a pair of word get close together or farther apart. If they are getting closer together, it means these two terms have started appearing more frequently in similar contexts, which leads to similar embeddings. In the paper, it is described as:

acc_plot_1

acc_plot_2

Below, we display the top few pairs between the given start and end year in dataframe, then one can select years and then select word-pairs in the plot parameters expander. A reduced dimension plot is displayed.

NOTE: They suggest using skip-gram method over CBOW for the model. They use t-SNE representation to view the embeddings. But their way of aligning the embeddings is different. They also use some stability measure to find the best Word2Vec model. The also use Word2Phrase which we are planning to add soon.

Semantic Drift

semantic_drift_usage

This plot represents the change in meaning of a word over time. This shift is represented on a 2-dimensional representation of the embedding space. To find the drift of a word, we calculate the distance between the embeddings of the word in the final year and in the initial year. We find the drift for all words and sort them in descending order to find the most drifted words. We give an option to use one of two distance metrics: Euclidean Distance and Cosine Distance.

sem_drift_1

sem_drift_2

We plot top-K (sim.) most similar words around the two representations of the selected word.

In the Plot Parameters expander, the user can select the range of years over which the drift will be computed. He/She can also select the dimensionality reduction method for plotting the embeddings.

Below the graph, we provide a list of most drifted words (from the top-K keywords). The user can also choose a custom word.

Tracking Clusters

track_clusters_usage

Word meanings change over time. They come closer or drift apart. In a certain year, words are clumped together, i.e., they belong to one cluster. But over time, clusters can break into two/coalesce together to form one. Unlike the previous module which tracks movement of one word at a time, here, we track the movement of clusters.

We plot the formed clusters for all the years lying in the selected range of years. NOTE: We give an option to use one of two libraries for clustering: sklearn or faiss. faiss' KMeans implementation is around 10 times faster than sklearn's.

Acceleration Heatmap

acceleration_heatmap_usage

This plot is based on the word-pair acceleration over time. Our inspiration for this method is this paper. Acceleration is a metric which calculates how quickly the word embeddings for a pair of word get close together or farther apart. If they are getting closer together, it means these two terms have started appearing more frequently in similar contexts, which leads to similar embeddings. In the paper, it is described as:

acc_hm_1

acc_hm_2

For all the selected keywords, we display a heatmap, where the brightness of the colour determines the value of the acceleration between that pair, i.e., the brightness is directly proportional to the acceleration value.

NOTE: They suggest using skip-gram method over CBOW for the model.

Track Trends with Similarity

track_trends_usage

In this method, we wish to chart the trajectory of a word/topic from year 1 to year 2.

To accomplish this, we allow the user to pick a word from year 1. At the same time, we ask the user to provide the desired stride. We search for the most similar word in the next stride years. We keep doing this iteratively till we reach year 2, updating the word at each step.

The user has to select a word and click on Generate Dataframe. This gives a list of most similar words in the next stride years. The user can now iteratively select the next word from the drop-down till the final year is reached.

Keyword Visualisation

keyword_viz_usage

Here, we use the YAKE Keyword Extraction method to extract keywords. You can read more about YAKE here.

In our code, we use an open source implementation of YAKE.

NOTE: Yake returns scores which are indirectly proportional to the keyword importance. Hence, we do the following to report the final scores:

keyword_viz

LDA Topic Modelling

lda_usage

Latent Dirichlet Allocation is a generative probabilistic model for an assortment of documents, generally used for topic modelling and extraction. LDA clusters the text data into imaginary topics.

Every topic can be represented as a probability distribution over ngrams and every document can be represented as a probability distribution over these generated topics.

We train LDA on a corpus where each document contains the abstracts of a particular year. We express every year as a probability distribution of topics.

In the first bar graph, we show how a year can be decomposed into topics. The graphs below the first one show a decomposition of the relevant topics.

Citation

You can cite our work as:

@misc{sharma2021drift,
      title={DRIFT: A Toolkit for Diachronic Analysis of Scientific Literature}, 
      author={Abheesht Sharma and Gunjan Chhablani and Harshit Pandey and Rajaswa Patil},
      year={2021},
      eprint={2107.01198},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

OR

Sharma, A., Chhablani, G., Pandey, H., & Patil, R. (2021). DRIFT: A Toolkit for Diachronic Analysis of Scientific Literature.
Comments
  • Add script for arranging anthology data

    Add script for arranging anthology data

    • Reading XML files from here
    • Data stored in the form of nested dictionaries Format: publisher ---> (month, url, booktitle, year ---> [papers]) paper ---> (author, abstract, ...other text elements)

    Issues:

    • Most conferences have data for very few years. Mistake in the code? Or lack of uniformity in the XML files?
    • Keep "year" at the top of the hierarchy.
    opened by abheesht17 3
  • Script for Clustering Word Embeddings

    Script for Clustering Word Embeddings

    • Use K-Means for clustering the diachronic word embeddings.
    • Rough Sketch:
      • The class can have multiple functions: train, predict, store, visualisation, etc.
      • The function(s) will take as input word vectors from a particular timestamp. They will also take as input parameters of K-Means like number of clusters, etc.
      • Add functionality for visualisation.
      • Return the centroids and the cluster to which the words belong.
    opened by abheesht17 2
  • LDA Topic Modelling error

    LDA Topic Modelling error

    When choosing the LDA Topic Modelling section , the following message appears : ValueError: list.remove(x): x not in list Traceback: File "c:\users\doub2420.virtualenvs\drift-qengzvvy\lib\site-packages\streamlit\script_runner.py", line 337, in run_script exec(code, module.dict) File "C:\drift\app.py", line 1726, in year_paths.remove(os.path.join(vars["data_path"], "compass.txt"))

    What would that mean ?

    Thanks !

    opened by doubianimehdi 1
  • Changes to Productivity Plot

    Changes to Productivity Plot

    1. Check if cluster labels are more or less correct, otherwise we will remove/change the cluster table.
    2. Formatting changes for cluster table might be required
    3. Labels in dataframe should be named
    opened by harsh4799 0
  • Stop Words Custom list and Remove digits not transform it to english numerals

    Stop Words Custom list and Remove digits not transform it to english numerals

    Hi,

    I would like to have an option to use custom stop words and also to not have digits transformed into english numerals (thousand, hundred , and so on ...) because it doesn't help with the purpose of tracking trends or analyzing the abstracts.

    Thank you !

    opened by doubianimehdi 0
Releases(v0.1.0)
Owner
Rajaswa Patil
Rajaswa Patil
PyTorch implementation of "Simple and Deep Graph Convolutional Networks"

Simple and Deep Graph Convolutional Networks This repository contains a PyTorch implementation of "Simple and Deep Graph Convolutional Networks".(http

chenm 253 Dec 08, 2022
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

chitra What is chitra? chitra (चित्र) is a multi-functional library for full-stack Deep Learning. It simplifies Model Building, API development, and M

Aniket Maurya 210 Dec 21, 2022
Coded illumination for improved lensless imaging

CodedCam Coded Illumination for Improved Lensless Imaging Paper | Supplementary results | Data and Code are available. Coded illumination for improved

Computational Sensing and Information Processing Lab 1 Nov 29, 2021
Face Mask Detection System built with OpenCV, TensorFlow using Computer Vision concepts

Face mask detection Face Mask Detection System built with OpenCV, TensorFlow using Computer Vision concepts in order to detect face masks in static im

Vaibhav Shukla 1 Oct 27, 2021
TreeSubstitutionCipher - Encryption system based on trees and substitution

Tree Substitution Cipher Generation Algorithm: Generate random tree. Tree nodes

stepa 1 Jan 08, 2022
Codebase for the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL BASALT Challenge.

KAIROS MineRL BASALT Codebase for the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL B

Vinicius G. Goecks 37 Oct 30, 2022
Active and Sample-Efficient Model Evaluation

Active Testing: Sample-Efficient Model Evaluation Hi, good to see you here! 👋 This is code for "Active Testing: Sample-Efficient Model Evaluation". P

Jannik Kossen 19 Oct 30, 2022
A module for solving and visualizing Schrödinger equation.

qmsolve This is an attempt at making a solid, easy to use solver, capable of solving and visualize the Schrödinger equation for multiple particles, an

506 Dec 28, 2022
使用yolov5训练自己数据集(详细过程)并通过flask部署

使用yolov5训练自己的数据集(详细过程)并通过flask部署 依赖库 torch torchvision numpy opencv-python lxml tqdm flask pillow tensorboard matplotlib pycocotools Windows,请使用 pycoc

HB.com 19 Dec 28, 2022
A NSFW content filter.

Project_Nfilter A NSFW content filter. With a motive of minimizing the spreads and leakage of NSFW contents on internet and access to others devices ,

1 Jan 20, 2022
Offline Reinforcement Learning with Implicit Q-Learning

Offline Reinforcement Learning with Implicit Q-Learning This repository contains the official implementation of Offline Reinforcement Learning with Im

Ilya Kostrikov 126 Jan 06, 2023
Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.

CycleGAN PyTorch | project page | paper Torch implementation for learning an image-to-image translation (i.e. pix2pix) without input-output pairs, for

Jun-Yan Zhu 11.5k Dec 30, 2022
Language model Prompt And Query Archive

LPAQA: Language model Prompt And Query Archive This repository contains data and code for the paper How Can We Know What Language Models Know? Install

127 Dec 20, 2022
My freqtrade strategies

My freqtrade-strategies Hi there! This is repo for my freqtrade-strategies. My name is Ilya Zelenchuk, I'm a lecturer at the SPbU university (https://

171 Dec 05, 2022
A Simple LSTM-Based Solution for "Heartbeat Signal Classification and Prediction" in Tianchi

LSTM-Time-Series-Prediction A Simple LSTM-Based Solution for "Heartbeat Signal Classification and Prediction" in Tianchi Contest. The Link of the Cont

KevinCHEN 1 Jun 13, 2022
SCAN: Learning to Classify Images without Labels, incl. SimCLR. [ECCV 2020]

Learning to Classify Images without Labels This repo contains the Pytorch implementation of our paper: SCAN: Learning to Classify Images without Label

Wouter Van Gansbeke 1.1k Dec 30, 2022
Devkit for 3D -- Some utils for 3D object detection based on Numpy and Pytorch

D3D Devkit for 3D: Some utils for 3D object detection and tracking based on Numpy and Pytorch Please consider siting my work if you find this library

Jacob Zhong 27 Jul 07, 2022
EMNLP 2021: Single-dataset Experts for Multi-dataset Question-Answering

MADE (Multi-Adapter Dataset Experts) This repository contains the implementation of MADE (Multi-adapter dataset experts), which is described in the pa

Princeton Natural Language Processing 68 Jul 18, 2022
2D Human Pose estimation using transformers. Implementation in Pytorch

PE-former: Pose Estimation Transformer Vision transformer architectures perform very well for image classification tasks. Efforts to solve more challe

Panteleris Paschalis 23 Oct 17, 2022
Joint-task Self-supervised Learning for Temporal Correspondence (NeurIPS 2019)

Joint-task Self-supervised Learning for Temporal Correspondence Project | Paper Overview Joint-task Self-supervised Learning for Temporal Corresponden

Sifei Liu 167 Dec 14, 2022