A Python framework for conversational search

Overview

Chatty Goose

Multi-stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting


PyPI LICENSE

Installation

  1. Make sure Java 11+ and Python 3.7+ are installed

  2. Install the chatty-goose PyPI module

pip install chatty-goose
  1. If you are using T5 or BERT, make sure to install PyTorch 1.4.0 - 1.7.1 using your specific platform instructions. Note that PyTorch 1.8 is currently incompatible due to the transformers version we currently use. Also make sure to install the corresponding torchtext version.

  2. Download the English model for spaCy

python -m spacy download en_core_web_sm

Quickstart Guide

The following example shows how to initialize a searcher and build a ConversationalQueryRewriter agent from scratch using HQE and T5 as first-stage retrievers, and a BERT reranker. To see a working example agent, see chatty_goose/agents/chat.py.

First, load a searcher

from pyserini.search import SimpleSearcher

# Option 1: load a prebuilt index
searcher = SimpleSearcher.from_prebuilt_index("INDEX_NAME_HERE")
# Option 2: load a local Lucene index
searcher = SimpleSearcher("PATH_TO_INDEX")

searcher.set_bm25(0.82, 0.68)

Next, initialize one or more first-stage CQR retrievers

from chatty_goose.cqr import Hqe, Ntr
from chatty_goose.settings import HqeSettings, NtrSettings

hqe = Hqe(searcher, HqeSettings())
ntr = Ntr(NtrSettings())

Load a reranker

from chatty_goose.util import build_bert_reranker

reranker = build_bert_reranker()

Create a new RetrievalPipeline

from chatty_goose.pipeline import RetrievalPipeline

rp = RetrievalPipeline(searcher, [hqe, ntr], searcher_num_hits=50, reranker=reranker)

And we're done! Simply call rp.retrieve(query) to retrieve passages, or call rp.reset_history() to reset the conversational history of the retrievers.

Running Experiments

  1. Clone the repo and all submodules (git submodule update --init --recursive)

  2. Clone and build Anserini for evaluation tools

  3. Install dependencies

pip install -r requirements.txt
  1. Follow the instructions under docs/cqr_experiments.md to run experiments using HQE, T5, or fusion.

Example Agent

To run an interactive conversational search agent with ParlAI, simply run chat.py. By default, we use the CAsT 2019 pre-built Pyserini index, but it is possible to specify other indexes using the --from_prebuilt flag. See the file for other possible arguments:

python -m chatty_goose.agents.chat

Alternatively, run the agent using ParlAI's command line interface:

python -m parlai interactive --model chatty_goose.agents.chat:ChattyGooseAgent

We also provide instructions to deploy the agent to Facebook Messenger using ParlAI under examples/messenger.

Comments
  • Add baselines for CAsT 2020

    Add baselines for CAsT 2020

    Need someone help to add CAsT 2020 baseline results:

    • [ ] Naive: CQR without canonical responses
    • [ ] Canonical: CQR with canonical (manual) response

    CQR methods: HQE /Ntr (T5)

    enhancement help wanted 
    opened by justram 2
  • Running HQE and getting the reformulated queries

    Running HQE and getting the reformulated queries

    Dear authors,

    I am trying to use your method in some of my work. For that, I need to get the reformulated queries (instead of only the generated ranked hits).

    I am trying to run the HQE experiment as indicated using:

    python -m experiments.run_retrieval \
          --experiment hqe \
          --hits 1000 \
          --sparse_index cast2019 \
          --qid_queries $input_query_json \
          --output ./output/hqe_bm25 
    

    However, when I print the arguments passed inside the retrieval pipeline (L101 of retrieval_pipeline.py) I get as query the raw/original/last-turn query string, and as manual_context_buffer[turn_id] simply None. If I'm not mistaken, that means that running the specific experiment equals to no reformulation being done at all. Can you check/confirm this?

    Digging more into the code, it seems to me that the queries I'd like to access are inside cqr_queries, but still, it seems to me that context should be empty/None in that case - probably resulting to no reformulation done at all.

    opened by littlewine 1
  • Query rewriting fix

    Query rewriting fix

    Thank you with the project.

    The fix to below will be hits = rp.retrieve(query, manual_context_buffer[turn_id-1] if turn_id!=0 else None), to pass the last previous canonical response.

    https://github.com/castorini/chatty-goose/blob/f9c21c8b7b6194d11d7aec5b4e218174cde98418/experiments/run_retrieval.py#L100

    opened by xeniaqian94 1
  • Update based on Pyserini==0.14.0 and fix canonical response bug

    Update based on Pyserini==0.14.0 and fix canonical response bug

    Main change:

    1. change --dense_index from temporary one to pyserini prebuilt index name
    2. fix canonical response bug, which previously add current response to context
    3. since now we have --dense_index, change option name --index to --sparse_index
    opened by jacklin64 0
  • Add chatty goose support for dense retrieval and hybrid search for T5 and CQE

    Add chatty goose support for dense retrieval and hybrid search for T5 and CQE

    New features added: (only for T5 and CQE, may consider HQE in the future) (1) Dense retrieval (2) Dense-sparse hybrid retrieval

    Some arg might be confused and may be changed in the future: (1) --index, --dense_index: may change to --sparse_index and --dense_index (2) --experiment now has options (hqe,cqe,t5,fusion,cqe_t5_fusion) may change to (hqe,cqe,t5,hqe_t5fusion,cqe_t5_fusion)

    opened by jacklin64 0
  • Add cast2020 baseline

    Add cast2020 baseline

    This PR adds both naive and canonical baselines for CAsT2020 topics. The results are overall lower as compared to CAst2019 and the results from the canonical run are only slightly better for some metrics as compared to results from the naive run.

    Resolves #23

    opened by saileshnankani 0
  • Add support for canonical response

    Add support for canonical response

    This PR adds support for using manual_canonical_result_id in the CAsT2020 data for both ntr and hqe (for #23).

    For ntr, rewrite uses the passage corresponding to the canonical document in the history. We only use 1 passage in the historical context as otherwise, it exceed 512 tokens limit. For e.g., it uses q1/P1/q2 and then q1/q2/P2/q3 and so on.

    enhancement 
    opened by saileshnankani 0
  • CQR Replication

    CQR Replication

    Add CQR replication for Fusion BM25

    Library versions used: torch==1.7.0 torchvision==0.8.1 torchtext==0.8


    Results:

    map                   	all	0.2584
    recall_1000           	all	0.8028
    ndcg_cut_1            	all	0.3353
    ndcg_cut_3            	all	0.3247
    

    Details and reproduction results can be found in the notebook

    opened by saileshnankani 0
  • Rename classes and update messenger bot

    Rename classes and update messenger bot

    Breaking changes:

    Renamed several classes to follow Python conventions / be more consistent

    • chatty_goose.agents.cqragent -> chatty_goose.agents.chat
    • HQE -> Hqe
    • T5_NTR -> Ntr
    • HQESettings -> HqeSettings
    • T5Settings -> NtrSettings
    • CQRType -> CqrType
    • CQRSettings -> CqrSettings
    • CQR -> ConversationalQueryRewriter
    opened by edwinzhng 0
  • document spaCy model dependency

    document spaCy model dependency

    With a fresh install, we get the following error if we try to run anything:

    OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
    

    Solution is:

    $ python -m spacy download en_core_web_sm
    

    We should document this.

    opened by lintool 0
  • PyTorch version: needs Torch 1.7 (won't work with 1.8)

    PyTorch version: needs Torch 1.7 (won't work with 1.8)

    With a from-scratch installation, the module pulls in Torch 1.8, which causes this error:

    ImportError: cannot import name 'SAVE_STATE_WARNING' from 'torch.optim.lr_scheduler' (/anaconda3/envs/chatty-goose-test/lib/python3.7/site-packages/torch/optim/lr_scheduler.py)
    

    Downgrading fixes the issue:

    $ pip install torch==1.7.1 torchtext==0.8.1
    

    Should we pin the version in our module dependencies? Or at the very least this needs to be documented.=

    opened by lintool 0
  • dependency conflict

    dependency conflict

    Hi,

    When I install chatty-goose from github using:

    python -m pip install git+https://github.com/castorini/chatty-goose.git
    
    

    I met this issue:

    ERROR: Cannot install chatty-goose and chatty-goose==0.2.0 because these package versions have conflicting dependencies.
    
    The conflict is caused by:
        chatty-goose 0.2.0 depends on pyserini==0.14.0
        pygaggle 0.0.3.1 depends on pyserini==0.10.1.0
    

    It seems that chatty-goose requires pyserini==0.14.0 as well as pygaggle 0.0.3.1. However, pygaggle 0.0.3.1 and pyserini==0.14.0 do not play nice with each other

    Could someone provide some help?

    Thanks!

    opened by dayuyang1999 1
  • Expansion to new datasets

    Expansion to new datasets

    Does it make sense to expand Chatty Goose to new datasets? For example:

    • MANtIS - a multi-domain information seeking dialogues dataset: https://guzpenha.github.io/MANtIS/
    • ClariQ - Search-oriented Conversational AI (SCAI) EMNLP https://github.com/aliannejadi/ClariQ
    enhancement 
    opened by lintool 1
  • Checkpoint transformation

    Checkpoint transformation

    According to @edwinzhng's replication log, we have a reranker checkpoint mismatch issue. Currently, we have diffs in our reranking model and the pygaggle's default model.

    Related to this issue: I think we need a folder to put/track our tf2torch ckpt transformation/sanity check scripts?

    opened by justram 0
Releases(0.2.0)
  • 0.2.0(May 7, 2021)

    Breaking changes

    Renamed several classes to follow Python conventions / be more consistent

    chatty_goose.agents.cqragent -> chatty_goose.agents.chat HQE -> Hqe T5_NTR -> Ntr HQESettings -> HqeSettings T5Settings -> NtrSettings CQRType -> CqrType CQRSettings -> CqrSettings CQR -> ConversationalQueryRewriter

    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Mar 8, 2021)

    BREAKING CHANGES

    • Integrate ParlAI Facebook Messenger example for a demo by @edwinzhng
    • Integrate Pyserini/Pygaggle for a reference implementation of multi-stage passage retrieval by @edwinzhng
    • Add replication log for TREC CAsT 2019 conversational passage retrieval task by @edwinzhng
    Source code(tar.gz)
    Source code(zip)
Owner
Castorini
Deep learning for natural language processing and information retrieval at the University of Waterloo
Castorini
pytorch implementation for PointNet

PointNet.pytorch This repo is implementation for PointNet in pytorch. The model is in pointnet/model.py. It is teste

Fei Xia 1.7k Dec 30, 2022
Official repository for "Orthogonal Projection Loss" (ICCV'21)

Orthogonal Projection Loss (ICCV'21) Kanchana Ranasinghe, Muzammal Naseer, Munawar Hayat, Salman Khan, & Fahad Shahbaz Khan Paper Link | Project Page

Kanchana Ranasinghe 83 Dec 26, 2022
Pytorch implementation of our paper accepted by NeurIPS 2021 -- Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021) (Link) Overview Prerequisites Linu

Shaojie Li 34 Mar 31, 2022
The code for our NeurIPS 2021 paper "Kernelized Heterogeneous Risk Minimization".

Kernelized-HRM Jiashuo Liu, Zheyuan Hu The code for our NeurIPS 2021 paper "Kernelized Heterogeneous Risk Minimization"[1]. This repo contains the cod

Liu Jiashuo 8 Nov 20, 2022
MDMM - Learning multi-domain multi-modality I2I translation

Multi-Domain Multi-Modality I2I translation Pytorch implementation of multi-modality I2I translation for multi-domains. The project is an extension to

Hsin-Ying Lee 107 Nov 04, 2022
Optimized Gillespie algorithm for simulating Stochastic sPAtial models of Cancer Evolution (OG-SPACE)

OG-SPACE Introduction Optimized Gillespie algorithm for simulating Stochastic sPAtial models of Cancer Evolution (OG-SPACE) is a computational framewo

Data and Computational Biology Group UNIMIB (was BI*oinformatics MI*lan B*icocca) 0 Nov 17, 2021
Keras Image Embeddings using Contrastive Loss

Keras-Image-Embeddings-using-Contrastive-Loss Image to Embedding projection in vector space. Implementation in keras and tensorflow for custom data. B

Shravan Anand K 5 Mar 21, 2022
[CVPR 2022] Deep Equilibrium Optical Flow Estimation

Deep Equilibrium Optical Flow Estimation This is the official repo for the paper Deep Equilibrium Optical Flow Estimation (CVPR 2022), by Shaojie Bai*

CMU Locus Lab 136 Dec 18, 2022
Predicting 10 different clothing types using Xception pre-trained model.

Predicting-Clothing-Types Predicting 10 different clothing types using Xception pre-trained model from Keras library. It is reimplemented version from

AbdAssalam Ahmad 3 Dec 29, 2021
Point-NeRF: Point-based Neural Radiance Fields

Point-NeRF: Point-based Neural Radiance Fields Project Sites | Paper | Primary c

Qiangeng Xu 662 Jan 01, 2023
FB-tCNN for SSVEP Recognition

FB-tCNN for SSVEP Recognition Here are the codes of the tCNN and FB-tCNN in the paper "Filter Bank Convolutional Neural Network for Short Time-Window

Wenlong Ding 12 Dec 14, 2022
Code & Data for the Paper "Time Masking for Temporal Language Models", WSDM 2022

Time Masking for Temporal Language Models This repository provides a reference implementation of the paper: Time Masking for Temporal Language Models

Guy Rosin 12 Jan 06, 2023
"Inductive Entity Representations from Text via Link Prediction" @ The Web Conference 2021

Inductive entity representations from text via link prediction This repository contains the code used for the experiments in the paper "Inductive enti

Daniel Daza 45 Jan 09, 2023
A minimal TPU compatible Jax implementation of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

NeRF Minimal Jax implementation of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Result of Tiny-NeRF RGB Depth

Soumik Rakshit 11 Jul 24, 2022
Junction Tree Variational Autoencoder for Molecular Graph Generation (ICML 2018)

Junction Tree Variational Autoencoder for Molecular Graph Generation Official implementation of our Junction Tree Variational Autoencoder https://arxi

Wengong Jin 418 Jan 07, 2023
Efficient Speech Processing Tookit for Automatic Speaker Recognition

Sugar Efficient Speech Processing Tookit for Automatic Speaker Recognition | HuggingFace | What's New EfficientTDNN: Efficient Architecture Search for

WangRui 14 Sep 14, 2022
Repository for MeshTalk supplemental material and code once the (already approved) 16 GHS captures our lab will make publicly available are released.

meshtalk This repository contains code to run MeshTalk for face animation from audio. If you use MeshTalk, please cite @inproceedings{richard2021mesht

Meta Research 221 Jan 06, 2023
Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Portrait Photo Retouching with PPR10K Paper | Supplementary Material PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask an

184 Dec 11, 2022
Using Tensorflow Object Detection API to detect Waymo open dataset

Waymo-2D-Object-Detection Using Tensorflow Object Detection API to detect Waymo open dataset Result CenterNet Training Loss SSD ResNet Training Loss C

76 Dec 12, 2022
Auto HMM: Automatic Discrete and Continous HMM including Model selection

Auto HMM: Automatic Discrete and Continous HMM including Model selection

Chess_champion 29 Dec 07, 2022