LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Circuit Training: An open-source framework for generating chip floor plans with distributed deep reinforcement learning

PyTorch implementation of our paper How robust are discriminatively trained zero-shot learning models?

[ICLR'19] Trellis Networks for Sequence Modeling

Sample Code for "Pessimism Meets Invariance: Provably Efficient Offline Mean-Field Multi-Agent RL"

🍅🍅🍅YOLOv5-Lite: lighter, faster and easier to deploy. Evolved from yolov5 and the size of model is only 1.7M (int8) and 3.3M (fp16). It can reach 10+ FPS on the Raspberry Pi 4B when the input size is 320×320~

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

A little Python application to auto tag your photos with the power of machine learning.

Dense Prediction Transformers

Code accompanying paper: Meta-Learning to Improve Pre-Training

Segmentation vgg16 fcn - cityscapes

An Ensemble of CNN (Python 3.5.1 Tensorflow 1.3 numpy 1.13)

A Simple Framwork for CV Pre-training Model (SOCO, VirTex, BEiT)

SoK: Vehicle Orientation Representations for Deep Rotation Estimation

Dataset and Source code of paper 'Enhancing Keyphrase Extraction from Academic Articles with their Reference Information'.

A library for graph deep learning research

Learning Open-World Object Proposals without Learning to Classify

RATCHET is a Medical Transformer for Chest X-ray Diagnosis and Reporting

PyTorch Implementation of ByteDance's Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech

Intel® Nervana™ reference deep learning framework committed to best performance on all hardware

Simple helper library to convert a collection of numpy data to tfrecord, and build a tensorflow dataset from the tfrecord.