LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Implementation of Sequence Generative Adversarial Nets with Policy Gradient

Collection of common code that's shared among different research projects in FAIR computer vision team.

Implementation of Bidirectional Recurrent Independent Mechanisms (Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules)

PyTorch Implementation of ECCV 2020 Spotlight TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images

automatic color-grading

Modified prey-predator system - Modified prey–predator model describes the rate of change for each species by adding coupling terms.

Official repository for "PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation"

SingleVC performs any-to-one VC, which is an important component of MediumVC project.

A mini-course offered to Undergrad chemistry students

Project page for the paper Semi-Supervised Raw-to-Raw Mapping 2021.

Official repo for SemanticGAN https://nv-tlabs.github.io/semanticGAN/

Planner_backend - Academic planner application designed for students and counselors.

The official PyTorch code for NeurIPS 2021 ML4AD Paper, "Does Thermal data make the detection systems more reliable?"

Styleformer - Official Pytorch Implementation

minimizer-space de Bruijn graphs (mdBG) for whole genome assembly

Official implementation of "DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation"

Speedy Implementation of Instance-based Learning (IBL) agents in Python

METER: Multimodal End-to-end TransformER

Python port of R's Comprehensive Dynamic Time Warp algorithm package

Yoga - Yoga asana classifier for python