LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Memoized coduals - Shows that it is possible to implement reverse mode autodiff using a variation on the dual numbers called the codual numbers

Official repository of DeMFI (arXiv.)

Exploring Relational Context for Multi-Task Dense Prediction [ICCV 2021]

Alignment Attention Fusion framework for Few-Shot Object Detection

Repository containing the PhD Thesis "Formal Verification of Deep Reinforcement Learning Agents"

Generate image analogies using neural matching and blending

Code for "Adversarial attack by dropping information." (ICCV 2021)

TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation, NeurIPS 2021 Spotlight

PyTorch implementation of MLP-Mixer

Funnels: Exact maximum likelihood with dimensionality reduction.

A framework for GPU based high-performance medical image processing and visualization

Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

CVPR '21: In the light of feature distributions: Moment matching for Neural Style Transfer

MAT: Mask-Aware Transformer for Large Hole Image Inpainting

Place holder for HOPE: a human-centric and task-oriented MT evaluation framework using professional post-editing

Code of our paper "Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning"

An onlinel learning to rank python codebase.

A simple implementation of Kalman filter in single object tracking

TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction.