LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe.

Python with OpenCV - MediaPip Framework Hand Detection

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Physical Anomalous Trajectory or Motion (PHANTOM) Dataset

Official implementation of the paper Momentum Capsule Networks (MoCapsNet)

Final project for Intro to CS class.

Complex Answer Generation For Conversational Search Systems.

naked is a Python tool which allows you to strip a model and only keep what matters for making predictions.

HyperaPy: An automatic hyperparameter optimization framework ⚡🚀

Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand

Elucidating Robust Learning with Uncertainty-Aware Corruption Pattern Estimation

Pytorch implementation of "MOSNet: Deep Learning based Objective Assessment for Voice Conversion"

ViSD4SA, a Vietnamese Span Detection for Aspect-based sentiment analysis dataset

A CV toolkit for my papers.

Investigating automatic navigation towards standard US views integrating MARL with the virtual US environment developed in CT2US simulation

Source code for models described in the paper "AudioCLIP: Extending CLIP to Image, Text and Audio" (https://arxiv.org/abs/2106.13043)

The Fundamental Clustering Problems Suite (FCPS) summaries 54 state-of-the-art clustering algorithms, common cluster challenges and estimations of the number of clusters as well as the testing for cluster tendency.

SSD: Single Shot MultiBox Detector pytorch implementation focusing on simplicity

Awesome AI Learning with +100 AI Cheat-Sheets, Free online Books, Top Courses, Best Videos and Lectures, Papers, Tutorials, +99 Researchers, Premium Websites, +121 Datasets, Conferences, Frameworks, Tools

Unofficial Implementation of MLP-Mixer, Image Classification Model