Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Last update: Jan 08, 2023

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Model	STS12	STS13	STS14	STS15	STS16	STSb	SICK-R	Avg.
unsup-prompt-bert-base Download	71.98	84.66	77.13	84.52	81.10	82.03	70.64	78.87
unsup-prompt-roberta-base Download	73.98	84.73	77.88	84.93	81.89	82.74	69.21	79.34
sup-prompt-bert-base Download	75.48	85.59	80.57	85.99	81.08	84.56	80.52	81.97
sup-prompt-roberta-base Download	76.75	85.93	82.28	86.69	82.80	86.14	80.04	82.95

Download Data

cd SentEval/data/downstream/
bash download_dataset.sh
cd -
cd ./data
bash download_wiki.sh
bash download_nli.sh
cd -

Static token embedding with removing embedding biases

robert-base, bert-base-cased and robert-base-uncased

./run.sh roberta-base-embedding-only-remove-baises
./run.sh bert-base-cased-embedding-only-remove-baises
./run.sh bert-base-uncased-embedding-only-remove-baises

Non fine-tuned BERT with Prompt

bert-base-uncased with prompt

./run.sh bert-prompt

bert-base-uncased with optiprompt

./run.sh bert-optiprompt

fine-tuned BERT with Prompt

unsupervised

SEED=0
./run.sh unsup-roberta $SEED

SEED=0
./run.sh unsup-bert $SEED

supervised

./run.sh sup-roberta

./run.sh sup-bert

Our Code is based on SimCSE

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

History Aware Multimodal Transformer for Vision-and-Language Navigation

LoL Runes Recommender With Python

A project studying the influence of communication in multi-objective normal-form games

Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer

Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration

HistoKT: Cross Knowledge Transfer in Computational Pathology

Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

Code for Massive-scale Decoding for Text Generation using Lattices

Easy-to-use library to boost AI inference leveraging state-of-the-art optimization techniques.

DUE: End-to-End Document Understanding Benchmark

An implementation of paper `Real-time Convolutional Neural Networks for Emotion and Gender Classification` with PaddlePaddle.

Author: Wenhao Yu ([email protected]). ACL 2022. Commonsense Reasoning on Knowledge Graph for Text Generation

Hierarchical Motion Encoder-Decoder Network for Trajectory Forecasting (HMNet)

Custom studies about block sparse attention.

A Structured Self-attentive Sentence Embedding

Train the HRNet model on ImageNet

deep learning model that learns to code with drawing in the Processing language

This is implementation of AlexNet(2012) with 3D Convolution on TensorFlow (AlexNet 3D).

hySLAM is a hybrid SLAM/SfM system designed for mapping

ParmeSan: Sanitizer-guided Greybox Fuzzing