Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Last update: Jan 08, 2023

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Model	STS12	STS13	STS14	STS15	STS16	STSb	SICK-R	Avg.
unsup-prompt-bert-base Download	71.98	84.66	77.13	84.52	81.10	82.03	70.64	78.87
unsup-prompt-roberta-base Download	73.98	84.73	77.88	84.93	81.89	82.74	69.21	79.34
sup-prompt-bert-base Download	75.48	85.59	80.57	85.99	81.08	84.56	80.52	81.97
sup-prompt-roberta-base Download	76.75	85.93	82.28	86.69	82.80	86.14	80.04	82.95

Download Data

cd SentEval/data/downstream/
bash download_dataset.sh
cd -
cd ./data
bash download_wiki.sh
bash download_nli.sh
cd -

Static token embedding with removing embedding biases

robert-base, bert-base-cased and robert-base-uncased

./run.sh roberta-base-embedding-only-remove-baises
./run.sh bert-base-cased-embedding-only-remove-baises
./run.sh bert-base-uncased-embedding-only-remove-baises

Non fine-tuned BERT with Prompt

bert-base-uncased with prompt

./run.sh bert-prompt

bert-base-uncased with optiprompt

./run.sh bert-optiprompt

fine-tuned BERT with Prompt

unsupervised

SEED=0
./run.sh unsup-roberta $SEED

SEED=0
./run.sh unsup-bert $SEED

supervised

./run.sh sup-roberta

./run.sh sup-bert

Our Code is based on SimCSE

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

A PyTorch implementation of "ANEMONE: Graph Anomaly Detection with Multi-Scale Contrastive Learning", CIKM-21

Server files for UltimateLabeling

Contains code for the paper "Vision Transformers are Robust Learners".

GLODISMO: Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

A JAX-based research framework for writing differentiable numerical simulators with arbitrary discretizations

Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

Includes PyTorch -> Keras model porting code for ConvNeXt family of models with fine-tuning and inference notebooks.

HODEmu, is both an executable and a python library that is based on Ragagnin 2021 in prep.

Hydra Lightning Template for Structured Configs

Torchyolo - Yolov3 ve Yolov4 modellerin Pytorch uygulamasıdır

Project Tugas Besar pertama Pengenalan Komputasi Institut Teknologi Bandung

Tutorial for the PERFECTING FACTORY 5.0 WITH EDGE-POWERED AI workshop

TorchIO is a Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

The code repository for EMNLP 2021 paper "Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization".

Image transformations designed for Scene Text Recognition (STR) data augmentation. Published at ICCV 2021 Workshop on Interactive Labeling and Data Augmentation for Vision.

PyTorch implementations of the paper: "Learning Independent Instance Maps for Crowd Localization"

A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population

A curated list of programmatic weak supervision papers and resources

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search