Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Last update: Jan 08, 2023

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Model	STS12	STS13	STS14	STS15	STS16	STSb	SICK-R	Avg.
unsup-prompt-bert-base Download	71.98	84.66	77.13	84.52	81.10	82.03	70.64	78.87
unsup-prompt-roberta-base Download	73.98	84.73	77.88	84.93	81.89	82.74	69.21	79.34
sup-prompt-bert-base Download	75.48	85.59	80.57	85.99	81.08	84.56	80.52	81.97
sup-prompt-roberta-base Download	76.75	85.93	82.28	86.69	82.80	86.14	80.04	82.95

Download Data

cd SentEval/data/downstream/
bash download_dataset.sh
cd -
cd ./data
bash download_wiki.sh
bash download_nli.sh
cd -

Static token embedding with removing embedding biases

robert-base, bert-base-cased and robert-base-uncased

./run.sh roberta-base-embedding-only-remove-baises
./run.sh bert-base-cased-embedding-only-remove-baises
./run.sh bert-base-uncased-embedding-only-remove-baises

Non fine-tuned BERT with Prompt

bert-base-uncased with prompt

./run.sh bert-prompt

bert-base-uncased with optiprompt

./run.sh bert-optiprompt

fine-tuned BERT with Prompt

unsupervised

SEED=0
./run.sh unsup-roberta $SEED

SEED=0
./run.sh unsup-bert $SEED

supervised

./run.sh sup-roberta

./run.sh sup-bert

Our Code is based on SimCSE

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

Code for our TKDE paper "Understanding WeChat User Preferences and “Wow” Diffusion"

The official implementation of Variable-Length Piano Infilling (VLI).

Code for our NeurIPS 2021 paper Mining the Benefits of Two-stage and One-stage HOI Detection

Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (ICCV 2021)

Working demo of the Multi-class and Anomaly classification model using the CLIP feature space

[CVPR 2020] Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

Semi-supervised Video Deraining with Dynamical Rain Generator (CVPR, 2021, Pytorch)

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.

Supervised Contrastive Learning for Downstream Optimized Sequence Representations

robomimic: A Modular Framework for Robot Learning from Demonstration

iNAS: Integral NAS for Device-Aware Salient Object Detection

Official Pytorch implementation of Meta Internal Learning

Multiview 3D object detection on MultiviewC dataset through moft3d.

FastyAPI is a Stack boilerplate optimised for heavy loads.

Emotional conditioned music generation using transformer-based model.

Human motion synthesis using Unity3D

Get started with Machine Learning with Python - An introduction with Python programming examples

PyTorch implementation of EGVSR: Efficcient & Generic Video Super-Resolution (VSR)

A Free and Open Source Python Library for Multiobjective Optimization

Complex Answer Generation For Conversational Search Systems.

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

Code for our TKDE paper "Understanding WeChat User Preferences and “Wow” Diffusion"

The official implementation of Variable-Length Piano Infilling (VLI).

Code for our NeurIPS 2021 paper Mining the Benefits of Two-stage and One-stage HOI Detection

Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (ICCV 2021)

Working demo of the Multi-class and Anomaly classification model using the CLIP feature space

[CVPR 2020] Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

Semi-supervised Video Deraining with Dynamical Rain Generator (CVPR, 2021, Pytorch)

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren*, Raymond A. Yeh*, Alexander G. Schwing.

Supervised Contrastive Learning for Downstream Optimized Sequence Representations

robomimic: A Modular Framework for Robot Learning from Demonstration

iNAS: Integral NAS for Device-Aware Salient Object Detection

Official Pytorch implementation of Meta Internal Learning

Multiview 3D object detection on MultiviewC dataset through moft3d.

FastyAPI is a Stack boilerplate optimised for heavy loads.

Emotional conditioned music generation using transformer-based model.

Human motion synthesis using Unity3D

Get started with Machine Learning with Python - An introduction with Python programming examples

PyTorch implementation of EGVSR: Efficcient & Generic Video Super-Resolution (VSR)

A Free and Open Source Python Library for Multiobjective Optimization

Complex Answer Generation For Conversational Search Systems.

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.