BARTScore: Evaluating Generated Text as Text Generation

Last update: Dec 17, 2022

Related tags

Deep Learning BARTScore

Overview

This is the Repo for the paper: BARTScore: Evaluating Generated Text as Text Generation

Updates

2021.06.28 Release online evaluation Demo
2021.06.25 Release online Explainable Leaderboard for Meta-evaluation
2021.06.22 Code will be released soon

Background

There is a recent trend that leverages neural models for automated evaluation in different ways, as shown in Fig.1.

(a) Evaluation as matching task. Unsupervised matching metrics aim to measure the semantic equivalence between the reference and hypothesis by using a token-level matching functions in distributed representation space (e.g. BERT) or discrete string space (e.g. ROUGE).

(b) Evaluation as regression task. Regression-based metrics (e.g. BLEURT) introduce a parameterized regression layer, which would be learned in a supervised fashion to accurately predict human judgments.

(c) Evaluation as ranking task. Ranking-based metrics (e.g. COMET) aim to learn a scoring function that assigns a higher score to better hypotheses than to worse ones.

(d) Evaluation as generation task. In this work, we formulate evaluating generated text as a text generation task from pre-trained language models.

BARTScore: Evaluating Generated Text as Text Generation

Related tags

Overview

Updates

Background

Owner

NeuLab

The PyTorch implementation for paper "Neural Texture Extraction and Distribution for Controllable Person Image Synthesis" (CVPR2022 Oral)

The code written during my Bachelor Thesis "Classification of Human Whole-Body Motion using Hidden Markov Models".

The materials used in the SaxonJS tutorial presented at Declarative Amsterdam, 2021

LBBA-boosted WSOD

Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning And private Server services

Self-Supervised Document-to-Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference

Automatic library of congress classification, using word embeddings from book titles and synopses.

Simple Dynamic Batching Inference

Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite.

Python periodic table module

Social Distancing Detector

The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

Code for ICMI2020 and ICMI2021 papers: "Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle" and "ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle"

Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)

This is a Machine Learning Based Hand Detector Project, It Uses Machine Learning Models and Modules Like Mediapipe, Developed By Google!

Code repo for "Cross-Scale Internal Graph Neural Network for Image Super-Resolution" (NeurIPS'20)

Official implementation of DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations in TensorFlow 2

Easy-to-use,Modular and Extendible package of deep-learning based CTR models .

A new test set for ImageNet

COLMAP - Structure-from-Motion and Multi-View Stereo