项目说明:

百度2021年语言与智能技术竞赛机器阅读理解Pytorch版baseline
比赛链接:https://aistudio.baidu.com/aistudio/competition/detail/66?isFromLuge=true

官方的baseline版本是基于paddlepaddle框架的,我把它改写成了Pytorch框架,其中大部分代码沿用的是官方提供的代码,只是有一些框架部分进行了修改,另外增加了早停策略/对抗训练等优化措施,习惯用Pytorch版本的可以基于此进行优化.

环境

python=3.6
torch=1.7
transformers=4.5.0

训练示例

训练

python run.py
--max_len=256
--model_name_or_path=下载的预训练模型路径
--per_gpu_train_batch_size=7
--per_gpu_eval_batch_size=40
--learning_rate=1e-5
--linear_learning_rate=1e-4
--num_train_epochs=100
--output_dir="./output"
--weight_decay=0.01
--early_stop=2

预测

python predict.py
--max_len=400
--model_name_or_path=下载的预训练模型路径
--per_gpu_eval_batch_size=120
--output_dir="./output"
--fine_tunning_model=微调后的模型路径

实验结果

用的baseline模型是base版MacBERT(具体请看https://github.com/ymcui/MacBERT)

后续优化策略

数据清洗，据官方工作人员讲解到，训练集的准确率只能确保92%以上
更多的数据
更细粒度的数据增强
模型结构的优化

百度2021年语言与智能技术竞赛机器阅读理解Pytorch版baseline

Related tags

Overview

项目说明:

环境

训练示例

实验结果

后续优化策略

Owner

周俊贤

Official PyTorch implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

A bare-bones Python library for quality diversity optimization.

Emotional conditioned music generation using transformer-based model.

MaskTrackRCNN for video instance segmentation based on mmdetection

Code for Paper: Self-supervised Learning of Motion Capture

[LREC] MMChat: Multi-Modal Chat Dataset on Social Media

Neon-erc20-example - Example of creating SPL token and wrapping it with ERC20 interface in Neon EVM

Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

Commonsense Ability Tests

This project uses reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can learn to read tape. The project is dedicated to hero in life great Jesse Livermore.

EMNLP 2021: Single-dataset Experts for Multi-dataset Question-Answering

The Pytorch implementation for "Video-Text Pre-training with Learned Regions"

Orchestrating Distributed Materials Acceleration Platform Tutorial

Official Code Release for Container : Context Aggregation Network

An Implicit Function Theorem (IFT) optimizer for bi-level optimizations

NeWT: Natural World Tasks

Space Invaders For Python

generate-2D-quadrilateral-mesh-with-neural-networks-and-tree-search

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

Tacotron 2 - PyTorch implementation with faster-than-realtime inference