DOP-Tuning(Domain-Oriented Prefix-tuning model)

Overview

DOP-Tuning

DOP-Tuning(Domain-Oriented Prefix-tuning model)代码基于Prefix-Tuning改进.

Files

├── seq2seq                       # Code for encoder-decoder architecture
│   ├── train_bart.py             # high-level scripts to train.
│   ├── prefixTuning.py           # code that implements prefix-tuning.
│   ├── finetune.py               # training code (contains data loading, model loading, and calls trainer)   
│   ├── lightning_base.py         # helper code
│   ├── utils.py                  # helper code
│   └── callbacks.py              # helper code
└── ...

To run the code for encoder-decoder architecture like BART, the code is in seq2seq. This corresponds to the summarization experiments in the paper.

Setup:

cd transformer; pip install -e .

Train via DOP-tuning

cd seq2seq; 

python train_bart.py --mode xsum --preseqlen 200 --do_train yes --fp16 yes --bsz 16  --epoch 30  --gradient_accumulation_step 3 --learning_rate 0.00005  --mid_dim 800 --use_lowdata_token 'yes' --lowdata_token 'summarize'

其中use_lowdata_token表示是否采用domain word初始化的方式;lowdata_token表示传入的domain word.

Decode

cd seq2seq; 

python train_bart.py --mode xsum --do_train no --prefix_model_path {checkpoint_path} --preseqlen {same as training} --mid_dim {same as training} --use_lowdata_token 'yes' --lowdata_token 'summarize'
Owner
Andrew Zeng
Andrew Zeng
Andrew Zeng
An evolutionary multi-agent platform based on mesa and NEAT

An evolutionary multi-agent platform based on mesa and NEAT

Valerio1988 6 Dec 04, 2022
A Python application that simulates the rolling of a dice, randomly picking one of the 6 faces and then displaying it.

dice-roller-app This is an application developed in Python that shuffles between the 6 faces of a dice, using buttons to shuffle and close the applica

Paddy Costelloe 0 Jul 20, 2021
Convert your Gyrosco.pe travels to GPX files

gyroscope2gpx This little python joint will do you a favor of taking your "Travel" export from Gyroscope (https://gyrosco.pe) and turn it into a bunch

nick g 4 Oct 02, 2022
Your Google Recon is Now Automated

GRecon : GRecon (Greei-Conn) is a simple python tool that automates the process of Google Based Recon AKA Google Dorking The current Version 1.0 Run 7

adnane-tebbaa 189 Dec 21, 2022
This is a survey of python's async concurrency features by example.

Survey of Python's Async Features This is a survey of python's async concurrency features by example. The purpose of this survey is to demonstrate tha

Tyler Lovely 4 Feb 10, 2022
🏆 A ranked list of awesome Python open-source libraries and tools. Updated weekly.

Best-of Python 🏆 A ranked list of awesome Python open-source libraries & tools. Updated weekly. This curated list contains 230 awesome open-source pr

Machine Learning Tooling 2.7k Jan 03, 2023
Experimental proxy for dumping the unencrypted packet data from Brawl Stars (WIP)

Brawl Stars Proxy Experimental proxy for version 39.99 of Brawl Stars. It allows you to capture the packets being sent between the Brawl Stars client

4 Oct 29, 2021
Library for mocking AsyncIOMotorClient built on top of mongomock.

mongomock-motor Best effort mock for AsyncIOMotorClient (Database, Collection, e.t.c) built on top of mongomock library. Example / Showcase from mongo

Michael Kryukov 43 Jan 04, 2023
A Python Based Utility for Processing GST-Return JSON Files to Multiple Formats

GSTR 1/2A Utility by Shan.tk Open Source GSTR 1/GSTR 2A JSON to Excel utility based on Python. Useful for Auditors in Verifying GSTR 1 Return Invoices

Sudharshan TK 1 Oct 08, 2022
Hypothesis strategies for generating Python programs, something like CSmith

hypothesmith Hypothesis strategies for generating Python programs, something like CSmith. This is definitely pre-alpha, but if you want to play with i

Zac Hatfield-Dodds 73 Dec 14, 2022
A nonebot2 plugin, send news information in a picture form.

A nonebot2 plugin, send news information in a picture form.

幼稚园园长 7 Nov 18, 2022
Ml-design-patterns - Source code accompanying O'Reilly book: Machine Learning Design Patterns

This is not an official Google product ml-design-patterns Source code accompanying O'Reilly book: Title: Machine Learning Design Patterns Authors: Val

Google Cloud Platform 1.5k Jan 05, 2023
Beacon Object File (BOF) to obtain a usable TGT for the current user.

Beacon Object File (BOF) to obtain a usable TGT for the current user.

Connor McGarr 109 Dec 25, 2022
Solutions to the language assignment for Internship in JALA Technologies.

Python Assignment Solutions (JALA Technologies) Solutions to the language assignment for Internship in JALA Technologies. Features Properly formatted

Samyak Jain 2 Jan 17, 2022
NES development tool made with Python and Lua

NES Builder NES development and romhacking tool made with Python and Lua Current Stage: Alpha Features Open source "Build" project, which exports vari

10 Aug 19, 2022
A Way to Use Python, Easier.

PyTools A Way to Use Python, Easier. How to Install Just copy this code, then make a new file in your project directory called PyTools.py, then paste

Kamran 2 Aug 15, 2022
Roman numeral conversion with python

Roman numeral conversion Discipline: Programming Languages Student: Paulo Henrique Diniz de Lima Alencar. Language: Python Description Responsible for

Paulo Alencar 1 Jul 11, 2022
A class to draw curves expressed as L-System production rules

A class to draw curves expressed as L-System production rules

Juna Salviati 6 Sep 09, 2022
Procscan is a quick and dirty python script used to look for potentially dangerous api call patterns in a Procmon PML file.

PROCSCAN Procscan is a quick and dirty python script used to look for potentially dangerous api call patterns in a Procmon PML file. Installation git

Daniel Santos 9 Sep 02, 2022
Integration of Hotwire's Turbo library with Flask.

turbo-flask Integration of Hotwire's Turbo library with Flask, to allow you to create applications that look and feel like single-page apps without us

Miguel Grinberg 240 Jan 06, 2023