Berkeley, Samsung | a fast post training converter pruning framework

title ： Berkeley 、 samsung |A Fast Post-Training Pruning Framework for Transformers（ A fast post training converter trim frame ）

author ：Woosuk Kwon, Sehoon Kim, Amir Gholami etc.

brief introduction ： This paper introduces a reasoning acceleration method . Pruning is an effective way to reduce large Transformer The huge reasoning cost of the model . However , Previous work on model pruning requires retraining the model . This may increase high costs and model deployment complexity , Make it difficult to use the actual situation in many cases . To solve this problem , A fast post training pruning framework for converters is proposed , You don't need any retraining . Given resource constraints and a sample dataset , The author's framework automatically prunes the converter model using the structured sparse method . To maintain high accuracy without retraining , The author introduces three new technologies ：（i） The lightweight mask search algorithm is based on Fisher Information ; (ii) Mask rearrangement supplementary search algorithm ; (iii) Mask adjustment reconstructs the output activation of each layer . The author of BERTBASE and DistilBERT Apply this method , The author evaluates it in GLUE and SQuAD Effectiveness in benchmarking . The author's framework achieves up to 2.0 Times FLOP Savings and reasoning delays are improved 1.56 times , While maintaining accuracy loss <1%. It is important to , The author's converter frame is trimmed in a single GPU Not up to 3 minute , This is more than two orders of magnitude faster than the existing retraining pruning methods .

Paper download ：https://arxiv.org/pdf/2204.09656.pdf