Distilling Knowledge via Knowledge Review, CVPR 2021

Overview

ReviewKD

Distilling Knowledge via Knowledge Review

Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia

This project provides an implementation for the CVPR 2021 paper "Distilling Knowledge via Knowledge Review"

CIFAR-100 Classification

Please refer to CIFAR-100 for more details.

ImageNet Classification

Please refer to ImageNet for more details.

COCO Detection

Coming soon

COCO Instance Segmentation

Coming soon

Citation

Please consider citing ReviewKD in your publications if it helps your research.

@inproceedings{chen2021reviewkd,
    title={Distilling Knowledge via Knowledge Review},
    author={Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia},
    booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2021},
}
Comments
  • Questions about detection pretrained weights

    Questions about detection pretrained weights

    I want to make sure that the file mv2-r50.pth in the detection pretrained weights you provided contains both teacher's and student's weights.

    Thank you!

    opened by Coldfire93 7
  • Log file of loss values

    Log file of loss values

    Hi Author, thanks for your excellent work. I want to ask whether you can release a log file that includes loss values. Based on this file, I can check what‘s the loss change? It is better for the detection model. It would be the best for retinanet. Thank you!

    opened by hdjsjyl 2
  • Can we find the teacher_weights somewhere?

    Can we find the teacher_weights somewhere?

    When I run your scripts "reviewKD.sh" and "baseline.sh" in Cifar100. There's FileNotFoundError:

    FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/cifar100_wrn-40-2__baseline1_best.pt'
    Namespace(T=4.0, batch_size=128, ce_loss_weight=1.0, dataset='cifar100', epochs=240, gamma=0.1, kd_loss_weight=5.0, kd_warm_up=20.0, kl_loss_weight=1.0, lr=0.1, lr_adjust_step=[150, 180, 210], model='wrn-40-1', resume='', seed=148, suffix='reviewkd1', teacher='wrn-40-2', teacher_weight='checkpoints/cifar100_wrn-40-2__baseline1_best.pt', test=False, use_kl=False, wd=0.0005)
    

    Where could I find those weights or can you release the related teacher weights so that we can download and better configure our experiment environment.

    opened by Luodian 2
  • Realization of the knowledge review

    Realization of the knowledge review

    Hi, thanks for your great job! I wrote a kr version using paddle, could you please help see is there any problems? thank you!

    https://github.com/littletomatodonkey/code_scipts/blob/main/knowledge_review/knowledge_review.py

    I used conv_1x1 for all the channel transform and adaptative avg pool for the size transform.

    opened by littletomatodonkey 2
  • about teacher net

    about teacher net

    Thank you very much for your work!

    I have noticed that before distillation, the teacher networks are loaded with a pre-trained model. Is the teacher network fixed during distillation, I didn't find where this part of the code (like detach or i.requires_grad = False)

    opened by yyuxin 1
  • Knowledge distillation on RetinaNet

    Knowledge distillation on RetinaNet

    Hi authors, thanks for the great work. But the repository only includes object detectors on Faster RCNN. I want to know when the knowledge distillation of the object detector based on RetinaNet will be released? Thank you!

    opened by hdjsjyl 1
  • Where is the mobilenet baseline from

    Where is the mobilenet baseline from

    Hi, thanks for your great job! Where is the mobilenet baseline from? I train the mobilenet for 100epochs and the top1-acc is 69.4%, which seems higher than that provided in the article(68.8%).

    opened by littletomatodonkey 1
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
  • apply it on yolox

    apply it on yolox

    Thanks for your great work! Have you ever apply it on yolox? When i do like this, the loss of it is unstable.I used the output of neck(3 layers),and adapt the teacher channel with student ones.Looking for your reply.Thanks a lot. image

    opened by Thatboy7 1
  • Will KL-Divergence loss further improve the performance?

    Will KL-Divergence loss further improve the performance?

    Thank you for the nice work! I wonder if you have tried to use ReviewKD loss and KL-divergence loss together? Will the combination further improve the performance? If yes, would you like to share the results or the hyperparameters?

    opened by LiuDongyang6 1
  • shapes and out_shapes values in ReviewKD

    shapes and out_shapes values in ReviewKD

    To me, it's confusing, how to set the "shapes" and "out_shapes" when the student is ResNet18 and the teacher is ResNet34 on CIFAR-100.

    Is it shapes = out_shapes = [1, 8, 16, 32, 32]

    or, shapes = out_shapes = [1, 4, 8, 16, 32]

    opened by Nandan91 1
Owner
DV Lab
Deep Vision Lab
DV Lab
([email protected]) Boosting Co-teaching with Compression Regularization for Label Noise

Nested-Co-teaching ([email protected]) Pytorch implementation of paper "Boosting Co-tea

YINGYI CHEN 41 Jan 03, 2023
SemTorch

SemTorch This repository contains different deep learning architectures definitions that can be applied to image segmentation. All the architectures a

David Lacalle Castillo 154 Dec 07, 2022
"Very simple but works well" Computer Vision based ID verification solution provided by LibraX.

ID Verification by LibraX.ai This is the first free Identity verification in the market. LibraX.ai is an identity verification platform for developers

LibraX.ai 46 Dec 06, 2022
Code for the ACL2021 paper "Combining Static Word Embedding and Contextual Representations for Bilingual Lexicon Induction"

CSCBLI Code for our ACL Findings 2021 paper, "Combining Static Word Embedding and Contextual Representations for Bilingual Lexicon Induction". Require

Jinpeng Zhang 12 Oct 08, 2022
Convolutional Recurrent Neural Network (CRNN) for image-based sequence recognition.

Convolutional Recurrent Neural Network This software implements the Convolutional Recurrent Neural Network (CRNN), a combination of CNN, RNN and CTC l

Baoguang Shi 2k Dec 31, 2022
Total Text Dataset. It consists of 1555 images with more than 3 different text orientations: Horizontal, Multi-Oriented, and Curved, one of a kind.

Total-Text-Dataset (Official site) Updated on April 29, 2020 (Detection leaderboard is updated - highlighted E2E methods. Thank you shine-lcy.) Update

Chee Seng Chan 671 Dec 27, 2022
OCR of Chicago 1909 Renumbering Plan

Requirements: Python 3 (probably at least 3.4) pipenv (pip3 install pipenv) tesseract (brew install tesseract, at least if you have a mac and homebrew

ted whalen 2 Nov 21, 2021
Lightning Fast Language Prediction 🚀

whatthelang Lightning Fast Language Prediction 🚀 Dependencies The dependencies can be installed using the requirements.txt file: $ pip install -r req

Indix 152 Oct 16, 2022
Face_mosaic - Mosaic blur processing is applied to multiple faces appearing in the video

動機 face_recognitionを使用して得られる顔座標は長方形であり、この座標をそのまま用いてぼかし処理を行った場合得られる画像は醜い。 それに対してモ

Yoshitsugu Kesamaru 6 Feb 03, 2022
document image degradation

ocrodeg The ocrodeg package is a small Python library implementing document image degradation for data augmentation for handwriting recognition and OC

NVIDIA Research Projects 134 Nov 18, 2022
【Auto】原神⭐钓鱼辅助工具 | 自动收竿、校准游标 | ✨您只需要抛出鱼竿,我们会帮你完成一切✨

原神钓鱼辅助工具 ✨ 作者正在努力重构代码中……会尽快带给大家一个更完美的脚本 ✨ 「您只需抛出鱼竿,然后我们会帮您搞定一切」 如果你觉得这个脚本好用,请点一个 Star ⭐ ,你的 Star 就是作者更新最大的动力 点击这里 查看演示视频 ✨ 欢迎大家在 Issues 中分享自己的配置文件 ✨ ✨

261 Jan 02, 2023
A curated list of awesome synthetic data for text location and recognition

awesome-SynthText A curated list of awesome synthetic data for text location and recognition and OCR datasets. Text location SynthText SynthText_Chine

Tianzhong 283 Jan 05, 2023
Extract tables from scanned image PDFs using Optical Character Recognition.

ocr-table This project aims to extract tables from scanned image PDFs using Optical Character Recognition. Install Requirements Tesseract OCR sudo apt

Abhijeet Singh 209 Dec 06, 2022
The code for “Oriented RepPoints for Aerail Object Detection”

Oriented RepPoints for Aerial Object Detection The code for the implementation of “Oriented RepPoints”, Under review. (arXiv preprint) Introduction Or

WentongLi 207 Dec 24, 2022
This is a GUI program which consist of 4 OpenCV projects

Tkinter-OpenCV Project Using Tkinter, Opencv, Mediapipe This is a python GUI program using Tkinter which consist of 4 OpenCV projects 1. Finger Counte

Arya Bagde 3 Feb 22, 2022
Captcha Recognition

The objective of this project is to recognize the target numbers in the captcha images correctly which would tell us how good or bad a captcha system has been built.

Mohit Kaushik 5 Feb 20, 2022
A Vietnamese personal card OCR website built with Django.

Django VietCardOCR Installation Creation of virtual environments is done by executing the command venv: python -m venv venv That will create a new fol

Truong Hoang Thuan 4 Sep 04, 2021
An Implementation of the FOTS: Fast Oriented Text Spotting with a Unified Network

FOTS: Fast Oriented Text Spotting with a Unified Network Introduction This is a pytorch re-implementation of FOTS: Fast Oriented Text Spotting with a

GeorgeJoe 171 Aug 04, 2022
Papers, Datasets, Algorithms, SOTA for STR. Long-time Maintaining

Scene Text Recognition Recommendations Everythin about Scene Text Recognition SOTA • Papers • Datasets • Code Contents 1. Papers 2. Datasets 2.1 Synth

Deep Learning and Vision Computing Lab, SCUT 197 Jan 05, 2023