Distilling Knowledge via Knowledge Review, CVPR 2021

Overview

ReviewKD

Distilling Knowledge via Knowledge Review

Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia

This project provides an implementation for the CVPR 2021 paper "Distilling Knowledge via Knowledge Review"

CIFAR-100 Classification

Please refer to CIFAR-100 for more details.

ImageNet Classification

Please refer to ImageNet for more details.

COCO Detection

Coming soon

COCO Instance Segmentation

Coming soon

Citation

Please consider citing ReviewKD in your publications if it helps your research.

@inproceedings{chen2021reviewkd,
    title={Distilling Knowledge via Knowledge Review},
    author={Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia},
    booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2021},
}
Comments
  • Questions about detection pretrained weights

    Questions about detection pretrained weights

    I want to make sure that the file mv2-r50.pth in the detection pretrained weights you provided contains both teacher's and student's weights.

    Thank you!

    opened by Coldfire93 7
  • Log file of loss values

    Log file of loss values

    Hi Author, thanks for your excellent work. I want to ask whether you can release a log file that includes loss values. Based on this file, I can check what‘s the loss change? It is better for the detection model. It would be the best for retinanet. Thank you!

    opened by hdjsjyl 2
  • Can we find the teacher_weights somewhere?

    Can we find the teacher_weights somewhere?

    When I run your scripts "reviewKD.sh" and "baseline.sh" in Cifar100. There's FileNotFoundError:

    FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/cifar100_wrn-40-2__baseline1_best.pt'
    Namespace(T=4.0, batch_size=128, ce_loss_weight=1.0, dataset='cifar100', epochs=240, gamma=0.1, kd_loss_weight=5.0, kd_warm_up=20.0, kl_loss_weight=1.0, lr=0.1, lr_adjust_step=[150, 180, 210], model='wrn-40-1', resume='', seed=148, suffix='reviewkd1', teacher='wrn-40-2', teacher_weight='checkpoints/cifar100_wrn-40-2__baseline1_best.pt', test=False, use_kl=False, wd=0.0005)
    

    Where could I find those weights or can you release the related teacher weights so that we can download and better configure our experiment environment.

    opened by Luodian 2
  • Realization of the knowledge review

    Realization of the knowledge review

    Hi, thanks for your great job! I wrote a kr version using paddle, could you please help see is there any problems? thank you!

    https://github.com/littletomatodonkey/code_scipts/blob/main/knowledge_review/knowledge_review.py

    I used conv_1x1 for all the channel transform and adaptative avg pool for the size transform.

    opened by littletomatodonkey 2
  • about teacher net

    about teacher net

    Thank you very much for your work!

    I have noticed that before distillation, the teacher networks are loaded with a pre-trained model. Is the teacher network fixed during distillation, I didn't find where this part of the code (like detach or i.requires_grad = False)

    opened by yyuxin 1
  • Knowledge distillation on RetinaNet

    Knowledge distillation on RetinaNet

    Hi authors, thanks for the great work. But the repository only includes object detectors on Faster RCNN. I want to know when the knowledge distillation of the object detector based on RetinaNet will be released? Thank you!

    opened by hdjsjyl 1
  • Where is the mobilenet baseline from

    Where is the mobilenet baseline from

    Hi, thanks for your great job! Where is the mobilenet baseline from? I train the mobilenet for 100epochs and the top1-acc is 69.4%, which seems higher than that provided in the article(68.8%).

    opened by littletomatodonkey 1
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
  • apply it on yolox

    apply it on yolox

    Thanks for your great work! Have you ever apply it on yolox? When i do like this, the loss of it is unstable.I used the output of neck(3 layers),and adapt the teacher channel with student ones.Looking for your reply.Thanks a lot. image

    opened by Thatboy7 1
  • Will KL-Divergence loss further improve the performance?

    Will KL-Divergence loss further improve the performance?

    Thank you for the nice work! I wonder if you have tried to use ReviewKD loss and KL-divergence loss together? Will the combination further improve the performance? If yes, would you like to share the results or the hyperparameters?

    opened by LiuDongyang6 1
  • shapes and out_shapes values in ReviewKD

    shapes and out_shapes values in ReviewKD

    To me, it's confusing, how to set the "shapes" and "out_shapes" when the student is ResNet18 and the teacher is ResNet34 on CIFAR-100.

    Is it shapes = out_shapes = [1, 8, 16, 32, 32]

    or, shapes = out_shapes = [1, 4, 8, 16, 32]

    opened by Nandan91 1
Owner
DV Lab
Deep Vision Lab
DV Lab
OCR engine for all the languages

Description kraken is a turn-key OCR system optimized for historical and non-Latin script material. kraken's main features are: Fully trainable layout

431 Jan 04, 2023
This is a project to detect gestures to zoom in or out, using the real-time distance between the index finger and the thumb. It's based on OpenCV and Mediapipe.

Pinch-zoom This is a python project based on real-time hand-gesture detection, to zoom in or out, using the distance between the index finger and the

Harshit Bhalla 6 Jul 11, 2022
Optical character recognition for Japanese text, with the main focus being Japanese manga

Manga OCR Optical character recognition for Japanese text, with the main focus being Japanese manga. It uses a custom end-to-end model built with Tran

Maciej Budyś 327 Jan 01, 2023
CNN+LSTM+CTC based OCR implemented using tensorflow.

CNN_LSTM_CTC_Tensorflow CNN+LSTM+CTC based OCR(Optical Character Recognition) implemented using tensorflow. Note: there is No restriction on the numbe

Watson Yang 356 Dec 08, 2022
Python rubik's cube solver

This program makes a 3D representation of a rubiks cube and solves it step by step.

Pablo QB 4 May 29, 2022
Awesome Spectral Indices in Python.

Awesome Spectral Indices in Python: Numpy | Pandas | GeoPandas | Xarray | Earth Engine | Planetary Computer | Dask GitHub: https://github.com/davemlz/

David Montero Loaiza 98 Jan 02, 2023
Pytorch implementation of PSEnet with Pyramid Attention Network as feature extractor

Scene Text-Spotting based on PSEnet+CRNN Pytorch implementation of an end to end Text-Spotter with a PSEnet text detector and CRNN text recognizer. We

azhar shaikh 62 Oct 10, 2022
A collection of resources (including the papers and datasets) of OCR (Optical Character Recognition).

OCR Resources This repository contains a collection of resources (including the papers and datasets) of OCR (Optical Character Recognition). Contents

Zuming Huang 363 Jan 03, 2023
Qrcode Attendence System with Opencv and Pyzbar

Setup process Creates a virtual environment (Scripts that ensure executed Python code uses the Python interpreter and site packages installed inside t

Ganesh 5 Aug 01, 2022
一键翻译各类图片内文字

一键翻译各类图片内文字 针对群内、各个图站上大量不太可能会有人去翻译的图片设计,让我这种日语小白能够勉强看懂图片 主要支持日语,不过也能识别汉语和小写英文 支持简单的涂白和嵌字

574 Dec 28, 2022
A simple python program to record security cam footage by detecting a face and body of a person in the frame.

SecurityCam A simple python program to record security cam footage by detecting a face and body of a person in the frame. This code was created by me,

1 Nov 08, 2021
Code for CVPR'2022 paper ✨ "Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model"

PPE ✨ Repository for our CVPR'2022 paper: Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-

Zipeng Xu 34 Nov 28, 2022
TedEval: A Fair Evaluation Metric for Scene Text Detectors

TedEval: A Fair Evaluation Metric for Scene Text Detectors Official Python 3 implementation of TedEval | paper | slides Chae Young Lee, Youngmin Baek,

Clova AI Research 167 Nov 20, 2022
A simple demo program for using OpenCV on Android

Kivy OpenCV Demo A simple demo program for using OpenCV on Android Build with: buildozer android debug deploy run Run (on desktop) with: python main.p

Andrea Ranieri 13 Dec 29, 2022
A simple Security Camera created using Opencv in Python where images gets saved in realtime in your Dropbox account at every 5 seconds

Security Camera using Opencv & Dropbox This is a simple Security Camera created using Opencv in Python where images gets saved in realtime in your Dro

Arpit Rath 1 Jan 31, 2022
Fully-automated scripts for collecting AI-related papers

AI-Paper-Collector Web demo: https://ai-paper-collector.vercel.app/ (recommended) Colab notebook: here Motivation Fully-automated scripts for collecti

772 Dec 30, 2022
A program that takes in the hand gesture displayed by the user and translates ASL.

Interactive-ASL-Recognition Using the framework mediapipe made by google, OpenCV library and through self teaching, I was able to create a program tha

Riddhi Bajaj 3 Nov 22, 2021
The first open-source library that detects the font of a text in a image.

Typefont Typefont is an experimental library that detects the font of a text in a image. Usage Import the main function and invoke it like in the foll

Vasile Pește 1.6k Feb 24, 2022
A set of workflows for corpus building through OCR, post-correction and normalisation

PICCL: Philosophical Integrator of Computational and Corpus Libraries PICCL offers a workflow for corpus building and builds on a variety of tools. Th

Language Machines 41 Dec 27, 2022
Document Image Dewarping

Document image dewarping using text-lines and line Segments Abstract Conventional text-line based document dewarping methods have problems when handli

Taeho Kil 268 Dec 23, 2022