yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

Last update: Dec 04, 2022

Overview

代码地址：

https://github.com/Sharpiless/yolov5-knowledge-distillation

教师模型：

python train.py --weights weights/yolov5m.pt \
        --cfg models/yolov5m.yaml --data data/voc.yaml --epochs 50 \
        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml

蒸馏训练：

python train.py --weights weights/yolov5s.pt \
        --cfg models/yolov5s.yaml --data data/voc.yaml --epochs 50 \
        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml \
        --t_weights yolov5m.pt --distill

训练参数:

--weights：预训练模型

--t_weights：教师模型权重

--distill：使用知识蒸馏进行训练

--dist_loss：l2或者kl

--temperature：使用知识蒸馏时的温度

使用《Object detection at 200 Frames Per Second》中的损失

这篇文章分别对这几个损失函数做出改进，具体思路为只有当teacher network的objectness value高时，才学习bounding box坐标和class probabilities。

实验结果：

这里假设VOC2012中新增加的数据为无标签数据（2k张）。

教师模型	训练方法	蒸馏损失	P	R	mAP50
无	正常训练	不使用	0.7756	0.7115	0.7609
Yolov5l	output based	l2	0.7585	0.7198	0.7644
Yolov5l	output based	KL	0.7417	0.7207	0.7536
Yolov5m	output based	l2	0.7682	0.7436	0.7976
Yolov5m	output based	KL	0.7731	0.7313	0.7931

参数和细节正在完善，支持KL散度、L2 logits损失和Sigmoid蒸馏损失等

1. 正常训练：

2. L2蒸馏损失：

我的公众号：

关于作者

B站：https://space.bilibili.com/470550823

CSDN：https://blog.csdn.net/weixin_44936889

AI Studio：https://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156

Github：https://github.com/Sharpiless

yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

Related tags

Overview

代码地址：

教师模型：

蒸馏训练：

训练参数:

实验结果：

1. 正常训练：

2. L2蒸馏损失：

我的公众号：

关于作者

Owner

A PyTorch implementation of "CoAtNet: Marrying Convolution and Attention for All Data Sizes".

Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers"

Python version of the amazing Reaction Mechanism Generator (RMG).

Codeflare - Scale complex AI/ML pipelines anywhere

PyTorch implementation of "Simple and Deep Graph Convolutional Networks"

Where2Act: From Pixels to Actions for Articulated 3D Objects

A simple baseline for the 2022 IEEE GRSS Data Fusion Contest (DFC2022)

Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing, Ant Colony Optimization Algorithm,Immune Algorithm, Artificial Fish Swarm Algorithm, Differential Evolution and TSP(Traveling salesman)

💃 VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena

GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition

Source code for our EMNLP'21 paper 《Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning》

This project provides an unsupervised framework for mining and tagging quality phrases on text corpora with pretrained language models (KDD'21).

Learning 3D Part Assembly from a Single Image

Implementation of SegNet: A Deep Convolutional Encoder-Decoder Architecture for Semantic Pixel-Wise Labelling

WHENet - ONNX, OpenVINO, TFLite, TensorRT, EdgeTPU, CoreML, TFJS, YOLOv4/YOLOv4-tiny-3L

Sign Language is detected in realtime using video sequences. Our approach involves MediaPipe Holistic for keypoints extraction and LSTM Model for prediction.

Semi-Supervised Learning, Object Detection, ICCV2021

VoxHRNet - Whole Brain Segmentation with Full Volume Neural Network

Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, as a standalone package for Pytorch

Neighborhood Contrastive Learning for Novel Class Discovery