yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

Last update: Dec 04, 2022

Overview

代码地址：

https://github.com/Sharpiless/yolov5-knowledge-distillation

教师模型：

python train.py --weights weights/yolov5m.pt \
        --cfg models/yolov5m.yaml --data data/voc.yaml --epochs 50 \
        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml

蒸馏训练：

python train.py --weights weights/yolov5s.pt \
        --cfg models/yolov5s.yaml --data data/voc.yaml --epochs 50 \
        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml \
        --t_weights yolov5m.pt --distill

训练参数:

--weights：预训练模型

--t_weights：教师模型权重

--distill：使用知识蒸馏进行训练

--dist_loss：l2或者kl

--temperature：使用知识蒸馏时的温度

使用《Object detection at 200 Frames Per Second》中的损失

这篇文章分别对这几个损失函数做出改进，具体思路为只有当teacher network的objectness value高时，才学习bounding box坐标和class probabilities。

实验结果：

这里假设VOC2012中新增加的数据为无标签数据（2k张）。

教师模型	训练方法	蒸馏损失	P	R	mAP50
无	正常训练	不使用	0.7756	0.7115	0.7609
Yolov5l	output based	l2	0.7585	0.7198	0.7644
Yolov5l	output based	KL	0.7417	0.7207	0.7536
Yolov5m	output based	l2	0.7682	0.7436	0.7976
Yolov5m	output based	KL	0.7731	0.7313	0.7931

参数和细节正在完善，支持KL散度、L2 logits损失和Sigmoid蒸馏损失等

1. 正常训练：

2. L2蒸馏损失：

我的公众号：

关于作者

B站：https://space.bilibili.com/470550823

CSDN：https://blog.csdn.net/weixin_44936889

AI Studio：https://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156

Github：https://github.com/Sharpiless

yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

Related tags

Overview

代码地址：

教师模型：

蒸馏训练：

训练参数:

实验结果：

1. 正常训练：

2. L2蒸馏损失：

我的公众号：

关于作者

Owner

State-Relabeling Adversarial Active Learning

This repository contains code and data for "On the Multimodal Person Verification Using Audio-Visual-Thermal Data"

Code for the paper "Zero-shot Natural Language Video Localization" (ICCV2021, Oral).

Aalto-cs-msc-theses - Listing of M.Sc. Theses of the Department of Computer Science at Aalto University

tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

A note taker for NVDA. Allows the user to create, edit, view, manage and export notes to different formats.

KoCLIP: Korean port of OpenAI CLIP, in Flax

[NeurIPS 2021] Garment4D: Garment Reconstruction from Point Cloud Sequences

Official Implementation of CVPR 2022 paper: "Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning"

Airborne magnetic data of the Osborne Mine and Lightning Creek sill complex, Australia

Code for "Hierarchical Skills for Efficient Exploration" HSD-3 Algorithm and Baselines

Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels

Website which uses Deep Learning to generate horror stories.

本项目是一个带有前端界面的垃圾分类项目，加载了训练好的模型参数，模型为efficientnetb4，暂时为40分类问题。

A collection of Google research projects related to Federated Learning and Federated Analytics.

a reimplementation of Optical Flow Estimation using a Spatial Pyramid Network in PyTorch

《Train in Germany, Test in The USA: Making 3D Object Detectors Generalize》(CVPR 2020)

A super lightweight Lagrangian model for calculating millions of trajectories using ERA5 data

Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX