The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Last update: Nov 09, 2022

Related tags

Deep Learning RBN

Overview

Representative Batch Normalization (RBN) with Feature Calibration

The official implementation of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

You only need to replace the BN with our RBN without any other adjustment.

Update

2021.4.9 The Jittor implementation is available now in Jittor.
2021.4.1 The training code of ImageNet classification using RBN is released.

Introduction

Batch Normalization (BatchNorm) has become the default component in modern neural networks to stabilize training. In BatchNorm, centering and scaling operations, along with mean and variance statistics, are utilized for feature standardization over the batch dimension. The batch dependency of BatchNorm enables stable training and better representation of the network, while inevitably ignores the representation differences among instances. We propose to add a simple yet effective feature calibration scheme into the centering and scaling operations of BatchNorm, enhancing the instance-specific representations with the negligible computational cost. The centering calibration strengthens informative features and reduces noisy features. The scaling calibration restricts the feature intensity to form a more stable feature distribution. Our proposed variant of BatchNorm, namely Representative BatchNorm, can be plugged into existing methods to boost the performance of various tasks such as classification, detection, and segmentation.

Applications

ImageNet classification

The training code of ImageNet classification is released in ImageNet_training folder.

Citation

If you find this work or code is helpful in your research, please cite:

@inproceedings{gao2021rbn,
  title={Representative Batch Normalization with Feature Calibration},
  author={Gao, Shang-Hua and Han, Qi and Li, Duo and Peng, Pai and Cheng, Ming-Ming and Pai Peng},
  booktitle=CVPR,
  year={2021}
}

Contact

If you have any questions, feel free to E-mail Shang-Hua Gao (shgao(at)live.com) and Qi Han(hqer(at)foxmail.com).

The official repo for CVPR2021——ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search.

ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search [paper] Introduction This is the official implementation of ViPNAS: Efficient V

42 Sep 26, 2022

[CVPR 2022] Official code for the paper: "A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration"

MDCA Calibration This is the official PyTorch implementation for the paper: "A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved

21 Dec 22, 2022

Code for CVPR2021 paper "Learning Salient Boundary Feature for Anchor-free Temporal Action Localization"

AFSD: Learning Salient Boundary Feature for Anchor-free Temporal Action Localization This is an official implementation in PyTorch of AFSD. Our paper

146 Dec 24, 2022

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information by Masato Tamura, Hiroki Ohashi, and Tomoaki Yosh

105 Dec 23, 2022

[CVPR2021 Oral] UP-DETR: Unsupervised Pre-training for Object Detection with Transformers

UP-DETR: Unsupervised Pre-training for Object Detection with Transformers This is the official PyTorch implementation and models for UP-DETR paper: @a

430 Dec 23, 2022

[CVPR2021 Oral] End-to-End Video Instance Segmentation with Transformers

VisTR: End-to-End Video Instance Segmentation with Transformers This is the official implementation of the VisTR paper: Installation We provide instru

687 Jan 7, 2023

[CVPR2021 Oral] FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation.

FFB6D This is the official source code for the CVPR2021 Oral work, FFB6D: A Full Flow Biderectional Fusion Network for 6D Pose Estimation. (Arxiv) Tab

201 Dec 28, 2022

Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces Installation After cloning the repo open

37 Dec 3, 2022

Official pytorch implementation of "Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization" ACMMM 2021 (Oral)

Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization This is an official implementation of "Feature Stylization and Domain-

22 Sep 22, 2022

Comments

关于scaling Calibration的可学习参数b初始化问题
您好，我有个问题想问下，关于scaling calibration中，您对偏置b的参数初始化为1，这是有什么根据吗

self.scale_weight.data.fill_(0) self.scale_bias.data.fill_(1)

因为根据你的公式在限制函数中（沿用你代码的sigmoid函数），你先让可学习参数w初始化为0，那么整个限制函数中一开始就是

R(wb)

而wb一开始为1的时候，对应sigmoid的值约为0.731，把他提到方差外部，则方差变为原始方差的0.73*0.73 = 0.5329，相当于方差减半了。若一开始训练就做这么剧烈的变化，是不是对后续训练有一定影响？

我能理解权重w初始化为0，可以根据centering calibration那一节有

When the absolute value of wm is close to zero, the centering operation still relies on the running statistics.

针对这两个可学习参数的初始值设定，有进行过相关实验探讨吗
opened by MARD1NO 2
使用fuse函数会报错
def fuse_conv_and_bn(conv, bn): # https://tehnokv.com/posts/fusing-batchnorm-and-conv/ with torch.no_grad(): # init fusedconv = torch.nn.Conv2d(conv.in_channels, conv.out_channels, kernel_size=conv.kernel_size, stride=conv.stride, padding=conv.padding, bias=True)

# prepare filters w_conv = conv.weight.clone().view(conv.out_channels, -1) w_bn = torch.diag(bn.weight.div(torch.sqrt(bn.eps + bn.running_var))) fusedconv.weight.copy_(torch.mm(w_bn, w_conv).view(fusedconv.weight.size())) # prepare spatial bias if conv.bias is not None: b_conv = conv.bias else: b_conv = torch.zeros(conv.weight.size(0)) b_bn = bn.bias - bn.weight.mul(bn.running_mean).div(torch.sqrt(bn.running_var + bn.eps)) fusedconv.bias.copy_(torch.mm(w_bn, b_conv.reshape(-1, 1)).reshape(-1) + b_bn) return fusedconv

Fusing layers... Traceback (most recent call last): File "test.py", line 263, in opt.augment) File "test.py", line 45, in test model.fuse() File "/home/zzf/Desktop/yolov3-dbb+representbatchnorm/models.py", line 402, in fuse fused = torch_utils.fuse_conv_and_bn(conv, b) File "/home/zzf/Desktop/yolov3-dbb+representbatchnorm/utils/torch_utils.py", line 83, in fuse_conv_and_bn w_bn = torch.diag(bn.weight.div(torch.sqrt(bn.eps + bn.running_var))) RuntimeError: matrix or a vector expected

把自己网络的batchnorm 改变后会报错麻烦解决以下。
opened by xiaowanzizz 1
论文中的一些疑惑

您好，感谢您的工作！论文里的一些地方我没有明白，希望您能解答一下，谢谢。 ① When the Km in Eqn.(5) is set to Uc，the running mean of Km is equal to E(X) 请问这句话应该怎么理解呢？ ②在Choice of Instance Statistics中，你提到的the mean and standard division over spatial dimensions, denoted by 请问这两个值具体怎么计算？ ③ ”Since scaling calibration only restricts the feature intensity while not changing the amount of information, scaling with both channel and spatial statistics results in a similar performance.”，请问改变信息的数量是什么意思呢？

opened by songyonger 1

Releases(pretrained)

pretrained(Apr 1, 2021)

The resnet101-rbn ImageNet pre-trained model.
Source code(tar.gz)
Source code(zip)
resnet101_rbn_best_model.pth(171.08 MB)

Owner

Open source projects of ShangHua-Gao

GitHub Repository

Implementation of "Selection via Proxy: Efficient Data Selection for Deep Learning" from ICLR 2020.

Selection via Proxy: Efficient Data Selection for Deep Learning This repository contains a refactored implementation of "Selection via Proxy: Efficien

70 Nov 16, 2022

A large-scale video dataset for the training and evaluation of 3D human pose estimation models

ASPset-510 (Australian Sports Pose Dataset) is a large-scale video dataset for the training and evaluation of 3D human pose estimation models. It contains 17 different amateur subjects performing 30

25 Jun 20, 2021

It is a system used to detect bone fractures. using techniques deep learning and image processing

MohammedHussiengadalla-Intelligent-Classification-System-for-Bone-Fractures It is a system used to detect bone fractures. using techniques deep learni

7 Nov 11, 2022

Neurolab is a simple and powerful Neural Network Library for Python

Neurolab Neurolab is a simple and powerful Neural Network Library for Python. Contains based neural networks, train algorithms and flexible framework

152 Dec 06, 2022

This repository is for DSA and CP scripts for reference.

dsa-script-collections This Repo is the collection of DSA and CP scripts for reference. Contents Python Bubble Sort Insertion Sort Merge Sort Quick So

9 Nov 22, 2022

Code & Data for Enhancing Photorealism Enhancement

Enhancing Photorealism Enhancement Stephan R. Richter, Hassan Abu AlHaija, Vladlen Koltun Paper | Website (with side-by-side comparisons) | Video (Pap

1.1k Dec 31, 2022

A hobby project which includes a hand-gesture based virtual piano using a mobile phone camera and OpenCV library functions

Overview This is a hobby project which includes a hand-gesture controlled virtual piano using an android phone camera and some OpenCV library. My moti

1 Nov 19, 2021

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target i

13 Oct 09, 2022

Code for "Long Range Probabilistic Forecasting in Time-Series using High Order Statistics"

Long Range Probabilistic Forecasting in Time-Series using High Order Statistics This is the code produced as part of the paper Long Range Probabilisti

16 Dec 06, 2022

PyTorch implementation for the paper Visual Representation Learning with Self-Supervised Attention for Low-Label High-Data Regime

Visual Representation Learning with Self-Supervised Attention for Low-Label High-Data Regime Created by Prarthana Bhattacharyya. Disclaimer: This is n

5 Nov 08, 2022

The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Related tags

Overview

Representative Batch Normalization (RBN) with Feature Calibration

Update

Introduction

Applications

ImageNet classification

Citation

Contact

You might also like...

The official repo for CVPR2021——ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search.

[CVPR 2022] Official code for the paper: "A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration"

Code for CVPR2021 paper "Learning Salient Boundary Feature for Anchor-free Temporal Action Localization"

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

[CVPR2021 Oral] UP-DETR: Unsupervised Pre-training for Object Detection with Transformers

[CVPR2021 Oral] End-to-End Video Instance Segmentation with Transformers

[CVPR2021 Oral] FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation.

Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

Official pytorch implementation of "Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization" ACMMM 2021 (Oral)

Comments

关于scaling Calibration的可学习参数b初始化问题

使用fuse函数会报错

论文中的一些疑惑

Releases(pretrained)

pretrained(Apr 1, 2021)

Owner

Open source projects of ShangHua-Gao

Implementation of "Selection via Proxy: Efficient Data Selection for Deep Learning" from ICLR 2020.

A large-scale video dataset for the training and evaluation of 3D human pose estimation models

It is a system used to detect bone fractures. using techniques deep learning and image processing

Neurolab is a simple and powerful Neural Network Library for Python

This repository is for DSA and CP scripts for reference.

Code & Data for Enhancing Photorealism Enhancement

A hobby project which includes a hand-gesture based virtual piano using a mobile phone camera and OpenCV library functions

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation

Code for "Long Range Probabilistic Forecasting in Time-Series using High Order Statistics"

PyTorch implementation for the paper Visual Representation Learning with Self-Supervised Attention for Low-Label High-Data Regime

Rethinking Nearest Neighbors for Visual Classification

Analysis of Smiles through reservoir sampling & RDkit

PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

Small little script to scrape, parse and check for active tor nodes. Can be used as proxies.

Code to reproduce the results in "Visually Grounded Reasoning across Languages and Cultures", EMNLP 2021.

Reference code for the paper "Cross-Camera Convolutional Color Constancy" (ICCV 2021)

History Aware Multimodal Transformer for Vision-and-Language Navigation

PyTorch Implementation of VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis.

Official Implementation of Few-shot Visual Relationship Co-localization

NaturalProofs: Mathematical Theorem Proving in Natural Language