CV backbones including GhostNet, TinyNet and TNT, developed by Huawei Noah's Ark Lab.

Last update: Jan 08, 2023

Overview

CV Backbones

including GhostNet, TinyNet, TNT (Transformer in Transformer) developed by Huawei Noah's Ark Lab.

GhostNet Code
TinyNet Code
TNT Code
PyramidTNT Code
LegoNet Code
Versatile Filters Code
Citation
Other versions

News

2022/01/05 PyramidTNT: An improved TNT baseline is released.

2021/09/28 The paper of TNT (Transformer in Transformer) is accepted by NeurIPS 2021.

2021/09/18 The extended version of Versatile Filters is accepted by T-PAMI.

2021/08/30 GhostNet paper is selected as the Most Influential CVPR 2020 Papers.

2021/08/26 The codes of LegoNet and Versatile Filters has been merged into this repo.

2021/06/15 The code of TNT (Transformer in Transformer) has been released in this repo.

2020/10/31 GhostNet+TinyNet achieves better performance. See details in our NeurIPS 2020 paper: arXiv.

2020/06/10 GhostNet is included in PyTorch Hub.

GhostNet Code

This repo provides GhostNet pretrained models and inference code for TensorFlow and PyTorch:

Tensorflow: ./ghostnet_tensorflow with pretrained model.
PyTorch: ./ghostnet_pytorch with pretrained model.
We also opensource code on MindSpore Hub and MindSpore Model Zoo.

For training, please refer to tinynet or timm.

TinyNet Code

This repo provides TinyNet pretrained models and inference code for PyTorch:

PyTorch: ./tinynet_pytorch with pretrained model.
We also opensource training code on MindSpore Model Zoo.

TNT Code

This repo provides training code and pretrained models of TNT (Transformer in Transformer) for PyTorch:

PyTorch: ./tnt_pytorch.
We also opensource code on MindSpore Model Zoo.

The code of PyramidTNT is also released:

PyTorch: ./tnt_pytorch.

LegoNet Code

This repo provides the implementation of paper LegoNet: Efficient Convolutional Neural Networks with Lego Filters (ICML 2019)

PyTorch: ./legonet_pytorch.

Versatile Filters Code

This repo provides the implementation of paper Learning Versatile Filters for Efficient Convolutional Neural Networks (NeurIPS 2018)

PyTorch: ./versatile_filters.

Citation

@inproceedings{ghostnet,
  title={GhostNet: More Features from Cheap Operations},
  author={Han, Kai and Wang, Yunhe and Tian, Qi and Guo, Jianyuan and Xu, Chunjing and Xu, Chang},
  booktitle={CVPR},
  year={2020}
}
@inproceedings{tinynet,
  title={Model Rubik’s Cube: Twisting Resolution, Depth and Width for TinyNets},
  author={Han, Kai and Wang, Yunhe and Zhang, Qiulin and Zhang, Wei and Xu, Chunjing and Zhang, Tong},
  booktitle={NeurIPS},
  year={2020}
}
@inproceedings{tnt,
  title={Transformer in transformer},
  author={Han, Kai and Xiao, An and Wu, Enhua and Guo, Jianyuan and Xu, Chunjing and Wang, Yunhe},
  booktitle={NeurIPS},
  year={2021}
}
@inproceedings{legonet,
    title={LegoNet: Efficient Convolutional Neural Networks with Lego Filters},
    author={Yang, Zhaohui and Wang, Yunhe and Liu, Chuanjian and Chen, Hanting and Xu, Chunjing and Shi, Boxin and Xu, Chao and Xu, Chang},
    booktitle={ICML},
    year={2019}
  }
@inproceedings{wang2018learning,
  title={Learning versatile filters for efficient convolutional neural networks},
  author={Wang, Yunhe and Xu, Chang and Chunjing, XU and Xu, Chao and Tao, Dacheng},
  booktitle={NeurIPS},
  year={2018}
}

Other versions of GhostNet

This repo provides the TensorFlow/PyTorch code of GhostNet. Other versions and applications can be found in the following:

timm: code with pretrained model
Darknet: cfg file, and description
Gluon/Keras/Chainer: code
Paddle: code
Bolt inference framework: benckmark
Human pose estimation: code
YOLO with GhostNet backbone: code
Face recognition: cavaface, FaceX-Zoo, TFace

Comments

TypeError: __init__() got an unexpected keyword argument 'bn_tf'

Hello, I want to ask what caused the following error when running the train.py file? thank you “ TypeError: init() got an unexpected keyword argument 'bn_tf' ”

opened by ModeSky 16
Counting ReLU vs HardSwish FLOPs

Thank you very much for sharing the source code. I have a question related to FLOPs counting for ReLU and HardSwish. I saw in the paper the flops are the same in ReLU and HardSwish. Can you explain this situation?

opened by jahongir7174 10
kernel size in primary convolution of Ghost module

Hi, It is said in your paper that the primary convolution in Ghost module can have customized kernel size, which is a major difference from existing efficient convolution schemes. However, it seems that in this code all the kernel size of primary convolution in Ghost module are set to [1, 1], and the kernel set in _CONV_DEFS_0 are only used in blocks of stride=2. Is it set intentionally?

opened by YUHAN666 9
用GhostModule替换Conv2d，loss降的很慢？

我直接将efficientnet里面的MBConvBlock中的Conv2d替换为GhostModule： Conv2d(in_channels=inp, out_channels=oup, kernel_size=1, bias=False) 替换为 GhostModule(inp, oup)，其他参数不变，为什么损失比以前收敛的更慢了，一直降不下来？请问需要修改其他什么参数吗？

opened by yc-cui 8
Training hyperparams on ImageNet

Hi, thanks for sharing such a wonderful work, I'd like to reproduce your results on ImageNet, could you please specify training parameters such as initial learning rate, how to decay it, batch size, etc. It would be even better if you can provide tricks to train GhostNet, such as label smoothing and data augmentation. Thx!
good first issue

opened by sean-zhuh 8
Why did you exclude EfficientNetB0 from Accuracy-Latency chart?
@iamhankai Hi,

Great work!

Why did you exclude EfficientNetB0 (0.390 BFlops - 76.3% Top1) from Accuracy-Latency chart?

Also what mini_batch_size did you use for training GhostNet?
opened by AlexeyAB 8
VIG pretrained weights

@huawei-noah-admin cna you please share the VIG pretraiend model on google drive or one drive as baidu is not accessible from our end

THank in advance

opened by abhigoku10 7
The implementation of Isotropic architecture

Hi, thanks for sharing this impressive work. The paper mentioned two architectures, Isotropic one and pyramid one. I noticed that in the code, this is a reduce_ratios, and this reduce_ratios are used by a avg_pooling operation to calculate before building the graph. I am wondering whether all I need to do is setting this reduce_ratios to [1,1,1,1] if I want to implement the Isotropic architecture. Thanks.

self.n_blocks = sum(blocks) channels = opt.channels reduce_ratios = [4, 2, 1, 1] dpr = [x.item() for x in torch.linspace(0, drop_path, self.n_blocks)] num_knn = [int(x.item()) for x in torch.linspace(k, k, self.n_blocks)]

opened by buptxiaofeng 6
Gradient overflow occurs while training tnt-ti model

^@^@Train: 41 [ 0/625 ( 0%)] Loss: 4.564162 (4.5642) Time: 96.744s, 21.17/s (96.744s, 21.17/s) LR: 8.284e-04 Data: 94.025 (94.025) ^@^@^@^@Train: 41 [ 50/625 ( 8%)] Loss: 4.395192 (4.4797) Time: 2.742s, 746.96/s (7.383s, 277.38/s) LR: 8.284e-04 Data: 0.057 (4.683) ^@^@^@^@Train: 41 [ 100/625 ( 16%)] Loss: 4.424296 (4.4612) Time: 2.741s, 747.15/s (6.529s, 313.66/s) LR: 8.284e-04 Data: 0.056 (3.831) Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0

And the top-1 acc is only 0.2 after 40 epochs.

Any tips available here, dear @iamhankai @yitongh

opened by jimmyflycv 6
Bloated model

Hi, I am using Ghostnet backbone for training YoloV3 model in Tensorflow, but I am getting a bloated model. The checkpoint data size is approx. 68MB, but the checkpoint given here is of approx 20MB https://github.com/huawei-noah/ghostnet/blob/master/tensorflow/models/ghostnet_checkpoint.data-00000-of-00001

I am also training EfficientNet model with YoloV3 and that seems to be working fine, without any bloated size.

Could anyone or the author please confirm if this is the correct architecture or anything seems weird? I have attached the Ghostnet architecture file out of the code.

Thanks. ghostnet_model_arch.txt

opened by ghost 6
Replace Conv2d in my network, however it becomes slower, why?

Above all, thanks for your great work! It really inspires me a lot! But now I have a question.

I replace all the Conv2d operations in my network except the final ones, the model parameters really becomes much more less. However, when testing, I found that the average forward time decreases a lot by the replacement (from 428FPS down to 354FPS). So, is this a normal phenomenon? Or is this because of the concat operation?

opened by FunkyKoki 6
VIG for segmenation

@iamhankai thanks for open-sourcing the code base . Can you please let me knw how to use the pvig for segmentation related activities its really helpful

THanks in advance

opened by abhigoku10 0
higher performance of ViG

I try to train ViG-S on ImageNet and get 80.54% top1 accuracy, which is higher than that in paper, 80.4%. I wonder if 80.4 is the average of multiple trainings? If yes, how many reps do you use?

opened by tdzdog 9