Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift (ICCV 2021)

Related tags

Deep LearningPi-NAS
Overview

Π-NAS

This repository provides the evaluation code of our submitted paper: Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift.

Our Trained Models

  • Here is a summary of our searched models:

    ImageNet FLOPs Params [email protected] [email protected]
    Π-NAS-cls 5.38G 27.1M 81.6% 95.7%
    Mask-RCNN on COCO 2017 APbb APmk
    Π-NAS-trans 44.07 39.50
    DeeplabV3 on ADE20K pixAcc mIoU
    Π-NAS-trans 81.27 45.47
    DeeplabV3 on Cityscapes mIoU
    Π-NAS-trans 80.70

Usage

1. Requirements

  • Install third-party requirements with command pip install -e .
  • Prepare ImageNet, COCO 2017, ADE20K and Cityscapes datasets
    • Our data paths are at /data/ImageNet, /data/coco, /data/ADEChallengeData2016 and /data/citys, respectively.
    • You can specify COCO's data path through environment variable DETECTRON2_DATASETS and others in experiments/recognition/verify.py, encoding/datasets/ade20k.py and encoding/datasets/cityscapes.py.
  • Download our checkpoint files

2. Evaluate our models

  • You can evaluate our models with the following command:

    ImageNet FLOPs Params [email protected] [email protected]
    Π-NAS-cls 5.38G 27.1M 81.6% 95.7%
    python experiments/recognition/verify.py --dataset imagenet --model alone_resnest50 --choice-indices 3 0 1 3 2 3 1 2 0 3 2 1 3 0 3 2 --resume /path/to/PiNAS_cls.pth.tar
    Mask-RCNN on COCO 2017 APbb APmk
    Π-NAS-trans 44.07 39.50
    DETECTRON2_DATASETS=/data python experiments/detection/plain_train_net.py --config-file experiments/detection/configs/mask_rcnn_ResNeSt_50_FPN_syncBN_1x.yaml --num-gpus 8 --eval-only MODEL.WEIGHTS /path/to/PiNAS_trans_COCO.pth MODEL.RESNETS.CHOICE_INDICES [3,3,3,3,1,1,3,3,3,0,0,1,1,0,2,1]
    DeeplabV3 on ADE20K pixAcc mIoU
    Π-NAS-trans 81.27 45.47
    python experiments/segmentation/test.py --dataset ADE20K --model deeplab --backbone alone_resnest50 --choice-indices 3 3 3 3 1 1 3 3 3 0 0 1 1 0 2 1 --aux --se-loss --resume /path/to/PiNAS_trans_ade.pth.tar --eval
    DeeplabV3 on Cityscapes mIoU
    Π-NAS-trans 80.70
    python experiments/segmentation/test.py --dataset citys --base-size 2048 --crop-size 768 --model deeplab --backbone alone_resnest50 --choice-indices 3 3 3 3 1 1 3 3 3 0 0 1 1 0 2 1 --aux --se-loss --resume /path/to/PiNAS_trans_citys.pth.tar --eval

Training and Searching

This reimplementation is based on OpenSelfSup and MoCo. Please acknowledge their contribution.

cd OpenSelfSup && pip install -v -e .

1. Π-NAS Learning

bash tools/dist_train.sh configs/pinas_learning.py 8 --work_dir /path/to/save/logs/and/models

2. Extract supernet backbone weights

python tools/extract_backbone_weights.py /checkpoint/of/1. /extracted/weight/of/1.

3. Linear Training

bash tools/dist_train.sh configs/pinas_linear_training.py 8 --pretrained /extracted/weight/of/1. --work_dir /path/to/save/logs/and/models

4. Linear Evaluation

bash tools/dist_train.sh configs/pinas_linear_evaluation.py 8 --resume_from /checkpoint/of/3. --work_dir /path/to/save/logs/and/models
Owner
Jiqi Zhang
Jiqi Zhang
TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification

TransPrompt This code is implement for our EMNLP 2021's paper 《TransPrompt:Towards an Automatic Transferable Prompting Framework for Few-shot Text Cla

WangJianing 23 Dec 21, 2022
Multi-Stage Spatial-Temporal Convolutional Neural Network (MS-GCN)

Multi-Stage Spatial-Temporal Convolutional Neural Network (MS-GCN) This code implements the skeleton-based action segmentation MS-GCN model from Autom

Benjamin Filtjens 8 Nov 29, 2022
source code for 'Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledge' by A. Shah, K. Shanmugam, K. Ahuja

Source code for "Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledge" Reference: Abhin Shah, Karthikeyan Shanmugam, Kartik Ahu

Abhin Shah 1 Jun 03, 2022
Red Team tool for exfiltrating files from a target's Google Drive that you have access to, via Google's API.

GD-Thief Red Team tool for exfiltrating files from a target's Google Drive that you(the attacker) has access to, via the Google Drive API. This includ

Antonio Piazza 39 Dec 27, 2022
Physics-informed convolutional-recurrent neural networks for solving spatiotemporal PDEs

PhyCRNet Physics-informed convolutional-recurrent neural networks for solving spatiotemporal PDEs Paper link: [ArXiv] By: Pu Ren, Chengping Rao, Yang

Pu Ren 11 Aug 23, 2022
[IROS2021] NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences

NYU-VPR This repository provides the experiment code for the paper Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymiza

Automation and Intelligence for Civil Engineering (AI4CE) Lab @ NYU 22 Sep 28, 2022
Labels4Free: Unsupervised Segmentation using StyleGAN

Labels4Free: Unsupervised Segmentation using StyleGAN ICCV 2021 Figure: Some segmentation masks predicted by Labels4Free Framework on real and synthet

70 Dec 23, 2022
Spontaneous Facial Micro Expression Recognition using 3D Spatio-Temporal Convolutional Neural Networks

Spontaneous Facial Micro Expression Recognition using 3D Spatio-Temporal Convolutional Neural Networks Abstract Facial expression recognition in video

Bogireddy Sai Prasanna Teja Reddy 103 Dec 29, 2022
Portfolio Optimization and Quantitative Strategic Asset Allocation in Python

Riskfolio-Lib Quantitative Strategic Asset Allocation, Easy for Everyone. Description Riskfolio-Lib is a library for making quantitative strategic ass

Riskfolio 1.7k Jan 07, 2023
a short visualisation script for pyvideo data

PyVideo Speakers A CLI that visualises repeat speakers from events listed in https://github.com/pyvideo/data Not terribly efficient, but you know. Ins

Katie McLaughlin 3 Nov 24, 2021
Analyzes your GitHub Profile and presents you with a report on how likely you are to become the next MLH Fellow!

Fellowship Prediction GitHub Profile Comparative Analysis Tool Built with BentoML Table of Contents: Features Disclaimer Technologies Used Contributin

Damir Temir 51 Dec 29, 2022
Toward Multimodal Image-to-Image Translation

BicycleGAN Project Page | Paper | Video Pytorch implementation for multimodal image-to-image translation. For example, given the same night image, our

Jun-Yan Zhu 1.4k Dec 22, 2022
Implementation of ICCV 2021 oral paper -- A Novel Self-Supervised Learning for Gaussian Mixture Model

SS-GMM Implementation of ICCV 2021 oral paper -- Self-Supervised Image Prior Learning with GMM from a Single Noisy Image with supplementary material R

HUST-The Tan Lab 4 Dec 05, 2022
Python library for loading and using triangular meshes.

Trimesh is a pure Python (2.7-3.4+) library for loading and using triangular meshes with an emphasis on watertight surfaces. The goal of the library i

Michael Dawson-Haggerty 2.2k Jan 07, 2023
Code for paper "Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs"

This is the codebase for the paper: Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs Directory Structur

Peter Hase 19 Aug 21, 2022
Use .csv files to record, play and evaluate motion capture data.

Purpose These scripts allow you to record mocap data to, and play from .csv files. This approach facilitates parsing of body movement data in statisti

21 Dec 12, 2022
Mengzi Pretrained Models

中文 | English Mengzi 尽管预训练语言模型在 NLP 的各个领域里得到了广泛的应用,但是其高昂的时间和算力成本依然是一个亟需解决的问题。这要求我们在一定的算力约束下,研发出各项指标更优的模型。 我们的目标不是追求更大的模型规模,而是轻量级但更强大,同时对部署和工业落地更友好的模型。

Langboat 424 Jan 04, 2023
Implementation of GGB color space

GGB Color Space This package is implementation of GGB color space from Development of a Robust Algorithm for Detection of Nuclei and Classification of

Resha Dwika Hefni Al-Fahsi 2 Oct 06, 2021
A pre-trained language model for social media text in Spanish

RoBERTuito A pre-trained language model for social media text in Spanish READ THE FULL PAPER Github Repository RoBERTuito is a pre-trained language mo

25 Dec 29, 2022
Paddle-Skeleton-Based-Action-Recognition - DecoupleGCN-DropGraph, ASGCN, AGCN, STGCN

Paddle-Skeleton-Action-Recognition DecoupleGCN-DropGraph, ASGCN, AGCN, STGCN. Yo

Chenxu Peng 3 Nov 02, 2022