This project hosts the code for implementing the ISAL algorithm for object detection and image classification

Related tags

Deep LearningISAL
Overview

Influence Selection for Active Learning (ISAL)

This project hosts the code for implementing the ISAL algorithm for object detection and image classification, as presented in our paper:

Influence Selection for Active Learning;
Zhuoming Liu, Hao Ding, Huaping Zhong, Weijia Li, Jifeng Dai, Conghui He;
In: Proc. Int. Conf. Computer Vision (ICCV), 2021.
arXiv preprint arXiv:2108.09331

The full paper is available at: https://arxiv.org/abs/2108.09331.

Implementation based on MMDetection is included in MMDetection.

Highlights

  • Task agnostic: We evaluate ISAL in both object detection and image classification. Compared with previous methods, ISAL decreases the annotation cost at least by 12%, 12%, 3%, 13% and 16% on CIFAR10, SVHN, CIFAR100, VOC2012 and COCO, respectively.

  • Model agnostic: We evaluate ISAL with different model in object detection. On COCO dataset, with one-stage anchor-free detector FCOS, ISAL decreases the annotation cost at least by 16%. With two-stage anchor-based detector Faster R-CNN, ISAL decreases the annotation cost at least by 10%.

ISAL just needs to use the model gradients, which can be easily obtained in a neural network no matter what task is and how complex the model structure is, our proposed ISAL is task-agnostic and model-agnostic.

Required hardware

We use 4 NVIDIA V100 GPUs for object detection. We use 1 NVIDIA TITAN Xp GPUs for image classification.

Installation

Our ISAL implementation for object detection is based on mmdetection v2.4.0 with mmcv v1.1.1. Their need Pytorch version = 1.5, CUDA version = 10.1, CUDNN version = 7. We provide a docker file (./detection/Dockerfile) to prepare the environment. Once the environment is prepared, please copy all the files under the folder ./detection into the directory /mmdetection in the docker.

Our ISAL implementation for image classification is based on pycls v0.1. It need Pytorch version = 1.6, CUDA version = 10.1, CUDNN version = 7.

Training

The following command line will perform the ISAL algorithm with FCOS detector on COCO dataset, the active learning algorithm will iterate 20 steps with 4 GPUS:

bash dist_run_isal.sh /workdir /datadir \
    /mmdetection/configs/mining_experiments/ \
    fcos/fcos_r50_caffe_fpn_1x_coco_influence_function.py \
    --mining-method=influence --seed=42 --deterministic \
    --noised-score-thresh=0.1

Note that:

  1. If you want to use fewer GPUs, please change GPUS in shell script. In addition, you may need to change the samples_per_gpu in the config file to mantain the total batch size is equal to 8.
  2. The models and all inference results will be saved into /workdir.
  3. The data should be place in /datadir.
  4. If you want to run our code on VOC or your own dataset, we suggest that you should change the data format into COCO format.
  5. If you want to change the active learning iteration steps, please change the TRAIN_STEP in shell script. If you want to change the image selected by step_0 or the following steps, please change the INIT_IMG_NUM or IMG_NUM in shell script, respectively.
  6. The shell script will delete all the trained models after all the active learning steps. If you want to maintain the models please change the DELETE_MODEL in shell script.

The following command line will perform the ISAL algorithm with ResNet-18 on CIFAR10 dataset, the active learning algorithm will iterate 10 steps with 1 GPU:

bash run_isal.sh /workdir /datadir \
    pycls/configs/archive/cifar/resnet/R-18_nds_1gpu_cifar10.yaml \
    --mining-method=influence --random-seed=0

Note that:

  1. The models and all inference results will be saved into /workdir.
  2. The data should be place in /datadir.
  3. If you want to train SHVN or your own dataset, we suggest that you should change the data format into CIFAR10 format.
  4. The STEP in shell script indicates that in each active learning step the algorithm will add (1/STEP)% of the whole dataset into labeled dataset. The TRAIN_STEP indicates the total steps of active learning algorithm.

Citations

Please consider citing our paper in your publications if the project helps your research. BibTeX reference is as follows.

@inproceedings{liu2021influence,
  title={Influence selection for active learning},
  author={Liu, Zhuoming and Ding, Hao and Zhong, Huaping and Li, Weijia and Dai, Jifeng and He, Conghui},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={9274--9283},
  year={2021}
}

Acknowledgments

We thank Zheng Zhu for implementing the classification pipeline. We thank Bin Wang and Xizhou Zhu for discussion and helping with the experiments. We thank Yuan Tian and Jiamin He for discussing the mathematic derivation.

License

For academic use only. For commercial use, please contact the authors.

PyTorch implementation of saliency map-aided GAN for Auto-demosaic+denosing

Saiency Map-aided GAN for RAW2RGB Mapping The PyTorch implementations and guideline for Saiency Map-aided GAN for RAW2RGB Mapping. 1 Implementations B

Yuzhi ZHAO 20 Oct 24, 2022
Laser device for neutralizing - mosquitoes, weeds and pests

Laser device for neutralizing - mosquitoes, weeds and pests (in progress) Here I will post information for creating a laser device. A warning!! How It

Ildaron 1k Jan 02, 2023
deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

63 Oct 17, 2022
Official implementation of the paper Label-Efficient Semantic Segmentation with Diffusion Models

Label-Efficient Semantic Segmentation with Diffusion Models Official implementation of the paper Label-Efficient Semantic Segmentation with Diffusion

Yandex Research 355 Jan 06, 2023
Kindle is an easy model build package for PyTorch.

Kindle is an easy model build package for PyTorch. Building a deep learning model became so simple that almost all model can be made by copy and paste from other existing model codes. So why code? wh

Jongkuk Lim 77 Nov 11, 2022
Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding (AAAI 2020) - PyTorch Implementation

Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding PyTorch implementation for the Scalable Attentive Sentence-Pair Modeling vi

Microsoft 25 Dec 02, 2022
Code for Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (CVPR 2021)

Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (CVPR 2021) Hang Zhou, Yasheng Sun, Wayne Wu, Chen Cha

Hang_Zhou 628 Dec 28, 2022
Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.

Machine Learning From Scratch About Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The purpose

Erik Linder-Norén 21.8k Jan 09, 2023
Behind the Curtain: Learning Occluded Shapes for 3D Object Detection

Behind the Curtain: Learning Occluded Shapes for 3D Object Detection Acknowledgement We implement our model, BtcDet, based on [OpenPcdet 0.3.0]. Insta

Qiangeng Xu 163 Dec 19, 2022
A framework for annotating 3D meshes using the predictions of a 2D semantic segmentation model.

Semantic Meshes A framework for annotating 3D meshes using the predictions of a 2D semantic segmentation model. Paper If you find this framework usefu

Florian 40 Dec 09, 2022
This is the official implement of paper "ActionCLIP: A New Paradigm for Action Recognition"

This is an official pytorch implementation of ActionCLIP: A New Paradigm for Video Action Recognition [arXiv] Overview Content Prerequisites Data Prep

268 Jan 09, 2023
This is the official implementation for the paper "(Almost) Free Incentivized Exploration from Decentralized Learning Agents" in NeurIPS 2021.

Observe then Incentivize Experiments This is the code used for the paper "(Almost) Free Incentivized Exploration from Decentralized Learning Agents",

Cong Shen Research Group 0 Mar 08, 2022
Revisiting Temporal Alignment for Video Restoration

Revisiting Temporal Alignment for Video Restoration [arXiv] Kun Zhou, Wenbo Li, Liying Lu, Xiaoguang Han, Jiangbo Lu We provide our results at Google

52 Dec 25, 2022
A cool little repl-based simulation written in Python

A cool little repl-based simulation written in Python planned to integrate machine-learning into itself to have AI battle to the death before your eye

Em 6 Sep 17, 2022
SNIPS: Solving Noisy Inverse Problems Stochastically

SNIPS: Solving Noisy Inverse Problems Stochastically This repo contains the official implementation for the paper SNIPS: Solving Noisy Inverse Problem

Bahjat Kawar 35 Nov 09, 2022
Bilinear attention networks for visual question answering

Bilinear Attention Networks This repository is the implementation of Bilinear Attention Networks for the visual question answering and Flickr30k Entit

Jin-Hwa Kim 506 Nov 29, 2022
Automated Evidence Collection for Fake News Detection

Automated Evidence Collection for Fake News Detection This is the code repo for the Automated Evidence Collection for Fake News Detection paper accept

Mrinal Rawat 2 Apr 12, 2022
Generating Band-Limited Adversarial Surfaces Using Neural Networks

Generating Band-Limited Adversarial Surfaces Using Neural Networks This is the official repository of the technical report that was published on arXiv

3 Jul 26, 2022
CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer

CycleTransGAN-EVC CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer Demo emotion CycleTransGAN CycleTransGAN Cycle

24 Dec 15, 2022
Official code release for: EditGAN: High-Precision Semantic Image Editing

Official code release for: EditGAN: High-Precision Semantic Image Editing

565 Jan 05, 2023