Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

Last update: Dec 27, 2022

Related tags

Overview

SNN_Calibration

Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

Feature Comparison of SNN calibration:

Features	SNN Direct Training	ANN-SNN Conversion	SNN Calibration
Accuract (T<100)	High	Low	High
Scalability to ImageNet	Tiny	Large	Large
Training Speed	Slow	Fast	Fast
# Required Data	Full-set (1.2M For ImageNet)	~1000	~1000
Inference Speed	Fast	Slow	Fast

Requirements

Pytorch 1.8

For ImageNet experiments, please be sure that you can initialize distributed environments

For CIFAR experiments, one GPU would suffice.

Pre-training ANN on CIFAR10&100

Train an ANN model with main_train.py

python CIFAR/main_train.py --dataset CIFAR10 --arch VGG16 --dpath PATH/TO/DATA --usebn

Pre-trained results:

Dataset	Model	Random Seed	Accuracy
CIFAR10	VGG16	1000	95.76
CIFAR10	ResNet-20	1000	95.68
CIFAR100	VGG16	1000	77.98
CIFAR100	ResNet-20	1000	76.52

SNN Calibration on CIFAR10&100

Calibrate an SNN with main_calibration.py.

python CIFAR/main_calibration.py --dataset CIFAR10 --arch VGG16 --T 16 --usebn --calib advanced --dpath PATH/TO/DATA

--T is the time step, --calib is the calibration method, please use none, light, advanced for experiments.

The calibration will run for 5 times, and return the mean accuracy as well as the standard deviation.

Example results:

Architecture	Datset	T	Random Seed	Calibration	Mean Acc	Std.
VGG16	CIFAR10	16	1000	None	64.52	4.12
VGG16	CIFAR10	16	1000	Light	93.30	0.08
VGG16	CIFAR10	16	1000	Advanced	93.65	0.25
ResNet-20	CIFAR10	16	1000	None	67.88	3.63
ResNet-20	CIFAR10	16	1000	Light	93.89	0.20
ResNet-20	CIFAR10	16	1000	Advanced	94.33	0.12
VGG16	CIFAR100	16	1000	None	2.69	0.76
VGG16	CIFAR100	16	1000	Light	65.26	0.99
VGG16	CIFAR100	16	1000	Advanced	70.91	0.65
ResNet-20	CIFAR100	16	1000	None	39.27	2.85
ResNet-20	CIFAR100	16	1000	Light	73.89	0.15
ResNet-20	CIFAR100	16	1000	Advanced	74.48	0.16

Pre-training ANN on ImageNet

To be updaed

Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

Related tags

Overview

SNN_Calibration

Requirements

Pre-training ANN on CIFAR10&100

SNN Calibration on CIFAR10&100

Pre-training ANN on ImageNet

Owner

Yuhang Li

Code accompanying "Evolving spiking neuron cellular automata and networks to emulate in vitro neuronal activity," accepted to IEEE SSCI ICES 2021

[EMNLP 2021] MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

DeiT: Data-efficient Image Transformers

The official PyTorch code for NeurIPS 2021 ML4AD Paper, "Does Thermal data make the detection systems more reliable?"

This repository contains Prior-RObust Bayesian Optimization (PROBO) as introduced in our paper "Accounting for Gaussian Process Imprecision in Bayesian Optimization"

The official code repository for examples in the O'Reilly book 'Generative Deep Learning'

Implementation of DropLoss for Long-Tail Instance Segmentation in Pytorch

TensorFlow implementation of ENet, trained on the Cityscapes dataset.

converts nominal survey data into a numerical value based on a dictionary lookup.

A pure PyTorch batched computation implementation of "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition"

A basic duplicate image detection service using perceptual image hash functions and nearest neighbor search, implemented using faiss, fastapi, and imagehash

Recurrent Variational Autoencoder that generates sequential data implemented with pytorch

🔪 Elimination based Lightweight Neural Net with Pretrained Weights

Relaxed-machines - explorations in neuro-symbolic differentiable interpreters

PINN Burgers - 1D Burgers equation simulated by PINN

CIFAR-10_train-test - training and testing codes for dataset CIFAR-10

Exploring the link between uncertainty estimates obtained via "exact" Bayesian inference and out-of-distribution (OOD) detection.

Experiments for Neural Flows paper

This is code of book "Learn Deep Learning with PyTorch"

A diff tool for language models