NALSM: Neuron-Astrocyte Liquid State Machine

Related tags

Deep LearningNALSM
Overview

NALSM: Neuron-Astrocyte Liquid State Machine

This package is a Tensorflow implementation of the Neuron-Astrocyte Liquid State Machine (NALSM) that introduces astrocyte-modulated STDP to the Liquid State Machine learning framework for improved accuracy performance and minimal tuning.

The paper has been accepted at NeurIPS 2021, available here.

Citation

Vladimir A. Ivanov and Konstantinos P. Michmizos. "Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity." 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

@inproceedings{ivanov_2021,
author = {Ivanov, Vladimir A. and Michmizos, Konstantinos P.},
title = {Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity},
year = {2021},
pages={1--10},
booktitle = {35th Conference on Neural Information Processing Systems (NeurIPS 2021)}
}

Software Installation

  • Python 3.6.9
  • Tensorflow 2.1 (with CUDA 11.2 using tensorflow.compat.v1)
  • Numpy
  • Multiprocessing

Usage

This code performs the following functions:

  1. Generate the 3D network
  2. Train NALSM
  3. Evaluate trained model accuracy
  4. Evaluate trained model branching factor
  5. Evaluate model kernel quality

Instructions for obtaining/setting up datasets can be accessed here.

Overview of all files can be accessed here.

1. Generate 3D Network

To generate the 3D network, enter the following command:

python generate_spatial_network.py

This will prompt for following inputs:

  • WHICH_DATASET_TO_GENERATE_NETWORK_FOR? [TYPE M FOR MNIST/ N FOR NMNIST] : enter M to make a network with an input layer sized for MNIST/Fashion-MNIST or N for N-MNIST.
  • NETWORK_NUMBER_TO_CREATE? [int] : enter an integer to label the network.
  • SIZE_OF_LIQUID_DIMENSION_1? [int] : enter an integer representing the number of neurons to be in dimension 1 of liquid.
  • SIZE_OF_LIQUID_DIMENSION_2? [int] : enter an integer representing the number of neurons to be in dimension 2 of liquid.
  • SIZE_OF_LIQUID_DIMENSION_3? [int] : enter an integer representing the number of neurons to be in dimension 3 of liquid.

The run file will generate the network and associated log file containing data about the liquid (i.e. connection densities) in sub-directory

/ /networks/ .

2. Train NALSM

2.1 MNIST

To train NALSM model on MNIST, enter the following command:

python NALSM_RUN_MAIN_SIM_MNIST.py

This will prompt for the following inputs:

  • GPU? : enter an integer specifying the gpu to use for training.
  • VERSION? [int] : enter an integer to label the training simulation.
  • NET_NUM_VAR? [int] : enter the number of the network created in Section 1.
  • BATCH_SIZE? [int] : specify the number of samples to train at same time (batch), for liquids with 1000 neurons, batch size of 250 will work on a 12gb gpu. For larger liquids(8000), smaller batch sizes of 50 should work.
  • BATCHS_PER_BLOCK? [int] : specify number of batchs to keep in memory for training output layer, we found 2500 samples works well in terms of speed and memory (so for batch size of 250, this should be set to 10 (10 x 250 = 2500), for batch size 50 set this to 50 (50 x 50 = 2500).
  • ASTRO_W_SCALING? [float] : specify the astrocyte weight detailed in equation 7 of paper. We used 0.015 for all 1000 neuron liquids, and 0.0075 for 8000 neuron liquids. Generally accuracy peaks with a value around 0.01 (See Appendix).

This will generate all output in sub-directory

/ /train_data/ver_XX/ where XX is VERSION number.

2.2 N-MNIST

To train NALSM model on N-MNIST, enter the following command:

python NALSM_RUN_MAIN_SIM_N_MNIST.py

All input prompts and output are the same as described above for run file NALSM_RUN_MAIN_SIM_MNIST.py.

2.3 Fashion-MNIST

To train NALSM model on Fashion-MNIST, enter the following command:

python NALSM_RUN_MAIN_SIM_F_MNIST.py

All input prompts and output are the same as described above for run file NALSM_RUN_MAIN_SIM_MNIST.py.

Instructions for training other benchmarked LSM models can be accessed here.

3. Evaluate Trained Model Accuracy

To get accuracy of a trained model, enter the following command:

python get_test_accuracy.py

The run file will prompt for following inputs:

  • VERSION? [int] : enter the version number of the trained model

This will find the epoch with maximum validation accuracy and return the test accuracy for that epoch.

4. Evaluate Model Branching Factor

To compute the branching factor of a trained model, enter the following command:

python compute_branching_factor.py

The run file will prompt for following inputs:

  • VERSION? [int] : enter the version number of the trained model.

The trained model directory must have atleast one .spikes file, which contains millisecond spike data of each neuron for 20 arbitrarily selected input samples in a batch. The run file will generate a .bf file with same name as the .spikes file.

To read the generated .bf file, enter the following command:

python get_branching_factor.py

The run file will prompt for following inputs:

  • VERSION? [int] : enter the version number of the trained model.

The run file will print the average branching factor over the 20 samples.

5. Evaluate Model Kernel Quality

Model liquid kernel quality was calculated from the linear speration (SP) and generalization (AP) metrics for MNIST and N-MNIST datasets. To compute SP and AP metrics, first noisy spike counts must be generated for the AP metric, as follows.

To generate noisy spike counts for NALSM model on MNIST, enter the following command:

python NALSM_RUN_MAIN_SIM_MNIST_NOISE.py

The run file requires a W_INI.wdata file (the initialized weights), which should have been generated during model training.

The run file will prompt for the following inputs:

  • GPU? : enter an integer to select the gpu for the training simulation.
  • VERSION? [int] : enter the version number of the trained model.
  • NET_NUM_VAR? [int] : enter the network number of the trained model.
  • BATCH_SIZE? [int] : use the same value used for training the model.
  • BATCHS_PER_BLOCK? [int] : use the same value used for training the model.

The run file will generate all output in sub-directory

/ /train_data/ver_XX/ where XX is VERSION number.

To generate noisy spike counts for NALSM model on N-MNIST, enter the following command:

python NALSM_RUN_MAIN_SIM_N_MNIST_NOISE.py

As above, the run file requires 'W_INI.wdata' file. All input prompts and output are the same as described above for run file NALSM_RUN_MAIN_SIM_MNIST_NOISE.py.

After generating the noisy spike counts, to compute the SP and AP metrics for each trained model enter the following command:

python compute_SP_AP_kernel_quality_measures.py

The run file will prompt for inputs:

  • VERSION? [int] : enter the version number of the trained model.
  • DATASET_MODEL_WAS_TRAINED_ON? [TYPE M FOR MNIST/ N FOR NMNIST] : enter dataset the model was trained on. The run file will print out the SP and AP metrics.

Instructions for evaluating kernel quality for other benchmarked LSM models can be accessed here.

Owner
Computational Brain Lab
Computational Brain Lab @ Rutgers University
Computational Brain Lab
StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking Datasets You can download datasets that have been pre-pr

25 May 29, 2022
Official PyTorch implementation of "Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition" in AAAI2022.

AimCLR This is an official PyTorch implementation of "Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Reco

Gty 44 Dec 17, 2022
DiAne is a smart fuzzer for IoT devices

Diane Diane is a fuzzer for IoT devices. Diane works by identifying fuzzing triggers in the IoT companion apps to produce valid yet under-constrained

seclab 28 Jan 04, 2023
The code for 'Deep Residual Fourier Transformation for Single Image Deblurring'

Deep Residual Fourier Transformation for Single Image Deblurring Xintian Mao, Yiming Liu, Wei Shen, Qingli Li and Yan Wang code will be released soon

145 Dec 13, 2022
GenshinMapAutoMarkTools - Tools To add/delete/refresh resources mark in Genshin Impact Map

使用说明 适配 windows7以上 64位 原神1920x1080窗口(其他分辨率后续适配) 待更新渊下宫 English version is to be

Zero_Circle 209 Dec 28, 2022
Companion code for the paper "Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks" by Yatsura et al.

META-RS This is the companion code for the paper "Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks" by Yatsu

Bosch Research 7 Dec 09, 2022
Lazy, a tool for running things in idle time

Lazy, a tool for running things in idle time Mostly used to stop CUDA ML model training from making my desktop unusable. Simply monitors keyboard/mous

N Shepperd 46 Nov 06, 2022
[CVPR 2021] NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning

NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning Project Page | Paper | Supplemental material #1 | Supplement

KAIST VCLAB 49 Nov 24, 2022
[CVPR 2021] Region-aware Adaptive Instance Normalization for Image Harmonization

RainNet — Official Pytorch Implementation Region-aware Adaptive Instance Normalization for Image Harmonization Jun Ling, Han Xue, Li Song*, Rong Xie,

130 Dec 11, 2022
Ontologysim: a Owlready2 library for applied production simulation

Ontologysim: a Owlready2 library for applied production simulation Ontologysim is an open-source deep production simulation framework, with an emphasi

10 Nov 30, 2022
IOT: Instance-wise Layer Reordering for Transformer Structures

Introduction This repository contains the code for Instance-wise Ordered Transformer (IOT), which is introduced in the ICLR2021 paper IOT: Instance-wi

IOT 19 Nov 15, 2022
In this project we combine techniques from neural voice cloning and musical instrument synthesis to achieve good results from as little as 16 seconds of target data.

Neural Instrument Cloning In this project we combine techniques from neural voice cloning and musical instrument synthesis to achieve good results fro

Erland 127 Dec 23, 2022
Neural Koopman Lyapunov Control

Neural-Koopman-Lyapunov-Control Code for our paper: Neural Koopman Lyapunov Control Requirements dReal4: v4.19.02.1 PyTorch: 1.2.0 The learning framew

Vrushabh Zinage 6 Dec 24, 2022
This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

Differentiable Volumetric Rendering Paper | Supplementary | Spotlight Video | Blog Entry | Presentation | Interactive Slides | Project Page This repos

697 Jan 06, 2023
Energy consumption estimation utilities for Jetson-based platforms

This repository contains a utility for measuring energy consumption when running various programs in NVIDIA Jetson-based platforms. Currently TX-2, NX, and AGX are supported.

OpenDR 10 Jun 17, 2022
Stitch it in Time: GAN-Based Facial Editing of Real Videos

STIT - Stitch it in Time [Project Page] Stitch it in Time: GAN-Based Facial Edit

1.1k Jan 04, 2023
Template repository to build PyTorch projects from source on any version of PyTorch/CUDA/cuDNN.

The Ultimate PyTorch Source-Build Template Translations: 한국어 TL;DR PyTorch built from source can be x4 faster than a naïve PyTorch install. This repos

Joonhyung Lee/이준형 651 Dec 12, 2022
Offical implementation for "Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation".

Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation (NeurIPS 2021) by Qiming Hu, Xiaojie Guo. Dependencies P

Qiming Hu 31 Dec 20, 2022
2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation Authors: Ge-Peng Ji*, Yu-Cheng Chou*, Deng-Ping Fan, Geng Che

Ge-Peng Ji (Daniel) 85 Dec 30, 2022
ToFFi - Toolbox for Frequency-based Fingerprinting of Brain Signals

ToFFi Toolbox This repository contains "before peer review" version of the software related to the preprint of the publication ToFFi - Toolbox for Fre

4 Aug 31, 2022