Cl datasets - PyTorch image dataloaders and utility functions to load datasets for supervised continual learning

Overview

Continual learning datasets

Introduction

This repository contains PyTorch image dataloaders and utility functions to load datasets for supervised continual learning. Currently supported datasets:

  • MNIST
  • Pairwise-MNIST
  • Fashion-MNIST
  • not-MNIST (letters version of MNIST, see EMNIST for more detail)
  • CIFAR-10
  • CIFAR-100
  • German Traffic Signs
  • Street View House Numbers (SVHN)
  • Incremental CIFAR-100
  • Incremental TinyImageNet

Features

The provided interface simplifies typical data loading for supervised continual learning scenarios.

  • Dataset order, additional training data (for replay buffers) and test data (for global metrics computation) can all be specified.

  • A batch balancing feature is also available to make sure data from all available classes are available in a training batch.

  • Training data size and channels can be specified. Transformations will be added to make sure input data always has the same size and number of channels. If a single channel is specified, grayscaling will be applied. Otherwise, if 3 channels are specified, single channels will be triplicated. Bicubic interpolation or linear subsampling will be applied to meet the specified size.

Installation

  1. Clone the repository to your machine.
  2. Install the package:
pip install -e cl_datasets/

Note: Please use Python 3.8 or above.

Example

from cl_datasets import getDatasets

datasets = ['svhn','cifar10','fashion','mnist']
batchSize = 32
dataSize = (32,32)
nChannels = 3

dataloaders = getDatasets(datasets,batchSize,dataSize,nChannels)

for train_test_loaders in dataloaders:
    trainLoader,testLoader = train_test_loaders
    ...

List of possible datasets for training tasks

Full datasets

Description Dataset string
MNIST "mnist" or "MNIST"
not-MNIST "notMnist" or "notMNIST"
Fashion MNIST "fashion"
SVHN "svhn"
Cifar-10 "cifar10"
Cifar-100 "cifar100"
German traffic signs "traffic"

Incremental datasets

Description Dataset string
Pairwise MNIST "mnist_xy" (e.g. "mnist_01")
Incremental Cifar-100 (10 classes per task) "cifar100_i" (e.g. "cifar100_4")
Incremental Tiny ImageNet (10 classes per task) "TIN_i" (e.g. "TIN_3")
Owner
berjaoui
Senior Research Engineer
berjaoui
Use stochastic processes to generate samples and use them to train a fully-connected neural network based on Keras

Use stochastic processes to generate samples and use them to train a fully-connected neural network based on Keras which will then be used to generate residuals

Federico Lopez 2 Jan 14, 2022
Project NII pytorch scripts

project-NII-pytorch-scripts By Xin Wang, National Institute of Informatics, since 2021 I am a new pytorch user. If you have any suggestions or questio

Yamagishi and Echizen Laboratories, National Institute of Informatics 184 Dec 23, 2022
Meli Data Challenge 2021 - First Place Solution

My solution for the Meli Data Challenge 2021

Matias Moreyra 23 Mar 09, 2022
Towhee is a flexible machine learning framework currently focused on computing deep learning embeddings over unstructured data.

Towhee is a flexible machine learning framework currently focused on computing deep learning embeddings over unstructured data.

1.7k Jan 08, 2023
A PyTorch re-implementation of Neural Radiance Fields

nerf-pytorch A PyTorch re-implementation Project | Video | Paper NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis Ben Mildenhall

Krishna Murthy 709 Jan 09, 2023
Tensorflow/Keras Plug-N-Play Deep Learning Models Compilation

DeepBay This project was created with the objective of compile Machine Learning Architectures created using Tensorflow or Keras. The architectures mus

Whitman Bohorquez 4 Sep 26, 2022
Demo notebooks for Qiskit application modules demo sessions (Oct 8 & 15):

qiskit-application-modules-demo-sessions This repo hosts demo notebooks for the Qiskit application modules demo sessions hosted on Qiskit YouTube. Par

Qiskit Community 46 Nov 24, 2022
Project page for the paper Semi-Supervised Raw-to-Raw Mapping 2021.

Project page for the paper Semi-Supervised Raw-to-Raw Mapping 2021.

Mahmoud Afifi 22 Nov 08, 2022
Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

ARAPReg Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators.. Installation The cod

Bo Sun 132 Nov 28, 2022
Tensorflow 2.x based implementation of EDSR, WDSR and SRGAN for single image super-resolution

Single Image Super-Resolution with EDSR, WDSR and SRGAN A Tensorflow 2.x based implementation of Enhanced Deep Residual Networks for Single Image Supe

Martin Krasser 1.3k Jan 06, 2023
This repository is an implementation of paper : Improving the Training of Graph Neural Networks with Consistency Regularization

CRGNN Paper : Improving the Training of Graph Neural Networks with Consistency Regularization Environments Implementing environment: GeForce RTX™ 3090

THUDM 28 Dec 09, 2022
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

ELECTRA Introduction ELECTRA is a method for self-supervised language representation learning. It can be used to pre-train transformer networks using

Google Research 2.1k Dec 28, 2022
To provide 100 JAX exercises over different sections structured as a course or tutorials to teach and learn for beginners, intermediates as well as experts

JaxTon 💯 JAX exercises Mission 🚀 To provide 100 JAX exercises over different sections structured as a course or tutorials to teach and learn for beg

Rohan Rao 512 Jan 01, 2023
Final project code: Implementing MAE with downscaled encoders and datasets, for ESE546 FA21 at University of Pennsylvania

546 Final Project: Masked Autoencoder Haoran Tang, Qirui Wu 1. Training To train the network, please run mae_pretraining.py. Please modify folder path

Haoran Tang 0 Apr 22, 2022
Deep Reinforcement Learning with pytorch & visdom

Deep Reinforcement Learning with pytorch & visdom Sample testings of trained agents (DQN on Breakout, A3C on Pong, DoubleDQN on CartPole, continuous A

Jingwei Zhang 783 Jan 04, 2023
PyTorch implementation of the paper: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features

Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features Estimate the noise transition matrix with f-mutual information. This co

<a href=[email protected]"> 1 Jun 05, 2022
A Kitti Road Segmentation model implemented in tensorflow.

KittiSeg KittiSeg performs segmentation of roads by utilizing an FCN based model. The model achieved first place on the Kitti Road Detection Benchmark

Marvin Teichmann 890 Jan 04, 2023
Deep learning with dynamic computation graphs in TensorFlow

TensorFlow Fold TensorFlow Fold is a library for creating TensorFlow models that consume structured data, where the structure of the computation graph

1.8k Dec 28, 2022
Object detection evaluation metrics using Python.

Object detection evaluation metrics using Python.

Louis Facun 2 Sep 06, 2022
Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

Intelligent Vision for Robotics in Complex Environment 12 Dec 18, 2022