On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

SIMD-accelerated bitwise hamming distance Python module for hexidecimal strings

PyHarmonize: Adding harmony lines to recorded melodies in Python

Avocado hass time series vs predict price

Decision tree is the most powerful and popular tool for classification and prediction

A comprehensive repository containing 30+ notebooks on learning machine learning!

This repository demonstrates the usage of hover to understand and supervise a machine learning task.

AtsPy: Automated Time Series Models in Python (by @firmai)

Code Repository for Machine Learning with PyTorch and Scikit-Learn

Model factory is a ML training platform to help engineers to build ML models at scale

Spark development environment for k8s

Class-imbalanced / Long-tailed ensemble learning in Python. Modular, flexible, and extensible

#30DaysOfStreamlit is a 30-day social challenge for you to build and deploy Streamlit apps.

This project impelemented for midterm of the Machine Learning #Zoomcamp #Alexey Grigorev

MiniTorch - a diy teaching library for machine learning engineers

💀mummify: a version control tool for machine learning

A repository for collating all the resources such as articles, blogs, papers, and books related to Bayesian Statistics.

Forecast dynamically at scale with this unique package. pip install scalecast

A unified framework for machine learning with time series

DistML is a Ray extension library to support large-scale distributed ML training on heterogeneous multi-node multi-GPU clusters

Generate music from midi files using BPE and markov model