On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

Gaussian Process Optimization using GPy

Xeasy-ml is a packaged machine learning framework.

Send rockets to Mars with artificial intelligence(Genetic algorithm) in python.

This project impelemented for midterm of the Machine Learning #Zoomcamp #Alexey Grigorev

Dieses Projekt ermöglicht es den Smartmeter der EVN (Netz Niederösterreich) über die Kundenschnittstelle auszulesen.

Lseng-iseng eksplor Machine Learning dengan menggunakan library Scikit-Learn

Machine learning that just works, for effortless production applications

Short PhD seminar on Machine Learning Security (Adversarial Machine Learning)

Relevance Vector Machine implementation using the scikit-learn API.

To-Be is a machine learning challenge on CodaLab Platform about Mortality Prediction

Evidently helps analyze machine learning models during validation or production monitoring

Reggy - Regressions with arbitrarily complex regularization terms

XGBoost + Optuna

scikit-learn models hyperparameters tuning and feature selection, using evolutionary algorithms.

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

Python based GBDT implementation

A simple and lightweight genetic algorithm for optimization of any machine learning model

Python factor analysis library (PCA, CA, MCA, MFA, FAMD)

CobraML: Completely Customizable A python ML library designed to give the end user full control

The code from the Machine Learning Bookcamp book and a free course based on the book