On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

A collection of video resources for machine learning

Warren - Stock Price Predictor

Learn how to responsibly deliver value with ML.

Open-Source CI/CD platform for ML teams. Deliver ML products, better & faster. ⚡️🧑‍🔧

This is a public repo where code samples are stored for the book Practical MLOps.

Scikit-learn compatible wrapper of the Random Bits Forest program written by (Wang et al., 2016)

Responsible AI Workshop: a series of tutorials & walkthroughs to illustrate how put responsible AI into practice

Model Validation Toolkit is a collection of tools to assist with validating machine learning models prior to deploying them to production and monitoring them after deployment to production.

A linear equation solver using gaussian elimination. Implemented for fun and learning/teaching.

GRaNDPapA: Generator of Rad Names from Decent Paper Acronyms

A high-performance topological machine learning toolbox in Python

Built on python (Mathematical straight fit line coordinates error predictor machine learning foundational model)

Microsoft contributing libraries, tools, recipes, sample codes and workshop contents for machine learning & deep learning.

Distributed Deep learning with Keras & Spark

Predict the income for each percentile of the population (Python) - FRENCH

Applied Machine Learning for Graduate Program in Computer Science (PPGCC)

Esse é o meu primeiro repo tratando de fim a fim, uma pipeline de dados abertos do governo brasileiro relacionado a compras de contrato e cronogramas anuais com spark, em pyspark e SQL!

Decision Weights in Prospect Theory

Bayesian optimization based on Gaussian processes (BO-GP) for CFD simulations.

moDel Agnostic Language for Exploration and eXplanation