Boosted CVaR Classification (NeurIPS 2021)

Overview

Boosted CVaR Classification

Runtian Zhai, Chen Dan, Arun Sai Suggala, Zico Kolter, Pradeep Ravikumar
NeurIPS 2021

Table of Contents

Quick Start

Before running the code, please install all the required packages in requirements.txt by running:

pip install -r requirements.txt

In the code, we solve linear programs with the MOSEK solver, which requires a license. You can acquire a free academic license from https://www.mosek.com/products/academic-licenses/. Please make sure that the license file is placed in the correct folder so that the solver could work.

Train

To train a set of base models with boosting, run the following shell command:

python train.py --dataset [DATASET] --data_root /path/to/dataset 
                --alg [ALGORITHM] --epochs [EPOCHS] --iters_per_epoch [ITERS]
                --scheduler [SCHEDULER] --warmup [WARMUP_EPOCHS] --seed [SEED]

Use the --download option to download the dataset if you are running for the first time. Use the --save_file option to save your training results into a .mat file. Set the training hyperparameters with --alpha, --beta and --eta.

For example, to train a set of base models on Cifar-10 with AdaLPBoost, use the following shell command:

python train.py --dataset cifar10 --data_root data --alg adalpboost 
                --eta 1.0 --epochs 100 --iters_per_epoch 5000
                --scheduler 2000,4000 --warmup 20 --seed 2021
                --save_file cifar10.mat

Evaluation

To evaluate the models trained with the above command, run:

python test.py --file cifar10.mat

Introduction

In this work, we study the CVaR classification problem, which requires a classifier to have low α-CVaR loss, i.e. low average loss over the worst α fraction of the samples in the dataset. While previous work showed that no deterministic model learning algorithm can achieve a lower α-CVaR loss than ERM, we address this issue by learning randomized models. Specifically we propose the Boosted CVaR Classification framework that learns ensemble models via Boosting. Our motivation comes from the direct relationship between the CVaR loss and the LPBoost objective. We implement two algorithms based on the framework: one uses LPBoost, and the other named AdaLPBoost uses AdaBoost to pick the sample weights and LPBoost to pick the model weights.

Algorithms

We implement three algorithms in algs.py:

Name Description
uniform All sample weight vectors are uniform distributions.
lpboost Regularized LPBoost (set --beta for regularization).
adalpboost α-AdaLPBoost.

train.py only trains the base models. After the base models are trained, use test.py to select the model weights by solving the dual LPBoost problem.

Parameters

All default training parameters can be found in config.py. For Regularized LPBoost we use β = 100 for all α. For AdaLPBoost we use η = 1.0.

Citation and Contact

To cite this work, please use the following BibTex entry:

@inproceedings{zhai2021boosted,
  author = {Zhai, Runtian and Dan, Chen and Suggala, Arun Sai and Kolter, Zico and Ravikumar, Pradeep},
  booktitle = {Advances in Neural Information Processing Systems},
  title = {Boosted CVaR Classification},
  volume = {34},
  year = {2021}
}

To contact us, please email to the following address: Runtian Zhai <[email protected]>

Owner
Runtian Zhai
2nd year PhD at CMU CSD.
Runtian Zhai
🤗 Paper Style Guide

🤗 Paper Style Guide (Work in progress, send a PR!) Libraries to Know booktabs natbib cleveref Either seaborn, plotly or altair for graphs algorithmic

Hugging Face 66 Dec 12, 2022
More than a hundred strange attractors

dysts Analyze more than a hundred chaotic systems. Basic Usage Import a model and run a simulation with default initial conditions and parameter value

William Gilpin 185 Dec 23, 2022
PyTorch implementation for our AAAI 2022 Paper "Graph-wise Common Latent Factor Extraction for Unsupervised Graph Representation Learning"

deepGCFX PyTorch implementation for our AAAI 2022 Paper "Graph-wise Common Latent Factor Extraction for Unsupervised Graph Representation Learning" Pr

Thilini Cooray 4 Aug 11, 2022
Multi-Task Deep Neural Networks for Natural Language Understanding

New Release We released Adversarial training for both LM pre-training/finetuning and f-divergence. Large-scale Adversarial training for LMs: ALUM code

Xiaodong 2.1k Dec 30, 2022
Software Platform for solving and manipulating multiparametric programs in Python

PPOPT Python Parametric OPtimization Toolbox (PPOPT) is a software platform for solving and manipulating multiparametric programs in Python. This pack

10 Sep 13, 2022
Source codes for "Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs"

Structure-Aware-BART This repo contains codes for the following paper: Jiaao Chen, Diyi Yang:Structure-Aware Abstractive Conversation Summarization vi

GT-SALT 56 Dec 08, 2022
FlingBot: The Unreasonable Effectiveness of Dynamic Manipulations for Cloth Unfolding

This repository contains code for training and evaluating FlingBot in both simulation and real-world settings on a dual-UR5 robot arm setup for Ubuntu 18.04

Columbia Artificial Intelligence and Robotics Lab 70 Dec 06, 2022
PyTorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision.

PyTorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision @misc{CV2018, author = {Donny You ( Donny You 40 Sep 14, 2022

Code for the paper: Sketch Your Own GAN

Sketch Your Own GAN Project | Paper | Youtube Our method takes in one or a few hand-drawn sketches and customizes an off-the-shelf GAN to match the in

677 Dec 28, 2022
Implementation of Barlow Twins paper

barlowtwins PyTorch Implementation of Barlow Twins paper: Barlow Twins: Self-Supervised Learning via Redundancy Reduction This is currently a work in

IgorSusmelj 86 Dec 20, 2022
3DV 2021: Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry

SynergyNet 3DV 2021: Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry Cho-Ying Wu, Qiangeng Xu, Ulrich Neumann, CGIT Lab at Unive

Cho-Ying Wu 239 Jan 06, 2023
Hierarchical User Intent Graph Network for Multimedia Recommendation

Hierarchical User Intent Graph Network for Multimedia Recommendation This is our Pytorch implementation for the paper: Hierarchical User Intent Graph

6 Jan 05, 2023
Hand gesture recognition model that can be used as a remote control for a smart tv.

Gesture_recognition The training data consists of a few hundred videos categorised into one of the five classes. Each video (typically 2-3 seconds lon

Pratyush Negi 1 Aug 11, 2022
5 Jan 05, 2023
nfelo: a power ranking, prediction, and betting model for the NFL

nfelo nfelo is a power ranking, prediction, and betting model for the NFL. Nfelo take's 538's Elo framework and further adapts it for the NFL, hence t

6 Nov 22, 2022
Official implementation for TTT++: When Does Self-supervised Test-time Training Fail or Thrive

TTT++ This is an official implementation for TTT++: When Does Self-supervised Test-time Training Fail or Thrive? TL;DR: Online Feature Alignment + Str

VITA lab at EPFL 39 Dec 25, 2022
Sionna: An Open-Source Library for Next-Generation Physical Layer Research

Sionna: An Open-Source Library for Next-Generation Physical Layer Research Sionna™ is an open-source Python library for link-level simulations of digi

NVIDIA Research Projects 313 Dec 22, 2022
Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data - Official PyTorch Implementation (CVPR 2022)

Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data (CVPR 2022) Potentials of primitive shapes f

31 Sep 27, 2022
Official implementation of Deep Convolutional Dictionary Learning for Image Denoising.

DCDicL for Image Denoising Hongyi Zheng*, Hongwei Yong*, Lei Zhang, "Deep Convolutional Dictionary Learning for Image Denoising," in CVPR 2021. (* Equ

Z80 91 Dec 21, 2022
Implementation of paper "Decision-based Black-box Attack Against Vision Transformers via Patch-wise Adversarial Removal"

Patch-wise Adversarial Removal Implementation of paper "Decision-based Black-box Attack Against Vision Transformers via Patch-wise Adversarial Removal

4 Oct 12, 2022