banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Last update: Dec 22, 2022

Overview

What's banditml?

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Specifically, this repo contains:

Feature engineering & preprocessing
Model implementations
Model training workflows
Model serving code for Python services

Supported models

Models supported:

Contextual Bandits (small datasets)
- Linear bandit w/ ε-greedy exploration
- Random forest bandit w/ ε-greedy exploration
- Gradient boosted decision tree bandit w/ ε-greedy exploration
Contextual Bandits (medium datasets)
- Neural bandit with ε-greedy exploration
- Neural bandit with UCB-based exploration (via. dropout exploration)
- Neural bandit with UCB-based exploration (via. mixture density networks)
Reinforcement Learning (large datasets)

4 feature types supported:

Numeric: standard floating point features
- e.g. {totalCartValue: 39.99}
Categorical: low-cardinality discrete features
- e.g. {currentlyViewingCategory: "men's jeans"}
ID list: high-cardinality discrete features
- e.g. {productsInCart: ["productId022", "productId109"...]}
- Handled via. learned embedding tables
"Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
- e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

pip install banditml

Get started

License

GNU General Public License v3.0 or later

See COPYING to see the full text.

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information by Masato Tamura, Hiroki Ohashi, and Tomoaki Yosh

105 Dec 23, 2022

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

48 Jan 2, 2023

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022

Generate Contextual Directory Wordlist For Target Org

PathPermutor Generate Contextual Directory Wordlist For Target Org This script generates contextual wordlist for any target org based on the set of UR

8 Jun 23, 2021

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

Introduction The official repository for "Mining Contextual Information Beyond Image for Semantic Segmentation". Our full code has been merged into ss

55 Nov 9, 2022

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

23 Dec 26, 2022

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

CSA: Contextual Similarity Aggregation with Self-attention for Visual Re-ranking PyTorch training code for CSA (Contextual Similarity Aggregation). We

19 Oct 21, 2022

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 3, 2022

Code and data for ImageCoDe, a contextual vison-and-language benchmark

ImageCoDe This repository contains code and data for ImageCoDe: Image Retrieval from Contextual Descriptions. Data All collected descriptions for the

27 Dec 2, 2022

Comments

Adapting ABTest data to contextual bandit setting

Hi, and thanks for open sourcing this project.

I wanted to dive into it by testing some ABTesting data with the implemented neural bandit.

In my setting I have only 2 choices, 121 features as context, a reward range of [0.0, 120], and only 11% rows have non-zero reward. After training for a few epoch I see the testing loss decreasing a bit. But at test time, scores of the two choices are always equals, and the ucb_scores always equal to 0.

opened by virgile-blg 0
Model input dimension does not update when keeping top n features

Setting : Neural Bandit

When setting keep_only_top_n to True, the model keeps the original number of features, resulting in a Pytorch matmul error for the first linear layer:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (256x10 and 121x64)

opened by virgile-blg 0

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Related tags

Overview

What's banditml?

Supported models

Docs

License

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

Generate Contextual Directory Wordlist For Target Org

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Code and data for ImageCoDe, a contextual vison-and-language benchmark

Comments

Adapting ABTest data to contextual bandit setting

Model input dimension does not update when keeping top n features

Releases(1.0.2)

1.0.2(Jun 4, 2021)

Owner

Bandit ML

Single Red Blood Cell Hydrodynamic Traps Via the Generative Design

A data-driven maritime port simulator

Pocsploit is a lightweight, flexible and novel open source poc verification framework

Galactic and gravitational dynamics in Python

Measuring and Improving Consistency in Pretrained Language Models

Release of the ConditionalQA dataset

PFLD pytorch Implementation

code for our paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

Repository for reproducing `Model-Based Robust Deep Learning`

Official Repo for Ground-aware Monocular 3D Object Detection for Autonomous Driving

Semantically Contrastive Learning for Low-light Image Enhancement

Complex Answer Generation For Conversational Search Systems.

Next-Best-View Estimation based on Deep Reinforcement Learning for Active Object Classification

Source code for EquiDock: Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking (ICLR 2022)

Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)

Diverse Object-Scene Compositions For Zero-Shot Action Recognition

🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

Powerful unsupervised domain adaptation method for dense retrieval.