banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Last update: Dec 22, 2022

Overview

What's banditml?

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Specifically, this repo contains:

Feature engineering & preprocessing
Model implementations
Model training workflows
Model serving code for Python services

Supported models

Models supported:

Contextual Bandits (small datasets)
- Linear bandit w/ ε-greedy exploration
- Random forest bandit w/ ε-greedy exploration
- Gradient boosted decision tree bandit w/ ε-greedy exploration
Contextual Bandits (medium datasets)
- Neural bandit with ε-greedy exploration
- Neural bandit with UCB-based exploration (via. dropout exploration)
- Neural bandit with UCB-based exploration (via. mixture density networks)
Reinforcement Learning (large datasets)

4 feature types supported:

Numeric: standard floating point features
- e.g. {totalCartValue: 39.99}
Categorical: low-cardinality discrete features
- e.g. {currentlyViewingCategory: "men's jeans"}
ID list: high-cardinality discrete features
- e.g. {productsInCart: ["productId022", "productId109"...]}
- Handled via. learned embedding tables
"Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
- e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

pip install banditml

Get started

License

GNU General Public License v3.0 or later

See COPYING to see the full text.

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information by Masato Tamura, Hiroki Ohashi, and Tomoaki Yosh

105 Dec 23, 2022

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

48 Jan 2, 2023

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022

Generate Contextual Directory Wordlist For Target Org

PathPermutor Generate Contextual Directory Wordlist For Target Org This script generates contextual wordlist for any target org based on the set of UR

8 Jun 23, 2021

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

Introduction The official repository for "Mining Contextual Information Beyond Image for Semantic Segmentation". Our full code has been merged into ss

55 Nov 9, 2022

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

23 Dec 26, 2022

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

CSA: Contextual Similarity Aggregation with Self-attention for Visual Re-ranking PyTorch training code for CSA (Contextual Similarity Aggregation). We

19 Oct 21, 2022

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 3, 2022

Code and data for ImageCoDe, a contextual vison-and-language benchmark

ImageCoDe This repository contains code and data for ImageCoDe: Image Retrieval from Contextual Descriptions. Data All collected descriptions for the

27 Dec 2, 2022

Comments

Adapting ABTest data to contextual bandit setting

Hi, and thanks for open sourcing this project.

I wanted to dive into it by testing some ABTesting data with the implemented neural bandit.

In my setting I have only 2 choices, 121 features as context, a reward range of [0.0, 120], and only 11% rows have non-zero reward. After training for a few epoch I see the testing loss decreasing a bit. But at test time, scores of the two choices are always equals, and the ucb_scores always equal to 0.

opened by virgile-blg 0
Model input dimension does not update when keeping top n features

Setting : Neural Bandit

When setting keep_only_top_n to True, the model keeps the original number of features, resulting in a Pytorch matmul error for the first linear layer:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (256x10 and 121x64)

opened by virgile-blg 0

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Related tags

Overview

What's banditml?

Supported models

Docs

License

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

Generate Contextual Directory Wordlist For Target Org

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Code and data for ImageCoDe, a contextual vison-and-language benchmark

Comments

Adapting ABTest data to contextual bandit setting

Model input dimension does not update when keeping top n features

Releases(1.0.2)

1.0.2(Jun 4, 2021)

Owner

Bandit ML

Trains an agent with stochastic policy gradient ascent to solve the Lunar Lander challenge from OpenAI

Repo for Photon-Starved Scene Inference using Single Photon Cameras, ICCV 2021

[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

DFM: A Performance Baseline for Deep Feature Matching

Applying curriculum to meta-learning for few shot classification

A tensorflow/keras implementation of StyleGAN to generate images of new Pokemon.

Codebase for testing whether hidden states of neural networks encode discrete structures.

Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding

Code for the Paper "Diffusion Models for Handwriting Generation"

[ICCV 2021 Oral] Mining Latent Classes for Few-shot Segmentation

Python periodic table module

clDice - a Novel Topology-Preserving Loss Function for Tubular Structure Segmentation

DRLib：A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.

Pytorch implementation for "Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets" (ECCV 2020 Spotlight)

Differentiable Simulation of Soft Multi-body Systems

A PyTorch-Based Framework for Deep Learning in Computer Vision

Learned model to estimate number of distinct values (NDV) of a population using a small sample.

Python implementation of Project Fluent

Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.