Multivariate Boosted TRee

Related tags

Deep Learningmbtr
Overview

Documentation Status Build Status codecov Latest Version License: MIT

Multivariate Boosted TRee

What is MBTR

MBTR is a python package for multivariate boosted tree regressors trained in parameter space. The package can handle arbitrary multivariate losses, as long as their gradient and Hessian are known. Gradient boosted trees are competition-winning, general-purpose, non-parametric regressors, which exploit sequential model fitting and gradient descent to minimize a specific loss function. The most popular implementations are tailored to univariate regression and classification tasks, precluding the possibility of capturing multivariate target cross-correlations and applying conditional penalties to the predictions. This package allows to arbitrarily regularize the predictions, so that properties like smoothness, consistency and functional relations can be enforced.

Installation

pip install --upgrade git+https://github.com/supsi-dacd-isaac/mbtr.git

Usage

MBT regressor follows the scikit-learn syntax for regressors. Creating a default instance and training it is as simple as:

m = MBT().fit(x,y)

while predictions for the test set are obtained through

y_hat = m.predict(x_te)

The most important parameters are the number of boosts n_boost, that is, the number of fitted trees, learning_rate and the loss_type. An extensive explanation of the different parameters can be found in the documentation.

Documentation

Documentation and examples on the usage can be found at docs.

Reference

If you make use of this software for your work, we would appreciate it if you would cite us:

Lorenzo Nespoli and Vasco Medici (2020). Multivariate Boosted Trees and Applications to Forecasting and Control arXiv

@article{nespoli2020multivariate,
  title={Multivariate Boosted Trees and Applications to Forecasting and Control},
  author={Nespoli, Lorenzo and Medici, Vasco},
  journal={arXiv preprint arXiv:2003.03835},
  year={2020}
}

Acknowledgments

The authors would like to thank the Swiss Federal Office of Energy (SFOE) and the Swiss Competence Center for Energy Research - Future Swiss Electrical Infrastructure (SCCER-FURIES), for their financial and technical support to this research work.

You might also like...
Grammar Induction using a Template Tree Approach

Gitta Gitta ("Grammar Induction using a Template Tree Approach") is a method for inducing context-free grammars. It performs particularly well on data

Code for "Learning Structural Edits via Incremental Tree Transformations" (ICLR'21)

Learning Structural Edits via Incremental Tree Transformations Code for "Learning Structural Edits via Incremental Tree Transformations" (ICLR'21) 1.

Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

Code for Graph-to-Tree Learning for Solving Math Word Problems (ACL 2020)

Graph-to-Tree Learning for Solving Math Word Problems PyTorch implementation of Graph based Math Word Problem solver described in our ACL 2020 paper G

NAS Benchmark in
NAS Benchmark in "Prioritized Architecture Sampling with Monto-Carlo Tree Search", CVPR2021

NAS-Bench-Macro This repository includes the benchmark and code for NAS-Bench-Macro in paper "Prioritized Architecture Sampling with Monto-Carlo Tree

The official code for paper "R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling".

R2D2 This is the official code for paper titled "R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Mode

Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks
Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks

SSTNet Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks(ICCV2021) by Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, Kui J

Implementation of fast algorithms for Maximum Spanning Tree (MST) parsing that includes fast ArcMax+Reweighting+Tarjan algorithm for single-root dependency parsing.

Fast MST Algorithm Implementation of fast algorithms for (Maximum Spanning Tree) MST parsing that includes fast ArcMax+Reweighting+Tarjan algorithm fo

This is the code repository implementing the paper
This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction".

TreePartNet This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction". Depende

Comments
  • Is it possible to define custom loss function ?

    Is it possible to define custom loss function ?

    Dear all, First thank you for developping this tool, that I believe is of great interest. I am working with:

    • environmental variables (e.g. temperature, salinity)
    • multi-dimensional targets, that are relative abundance, with their sum = 1 for each site

    Therefore, I was wondering if it is possible to implement a custom loss function in the mbtr framework, that would be adapted for proportions. Please note that I am quite new to python.

    To do some testing, I tryed to dupplicate the mse loss function with another name in the losses.py file and adding the new loss in the LOSS_MAP in __inits__.py. Then I compiled the files. However, I have this error when trying to run the model from the multi_reg.py example:

    >>> m = MBT(loss_type = 'mse', n_boosts=30,  min_leaf=100, lambda_weights=1e-3).fit(x_tr, y_tr, do_plot=True)
      3%|▎         | 1/30 [00:03<01:45,  3.63s/it]
    >>> m = MBT(loss_type = 'custom_mse', n_boosts=30,  min_leaf=100, lambda_weights=1e-3).fit(x_tr, y_tr, do_plot=True)
      0%|          | 0/30 [00:00<?, ?it/s]KeyError: 'custom_mse'
    

    It seems that the new loss is not recognised in LOSS_MAP:

    >>> LOSS_MAP = {'custom_mse': losses.custom_MSE,
    ...             'mse': losses.MSE,
    ...             'time_smoother': losses.TimeSmoother,
    ...             'latent_variable': losses.LatentVariable,
    ...             'linear_regression': losses.LinRegLoss,
    ...             'fourier': losses.FourierLoss,
    ...             'quantile': losses.QuantileLoss,
    ...             'quadratic_quantile': losses.QuadraticQuantileLoss}
    AttributeError: module 'mbtr.losses' has no attribute 'custom_MSE'
    

    I guess that I missed something when trying to dupplicate and rename the mse loss. I would appreciate any help if the definition of a custom loss function is possible.

    Best regards,

    opened by alexschickele 2
  • Dataset cannot be reached

    Dataset cannot be reached

    Hi thank you for your effort to create this. I want to try this but i cannot download nor visit the web that you provided in example multivariate_forecas.py

    Is there any alternative link for that dataset? thank you regards!

    opened by kristfrizh 1
  • Error at import time with python 3.10.*

    Error at import time with python 3.10.*

    I want to use MBTR in a teaching module and I need to use jupyter-lab inside a conda environment for teaching purposes. While MBTR works as expected in a vanilla python 3.8, it errors out (on the same machine) in a conda environment using python 3.10

    Steps to reproduce

    conda create --name testenv
    conda activate testenv
    
    conda install -c conda-forge jupyterlab
    pip install --upgrade git+https://github.com/supsi-dacd-isaac/mbtr.git
    # to make sure to get the latest version; but the version on pypi gives the same error 
    

    Then

    python
    

    and in python

    from mbtr.mbtr import MBT
    

    which outputs the following error

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/mbtr/mbtr.py", line 317, in <module>
        def leaf_stats(y, edges, x, order):
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/decorators.py", line 219, in wrapper
        disp.compile(sig)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/dispatcher.py", line 965, in compile
        cres = self._compiler.compile(args, return_type)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/dispatcher.py", line 129, in compile
        raise retval
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/dispatcher.py", line 139, in _compile_cached
        retval = self._compile_core(args, return_type)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/dispatcher.py", line 152, in _compile_core
        cres = compiler.compile_extra(self.targetdescr.typing_context,
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler.py", line 716, in compile_extra
        return pipeline.compile_extra(func)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler.py", line 452, in compile_extra
        return self._compile_bytecode()
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler.py", line 520, in _compile_bytecode
        return self._compile_core()
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler.py", line 499, in _compile_core
        raise e
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler.py", line 486, in _compile_core
        pm.run(self.state)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler_machinery.py", line 368, in run
        raise patched_exception
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler_machinery.py", line 356, in run
        self._runPass(idx, pass_inst, state)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
        return func(*args, **kwargs)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler_machinery.py", line 311, in _runPass
        mutated |= check(pss.run_pass, internal_state)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/compiler_machinery.py", line 273, in check
        mangled = func(compiler_state)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/typed_passes.py", line 105, in run_pass
        typemap, return_type, calltypes, errs = type_inference_stage(
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/typed_passes.py", line 83, in type_inference_stage
        errs = infer.propagate(raise_errors=raise_errors)
      File "/home/myself/.conda/envs/testenv/lib/python3.10/site-packages/numba/core/typeinfer.py", line 1086, in propagate
        raise errors[0]
    numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
    No conversion from UniTuple(none x 2) to UniTuple(array(float64, 2d, A) x 2) for '$116return_value.7', defined at None
    
    File ".conda/envs/testenv/lib/python3.10/site-packages/mbtr/mbtr.py", line 327:
    def leaf_stats(y, edges, x, order):
        <source elided>
            s_left, s_right = None, None
        return s_left, s_right
        ^
    
    During: typing of assignment at /home/myself/.conda/envs/testenv/lib/python3.10/site-packages/mbtr/mbtr.py (327)
    
    File ".conda/envs/test/lib/python3.10/site-packages/mbtr/mbtr.py", line 327:
    def leaf_stats(y, edges, x, order):
        <source elided>
            s_left, s_right = None, None
        return s_left, s_right
        ^
    

    Thanks in advance for any pointer/help. The course where I want to present this is a summer course and is closing in on me 😉

    opened by jiho 0
Releases(v0.1.3)
Owner
SUPSI-DACD-ISAAC
SUPSI-DACD-ISAAC
using yolox+deepsort for object-tracker

YOLOX_deepsort_tracker yolox+deepsort实现目标跟踪 最新的yolox尝尝鲜~~(yolox正处在频繁更新阶段,因此直接链接yolox仓库作为子模块) Install Clone the repository recursively: git clone --rec

245 Dec 26, 2022
This GitHub repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.'

About Repository This repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.' About Code

Arun Verma 1 Nov 09, 2021
Hidden-Fold Networks (HFN): Random Recurrent Residuals Using Sparse Supermasks

Hidden-Fold Networks (HFN): Random Recurrent Residuals Using Sparse Supermasks by Ángel López García-Arias, Masanori Hashimoto, Masato Motomura, and J

Ángel López García-Arias 4 May 19, 2022
A method to perform unsupervised cross-region adaptation of crop classifiers trained with satellite image time series.

TimeMatch Official source code of TimeMatch: Unsupervised Cross-region Adaptation by Temporal Shift Estimation by Joachim Nyborg, Charlotte Pelletier,

Joachim Nyborg 17 Nov 01, 2022
This repo contains the code for paper Inverse Weighted Survival Games

Inverse-Weighted-Survival-Games This repo contains the code for paper Inverse Weighted Survival Games instructions general loss function (--lfn) can b

3 Jan 12, 2022
LabelImg is a graphical image annotation tool.

LabelImgPlus LabelImg is a graphical image annotation tool. This project is not updated with new functions now. More functions are supported with Labe

lzx1413 200 Dec 20, 2022
A tf.keras implementation of Facebook AI's MadGrad optimization algorithm

MADGRAD Optimization Algorithm For Tensorflow This package implements the MadGrad Algorithm proposed in Adaptivity without Compromise: A Momentumized,

20 Aug 18, 2022
This is my codes that can visualize the psnr image in testing videos.

CVPR2018-Baseline-PSNRplot This is my codes that can visualize the psnr image in testing videos. Future Frame Prediction for Anomaly Detection – A New

Wenhao Yang 12 May 29, 2021
🔥 Real-time Super Resolution enhancement (4x) with content loss and relativistic adversarial optimization 🔥

🔥 Real-time Super Resolution enhancement (4x) with content loss and relativistic adversarial optimization 🔥

Rishik Mourya 48 Dec 20, 2022
Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

UniSpeech The family of UniSpeech: UniSpeech (ICML 2021): Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR UniSpeech-

Microsoft 282 Jan 09, 2023
You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling

You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling Transformer-based models are widely used in natural language processi

Zhanpeng Zeng 12 Jan 01, 2023
Self-labelling via simultaneous clustering and representation learning. (ICLR 2020)

Self-labelling via simultaneous clustering and representation learning 🆗 🆗 🎉 NEW models (20th August 2020): Added standard SeLa pretrained torchvis

Yuki M. Asano 469 Jan 02, 2023
Open-source code for Generic Grouping Network (GGN, CVPR 2022)

Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity Pytorch implementation for "Open-World Instance Segmen

Meta Research 99 Dec 06, 2022
[2021][ICCV][FSNet] Full-Duplex Strategy for Video Object Segmentation

Full-Duplex Strategy for Video Object Segmentation (ICCV, 2021) Authors: Ge-Peng Ji, Keren Fu, Zhe Wu, Deng-Ping Fan*, Jianbing Shen, & Ling Shao This

Daniel-Ji 55 Dec 22, 2022
Code for "Learning the Best Pooling Strategy for Visual Semantic Embedding", CVPR 2021

Learning the Best Pooling Strategy for Visual Semantic Embedding Official PyTorch implementation of the paper Learning the Best Pooling Strategy for V

Jiacheng Chen 106 Jan 06, 2023
Info and sample codes for "NTU RGB+D Action Recognition Dataset"

"NTU RGB+D" Action Recognition Dataset "NTU RGB+D 120" Action Recognition Dataset "NTU RGB+D" is a large-scale dataset for human action recognition. I

Amir Shahroudy 578 Dec 30, 2022
Includes PyTorch -> Keras model porting code for ConvNeXt family of models with fine-tuning and inference notebooks.

ConvNeXt-TF This repository provides TensorFlow / Keras implementations of different ConvNeXt [1] variants. It also provides the TensorFlow / Keras mo

Sayak Paul 87 Dec 06, 2022
Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Documentation: https://raamana.github.io/neuropredict/ News As of v0.6, neuropredict now supports regression applications i.e. predicting continuous t

Pradeep Reddy Raamana 93 Nov 29, 2022
Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021)

TDEER (WIP) Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021) Overview TDEER is an e

Alipay 6 Dec 17, 2022
RRxIO - Robust Radar Visual/Thermal Inertial Odometry: Robust and accurate state estimation even in challenging visual conditions.

RRxIO - Robust Radar Visual/Thermal Inertial Odometry RRxIO offers robust and accurate state estimation even in challenging visual conditions. RRxIO c

Christopher Doer 64 Dec 29, 2022