Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Overview

Build Status Travis Build Status AppVeyor DOI arXiv.org Python Versions PyPI Version CRAN Version

Regularized Greedy Forest

Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better results than gradient boosted decision trees (GBDT) on a number of datasets and it has been used to win a few Kaggle competitions. Unlike the traditional boosted decision tree approach, RGF works directly with the underlying forest structure. RGF integrates two ideas: one is to include tree-structured regularization into the learning formulation; and the other is to employ the fully-corrective regularized greedy algorithm.

This repository contains the following implementations of the RGF algorithm:

  • RGF: original implementation from the paper;
  • FastRGF: multi-core implementation with some simplifications;
  • rgf_python: wrapper of both RGF and FastRGF implementations for Python;
  • R package: wrapper of rgf_python for R.

You may want to get interesting information about RGF from the posts collected in Awesome RGF.

Comments
  • Support wheels

    Support wheels

    Since rgf_python hasn't any special requirements (for compiler, environment, etc.), I think it good idea to have wheels on PyPI site (and the sources in .tar.gz, of course). I believe providing successfully compiled binaries will prevent many strange errors like recent ones.

    We need wheels for two platforms: first for macOS and Linux and second for Windows.

    The final result should be similar to this one: image

    But each wheel for each platform should have 32bit and 64bit version.

    Binaries we could get from Travis and Appveyor as artifacts (I can do this). The one problem I see now is that Travis hasn't 32bit machines, but I believe we'll overcome this problem 😃 .

    @fukatani When you'll have time, please search how to appropriate name wheels according to target platforms and how to post them at PyPI. Or I can do it more later.

    enhancement 
    opened by StrikerRUS 35
  • error:Exception: Model learning result is not found in /tmp/rgf. This is rgf_python error.

    error:Exception: Model learning result is not found in /tmp/rgf. This is rgf_python error.

    How to deal with this error:

    Ran 0 examples: 0 success, 0 failure, 0 error

    None Ran 0 examples: 0 success, 0 failure, 0 error

    None Ran 0 examples: 0 success, 0 failure, 0 error

    None Traceback (most recent call last): File "/Users/k.den/Desktop/For_Submission/1_source_code/test.py", line 25, in pred = rgf_model.predict_proba(X_eval)[:, 1] File "/usr/local/lib/python3.6/site-packages/rgf/sklearn.py", line 652, in predict_proba class_proba = clf.predict_proba(X) File "/usr/local/lib/python3.6/site-packages/rgf/sklearn.py", line 798, in predict_proba 'This is rgf_python error.'.format(_TEMP_PATH)) Exception: Model learning result is not found in /tmp/rgf. This is rgf_python error.

    Process finished with exit code 1

    opened by tianke0711 34
  • ModuleNotFoundError: No module named 'rgf.sklearn'; 'rgf' is not a package

    ModuleNotFoundError: No module named 'rgf.sklearn'; 'rgf' is not a package

    For bugs and unexpected issues, please provide the following information, so that we could reproduce them on our system.

    Environment Info

    Operating System: MacOS Sierra 10.12 | Ubuntu 16.04.3 LTS

    Python version: 3.6.1

    rgf_python version: HEAD (pulled from github)

    Whether test.py is passed or not: FAILED (errors=24)

    Error Message

    ModuleNotFoundError: No module named 'rgf.sklearn'; 'rgf' is not a package

    Reproducible Example

    from rgf.sklearn import RGFClassifier

    opened by vsedelnik 30
  • suggestion to integrate the R wrapper in the repository

    suggestion to integrate the R wrapper in the repository

    This issue is related with a previous one. A month ago I wrapped rgf_python using the reticulate package in R. It can be installed on Linux, and somehow cumbersome on Macintosh and Windows (on Windows currently it works only from the command prompt). I opened the issue as suggested by @fukatani

    opened by mlampros 20
  • Model learning result is not found in C:\Users\hp\temp\rgf. This is rgf_python error.

    Model learning result is not found in C:\Users\hp\temp\rgf. This is rgf_python error.

    Hello,

    i have read the previous thread on the same post, but it does not seem to solve my problem, because the previous case had string included in dataset and all i have got are all numbers. Could you please let me know what could be the problem??

    Much appreciated !

    skf = StratifiedKFold(n_splits = kfold, random_state=1)
    for i, (train_index, test_index) in enumerate(skf.split(X, y)):
        X_train, X_eval = X[train_index], X[test_index]
        y_train, y_eval = y[train_index], y[test_index]
       
        rgf_model = RGFClassifier(max_leaf=400,
                        algorithm="RGF_Sib",
                        test_interval=100,
                        verbose=True).fit( X_train, y_train)
        pred = rgf_model.predict_proba(X_eval)[:,1]
        print( "Gini = ", eval_gini(y_eval, pred) )
    

    and

    ---------------------------------------------------------------------------
    Exception                                 Traceback (most recent call last)
    <ipython-input-17-b27ba3506d06> in <module>()
         12                     test_interval=100,
         13                     verbose=True).fit( X_train, y_train)
    ---> 14     pred = rgf_model.predict_proba(X_eval)[:,1]
         15     print( "Gini = ", eval_gini(y_eval, pred) )
    
    C:\Anaconda3\lib\site-packages\rgf\sklearn.py in predict_proba(self, X)
        644                              % (self._n_features, n_features))
        645         if self._n_classes == 2:
    --> 646             y = self._estimators[0].predict_proba(X)
        647             y = _sigmoid(y)
        648             y = np.c_[y, 1 - y]
    
    C:\Anaconda3\lib\site-packages\rgf\sklearn.py in predict_proba(self, X)
        796         if not model_files:
        797             raise Exception('Model learning result is not found in {0}. '
    --> 798                             'This is rgf_python error.'.format(_TEMP_PATH))
        799         latest_model_loc = sorted(model_files, reverse=True)[0]
        800 
    
    Exception: Model learning result is not found in C:\Users\hp\temp\rgf. This is rgf_python error.
    
    
    opened by mike-m123 20
  • migrate from Appveyor to GitHub Actions

    migrate from Appveyor to GitHub Actions

    Fixed #122. Appveyor suggests only 1 parallel job at free tier, GitHub Actions - 20.

    Should be considered as a continuation of #328. Same changes as for *nix OSes: latest R version; stop producing 32bit artifacts.

    opened by StrikerRUS 16
  • New release

    New release

    I suppose it's time to release a new version with the support of warm start.

    @fukatani Please release new Python version, and then @mlampros please upload to CRAN new R version.

    opened by StrikerRUS 16
  • updated wheels building

    updated wheels building

    @fukatani Please attach Linux i686 executable file to GitHub release - I've just tested replacing files into wheels and it works locally, so should work on Travis too! :-)

    Refer to https://github.com/fukatani/rgf_python/issues/81#issuecomment-348662123.

    opened by StrikerRUS 15
  • More Travis tests

    More Travis tests

    Hi @fukatani ! Can you add more platforms (Windows, MacOS) to Travis? I don't know how, but it's possible 😄 : image [Screenshot from xgboost repo] Maybe it can help: https://github.com/dmlc/xgboost/blob/master/.travis.yml

    If there is a limitation to number of tests, maybe it's better to split Python version tests between platforms: Windows + 2.7, Linux + 3.4, MacOS + 3.5 (I think you understand me).

    opened by StrikerRUS 15
  • Cannot import name 'RGFClassifier'

    Cannot import name 'RGFClassifier'

    I am having the above error. I have made rgf1.2 and have tested using rgf1.2's own perl test script. This works. I have installed rgf_python and run the python setup as specified. I have changed the two folder locations to rgf1.2..\rgf executable and a temp folder that exist.

    In python when I try to import I get the error Cannot import name 'RGFClassifier'. I tried to run the exact code in the test.py script provided in with rgf_python and this same error occurs.

    Strangely, I have /usr/local/lib/python3.5/dist-packages/rgf_sklearn-0.0.0-py3.5.egg/rgf in my path when I do run

    import sys
    sys.path
    

    in python. I also in /usr/local/lib/python3.5/dist-packages I only have the rfg-sklearn-0.0.0-py3.5.egg and no rgf-sklearn as I would expect as the following appeared towards the end of the setup.py,

    Extracting rgf_sklearn-0.0.0-py3.5.egg to /usr/local/lib/python3.5/dist-packages
    Adding rgf-sklearn 0.0.0 to easy-install.pth file
    
    opened by JoshuaC3 15
  • [rgf_python] add warm-start

    [rgf_python] add warm-start

    Fixed #184.

    This PR adds the support of warm-start in RGF estimators, save_model() method which is needed to obtain binary model file and for further passing in init_model argument.

    Also, this PR adds tests with analysis of exception message (as I promised in https://github.com/RGF-team/rgf/pull/258#issuecomment-439685042).

    opened by StrikerRUS 14
  • Running RGF from R cmd

    Running RGF from R cmd

    For bugs and unexpected issues, please provide the following information, so that we could reproduce them on our system.

    Environment Info

    Operating System: Windows 10

    RGF/FastRGF/rgf_python version: 3.5.0-9

    Python version (for rgf_python errors): 3.5.0-9

    Error Message

    image

    image

    Reproducible Example

    Error when running RGF from R console as shown in the pic. Installation of RGF should be working fine as shown in the pic. RGF was installed via devtools.

    help wanted 
    opened by similang 2
  • Python cant find executables

    Python cant find executables

    Hi there

    I'm trying to install rgf/fastrgf and use the python wrapper to launch the executables.

    I've installed using pip install rgf_python

    However when i import the rgf module i get a user warning

    UserWarning: Cannot find FastRGF executable files. FastRGF estimators will be unavailable for usage.
      warnings.warn("Cannot find FastRGF executable files. FastRGF estimators will be unavailable for usage.")
    

    To fix this issue i've compiled the rgf and fastrgf binaries* and added them to my $PATH variable (confirmed in bash that they are in the PATH) however i still get the same error. I've looked a bit into the rgf/utils get_paths and is_fastrgf_executable functions however i'm not completely sure why it fails?

    *binaries: i was not sure which binaries are needed so i've added the following rgf, forest_predict, forest_train, discretized_trainer, discretized_gendata, auc

    System Python: conda 3.6.1 OS: ubuntu 16.04

    opened by casperkaae 29
  • dump RGF and FastRGF to the JSON file

    dump RGF and FastRGF to the JSON file

    Initial support for dumping the RGF model is already implemented in #161. At present it's possible to print the model to the console. But it's good idea to bring the possibility of dumping the model to the file (e.g. JSON).

    @StrikerRUS:

    Really like new features introduced in this PR. But please think about "real dump" of a model. I suppose it'll be more useful than just printing to the console.

    @fukatani:

    For example dump in JSON format like lightGBM. It's convenient and we may support it in the future, but we should do it with another PR.

    enhancement 
    opened by StrikerRUS 6
  • Support f_ratio?

    Support f_ratio?

    I found not documented parameter f_ratio in RGF. This corresponding to LightGBM feature_fraction and XGB colsample_bytree.

    I tried these parameter with boston regression example. In small max_leaf(300), f_ratio=0.9 improves score to 11.0 from 11.8, but in many max_leaf(5000), f_ratio=0.95 degrared score to 10.34 from 10.19810.

    After all, is there no value to use f_ratio < 1.0?

    opened by fukatani 10
  • [FastRGF] FastRGF doesn't work for small sample and need to fix integration test for FastRGF

    [FastRGF] FastRGF doesn't work for small sample and need to fix integration test for FastRGF

    #Now, sklearn integration tests for FastRGFClassifier and FastRGFClassifier.

    FastRGF doesn't work well for small samples, that is reason for test failed. I doubt inside Fast RGF executable inside. I inspect Fast RGF by debugger, discretization boundaries are invalid.

    At least we should raise understandable error from RGF python if discretization failed.

    bug 
    opened by fukatani 18
Releases(3.12.0)
Owner
RGF-team
RGF-team
10x faster matrix and vector operations

Bolt is an algorithm for compressing vectors of real-valued data and running mathematical operations directly on the compressed representations. If yo

2.3k Jan 09, 2023
基于Paddle框架的fcanet复现

fcanet-Paddle 基于Paddle框架的fcanet复现 fcanet 本项目基于paddlepaddle框架复现fcanet,并参加百度第三届论文复现赛,将在2021年5月15日比赛完后提供AIStudio链接~敬请期待 参考项目: frazerlin-fcanet 数据准备 本项目已挂

QuanHao Guo 7 Mar 07, 2022
Code and data for ImageCoDe, a contextual vison-and-language benchmark

ImageCoDe This repository contains code and data for ImageCoDe: Image Retrieval from Contextual Descriptions. Data All collected descriptions for the

McGill NLP 27 Dec 02, 2022
A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution

DRSAN A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution Karam Park, Jae Woong Soh, and Nam Ik Cho Environments U

4 May 10, 2022
Sandbox for training deep learning networks

Deep learning networks This repo is used to research convolutional networks primarily for computer vision tasks. For this purpose, the repo contains (

Oleg Sémery 2.7k Jan 01, 2023
Continuous Security Group Rule Change Detection & Response at scale

Introduction Get notified of Security Group Changes across all AWS Accounts & Regions in an AWS Organization, with the ability to respond/revert those

Raajhesh Kannaa Chidambaram 3 Aug 13, 2022
Deep Learning Theory

Deep Learning Theory 整理了一些深度学习的理论相关内容,持续更新。 Overview Recent advances in deep learning theory 总结了目前深度学习理论研究的六个方向的一些结果,概述型,没做深入探讨(2021)。 1.1 complexity

fq 103 Jan 04, 2023
Codes accompanying the paper "Learning Nearly Decomposable Value Functions with Communication Minimization" (ICLR 2020)

NDQ: Learning Nearly Decomposable Value Functions with Communication Minimization Note This codebase accompanies paper Learning Nearly Decomposable Va

Tonghan Wang 69 Nov 26, 2022
Trash Sorter Extraordinaire is a software which efficiently detects the different types of waste in a pile of random trash through feeding it pictures or videos.

Trash-Sorter-Extraordinaire Trash Sorter Extraordinaire is a software which efficiently detects the different types of waste in a pile of random trash

Rameen Mahmood 1 Nov 07, 2021
Keyword-BERT: Keyword-Attentive Deep Semantic Matching

project discription An implementation of the Keyword-BERT model mentioned in my paper Keyword-Attentive Deep Semantic Matching (Plz cite this github r

1 Nov 14, 2021
Official PyTorch Implementation of Mask-aware IoU and maYOLACT Detector [BMVC2021]

The official implementation of Mask-aware IoU and maYOLACT detector. Our implementation is based on mmdetection. Mask-aware IoU for Anchor Assignment

Kemal Oksuz 46 Sep 29, 2022
Learning Tracking Representations via Dual-Branch Fully Transformer Networks

Learning Tracking Representations via Dual-Branch Fully Transformer Networks DualTFR ⭐ We achieves the runner-ups for both VOT2021ST (short-term) and

phiphi 19 May 04, 2022
Scenarios, tutorials and demos for Autonomous Driving

The Autonomous Driving Cookbook (Preview) NOTE: This project is developed and being maintained by Project Road Runner at Microsoft Garage. This is cur

Microsoft 2.1k Jan 02, 2023
Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods

Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods Introduction Graph Neural Networks (GNNs) have demonstrated

37 Dec 15, 2022
particle tracking model, works with the ROMS output file(qck.nc, his.nc)

particle-tracking-model-for-ROMS particle tracking model, works with the ROMS output file(qck.nc, his.nc) description this is a 2-dimensional particle

xusheng 1 Jan 11, 2022
This is the solution for 2nd rank in Kaggle competition: Feedback Prize - Evaluating Student Writing.

Feedback Prize - Evaluating Student Writing This is the solution for 2nd rank in Kaggle competition: Feedback Prize - Evaluating Student Writing. The

Udbhav Bamba 41 Dec 14, 2022
HugsVision is a easy to use huggingface wrapper for state-of-the-art computer vision

HugsVision is an open-source and easy to use all-in-one huggingface wrapper for computer vision. The goal is to create a fast, flexible and user-frien

Labrak Yanis 166 Nov 27, 2022
Cobalt Strike teamserver detection.

Cobalt-Strike-det Cobalt Strike teamserver detection. usage: cobaltstrike_verify.py [-l TARGETS] [-t THREADS] optional arguments: -h, --help show this

TimWhite 17 Sep 27, 2022
Discovering and Achieving Goals via World Models

Discovering and Achieving Goals via World Models [Project Website] [Benchmark Code] [Video (2min)] [Oral Talk (13min)] [Paper] Russell Mendonca*1, Ole

Oleg Rybkin 71 Dec 22, 2022
Differentiable Simulation of Soft Multi-body Systems

Differentiable Simulation of Soft Multi-body Systems Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, Ming C. Lin [Paper] [Code] Updates The C++ backend s

YilingQiao 26 Dec 23, 2022