Tools for Optuna, MLflow and the integration of both.

Related tags

Machine Learningpythonmachine-learningloggingtransformershyperparameter-optimizationhyperparametermlflowoptuna
Overview

HPOflow - Sphinx DOC

DOC MIT License Contributor Covenant Python Version pypi
pytest status Static Code Checks status Build & Deploy Doc GitHub issues

Tools for Optuna, MLflow and the integration of both.

Detailed documentation with examples can be found here: Sphinx DOC

Table of Contents

Maintainers

One Conversation
This project is maintained by the One Conversation team of Deutsche Telekom AG.

The main components are:

Installation

HPOflow is available at the Python Package Index (PyPI). It can be installed with pip:

$ pip install hpoflow

Some additional dependencies might be necessary.

To use hpoflow.optuna_mlflow.OptunaMLflow:

$ pip install mlflow GitPython

To use hpoflow.optuna_transformers.OptunaMLflowCallback:

$ pip install mlflow GitPython transformers

To install all optional dependencies use:

$ pip install hpoflow[optional]

Support and Feedback

The following channels are available for discussions, feedback, and support requests:

Reporting Security Vulnerabilities

This project is built with security and data privacy in mind to ensure your data is safe. We are grateful for security researchers and users reporting a vulnerability to us, first. To ensure that your request is handled in a timely manner and non-disclosure of vulnerabilities can be assured, please follow the below guideline.

Please do not report security vulnerabilities directly on GitHub. GitHub Issues can be publicly seen and therefore would result in a direct disclosure.

Please address questions about data privacy, security concepts, and other media requests to the [email protected] mailbox.

Contribution

Our commitment to open source means that we are enabling - in fact encouraging - all interested parties to contribute and become part of our developer community.

Contribution and feedback is encouraged and always welcome. For more information about how to contribute, as well as additional contribution information, see our Contribution Guidelines.

Code of Conduct

This project has adopted the Contributor Covenant as our code of conduct. Please see the details in our Contributor Covenant Code of Conduct. All contributors must abide by the code of conduct.

Licensing

Copyright (c) 2021 Philip May, Deutsche Telekom AG
Copyright (c) 2021 Philip May
Copyright (c) 2021 Timothy Wolff-Piggott

Licensed under the MIT License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License by reviewing the file LICENSE in the repository.

Comments
  • review README.md and CONTRIBUTING.md

    review README.md and CONTRIBUTING.md

    Review README.md and CONTRIBUTING.md

    • is there something missing? maybe compare with optuna and transformers
    • spelling
    • idiomatic english
    • consistency
    • correctness
    • links ok?
    • ...

    PS: The real documentation is still missing and a know issue.

    opened by PhilipMay 12
  • add typing in optuna_transformers

    add typing in optuna_transformers

    @twolffpiggott can you please tell me the type of this?

    https://github.com/telekom/HPOflow/blob/e2b0943218af419a79ce95e60b67c9a4c2477349/hpoflow/optuna_transformers.py#L47

    opened by PhilipMay 6
  • add `transformers.py`

    add `transformers.py`

    @twolffpiggott should we add this here or to an other project we open source?

    https://github.com/PhilipMay/mltb/blob/master/mltb/integration/transformers.py

    enhancement 
    opened by PhilipMay 6
  • Create Sphinx documentation page

    Create Sphinx documentation page

    • [x] setup
    • [x] make GH action
    • [x] setup page
    • [x] change styling to telekom style
    • switch to MD
    • [x] add more content
    • [x] link from README to page
    • [x] link from pypi to GH page
    • [x] add impressum
    • [x] remove strange mouse over image effect
    • add version info
    documentation 
    opened by PhilipMay 4
  • Problems with direct `_imports.check()` call

    Problems with direct `_imports.check()` call

    When the __init__.py imports OMLflowCallback the optuna_transformers.py script is executed. That executes the _imports.check() call which then throws an exception if transformers or mlflow is not installed. But that should be avoided.

    See here: https://github.com/telekom/HPOflow/blob/d1cce5cbc2a84634d1484a053286000dda05b681/hpoflow/optuna_transformers.py#L11-L17

    The solution would be to put the _imports.check() call into the constructor. But that is not possible because OMLflowCallback inherits from transformers.

    The only solution I have is to put OMLflowCallback into an factory function that creates an OMLflowCallback and does the _imports.check() in there.

    @twolffpiggott what do you think?

    bug 
    opened by PhilipMay 3
  • Flake8 ignore list for Black compatibility

    Flake8 ignore list for Black compatibility

    Flake8 raises a warning for "E203" when it encounters a Black decision to insert whitespace before : in slicing syntax.

    Black's behaviour is more correct here, so my suggestion is to add "E203" to the flake8 config ignore list.

    i.e. in setup.cfg:

    [flake8]
...
extend-ignore = E203
    opened by twolffpiggott 3
  • Simple Example?

    Simple Example?

    I don't understand how to use this package. Could you provide a basic example? I don't understand the import_structure and how it relates to importing the modules? Thanks

    opened by jmrichardson 2
  • WIP prefix in contrib file

    WIP prefix in contrib file

    Should this

    Create Work In Progress [WIP] pull requests only if you need clarification or an explicit review before you can continue your work item.

    be more like this

    Add a [WIP] prefix on your pull request name if you need clarification or an explicit review before you can continue your work item.

    documentation 
    opened by PhilipMay 2
Releases(0.1.4)
Owner
Telekom Open Source Software
published by Deutsche Telekom AG and partner companies
Telekom Open Source Software
GitHub Repository https://telekom.github.io/HPOflow/
stability-selection - A scikit-learn compatible implementation of stability selection

stability-selection - A scikit-learn compatible implementation of stability selection stability-selection is a Python implementation of the stability

185 Dec 03, 2022
PLUR is a collection of source code datasets suitable for graph-based machine learning.

PLUR (Programming-Language Understanding and Repair) is a collection of source code datasets suitable for graph-based machine learning. We provide scripts for downloading, processing, and loading the

Google Research 76 Nov 25, 2022
hgboost - Hyperoptimized Gradient Boosting

hgboost is short for Hyperoptimized Gradient Boosting and is a python package for hyperparameter optimization for xgboost, catboost and lightboost using cross-validation, and evaluating the results o

Erdogan Taskesen 34 Jan 03, 2023
PySpark ML Bank Churn Prediction

PySpark-Bank-Churn Surname: corresponds to the record (row) number and has no effect on the output. CreditScore: contains random values and has no eff

kemalgunay 2 Nov 11, 2021
OptaPy is an AI constraint solver for Python to optimize planning and scheduling problems.

OptaPy is an AI constraint solver for Python to optimize the Vehicle Routing Problem, Employee Rostering, Maintenance Scheduling, Task Assignment, School Timetabling, Cloud Optimization, Conference S

OptaPy 208 Dec 27, 2022
Automatic extraction of relevant features from time series:

tsfresh This repository contains the TSFRESH python package. The abbreviation stands for "Time Series Feature extraction based on scalable hypothesis

Blue Yonder GmbH 7k Jan 06, 2023
A handy tool for common machine learning models' hyper-parameter tuning.

Common machine learning models' hyperparameter tuning This repo is for a collection of hyper-parameter tuning for "common" machine learning models, in

Kevin Hu 2 Jan 27, 2022
Required for a machine learning pipeline data preprocessing and variable engineering script needs to be prepared

Feature-Engineering Required for a machine learning pipeline data preprocessing and variable engineering script needs to be prepared. When the dataset

kemalgunay 5 Apr 21, 2022
Fit interpretable models. Explain blackbox machine learning.

InterpretML - Alpha Release In the beginning machines learned in darkness, and data scientists struggled in the void to explain them. Let there be lig

InterpretML 5.2k Jan 09, 2023
Microsoft Machine Learning for Apache Spark

Microsoft Machine Learning for Apache Spark MMLSpark is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark

Microsoft Azure 3.9k Dec 30, 2022
Code base of KU AIRS: SPARK Autonomous Vehicle Team

KU AIRS: SPARK Autonomous Vehicle Project Check this link for the blog post describing this project and the video of SPARK in simulation and on parkou

Mehmet Enes Erciyes 1 Nov 23, 2021
Esse é o meu primeiro repo tratando de fim a fim, uma pipeline de dados abertos do governo brasileiro relacionado a compras de contrato e cronogramas anuais com spark, em pyspark e SQL!

Olá! Esse é o meu primeiro repo tratando de fim a fim, uma pipeline de dados abertos do governo brasileiro relacionado a compras de contrato e cronogr

Henrique de Paula 10 Apr 04, 2022
Predicting diabetes over a five year period using logistic regression and the Pima First-Nation dataset

Diabetes This script uses the Pima First Nations dataset to create a model to predict whether or not an individual will develop Diabetes Mellitus Type

1 Mar 28, 2022
AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。 它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

English | 简体中文 AutoX是什么？ AutoX一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。 它的特点包括: 效果出色: AutoX在多个kaggle数据集上，效果显著优于其他解决方案(见效果对比)。 简单易用: AutoX的接口和sklearn类似，方便上手使用。

4Paradigm 431 Dec 28, 2022
Drug prediction

I have collected data about a set of patients, all of whom suffered from the same illness. During their course of treatment, each patient responded to one of 5 medications, Drug A, Drug B, Drug c, Dr

Khazar 1 Jan 28, 2022
This is my implementation on the K-nearest neighbors algorithm from scratch using Python

K Nearest Neighbors (KNN) algorithm In this Machine Learning world, there are various algorithms designed for classification problems such as Logistic

sonny1902 1 Jan 08, 2022
Greykite: A flexible, intuitive and fast forecasting library

The Greykite library provides flexible, intuitive and fast forecasts through its flagship algorithm, Silverkite.

LinkedIn 1.7k Jan 04, 2023
Feature-engine is a Python library with multiple transformers to engineer and select features for use in machine learning models.

Feature-engine is a Python library with multiple transformers to engineer and select features for use in machine learning models. Feature-engine's transformers follow scikit-learn's functionality wit

Soledad Galli 33 Dec 27, 2022
NumPy-based implementation of a multilayer perceptron (MLP)

My own NumPy-based implementation of a multilayer perceptron (MLP). Several of its components can be tuned and played with, such as layer depth and size, hidden and output layer activation functions,

1 Feb 10, 2022
A chain of stores, 10 different stores and 50 different requests a 3-month demand forecast for its product.

Demand-Forecasting Business Problem A chain of stores, 10 different stores and 50 different requests a 3-month demand forecast for its product.

Ayşe Nur Türkaslan 3 Mar 06, 2022