reluMIP
Embed ReLU neural networks into mixed-integer programs.
About
With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified liner unit (ReLU) activation function. At the moment, only TensorFlow sequential models are supported. Interfaces to either the Pyomo or Gurobi modelling environments are offered.
ReLU ANNs can be used to approximate complex functions from data. In order to embed these functions into optimization problems, strong formulations of the network are needed. This package employs progressive bound tightening procedures to produce MIP encodings for ReLU networks. This allows the user to embed complex and nonlinear functions into mixed-integer programs. Note that the training of ReLU ANNs is not part of this package and has to be done by the user beforehand. A number of illustrative examples are provided to showcase the functionality of this package. For more detailed information, see the slide show in the docs/ folder of the reluMIP Git repository.
Installation
This package is part of PyPI. It can be installed through pip:
pip install reluMIP
After installing, you can use the examples provided in the examples/ folder to become familiar with the package.
Alternatively, you can clone the github repository:
git clone https://github.com/ChemEngAI/ReLU_ANN_MILP.git
You can install all requirements from the project root folder by calling:
pip install -r requirements.txt
You can add the root folder of the repository to your PYTHON_PATH, so that the package can be accessed from anywhere.
Note that in order to use the package, a compatible solver has to be installed. This can be Gurobi (with a valid license) or any MIP solver compatible with Pyomo (we recommend glpk). In our experience, the best performance is obtained when using the Gurobi interface.
Example usages
Two jupyter notebooks describing the use of the package are supplied in the examples/ folder of the Git repository. There, an MIP formulation of a ReLU ANN - trained on a nonliner, nonconvex function - is used to find the global minimum of the network response surface. If you installed the package through pip, you can simply download the example files that you are interested in.
Related work on NLP formulations of ANNs
In this tool, ReLU ANNs are formulated as MILPs. Notably, ANNs can also be formulated as nonlinear problems (NLPs) and solved through deterministic gloabl optimization (Schweidtmann and Mitsos (2019)). Please visit the MeLOn toolbox for more information.
How to cite this work
Please cite our Zenodo DOI if you use this code:
@software{reluMIP.2021,
title={reluMIP: Open Source Tool for MILP Optimization of ReLU Neural Networks},
author={Lueg, Laurens and Grimstad, Bjarne and Mitsos, Alexander and Schweidtmann, Artur M.},
year={2021},
doi={https://doi.org/10.5281/zenodo.5601907},
url = {https://github.com/ChemEngAI/ReLU_ANN_MILP},
version = {1.0.0}
}
References
Grimstad, B., Andersson, H. (2019). ReLU networks as surrogate models in mixed-integer linear programs. Computers & Chemical Engineering (Volume 131, 106580).
Schweidtmann, A. M., & Mitsos, A. (2019). Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications (Volume 180(3), 925-948).
4 Dec 26, 2021
159 Dec 23, 2022
4 Dec 3, 2022
1 Feb 23, 2022
5 Nov 23, 2022
5 Oct 9, 2022
1 Jan 6, 2022
47 Dec 30, 2022
13 Dec 23, 2022
23 Dec 02, 2022
2 Jan 16, 2022
1.7k Jan 06, 2023
36 Oct 30, 2022
324 Dec 27, 2022
75 Nov 24, 2022
11 Sep 25, 2022
46 Dec 27, 2022
18 Nov 01, 2022
440 Dec 18, 2022
159 Dec 21, 2022
38 Jan 02, 2023
47 Dec 18, 2022
82 Nov 17, 2022
19 Dec 15, 2022
3.2k Dec 23, 2022
967 Jan 04, 2023
95 Nov 25, 2022
26 Dec 23, 2022
48 Dec 08, 2022