Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Last update: Dec 29, 2021

Overview

Constrained Logistic Regression

Sample implementation of constructing a logistic regression with given ranges on each of the feature's coefficients (via clogistic library).

The Data

We will use the processed version of telco customer churn data from Kaggle. The data can be downloaded here.

Steps

Define the constraints

For example:

# define constraints as dataframe
import numpy as np
constraint_df = pd.DataFrame(data=[
                                   ['gender',-np.inf,np.inf],
                                   ['SeniorCitizen',-np.inf,np.inf],
                                   ['Partner',-np.inf, 0],
                                   ['Dependents',-np.inf,0],
                                   ['tenure',-np.inf,0],
                                   ['PhoneService',-np.inf,0],
                                   ['PaperlessBilling',-np.inf,np.inf],
                                   ['MonthlyCharges',-np.inf,np.inf],
                                   ['intercept',-np.inf,np.inf]],
                             columns=['feature','lower_bound','upper_bound'])
constraint_df

|    | feature          |   lower_bound |   upper_bound |
|---:|:-----------------|--------------:|--------------:|
|  0 | gender           |          -inf |           inf |
|  1 | SeniorCitizen    |          -inf |           inf |
|  2 | Partner          |          -inf |             0 |
|  3 | Dependents       |          -inf |             0 |
|  4 | tenure           |          -inf |             0 |
|  5 | PhoneService     |          -inf |             0 |
|  6 | PaperlessBilling |          -inf |           inf |
|  7 | MonthlyCharges   |          -inf |           inf |
|  8 | intercept        |          -inf |           inf |

Model training via clogistic

# train using clogistic
from scipy.optimize import Bounds
from clogistic import LogisticRegression as clLogisticRegression

lower_bounds = constraint_df['lower_bound'].to_numpy()
upper_bounds = constraint_df['upper_bound'].to_numpy()
bounds = Bounds(lower_bounds, upper_bounds)

cl_logreg = clLogisticRegression(penalty='none')
cl_logreg.fit(X_train, y_train, bounds=bounds)

Retrieve the model coefficients

# coefficients as dataframe
cl_coef = pd.DataFrame({
    'feature': df.drop(columns='Churn').columns.tolist() + ['intercept'],
    'coefficient': list(cl_logreg.coef_[0]) + [cl_logreg.intercept_[0]]
})

cl_coef

|    | feature          |   coefficient |
|---:|:-----------------|--------------:|
|  0 | gender           |   0.0184168   |
|  1 | SeniorCitizen    |   0.506692    |
|  2 | Partner          |   3.85603e-09 |
|  3 | Dependents       |  -0.35721     |
|  4 | tenure           |  -0.0557211   |
|  5 | PhoneService     |  -0.796233    |
|  6 | PaperlessBilling |   0.398824    |
|  7 | MonthlyCharges   |   0.033197    |
|  8 | intercept        |  -1.36086     |

Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Related tags

Overview

Constrained Logistic Regression

The Data

Steps

Define the constraints

Model training via clogistic

Owner

This is the repository for the paper "Have I done enough planning or should I plan more?"

This is the repository of the NeurIPS 2021 paper "Curriculum Disentangled Recommendation withNoisy Multi-feedback"

Network Enhancement implementation in pytorch

The official pytorch implemention of the CVPR paper "Temporal Modulation Network for Controllable Space-Time Video Super-Resolution".

Repo for WWW 2022 paper: Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval

A small library of 3D related utilities used in my research.

An Unsupervised Detection Framework for Chinese Jargons in the Darknet

Finding an Unsupervised Image Segmenter in each of your Deep Generative Models

CARL provides highly configurable contextual extensions to several well-known RL environments.

👨‍💻 run nanosaur in simulation with Gazebo/Ingnition

PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)

Much faster than SORT(Simple Online and Realtime Tracking), a little worse than SORT

This repo. is an implementation of ACFFNet, which is accepted for in Image and Vision Computing.

a simple, efficient, and intuitive text editor

PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World [ACL 2021]

Some pre-commit hooks for OpenMMLab projects

This code is for our paper "VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers"

Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics.

Bringing Characters to Life with Computer Brains in Unity

Local trajectory planner based on a multilayer graph framework for autonomous race vehicles.