Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Last update: Dec 29, 2021

Overview

Constrained Logistic Regression

Sample implementation of constructing a logistic regression with given ranges on each of the feature's coefficients (via clogistic library).

The Data

We will use the processed version of telco customer churn data from Kaggle. The data can be downloaded here.

Steps

Define the constraints

For example:

# define constraints as dataframe
import numpy as np
constraint_df = pd.DataFrame(data=[
                                   ['gender',-np.inf,np.inf],
                                   ['SeniorCitizen',-np.inf,np.inf],
                                   ['Partner',-np.inf, 0],
                                   ['Dependents',-np.inf,0],
                                   ['tenure',-np.inf,0],
                                   ['PhoneService',-np.inf,0],
                                   ['PaperlessBilling',-np.inf,np.inf],
                                   ['MonthlyCharges',-np.inf,np.inf],
                                   ['intercept',-np.inf,np.inf]],
                             columns=['feature','lower_bound','upper_bound'])
constraint_df

|    | feature          |   lower_bound |   upper_bound |
|---:|:-----------------|--------------:|--------------:|
|  0 | gender           |          -inf |           inf |
|  1 | SeniorCitizen    |          -inf |           inf |
|  2 | Partner          |          -inf |             0 |
|  3 | Dependents       |          -inf |             0 |
|  4 | tenure           |          -inf |             0 |
|  5 | PhoneService     |          -inf |             0 |
|  6 | PaperlessBilling |          -inf |           inf |
|  7 | MonthlyCharges   |          -inf |           inf |
|  8 | intercept        |          -inf |           inf |

Model training via clogistic

# train using clogistic
from scipy.optimize import Bounds
from clogistic import LogisticRegression as clLogisticRegression

lower_bounds = constraint_df['lower_bound'].to_numpy()
upper_bounds = constraint_df['upper_bound'].to_numpy()
bounds = Bounds(lower_bounds, upper_bounds)

cl_logreg = clLogisticRegression(penalty='none')
cl_logreg.fit(X_train, y_train, bounds=bounds)

Retrieve the model coefficients

# coefficients as dataframe
cl_coef = pd.DataFrame({
    'feature': df.drop(columns='Churn').columns.tolist() + ['intercept'],
    'coefficient': list(cl_logreg.coef_[0]) + [cl_logreg.intercept_[0]]
})

cl_coef

|    | feature          |   coefficient |
|---:|:-----------------|--------------:|
|  0 | gender           |   0.0184168   |
|  1 | SeniorCitizen    |   0.506692    |
|  2 | Partner          |   3.85603e-09 |
|  3 | Dependents       |  -0.35721     |
|  4 | tenure           |  -0.0557211   |
|  5 | PhoneService     |  -0.796233    |
|  6 | PaperlessBilling |   0.398824    |
|  7 | MonthlyCharges   |   0.033197    |
|  8 | intercept        |  -1.36086     |

Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Related tags

Overview

Constrained Logistic Regression

The Data

Steps

Define the constraints

Model training via clogistic

Owner

LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION

Pytorch implementation for the EMNLP 2020 (Findings) paper: Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering

git《Investigating Loss Functions for Extreme Super-Resolution》(CVPR 2020) GitHub:

Fuzzing the Kernel Using Unicornafl and AFL++

Implementation of QuickDraw - an online game developed by Google, combined with AirGesture - a simple gesture recognition application

CondenseNet V2: Sparse Feature Reactivation for Deep Networks

Using Clinical Drug Representations for Improving Mortality and Length of Stay Predictions

This is the solution for 2nd rank in Kaggle competition: Feedback Prize - Evaluating Student Writing.

Developed an optimized algorithm which finds the most optimal path between 2 points in a 3D Maze using various AI search techniques like BFS, DFS, UCS, Greedy BFS and A*

Unsupervised Image-to-Image Translation

Code for NeurIPS 2021 paper 'Spatio-Temporal Variational Gaussian Processes'

《Single Image Reflection Removal Beyond Linearity》(CVPR 2019)

Spline is a tool that is capable of running locally as well as part of well known pipelines like Jenkins (Jenkinsfile), Travis CI (.travis.yml) or similar ones.

The modify PyTorch version of Siam-trackers which are speed-up by TensorRT.

This repo holds codes of the ICCV21 paper: Visual Alignment Constraint for Continuous Sign Language Recognition.

Constructing interpretable quadratic accuracy predictors to serve as an objective function for an IQCQP problem that represents NAS under latency constraints and solve it with efficient algorithms.

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

A PyTorch Implementation of Neural IMage Assessment

official implementation for the paper "Simplifying Graph Convolutional Networks"

Roadmap to becoming a machine learning engineer in 2020