Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Last update: Dec 29, 2021

Overview

Constrained Logistic Regression

Sample implementation of constructing a logistic regression with given ranges on each of the feature's coefficients (via clogistic library).

The Data

We will use the processed version of telco customer churn data from Kaggle. The data can be downloaded here.

Steps

Define the constraints

For example:

# define constraints as dataframe
import numpy as np
constraint_df = pd.DataFrame(data=[
                                   ['gender',-np.inf,np.inf],
                                   ['SeniorCitizen',-np.inf,np.inf],
                                   ['Partner',-np.inf, 0],
                                   ['Dependents',-np.inf,0],
                                   ['tenure',-np.inf,0],
                                   ['PhoneService',-np.inf,0],
                                   ['PaperlessBilling',-np.inf,np.inf],
                                   ['MonthlyCharges',-np.inf,np.inf],
                                   ['intercept',-np.inf,np.inf]],
                             columns=['feature','lower_bound','upper_bound'])
constraint_df

|    | feature          |   lower_bound |   upper_bound |
|---:|:-----------------|--------------:|--------------:|
|  0 | gender           |          -inf |           inf |
|  1 | SeniorCitizen    |          -inf |           inf |
|  2 | Partner          |          -inf |             0 |
|  3 | Dependents       |          -inf |             0 |
|  4 | tenure           |          -inf |             0 |
|  5 | PhoneService     |          -inf |             0 |
|  6 | PaperlessBilling |          -inf |           inf |
|  7 | MonthlyCharges   |          -inf |           inf |
|  8 | intercept        |          -inf |           inf |

Model training via clogistic

# train using clogistic
from scipy.optimize import Bounds
from clogistic import LogisticRegression as clLogisticRegression

lower_bounds = constraint_df['lower_bound'].to_numpy()
upper_bounds = constraint_df['upper_bound'].to_numpy()
bounds = Bounds(lower_bounds, upper_bounds)

cl_logreg = clLogisticRegression(penalty='none')
cl_logreg.fit(X_train, y_train, bounds=bounds)

Retrieve the model coefficients

# coefficients as dataframe
cl_coef = pd.DataFrame({
    'feature': df.drop(columns='Churn').columns.tolist() + ['intercept'],
    'coefficient': list(cl_logreg.coef_[0]) + [cl_logreg.intercept_[0]]
})

cl_coef

|    | feature          |   coefficient |
|---:|:-----------------|--------------:|
|  0 | gender           |   0.0184168   |
|  1 | SeniorCitizen    |   0.506692    |
|  2 | Partner          |   3.85603e-09 |
|  3 | Dependents       |  -0.35721     |
|  4 | tenure           |  -0.0557211   |
|  5 | PhoneService     |  -0.796233    |
|  6 | PaperlessBilling |   0.398824    |
|  7 | MonthlyCharges   |   0.033197    |
|  8 | intercept        |  -1.36086     |

Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Related tags

Overview

Constrained Logistic Regression

The Data

Steps

Define the constraints

Model training via clogistic

Owner

Social Fabric: Tubelet Compositions for Video Relation Detection

OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

SysWhispers Shellcode Loader

This repository provides an unified frameworks to train and test the state-of-the-art few-shot font generation (FFG) models.

RL and distillation in CARLA using a factorized world model

PyTorch implementation of DARDet: A Dense Anchor-free Rotated Object Detector in Aerial Images

Official page of Patchwork (RA-L'21 w/ IROS'21)

Final project code: Implementing BicycleGAN, for CIS680 FA21 at University of Pennsylvania

Turn based roguelike in python

DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort

A Python implementation of the Locality Preserving Matching (LPM) method for pruning outliers in image matching.

DynaTune: Dynamic Tensor Program Optimization in Deep Neural Network Compilation

An extremely simple, intuitive, hardware-friendly, and well-performing network structure for LiDAR semantic segmentation on 2D range image. IROS21

这是一个yolox-pytorch的源码，可以用于训练自己的模型。

Check out the StyleGAN repo and place it in the same directory hierarchy as the present repo

Learning Confidence for Out-of-Distribution Detection in Neural Networks

Low-dose Digital Mammography with Deep Learning

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset of 53,000+ Legal Holdings

Discovering Explanatory Sentences in Legal Case Decisions Using Pre-trained Language Models.

A hobby project which includes a hand-gesture based virtual piano using a mobile phone camera and OpenCV library functions