Estimating local and global feature importance scores using DiCE

Summaries of counterfactual examples can be used to estimate importance of features. Intuitively, a feature that is changed more often to generate a proximal counterfactual is an important feature. We use this intuition to build a feature importance score.

This score can be interpreted as a measure of the necessity of a feature to cause a particular model output. That is, if the feature’s value changes, then it is likely that the model’s output class will also change (or the model’s output will significantly change in case of regression model).

Below we show how counterfactuals can be used to provide local feature importance scores for any input, and how those scores can be combined to yield a global importance score for each feature.

[1]:
%load_ext autoreload
%autoreload 2
[2]:
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
import pandas as pd
import numpy as np

Preliminaries: Loading the data and ML model

[3]:
from dice_ml.utils import helpers # helper functions
dataset = helpers.load_adult_income_dataset().sample(5000) # downsampling to reduce ML model fitting time
helpers.get_adult_data_info()
[3]:
{'age': 'age',
 'workclass': 'type of industry (Government, Other/Unknown, Private, Self-Employed)',
 'education': 'education level (Assoc, Bachelors, Doctorate, HS-grad, Masters, Prof-school, School, Some-college)',
 'marital_status': 'marital status (Divorced, Married, Separated, Single, Widowed)',
 'occupation': 'occupation (Blue-Collar, Other/Unknown, Professional, Sales, Service, White-Collar)',
 'race': 'white or other race?',
 'gender': 'male or female?',
 'hours_per_week': 'total work hours per week',
 'income': '0 (<=50K) vs 1 (>50K)'}
[4]:
target = dataset["income"]
# Split data into train and test
from sklearn.model_selection import train_test_split
datasetX = dataset.drop("income", axis=1)
x_train, x_test, y_train, y_test = train_test_split(datasetX,
                                                    target,
                                                    test_size = 0.2,
                                                    random_state=0,
                                                    stratify=target)

numerical=["age", "hours_per_week"]
categorical = x_train.columns.difference(numerical)
from sklearn.compose import ColumnTransformer

# We create the preprocessing pipelines for both numeric and categorical data.
numeric_transformer = Pipeline(steps=[
    ('scaler', StandardScaler())])

categorical_transformer = Pipeline(steps=[
    ('onehot', OneHotEncoder(handle_unknown='ignore'))])

transformations = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, numerical),
        ('cat', categorical_transformer, categorical)])

# Append classifier to preprocessing pipeline.
# Now we have a full prediction pipeline.
clf = Pipeline(steps=[('preprocessor', transformations),
                      ('classifier', RandomForestClassifier())])
model = clf.fit(x_train, y_train)
[5]:
import dice_ml
from dice_ml import Dice

d = dice_ml.Data(dataframe=dataset, continuous_features=['age', 'hours_per_week'], outcome_name='income')
m = dice_ml.Model(model=model, backend="sklearn")

Local feature importance

We first generate counterfactuals for a given input point.

[11]:
exp = Dice(d, m, method="random")
query_instance = x_train[1:2]
e1 = exp.generate_counterfactuals(query_instance, total_CFs=10, desired_range=None,desired_class="opposite",
                                   permitted_range=None, features_to_vary="all")
e1.visualize_as_dataframe(show_only_changes=True)
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 49 Private HS-grad Divorced White-Collar White Female 40 0

Diverse Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 - - - Married - - - - 1
1 - - Bachelors - - - Male - 1
2 - - Masters Married - - - - 1
3 - - - - - - Male 48.0 1
4 - - Masters - - - - 97.0 1
5 - Government - Married - - - - 1
6 - - Masters - - - Male - 1
7 - - - Married - - Male - 1
8 - - Masters Widowed - - - - -
9 - - Prof-school - - - Male - 1

These can now be used to calculate the feature importance scores.

[7]:
imp = exp.local_feature_importance(query_instance, cf_examples_list=e1.cf_examples_list)
print(imp.local_importance)
[{'education': 0.6, 'marital_status': 0.5, 'hours_per_week': 0.3, 'age': 0.2, 'workclass': 0.1, 'occupation': 0.1, 'race': 0.1, 'gender': 0.1}]

Feature importance can also be estimated directly, by leaving the cf_examples_list argument blank.

[8]:
imp = exp.local_feature_importance(query_instance, posthoc_sparsity_param=None)
print(imp.local_importance)
[{'education': 0.6, 'marital_status': 0.6, 'gender': 0.2, 'age': 0.2, 'workclass': 0.1, 'occupation': 0.1, 'race': 0.1, 'hours_per_week': 0.1}]

Global importance

For global importance, we need to generate counterfactuals for a representative sample of the dataset.

[12]:
cobj=exp.global_feature_importance(x_train[0:10], total_CFs=10, posthoc_sparsity_param=None)
print(cobj.summary_importance)
{'age': 0.57, 'marital_status': 0.4, 'education': 0.37, 'occupation': 0.27, 'hours_per_week': 0.23, 'race': 0.16, 'workclass': 0.13, 'gender': 0.13}

Convert the counterfactual output to json

[13]:
json_str = cobj.to_json()
print(json_str)
{
  "cf_examples_list": [
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":35},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Blue-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":50.0,\\\"1\\\":35,\\\"2\\\":59.0,\\\"3\\\":48.0,\\\"4\\\":56.0,\\\"5\\\":59.0,\\\"6\\\":35,\\\"7\\\":35,\\\"8\\\":55.0,\\\"9\\\":35},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Private\\\",\\\"5\\\":\\\"Private\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Government\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\",\\\"1\\\":\\\"HS-grad\\\",\\\"2\\\":\\\"HS-grad\\\",\\\"3\\\":\\\"HS-grad\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"HS-grad\\\",\\\"6\\\":\\\"HS-grad\\\",\\\"7\\\":\\\"Masters\\\",\\\"8\\\":\\\"HS-grad\\\",\\\"9\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Married\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Blue-Collar\\\",\\\"1\\\":\\\"Professional\\\",\\\"2\\\":\\\"White-Collar\\\",\\\"3\\\":\\\"Other\\\\/Unknown\\\",\\\"4\\\":\\\"Blue-Collar\\\",\\\"5\\\":\\\"Blue-Collar\\\",\\\"6\\\":\\\"Professional\\\",\\\"7\\\":\\\"Sales\\\",\\\"8\\\":\\\"Blue-Collar\\\",\\\"9\\\":\\\"Blue-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"Other\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"Other\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"Other\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\",\\\"1\\\":\\\"Male\\\",\\\"2\\\":\\\"Male\\\",\\\"3\\\":\\\"Male\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Male\\\",\\\"6\\\":\\\"Male\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Male\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40,\\\"1\\\":40,\\\"2\\\":40,\\\"3\\\":40,\\\"4\\\":40,\\\"5\\\":40,\\\"6\\\":40,\\\"7\\\":40,\\\"8\\\":40,\\\"9\\\":40},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":1,\\\"7\\\":1,\\\"8\\\":1,\\\"9\\\":1}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":49},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Divorced\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":41.0,\\\"1\\\":49,\\\"2\\\":49,\\\"3\\\":49,\\\"4\\\":49,\\\"5\\\":49,\\\"6\\\":49,\\\"7\\\":49,\\\"8\\\":86.0,\\\"9\\\":49},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Private\\\",\\\"5\\\":\\\"Private\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\",\\\"1\\\":\\\"Prof-school\\\",\\\"2\\\":\\\"HS-grad\\\",\\\"3\\\":\\\"Masters\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"Masters\\\",\\\"6\\\":\\\"HS-grad\\\",\\\"7\\\":\\\"Masters\\\",\\\"8\\\":\\\"Bachelors\\\",\\\"9\\\":\\\"Doctorate\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Divorced\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Divorced\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Widowed\\\",\\\"8\\\":\\\"Divorced\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\",\\\"1\\\":\\\"White-Collar\\\",\\\"2\\\":\\\"Other\\\\/Unknown\\\",\\\"3\\\":\\\"White-Collar\\\",\\\"4\\\":\\\"White-Collar\\\",\\\"5\\\":\\\"White-Collar\\\",\\\"6\\\":\\\"White-Collar\\\",\\\"7\\\":\\\"White-Collar\\\",\\\"8\\\":\\\"White-Collar\\\",\\\"9\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"Other\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\",\\\"1\\\":\\\"Male\\\",\\\"2\\\":\\\"Female\\\",\\\"3\\\":\\\"Female\\\",\\\"4\\\":\\\"Female\\\",\\\"5\\\":\\\"Male\\\",\\\"6\\\":\\\"Female\\\",\\\"7\\\":\\\"Female\\\",\\\"8\\\":\\\"Female\\\",\\\"9\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40,\\\"1\\\":40,\\\"2\\\":40,\\\"3\\\":40,\\\"4\\\":45.0,\\\"5\\\":40,\\\"6\\\":40,\\\"7\\\":40,\\\"8\\\":40,\\\"9\\\":40},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":1,\\\"7\\\":0,\\\"8\\\":1,\\\"9\\\":1}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":57},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"Some-college\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Blue-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":45},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":57,\\\"1\\\":57,\\\"2\\\":47.0,\\\"3\\\":72.0,\\\"4\\\":57,\\\"5\\\":63.0,\\\"6\\\":46.0,\\\"7\\\":57,\\\"8\\\":57,\\\"9\\\":57},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Private\\\",\\\"5\\\":\\\"Private\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Self-Employed\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"Some-college\\\",\\\"1\\\":\\\"Prof-school\\\",\\\"2\\\":\\\"Some-college\\\",\\\"3\\\":\\\"Some-college\\\",\\\"4\\\":\\\"Bachelors\\\",\\\"5\\\":\\\"Some-college\\\",\\\"6\\\":\\\"Some-college\\\",\\\"7\\\":\\\"Doctorate\\\",\\\"8\\\":\\\"Some-college\\\",\\\"9\\\":\\\"Doctorate\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Separated\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Separated\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Married\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Blue-Collar\\\",\\\"1\\\":\\\"Professional\\\",\\\"2\\\":\\\"White-Collar\\\",\\\"3\\\":\\\"Blue-Collar\\\",\\\"4\\\":\\\"Blue-Collar\\\",\\\"5\\\":\\\"Blue-Collar\\\",\\\"6\\\":\\\"Blue-Collar\\\",\\\"7\\\":\\\"Blue-Collar\\\",\\\"8\\\":\\\"Blue-Collar\\\",\\\"9\\\":\\\"Service\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"Other\\\",\\\"5\\\":\\\"Other\\\",\\\"6\\\":\\\"Other\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\",\\\"1\\\":\\\"Male\\\",\\\"2\\\":\\\"Male\\\",\\\"3\\\":\\\"Male\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Male\\\",\\\"6\\\":\\\"Male\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Male\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":93.0,\\\"1\\\":45,\\\"2\\\":45,\\\"3\\\":45,\\\"4\\\":45,\\\"5\\\":45,\\\"6\\\":45,\\\"7\\\":45,\\\"8\\\":27.0,\\\"9\\\":45},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":1,\\\"7\\\":1,\\\"8\\\":1,\\\"9\\\":1}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":37},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":50},\\\"income\\\":{\\\"0\\\":1}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":28.0,\\\"1\\\":37,\\\"2\\\":37,\\\"3\\\":90.0,\\\"4\\\":37,\\\"5\\\":37,\\\"6\\\":84.0,\\\"7\\\":37,\\\"8\\\":37,\\\"9\\\":32.0},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Other\\\\/Unknown\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Private\\\",\\\"5\\\":\\\"Private\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\",\\\"1\\\":\\\"HS-grad\\\",\\\"2\\\":\\\"HS-grad\\\",\\\"3\\\":\\\"HS-grad\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"HS-grad\\\",\\\"6\\\":\\\"HS-grad\\\",\\\"7\\\":\\\"HS-grad\\\",\\\"8\\\":\\\"HS-grad\\\",\\\"9\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Widowed\\\",\\\"1\\\":\\\"Widowed\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Married\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\",\\\"1\\\":\\\"Professional\\\",\\\"2\\\":\\\"White-Collar\\\",\\\"3\\\":\\\"White-Collar\\\",\\\"4\\\":\\\"White-Collar\\\",\\\"5\\\":\\\"White-Collar\\\",\\\"6\\\":\\\"White-Collar\\\",\\\"7\\\":\\\"White-Collar\\\",\\\"8\\\":\\\"White-Collar\\\",\\\"9\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\",\\\"1\\\":\\\"Male\\\",\\\"2\\\":\\\"Male\\\",\\\"3\\\":\\\"Male\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Male\\\",\\\"6\\\":\\\"Male\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Male\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":50,\\\"1\\\":50,\\\"2\\\":5.0,\\\"3\\\":13.0,\\\"4\\\":12.0,\\\"5\\\":38.0,\\\"6\\\":32.0,\\\"7\\\":32.0,\\\"8\\\":44.0,\\\"9\\\":50},\\\"income\\\":{\\\"0\\\":0,\\\"1\\\":0,\\\"2\\\":0,\\\"3\\\":0,\\\"4\\\":0,\\\"5\\\":0,\\\"6\\\":0,\\\"7\\\":0,\\\"8\\\":0,\\\"9\\\":0}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":48},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40},\\\"income\\\":{\\\"0\\\":1}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":20.0,\\\"1\\\":48,\\\"2\\\":18.0,\\\"3\\\":48,\\\"4\\\":51.0,\\\"5\\\":48,\\\"6\\\":48,\\\"7\\\":83.0,\\\"8\\\":25.0,\\\"9\\\":68.0},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Private\\\",\\\"5\\\":\\\"Private\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\",\\\"1\\\":\\\"HS-grad\\\",\\\"2\\\":\\\"HS-grad\\\",\\\"3\\\":\\\"Masters\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"HS-grad\\\",\\\"6\\\":\\\"HS-grad\\\",\\\"7\\\":\\\"HS-grad\\\",\\\"8\\\":\\\"HS-grad\\\",\\\"9\\\":\\\"Masters\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Widowed\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Single\\\",\\\"8\\\":\\\"Widowed\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\",\\\"1\\\":\\\"White-Collar\\\",\\\"2\\\":\\\"Sales\\\",\\\"3\\\":\\\"Professional\\\",\\\"4\\\":\\\"White-Collar\\\",\\\"5\\\":\\\"White-Collar\\\",\\\"6\\\":\\\"White-Collar\\\",\\\"7\\\":\\\"White-Collar\\\",\\\"8\\\":\\\"White-Collar\\\",\\\"9\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\",\\\"1\\\":\\\"Female\\\",\\\"2\\\":\\\"Female\\\",\\\"3\\\":\\\"Female\\\",\\\"4\\\":\\\"Female\\\",\\\"5\\\":\\\"Male\\\",\\\"6\\\":\\\"Female\\\",\\\"7\\\":\\\"Female\\\",\\\"8\\\":\\\"Female\\\",\\\"9\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40,\\\"1\\\":76.0,\\\"2\\\":40,\\\"3\\\":40,\\\"4\\\":40,\\\"5\\\":40,\\\"6\\\":8.0,\\\"7\\\":40,\\\"8\\\":40,\\\"9\\\":40},\\\"income\\\":{\\\"0\\\":0,\\\"1\\\":0,\\\"2\\\":0,\\\"3\\\":0,\\\"4\\\":0,\\\"5\\\":0,\\\"6\\\":0,\\\"7\\\":0,\\\"8\\\":0,\\\"9\\\":0}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":50},\\\"workclass\\\":{\\\"0\\\":\\\"Government\\\"},\\\"education\\\":{\\\"0\\\":\\\"Masters\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Professional\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":38.0,\\\"1\\\":50,\\\"2\\\":50,\\\"3\\\":65.0,\\\"4\\\":83.0,\\\"5\\\":50,\\\"6\\\":43.0,\\\"7\\\":50,\\\"8\\\":50,\\\"9\\\":30.0},\\\"workclass\\\":{\\\"0\\\":\\\"Government\\\",\\\"1\\\":\\\"Government\\\",\\\"2\\\":\\\"Self-Employed\\\",\\\"3\\\":\\\"Government\\\",\\\"4\\\":\\\"Government\\\",\\\"5\\\":\\\"Government\\\",\\\"6\\\":\\\"Government\\\",\\\"7\\\":\\\"Government\\\",\\\"8\\\":\\\"Government\\\",\\\"9\\\":\\\"Government\\\"},\\\"education\\\":{\\\"0\\\":\\\"Masters\\\",\\\"1\\\":\\\"Bachelors\\\",\\\"2\\\":\\\"HS-grad\\\",\\\"3\\\":\\\"Prof-school\\\",\\\"4\\\":\\\"Masters\\\",\\\"5\\\":\\\"Masters\\\",\\\"6\\\":\\\"Masters\\\",\\\"7\\\":\\\"Masters\\\",\\\"8\\\":\\\"Masters\\\",\\\"9\\\":\\\"Masters\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Married\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Professional\\\",\\\"1\\\":\\\"Service\\\",\\\"2\\\":\\\"Professional\\\",\\\"3\\\":\\\"Professional\\\",\\\"4\\\":\\\"Professional\\\",\\\"5\\\":\\\"Professional\\\",\\\"6\\\":\\\"Professional\\\",\\\"7\\\":\\\"Professional\\\",\\\"8\\\":\\\"Sales\\\",\\\"9\\\":\\\"Professional\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"Other\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"Other\\\",\\\"8\\\":\\\"Other\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\",\\\"1\\\":\\\"Female\\\",\\\"2\\\":\\\"Female\\\",\\\"3\\\":\\\"Female\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Female\\\",\\\"6\\\":\\\"Female\\\",\\\"7\\\":\\\"Female\\\",\\\"8\\\":\\\"Female\\\",\\\"9\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40,\\\"1\\\":40,\\\"2\\\":40,\\\"3\\\":40,\\\"4\\\":40,\\\"5\\\":45.0,\\\"6\\\":40,\\\"7\\\":95.0,\\\"8\\\":40,\\\"9\\\":40},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":1,\\\"7\\\":1,\\\"8\\\":1,\\\"9\\\":1}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":34},\\\"workclass\\\":{\\\"0\\\":\\\"Government\\\"},\\\"education\\\":{\\\"0\\\":\\\"Doctorate\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Professional\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":45},\\\"income\\\":{\\\"0\\\":1}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":30.0,\\\"1\\\":34,\\\"2\\\":34,\\\"3\\\":34,\\\"4\\\":34,\\\"5\\\":34,\\\"6\\\":34,\\\"7\\\":34,\\\"8\\\":34,\\\"9\\\":34},\\\"workclass\\\":{\\\"0\\\":\\\"Government\\\",\\\"1\\\":\\\"Government\\\",\\\"2\\\":\\\"Government\\\",\\\"3\\\":\\\"Government\\\",\\\"4\\\":\\\"Government\\\",\\\"5\\\":\\\"Government\\\",\\\"6\\\":\\\"Government\\\",\\\"7\\\":\\\"Government\\\",\\\"8\\\":\\\"Government\\\",\\\"9\\\":\\\"Other\\\\/Unknown\\\"},\\\"education\\\":{\\\"0\\\":\\\"Doctorate\\\",\\\"1\\\":\\\"Doctorate\\\",\\\"2\\\":\\\"Doctorate\\\",\\\"3\\\":\\\"Masters\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"Doctorate\\\",\\\"6\\\":\\\"School\\\",\\\"7\\\":\\\"Doctorate\\\",\\\"8\\\":\\\"Doctorate\\\",\\\"9\\\":\\\"Doctorate\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Separated\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Divorced\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Single\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Widowed\\\",\\\"9\\\":\\\"Married\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Professional\\\",\\\"1\\\":\\\"Professional\\\",\\\"2\\\":\\\"Service\\\",\\\"3\\\":\\\"Professional\\\",\\\"4\\\":\\\"Professional\\\",\\\"5\\\":\\\"Professional\\\",\\\"6\\\":\\\"Sales\\\",\\\"7\\\":\\\"Professional\\\",\\\"8\\\":\\\"Sales\\\",\\\"9\\\":\\\"Service\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"Other\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\",\\\"1\\\":\\\"Male\\\",\\\"2\\\":\\\"Male\\\",\\\"3\\\":\\\"Male\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Male\\\",\\\"6\\\":\\\"Male\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Male\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":45,\\\"1\\\":1.0,\\\"2\\\":45,\\\"3\\\":45,\\\"4\\\":84.0,\\\"5\\\":34.0,\\\"6\\\":45,\\\"7\\\":15.0,\\\"8\\\":45,\\\"9\\\":45},\\\"income\\\":{\\\"0\\\":0,\\\"1\\\":0,\\\"2\\\":0,\\\"3\\\":0,\\\"4\\\":0,\\\"5\\\":0,\\\"6\\\":0,\\\"7\\\":0,\\\"8\\\":0,\\\"9\\\":0}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":18},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"School\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Single\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Service\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":25},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":18,\\\"1\\\":44.0,\\\"2\\\":74.0,\\\"3\\\":83.0,\\\"4\\\":42.0,\\\"5\\\":84.0,\\\"6\\\":42.0,\\\"7\\\":47.0,\\\"8\\\":60.0,\\\"9\\\":79.0},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Other\\\\/Unknown\\\",\\\"5\\\":\\\"Private\\\",\\\"6\\\":\\\"Other\\\\/Unknown\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"Masters\\\",\\\"1\\\":\\\"Some-college\\\",\\\"2\\\":\\\"Bachelors\\\",\\\"3\\\":\\\"Bachelors\\\",\\\"4\\\":\\\"Prof-school\\\",\\\"5\\\":\\\"Masters\\\",\\\"6\\\":\\\"Prof-school\\\",\\\"7\\\":\\\"Bachelors\\\",\\\"8\\\":\\\"Prof-school\\\",\\\"9\\\":\\\"Doctorate\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Single\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Single\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Separated\\\",\\\"9\\\":\\\"Single\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\",\\\"1\\\":\\\"Service\\\",\\\"2\\\":\\\"Service\\\",\\\"3\\\":\\\"Service\\\",\\\"4\\\":\\\"Other\\\\/Unknown\\\",\\\"5\\\":\\\"White-Collar\\\",\\\"6\\\":\\\"Service\\\",\\\"7\\\":\\\"Service\\\",\\\"8\\\":\\\"Professional\\\",\\\"9\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"Other\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Male\\\",\\\"1\\\":\\\"Male\\\",\\\"2\\\":\\\"Male\\\",\\\"3\\\":\\\"Male\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Female\\\",\\\"6\\\":\\\"Male\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Male\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":52.0,\\\"1\\\":25,\\\"2\\\":70.0,\\\"3\\\":25,\\\"4\\\":25.0,\\\"5\\\":25,\\\"6\\\":25.0,\\\"7\\\":25,\\\"8\\\":25,\\\"9\\\":25},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":0,\\\"7\\\":1,\\\"8\\\":1,\\\"9\\\":1}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":21},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Single\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":48.0,\\\"1\\\":31.0,\\\"2\\\":48.0,\\\"3\\\":36.0,\\\"4\\\":54.0,\\\"5\\\":31.0,\\\"6\\\":31.0,\\\"7\\\":71.0,\\\"8\\\":55.0,\\\"9\\\":46.0},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\",\\\"1\\\":\\\"Private\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Private\\\",\\\"5\\\":\\\"Other\\\\/Unknown\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\",\\\"1\\\":\\\"HS-grad\\\",\\\"2\\\":\\\"HS-grad\\\",\\\"3\\\":\\\"Some-college\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"HS-grad\\\",\\\"6\\\":\\\"Doctorate\\\",\\\"7\\\":\\\"Doctorate\\\",\\\"8\\\":\\\"Assoc\\\",\\\"9\\\":\\\"Prof-school\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Married\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Single\\\",\\\"7\\\":\\\"Single\\\",\\\"8\\\":\\\"Married\\\",\\\"9\\\":\\\"Single\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"White-Collar\\\",\\\"1\\\":\\\"White-Collar\\\",\\\"2\\\":\\\"White-Collar\\\",\\\"3\\\":\\\"White-Collar\\\",\\\"4\\\":\\\"White-Collar\\\",\\\"5\\\":\\\"White-Collar\\\",\\\"6\\\":\\\"White-Collar\\\",\\\"7\\\":\\\"White-Collar\\\",\\\"8\\\":\\\"White-Collar\\\",\\\"9\\\":\\\"White-Collar\\\"},\\\"race\\\":{\\\"0\\\":\\\"Other\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"White\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"White\\\",\\\"7\\\":\\\"White\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\",\\\"1\\\":\\\"Female\\\",\\\"2\\\":\\\"Female\\\",\\\"3\\\":\\\"Female\\\",\\\"4\\\":\\\"Male\\\",\\\"5\\\":\\\"Female\\\",\\\"6\\\":\\\"Female\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Female\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40,\\\"1\\\":40,\\\"2\\\":40,\\\"3\\\":40,\\\"4\\\":40,\\\"5\\\":40,\\\"6\\\":72.0,\\\"7\\\":40,\\\"8\\\":40,\\\"9\\\":40},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":1,\\\"7\\\":1,\\\"8\\\":1,\\\"9\\\":1}}\"}",
    "{\"data_interface\": {\"outcome_name\": \"income\", \"data_df\": \"dummy_data\"}, \"model_type\": \"classifier\", \"desired_class\": \"opposite\", \"desired_range\": null, \"test_instance_df\": \"{\\\"age\\\":{\\\"0\\\":20},\\\"workclass\\\":{\\\"0\\\":\\\"Private\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Single\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Service\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40},\\\"income\\\":{\\\"0\\\":0}}\", \"final_cfs_df\": \"{\\\"age\\\":{\\\"0\\\":78.0,\\\"1\\\":45.0,\\\"2\\\":73.0,\\\"3\\\":79.0,\\\"4\\\":63.0,\\\"5\\\":63.0,\\\"6\\\":49.0,\\\"7\\\":49.0,\\\"8\\\":52.0,\\\"9\\\":85.0},\\\"workclass\\\":{\\\"0\\\":\\\"Self-Employed\\\",\\\"1\\\":\\\"Government\\\",\\\"2\\\":\\\"Private\\\",\\\"3\\\":\\\"Private\\\",\\\"4\\\":\\\"Government\\\",\\\"5\\\":\\\"Government\\\",\\\"6\\\":\\\"Private\\\",\\\"7\\\":\\\"Private\\\",\\\"8\\\":\\\"Private\\\",\\\"9\\\":\\\"Government\\\"},\\\"education\\\":{\\\"0\\\":\\\"HS-grad\\\",\\\"1\\\":\\\"HS-grad\\\",\\\"2\\\":\\\"Doctorate\\\",\\\"3\\\":\\\"Prof-school\\\",\\\"4\\\":\\\"HS-grad\\\",\\\"5\\\":\\\"HS-grad\\\",\\\"6\\\":\\\"HS-grad\\\",\\\"7\\\":\\\"HS-grad\\\",\\\"8\\\":\\\"HS-grad\\\",\\\"9\\\":\\\"Prof-school\\\"},\\\"marital_status\\\":{\\\"0\\\":\\\"Married\\\",\\\"1\\\":\\\"Married\\\",\\\"2\\\":\\\"Single\\\",\\\"3\\\":\\\"Married\\\",\\\"4\\\":\\\"Married\\\",\\\"5\\\":\\\"Married\\\",\\\"6\\\":\\\"Married\\\",\\\"7\\\":\\\"Married\\\",\\\"8\\\":\\\"Married\\\",\\\"9\\\":\\\"Single\\\"},\\\"occupation\\\":{\\\"0\\\":\\\"Service\\\",\\\"1\\\":\\\"Blue-Collar\\\",\\\"2\\\":\\\"White-Collar\\\",\\\"3\\\":\\\"Service\\\",\\\"4\\\":\\\"Service\\\",\\\"5\\\":\\\"Other\\\\/Unknown\\\",\\\"6\\\":\\\"Service\\\",\\\"7\\\":\\\"Sales\\\",\\\"8\\\":\\\"Service\\\",\\\"9\\\":\\\"Service\\\"},\\\"race\\\":{\\\"0\\\":\\\"White\\\",\\\"1\\\":\\\"White\\\",\\\"2\\\":\\\"White\\\",\\\"3\\\":\\\"White\\\",\\\"4\\\":\\\"Other\\\",\\\"5\\\":\\\"White\\\",\\\"6\\\":\\\"Other\\\",\\\"7\\\":\\\"Other\\\",\\\"8\\\":\\\"White\\\",\\\"9\\\":\\\"White\\\"},\\\"gender\\\":{\\\"0\\\":\\\"Female\\\",\\\"1\\\":\\\"Female\\\",\\\"2\\\":\\\"Male\\\",\\\"3\\\":\\\"Male\\\",\\\"4\\\":\\\"Female\\\",\\\"5\\\":\\\"Female\\\",\\\"6\\\":\\\"Female\\\",\\\"7\\\":\\\"Male\\\",\\\"8\\\":\\\"Male\\\",\\\"9\\\":\\\"Male\\\"},\\\"hours_per_week\\\":{\\\"0\\\":40,\\\"1\\\":36.0,\\\"2\\\":40,\\\"3\\\":60.0,\\\"4\\\":40,\\\"5\\\":40,\\\"6\\\":40,\\\"7\\\":40,\\\"8\\\":40,\\\"9\\\":40},\\\"income\\\":{\\\"0\\\":1,\\\"1\\\":1,\\\"2\\\":1,\\\"3\\\":1,\\\"4\\\":1,\\\"5\\\":1,\\\"6\\\":1,\\\"7\\\":1,\\\"8\\\":1,\\\"9\\\":0}}\"}"
  ],
  "local_importance": [
    {
      "age": 0.6,
      "occupation": 0.5,
      "race": 0.3,
      "workclass": 0.1,
      "education": 0.1,
      "marital_status": 0.0,
      "gender": 0.0,
      "hours_per_week": 0.0
    },
    {
      "marital_status": 0.7,
      "education": 0.6,
      "gender": 0.2,
      "age": 0.2,
      "occupation": 0.1,
      "race": 0.1,
      "hours_per_week": 0.1,
      "workclass": 0.0
    },
    {
      "education": 0.4,
      "age": 0.4,
      "occupation": 0.3,
      "race": 0.3,
      "marital_status": 0.2,
      "hours_per_week": 0.2,
      "workclass": 0.1,
      "gender": 0.0
    },
    {
      "hours_per_week": 0.7,
      "age": 0.4,
      "marital_status": 0.2,
      "workclass": 0.1,
      "occupation": 0.1,
      "education": 0.0,
      "race": 0.0,
      "gender": 0.0
    },
    {
      "age": 0.6,
      "marital_status": 0.3,
      "education": 0.2,
      "occupation": 0.2,
      "hours_per_week": 0.2,
      "gender": 0.1,
      "workclass": 0.0,
      "race": 0.0
    },
    {
      "age": 0.5,
      "education": 0.3,
      "race": 0.3,
      "occupation": 0.2,
      "hours_per_week": 0.2,
      "workclass": 0.1,
      "gender": 0.1,
      "marital_status": 0.0
    },
    {
      "marital_status": 0.4,
      "occupation": 0.4,
      "hours_per_week": 0.4,
      "education": 0.3,
      "workclass": 0.1,
      "race": 0.1,
      "age": 0.1,
      "gender": 0.0
    },
    {
      "education": 1.0,
      "age": 0.9,
      "marital_status": 0.7,
      "occupation": 0.5,
      "workclass": 0.2,
      "hours_per_week": 0.2,
      "race": 0.1,
      "gender": 0.1
    },
    {
      "age": 1.0,
      "marital_status": 0.7,
      "education": 0.5,
      "gender": 0.3,
      "workclass": 0.1,
      "race": 0.1,
      "hours_per_week": 0.1,
      "occupation": 0.0
    },
    {
      "age": 1.0,
      "marital_status": 0.8,
      "workclass": 0.5,
      "gender": 0.5,
      "occupation": 0.4,
      "education": 0.3,
      "race": 0.3,
      "hours_per_week": 0.2
    }
  ],
  "summary_importance": {
    "age": 0.57,
    "marital_status": 0.4,
    "education": 0.37,
    "occupation": 0.27,
    "hours_per_week": 0.23,
    "race": 0.16,
    "workclass": 0.13,
    "gender": 0.13
  },
  "metadata": {
    "version": "1.0"
  }
}

Convert the json output to a counterfactual object

[14]:
imp_r=imp.from_json(json_str)
print([o.visualize_as_dataframe(show_only_changes=True) for o in imp_r.cf_examples_list])
print(imp_r.local_importance)
print(imp_r.summary_importance)
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 35 Private HS-grad Married Blue-Collar White Male 40 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 50 - - - - - - - 1
1 - - - - Professional Other - - 1
2 59 - - - White-Collar - - - 1
3 48 - - - Other/Unknown - - - 1
4 56 - - - - Other - - 1
5 59 - - - - - - - 1
6 - - - - Professional - - - 1
7 - - Masters - Sales - - - 1
8 55 - - - - - - - 1
9 - Government - - - Other - - 1
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 49 Private HS-grad Divorced White-Collar White Female 40 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 41 - - Married - - - - 1
1 - - Prof-school - - - Male - 1
2 - - - Married Other/Unknown - - - 1
3 - - Masters Married - - - - 1
4 - - - Married - - - 45 1
5 - - Masters - - - Male - 1
6 - - - Married - Other - - 1
7 - - Masters Widowed - - - - -
8 86 - Bachelors - - - - - 1
9 - - Doctorate Married - - - - 1
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 57 Private Some-college Married Blue-Collar White Male 45 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 - - - Separated - - - 93 1
1 - - Prof-school - Professional - - - 1
2 47 - - - White-Collar - - - 1
3 72 - - Separated - - - - 1
4 - - Bachelors - - Other - - 1
5 63 - - - - Other - - 1
6 46 - - - - Other - - 1
7 - Self-Employed Doctorate - - - - - 1
8 - - - - - - - 27 1
9 - - Doctorate - Service - - - 1
Query instance (original outcome : 1)
age workclass education marital_status occupation race gender hours_per_week income
0 37 Private HS-grad Married White-Collar White Male 50 1

Counterfactual set (new outcome: 0.0)
age workclass education marital_status occupation race gender hours_per_week income
0 28 - - Widowed - - - - 0
1 - - - Widowed Professional - - - 0
2 - Other/Unknown - - - - - 5 0
3 90 - - - - - - 13 0
4 - - - - - - - 12 0
5 - - - - - - - 38 0
6 84 - - - - - - 32 0
7 - - - - - - - 32 0
8 - - - - - - - 44 0
9 32 - - - - - - - 0
Query instance (original outcome : 1)
age workclass education marital_status occupation race gender hours_per_week income
0 48 Private HS-grad Married White-Collar White Female 40 1

Counterfactual set (new outcome: 0.0)
age workclass education marital_status occupation race gender hours_per_week income
0 20 - - - - - - - 0
1 - - - - - - - 76 0
2 18 - - - Sales - - - 0
3 - - Masters - Professional - - - 0
4 51 - - - - - - - 0
5 - - - Widowed - - Male - 0
6 - - - - - - - 8 0
7 83 - - Single - - - - 0
8 25 - - Widowed - - - - 0
9 68 - Masters - - - - - 0
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 50 Government Masters Married Professional White Female 40 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 38 - - - - - - - 1
1 - - Bachelors - Service - - - 1
2 - Self-Employed HS-grad - - - - - 1
3 65 - Prof-school - - - - - 1
4 83 - - - - - Male - 1
5 - - - - - Other - 45 1
6 43 - - - - - - - 1
7 - - - - - Other - 95 1
8 - - - - Sales Other - - 1
9 30 - - - - - - - 1
Query instance (original outcome : 1)
age workclass education marital_status occupation race gender hours_per_week income
0 34 Government Doctorate Married Professional White Male 45 1

Counterfactual set (new outcome: 0.0)
age workclass education marital_status occupation race gender hours_per_week income
0 30 - - Separated - - - - 0
1 - - - - - - - 1 0
2 - - - - Service Other - - 0
3 - - Masters Divorced - - - - 0
4 - - HS-grad - - - - 84 0
5 - - - Single - - - 34 0
6 - - School - Sales - - - 0
7 - - - - - - - 15 0
8 - - - Widowed Sales - - - 0
9 - Other/Unknown - - Service - - - 0
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 18 Private School Single Service White Male 25 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 - - Masters Married White-Collar - - 52 1
1 44 - Some-college Married - - - - 1
2 74 - Bachelors Married - Other - 70 1
3 83 - Bachelors Married - - - - 1
4 42 Other/Unknown Prof-school - Other/Unknown - - - 1
5 84 - Masters Married White-Collar - Female - 1
6 42 Other/Unknown Prof-school - - - - - -
7 47 - Bachelors Married - - - - 1
8 60 - Prof-school Separated Professional - - - 1
9 79 - Doctorate - White-Collar - - - 1
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 21 Private HS-grad Single White-Collar White Female 40 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 48 - - Married - Other - - 1
1 31 - - Married - - - - 1
2 48 - - Married - - - - 1
3 36 - Some-college Married - - - - 1
4 54 - - Married - - Male - 1
5 31 Other/Unknown - Married - - - - 1
6 31 - Doctorate - - - - 72 1
7 71 - Doctorate - - - Male - 1
8 55 - Assoc Married - - - - 1
9 46 - Prof-school - - - Male - 1
Query instance (original outcome : 0)
age workclass education marital_status occupation race gender hours_per_week income
0 20 Private HS-grad Single Service White Female 40 0

Counterfactual set (new outcome: 1.0)
age workclass education marital_status occupation race gender hours_per_week income
0 78 Self-Employed - Married - - - - 1
1 45 Government - Married Blue-Collar - - 36 1
2 73 - Doctorate - White-Collar - Male - 1
3 79 - Prof-school Married - - Male 60 1
4 63 Government - Married - Other - - 1
5 63 Government - Married Other/Unknown - - - 1
6 49 - - Married - Other - - 1
7 49 - - Married Sales Other Male - 1
8 52 - - Married - - Male - 1
9 85 Government Prof-school - - - Male - -
[None, None, None, None, None, None, None, None, None, None]
[{'age': 0.6, 'occupation': 0.5, 'race': 0.3, 'workclass': 0.1, 'education': 0.1, 'marital_status': 0.0, 'gender': 0.0, 'hours_per_week': 0.0}, {'marital_status': 0.7, 'education': 0.6, 'gender': 0.2, 'age': 0.2, 'occupation': 0.1, 'race': 0.1, 'hours_per_week': 0.1, 'workclass': 0.0}, {'education': 0.4, 'age': 0.4, 'occupation': 0.3, 'race': 0.3, 'marital_status': 0.2, 'hours_per_week': 0.2, 'workclass': 0.1, 'gender': 0.0}, {'hours_per_week': 0.7, 'age': 0.4, 'marital_status': 0.2, 'workclass': 0.1, 'occupation': 0.1, 'education': 0.0, 'race': 0.0, 'gender': 0.0}, {'age': 0.6, 'marital_status': 0.3, 'education': 0.2, 'occupation': 0.2, 'hours_per_week': 0.2, 'gender': 0.1, 'workclass': 0.0, 'race': 0.0}, {'age': 0.5, 'education': 0.3, 'race': 0.3, 'occupation': 0.2, 'hours_per_week': 0.2, 'workclass': 0.1, 'gender': 0.1, 'marital_status': 0.0}, {'marital_status': 0.4, 'occupation': 0.4, 'hours_per_week': 0.4, 'education': 0.3, 'workclass': 0.1, 'race': 0.1, 'age': 0.1, 'gender': 0.0}, {'education': 1.0, 'age': 0.9, 'marital_status': 0.7, 'occupation': 0.5, 'workclass': 0.2, 'hours_per_week': 0.2, 'race': 0.1, 'gender': 0.1}, {'age': 1.0, 'marital_status': 0.7, 'education': 0.5, 'gender': 0.3, 'workclass': 0.1, 'race': 0.1, 'hours_per_week': 0.1, 'occupation': 0.0}, {'age': 1.0, 'marital_status': 0.8, 'workclass': 0.5, 'gender': 0.5, 'occupation': 0.4, 'education': 0.3, 'race': 0.3, 'hours_per_week': 0.2}]
{'age': 0.57, 'marital_status': 0.4, 'education': 0.37, 'occupation': 0.27, 'hours_per_week': 0.23, 'race': 0.16, 'workclass': 0.13, 'gender': 0.13}