{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.9.0"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "id": "223a0201",
   "metadata": {},
   "source": "# Fair Code \u2014 Audit 06: Healthcare Readmission \u2014 Clinical Bias\n\n> *A hospital readmission model flags patients for high clinical risk using payer code and discharge destination \u2014 variables that measure insurance access, not medical severity.*\n\n**Dataset:** Diabetes 130-US Hospitals 1999\u20132008 \u2014 `diabetic_data.csv` (101,766 records)  \n**Protected attributes:** Race, Gender, Age  \n**Proxy variables:** `payer_code` (Medicaid rate differs by race), `discharge_disposition_id` (SNF access differs by race), `medical_specialty` (encodes insurance access), `number_inpatient` (prior hospitalisations encode preventive care gaps)  \n**Fairness metric:** Demographic Parity  \n**Model:** Random Forest Classifier  \n\n---\n\nPipeline:\n1. Load and explore the dataset\n2. Identify the proxy variables (four of them)\n3. Train the biased model (protected attributes + proxies included)\n4. Measure the fairness gaps across three dimensions\n5. Train the fair model (all removed)\n6. Compare results"
  },
  {
   "cell_type": "markdown",
   "id": "1c4a57ea",
   "metadata": {},
   "source": "## 1. Setup"
  },
  {
   "cell_type": "code",
   "id": "c588824d",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as mpatches\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.preprocessing import LabelEncoder\nfrom scipy.stats import chi2_contingency\n\n# Consistent styling across all Fair Code notebooks\nplt.rcParams.update({\n    'figure.facecolor': '#0d0f14',\n    'axes.facecolor': '#131620',\n    'axes.edgecolor': '#1e2130',\n    'axes.labelcolor': '#b0aec0',\n    'xtick.color': '#b0aec0',\n    'ytick.color': '#b0aec0',\n    'text.color': '#d4cfc0',\n    'grid.color': '#1e2130',\n    'grid.linestyle': '--',\n    'font.family': 'monospace',\n    'figure.dpi': 120\n})\n\nACCENT = '#c9a84c'   # Fair Code gold\nDANGER = '#9b2335'   # red \u2014 bias\nSAFE   = '#4a7c6f'   # teal \u2014 mitigated\nMUTED  = '#b0aec0'\n\nprint('Libraries loaded.')"
  },
  {
   "cell_type": "markdown",
   "id": "079ce54b",
   "metadata": {},
   "source": "## 2. Load and Explore the Dataset"
  },
  {
   "cell_type": "code",
   "id": "51a7828c",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "df_raw = pd.read_csv('Healthcare Readmission/diabetic_data.csv')\nprint(f'Dataset: {df_raw.shape[0]:,} rows, {df_raw.shape[1]} columns')\nprint(f'\\nColumns: {list(df_raw.columns)}')\ndf_raw.head(3)"
  },
  {
   "cell_type": "code",
   "id": "3368ba3c",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "df = df_raw.copy()\n\n# Remove invalid entries\ndf = df[~df['race'].isin(['?'])]\ndf = df[df['gender'] != 'Unknown/Invalid']\n\n# Target: 1 = readmitted within 30 days (flagged as high clinical risk)\n#         0 = not readmitted within 30 days\n# In a real hospital tool this flag drives discharge planning and resource allocation\ndf['target'] = (df['readmitted'] == '<30').astype(int)\n\n# Protected attribute flags \u2014 retained for fairness measurement only\ndf['is_female']   = (df['gender'] == 'Female').astype(int)\ndf['is_minority'] = (~df['race'].isin(['Caucasian', 'Asian'])).astype(int)\ndf['age_numeric'] = df['age'].str.extract(r'\\[(\\d+)').astype(int)\ndf['is_elderly']  = (df['age_numeric'] >= 70).astype(int)\n\nprint('Protected group breakdown:')\nprint(f'  Female patients   : {df[\"is_female\"].mean():.1%}')\nprint(f'  Racial minorities : {df[\"is_minority\"].mean():.1%}')\nprint(f'  Elderly (70+)     : {df[\"is_elderly\"].mean():.1%}')\n\nprint(f'\\nOverall 30-day readmission rate: {df[\"target\"].mean():.1%}')\n\nprint('\\n30-day readmission rate by gender (raw data):')\ngender_raw = df.groupby('gender')['target'].mean() * 100\nfor g, v in gender_raw.items():\n    print(f'  {g:<10}: {v:.2f}%')\nprint(f'  Raw gap: {abs(gender_raw[\"Male\"] - gender_raw[\"Female\"]):.2f} percentage points')\n\nprint('\\n30-day readmission rate by race (raw data):')\nrace_raw = df.groupby('is_minority')['target'].mean() * 100\nprint(f'  Caucasian/Asian : {race_raw[0]:.2f}%')\nprint(f'  Other minorities: {race_raw[1]:.2f}%')\nprint(f'  Raw gap: {abs(race_raw[0] - race_raw[1]):.2f} percentage points')"
  },
  {
   "cell_type": "code",
   "id": "7bd4759e",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "fig, axes = plt.subplots(1, 3, figsize=(14, 3.5))\n\n# Readmission rate by gender\ngender_plot = df.groupby('gender')['target'].mean() * 100\nlabels_g = ['Male', 'Female']\nvals_g = [gender_plot['Male'], gender_plot['Female']]\nbars = axes[0].bar(labels_g, vals_g, color=[MUTED, DANGER], width=0.4)\nfor bar, val in zip(bars, vals_g):\n    axes[0].text(bar.get_x() + bar.get_width()/2, val + 0.03,\n                 f'{val:.2f}%', ha='center', color=ACCENT, fontsize=10)\naxes[0].set_title('30-day readmission by gender (raw)', color=MUTED, fontsize=10)\naxes[0].set_ylabel('% readmitted within 30 days')\naxes[0].set_ylim(0, 1.5)\naxes[0].grid(axis='y', alpha=0.3)\n\n# Readmission rate by race\nrace_plot = df.groupby('is_minority')['target'].mean() * 100\nlabels_r = ['Caucasian/Asian', 'Other minorities']\nvals_r = [race_plot[0], race_plot[1]]\nbars2 = axes[1].bar(labels_r, vals_r, color=[MUTED, DANGER], width=0.4)\nfor bar, val in zip(bars2, vals_r):\n    axes[1].text(bar.get_x() + bar.get_width()/2, val + 0.03,\n                 f'{val:.2f}%', ha='center', color=ACCENT, fontsize=10)\naxes[1].set_title('30-day readmission by race (raw)', color=MUTED, fontsize=10)\naxes[1].set_ylabel('% readmitted within 30 days')\naxes[1].set_ylim(0, 1.5)\naxes[1].grid(axis='y', alpha=0.3)\n\n# Readmission rate by age group\nage_plot = df.groupby('is_elderly')['target'].mean() * 100\nlabels_a = ['Under 70', '70+ (elderly)']\nvals_a = [age_plot[0], age_plot[1]]\nbars3 = axes[2].bar(labels_a, vals_a, color=[MUTED, DANGER], width=0.4)\nfor bar, val in zip(bars3, vals_a):\n    axes[2].text(bar.get_x() + bar.get_width()/2, val + 0.03,\n                 f'{val:.2f}%', ha='center', color=ACCENT, fontsize=10)\naxes[2].set_title('30-day readmission by age (raw)', color=MUTED, fontsize=10)\naxes[2].set_ylabel('% readmitted within 30 days')\naxes[2].set_ylim(0, 1.5)\naxes[2].grid(axis='y', alpha=0.3)\n\nplt.tight_layout()\nplt.show()\nprint('Note: the raw gaps exist in the data before any model is trained.')"
  },
  {
   "cell_type": "markdown",
   "id": "64ebd294",
   "metadata": {},
   "source": "## 3. Identify the Proxy Variables\n\nA proxy variable correlates with a protected attribute strongly enough to smuggle the bias back through the model \u2014 even after the protected column is removed.\n\nThis audit has **four** proxy variables:\n\n| Proxy | Protected attribute | Mechanism |\n|---|---|---|\n| `payer_code` | Race | Medicaid rates: Hispanic 9.0%, AfricanAmerican 5.5%, Caucasian 2.7%. Insurance type encodes income, which is racially stratified. |\n| `discharge_disposition_id` | Race | SNF discharge rates: Caucasian 17.3% vs AfricanAmerican 10.7%. Post-acute care access is determined by insurance and geography, not clinical need. |\n| `medical_specialty` | Race, Income | Specialty access is stratified by insurance type and zip code. |\n| `number_inpatient` | Race | AfricanAmerican patients average 0.70 prior inpatient visits vs 0.48 for Asian patients \u2014 a gap driven by preventive care access, not clinical severity alone. |"
  },
  {
   "cell_type": "code",
   "id": "7a7c1651",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def check_proxy(df, feature, protected_col):\n    \"\"\"Chi-squared test of independence. p < 0.05 = likely proxy.\"\"\"\n    contingency = pd.crosstab(df[feature], df[protected_col])\n    chi2, p, dof, _ = chi2_contingency(contingency)\n    return {'feature': feature, 'protected_attr': protected_col,\n            'p_value': round(p, 6), 'is_proxy': p < 0.05}\n\n# Test categorical proxies against race\nfor feat in ['payer_code', 'medical_specialty']:\n    r = check_proxy(df, feat, 'race')\n    print(f\"{r['feature']:<25} p={r['p_value']:<12} proxy={r['is_proxy']}\")\n\n# Test discharge_disposition_id against race\nr = check_proxy(df, 'discharge_disposition_id', 'race')\nprint(f\"{r['feature']:<25} p={r['p_value']:<12} proxy={r['is_proxy']}\")\n\n# number_inpatient \u2014 continuous, use correlation\ninpt_corr = df[['number_inpatient', 'is_minority']].corr().iloc[0, 1]\nprint(f'number_inpatient          Pearson r={inpt_corr:.4f} (positive = minorities have higher prior counts)')"
  },
  {
   "cell_type": "code",
   "id": "5b8968db",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# payer_code x race \u2014 Medicaid rate\nprint('Medicaid payer rate by race:')\ndf['is_medicaid'] = (df['payer_code'] == 'MD').astype(int)\nmedicaid_race = df.groupby('race')['is_medicaid'].mean().round(3)\nfor r, v in medicaid_race.items():\n    print(f'  {r:<22} {v:.1%}')\nprint('  \u2192 Medicaid encodes low income, which is racially stratified.')\nprint('    Hispanic: 9.0%, AfricanAmerican: 5.5%, Caucasian: 2.7%')\n\n# discharge_disposition_id x race \u2014 SNF access\nprint('\\nSNF (skilled nursing) discharge rate by race:')\ndf['discharged_to_snf'] = df['discharge_disposition_id'].isin([2, 3]).astype(int)\nsnf_race = df.groupby('race')['discharged_to_snf'].mean().round(3)\nfor r, v in snf_race.items():\n    print(f'  {r:<22} {v:.1%}')\nprint('  \u2192 SNF access requires insurance and nearby facility.')\nprint('    Caucasian: 17.3% vs AfricanAmerican: 10.7%.')\nprint('    Lower SNF access \u2192 higher home readmission risk \u2014 encoding')\nprint('    structural inequality as individual clinical risk.')\n\n# number_inpatient x race \u2014 prior hospitalisation gap\nprint('\\nMean prior inpatient visits by race:')\nprior_in = df.groupby('race')['number_inpatient'].mean().round(3)\nfor r, v in prior_in.items():\n    print(f'  {r:<22} {v:.3f}')\nprint('  \u2192 AfricanAmerican patients average 0.70 prior visits vs')\nprint('    0.48 for Asian patients. Gap reflects differential access')\nprint('    to preventive care, not higher inherent clinical risk.')"
  },
  {
   "cell_type": "code",
   "id": "31b4f354",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "fig, axes = plt.subplots(1, 2, figsize=(12, 4))\nfig.suptitle('Proxy variable analysis \u2014 what the model learns without being told', color=ACCENT, fontsize=12, y=1.02)\n\n# Medicaid rate by race\nmed_plot = df.groupby('race')['is_medicaid'].mean().sort_values() * 100\nbar_colors_m = [DANGER if r not in ['Caucasian', 'Asian'] else MUTED for r in med_plot.index]\naxes[0].barh(med_plot.index, med_plot.values, color=bar_colors_m, alpha=0.85)\naxes[0].set_xlabel('% on Medicaid')\naxes[0].set_title('Medicaid rate by race \u2014 payer_code is a race proxy', color=MUTED, fontsize=10)\naxes[0].grid(axis='x', alpha=0.3)\n\n# SNF discharge rate by race\nsnf_plot = df.groupby('race')['discharged_to_snf'].mean().sort_values() * 100\nbar_colors_s = [DANGER if r not in ['Caucasian', 'Asian'] else MUTED for r in snf_plot.index]\naxes[1].barh(snf_plot.index, snf_plot.values, color=bar_colors_s, alpha=0.85)\naxes[1].set_xlabel('% discharged to SNF')\naxes[1].set_title('SNF discharge rate by race \u2014 discharge_disposition is a race proxy', color=MUTED, fontsize=10)\naxes[1].grid(axis='x', alpha=0.3)\n\nplt.tight_layout()\nplt.show()"
  },
  {
   "cell_type": "markdown",
   "id": "bc9e4d89",
   "metadata": {},
   "source": "## 4. Train the Biased Model\n\nFeatures include `race`, `gender`, and `age` directly **and** all four proxy variables."
  },
  {
   "cell_type": "code",
   "id": "13682e05",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Encode categoricals\ncat_cols = [\n    'race', 'gender', 'age', 'payer_code', 'medical_specialty',\n    'diag_1', 'diag_2', 'diag_3', 'max_glu_serum', 'A1Cresult',\n    'metformin', 'insulin', 'change', 'diabetesMed'\n]\ndf_enc = df.copy()\nle = LabelEncoder()\nfor col in cat_cols:\n    df_enc[col] = le.fit_transform(df_enc[col].astype(str))\n\nprint('Categorical columns encoded.')"
  },
  {
   "cell_type": "code",
   "id": "195e0026",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "biased_features = [\n    'race',                      # protected attribute \u2717\n    'gender',                    # protected attribute \u2717\n    'age',                       # protected attribute \u2717\n    'payer_code',                # proxy: encodes income + race \u2717\n    'discharge_disposition_id',  # proxy: encodes insurance/wealth \u2717\n    'medical_specialty',         # proxy: encodes insurance access \u2717\n    'number_inpatient',          # proxy: encodes preventive care access \u2717\n    'admission_type_id',\n    'admission_source_id',\n    'time_in_hospital',\n    'num_lab_procedures',\n    'num_procedures',\n    'num_medications',\n    'number_outpatient',\n    'number_emergency',\n    'number_diagnoses',\n    'max_glu_serum',\n    'A1Cresult',\n    'insulin',\n    'change',\n    'diabetesMed',\n    'diag_1',\n    'diag_2',\n    'diag_3',\n]\n\nX_biased = df_enc[biased_features]\ny = df_enc['target']\n\nX_train, X_test, y_train, y_test = train_test_split(\n    X_biased, y, test_size=0.2, random_state=42\n)\n\nbiased_model = RandomForestClassifier(n_estimators=100, random_state=42)\nbiased_model.fit(X_train, y_train)\nbiased_preds    = biased_model.predict(X_test)\nbiased_accuracy = accuracy_score(y_test, biased_preds)\n\nresults_b = X_test.copy()\nresults_b['pred']        = biased_preds\nresults_b['is_female']   = df.loc[X_test.index, 'is_female'].values\nresults_b['is_minority'] = df.loc[X_test.index, 'is_minority'].values\nresults_b['is_elderly']  = df.loc[X_test.index, 'is_elderly'].values\n\nsex_b  = results_b.groupby('is_female')['pred'].mean() * 100\nrace_b = results_b.groupby('is_minority')['pred'].mean() * 100\nage_b  = results_b.groupby('is_elderly')['pred'].mean() * 100\n\nprint(f'Model Accuracy: {biased_accuracy:.2%}')\nprint()\nprint('\u2014 High-Risk Flag Rate by Gender \u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014')\nprint(f'  Male patients      : {sex_b[0]:.2f}% flagged high-risk')\nprint(f'  Female patients    : {sex_b[1]:.2f}% flagged high-risk')\nsex_gap_b = abs(sex_b[0] - sex_b[1])\nprint(f'\\n  Fairness Gap (Gender): {sex_gap_b:.2f}%')\nprint()\nprint('\u2014 High-Risk Flag Rate by Race \u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014')\nprint(f'  Caucasian/Asian    : {race_b[0]:.2f}% flagged high-risk')\nprint(f'  Other minorities   : {race_b[1]:.2f}% flagged high-risk')\nrace_gap_b = abs(race_b[0] - race_b[1])\nprint(f'\\n  Fairness Gap (Race): {race_gap_b:.2f}%')\nprint()\nprint('\u2014 High-Risk Flag Rate by Age \u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014')\nprint(f'  Under 70           : {age_b[0]:.2f}% flagged high-risk')\nprint(f'  70+ (elderly)      : {age_b[1]:.2f}% flagged high-risk')\nage_gap_b = abs(age_b[0] - age_b[1])\nprint(f'\\n  Fairness Gap (Age): {age_gap_b:.2f}%')"
  },
  {
   "cell_type": "markdown",
   "id": "db9530e6",
   "metadata": {},
   "source": "## 5. Train the Fair Model\n\nAll three protected attributes **and** all four proxy variables are removed. Only objective clinical signals from the current admission remain."
  },
  {
   "cell_type": "code",
   "id": "6fb410b8",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "fair_features = [\n    # race                    removed (protected attribute)\n    # gender                  removed (protected attribute)\n    # age                     removed (protected attribute)\n    # payer_code              removed (proxy: encodes income + race)\n    # discharge_disposition_id removed (proxy: encodes post-acute access)\n    # medical_specialty       removed (proxy: encodes insurance/geography)\n    # number_inpatient        removed (proxy: encodes preventive care access gap)\n    'admission_type_id',       # retained: emergency vs elective admission\n    'admission_source_id',     # retained: ER vs referral vs transfer\n    'time_in_hospital',        # retained: length of stay \u2014 severity proxy\n    'num_lab_procedures',      # retained: diagnostic intensity this visit\n    'num_procedures',          # retained: clinical procedures this visit\n    'num_medications',         # retained: medication burden this visit\n    'number_outpatient',       # retained: outpatient visits\n    'number_emergency',        # retained: emergency visits (acute events)\n    'number_diagnoses',        # retained: comorbidity count\n    'max_glu_serum',           # retained: glucose reading this admission\n    'A1Cresult',               # retained: HbA1c \u2014 direct diabetes control measure\n    'insulin',                 # retained: treatment decision this visit\n    'change',                  # retained: medication change flag\n    'diabetesMed',             # retained: on diabetes medication flag\n    'diag_1',                  # retained: primary ICD diagnosis code\n    'diag_2',                  # retained: secondary ICD diagnosis code\n    'diag_3',                  # retained: tertiary ICD diagnosis code\n]\n\n# Encode only the retained categorical columns for fair model\ncat_cols_fair = [\n    'diag_1', 'diag_2', 'diag_3',\n    'max_glu_serum', 'A1Cresult',\n    'insulin', 'change', 'diabetesMed'\n]\ndf_enc_fair = df.copy()\nle2 = LabelEncoder()\nfor col in cat_cols_fair:\n    df_enc_fair[col] = le2.fit_transform(df_enc_fair[col].astype(str))\n\nX_fair = df_enc_fair[fair_features]\n\nX_train_f, X_test_f, y_train_f, y_test_f = train_test_split(\n    X_fair, y, test_size=0.2, random_state=42\n)\n\nfair_model = RandomForestClassifier(n_estimators=100, random_state=42)\nfair_model.fit(X_train_f, y_train_f)\nfair_preds    = fair_model.predict(X_test_f)\nfair_accuracy = accuracy_score(y_test_f, fair_preds)\n\nresults_f = X_test_f.copy()\nresults_f['pred']        = fair_preds\nresults_f['is_female']   = df.loc[X_test_f.index, 'is_female'].values\nresults_f['is_minority'] = df.loc[X_test_f.index, 'is_minority'].values\nresults_f['is_elderly']  = df.loc[X_test_f.index, 'is_elderly'].values\n\nsex_f  = results_f.groupby('is_female')['pred'].mean() * 100\nrace_f = results_f.groupby('is_minority')['pred'].mean() * 100\nage_f  = results_f.groupby('is_elderly')['pred'].mean() * 100\n\nprint(f'Model Accuracy: {fair_accuracy:.2%}')\nprint()\nprint('\u2014 High-Risk Flag Rate by Gender \u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014')\nprint(f'  Male patients      : {sex_f[0]:.2f}% flagged high-risk')\nprint(f'  Female patients    : {sex_f[1]:.2f}% flagged high-risk')\nsex_gap_f = abs(sex_f[0] - sex_f[1])\nprint(f'\\n  New Fairness Gap (Gender): {sex_gap_f:.2f}%')\nprint()\nprint('\u2014 High-Risk Flag Rate by Race \u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014')\nprint(f'  Caucasian/Asian    : {race_f[0]:.2f}% flagged high-risk')\nprint(f'  Other minorities   : {race_f[1]:.2f}% flagged high-risk')\nrace_gap_f = abs(race_f[0] - race_f[1])\nprint(f'\\n  New Fairness Gap (Race): {race_gap_f:.2f}%')\nprint()\nprint('\u2014 High-Risk Flag Rate by Age \u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014')\nprint(f'  Under 70           : {age_f[0]:.2f}% flagged high-risk')\nprint(f'  70+ (elderly)      : {age_f[1]:.2f}% flagged high-risk')\nage_gap_f = abs(age_f[0] - age_f[1])\nprint(f'\\n  New Fairness Gap (Age): {age_gap_f:.2f}%')"
  },
  {
   "cell_type": "markdown",
   "id": "1468176f",
   "metadata": {},
   "source": "## 6. Compare Results"
  },
  {
   "cell_type": "code",
   "id": "048ae6ac",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Recompute gaps using the actual scalar values from each model's results\ngap_sex_b  = abs(sex_b[0]  - sex_b[1])\ngap_sex_f  = abs(sex_f[0]  - sex_f[1])\ngap_race_b = abs(race_b[0] - race_b[1])\ngap_race_f = abs(race_f[0] - race_f[1])\ngap_age_b  = abs(age_b[0]  - age_b[1])\ngap_age_f  = abs(age_f[0]  - age_f[1])\n\n# Gender gap increased \u2014 show direction correctly\ndef gap_change(before, after):\n    if before == 0:\n        return 'N/A'\n    pct = (after - before) / before * 100\n    if pct < 0:\n        return f'{abs(pct):.1f}% reduction'\n    else:\n        return f'+{pct:.1f}% increase'\n\nfig, axes = plt.subplots(1, 3, figsize=(15, 4))\nfig.suptitle('Healthcare Readmission \u2014 Biased vs Mitigated Model', color=ACCENT, fontsize=13, y=1.02)\n\ndims = [\n    (axes[0], [sex_b[0],  sex_b[1]],  [sex_f[0],  sex_f[1]],\n     ['Male', 'Female'],               gap_sex_b,  gap_sex_f,  'Gender'),\n    (axes[1], [race_b[0], race_b[1]], [race_f[0], race_f[1]],\n     ['Caucasian/Asian', 'Minorities'], gap_race_b, gap_race_f, 'Race'),\n    (axes[2], [age_b[0],  age_b[1]],  [age_f[0],  age_f[1]],\n     ['Under 70', '70+ elderly'],      gap_age_b,  gap_age_f,  'Age'),\n]\n\nfor ax, vals_b, vals_f, groups, gb, gf, label in dims:\n    x = np.arange(len(groups))\n    width = 0.35\n    bars_b = ax.bar(x - width/2, vals_b, width, color=DANGER, label='Biased',    alpha=0.85)\n    bars_f = ax.bar(x + width/2, vals_f, width, color=SAFE,   label='Mitigated', alpha=0.85)\n    for bar, val in list(zip(bars_b, vals_b)) + list(zip(bars_f, vals_f)):\n        ax.text(bar.get_x() + bar.get_width()/2, val + 0.005,\n                f'{val:.2f}%', ha='center', fontsize=9, color=ACCENT)\n    ax.set_xticks(x)\n    ax.set_xticklabels(groups, fontsize=9)\n    ax.set_ylim(0, max(max(vals_b), max(vals_f)) * 1.4)\n    ax.set_ylabel('% flagged high-risk')\n    ax.set_title(f'{label}: {gb:.2f}% \u2192 {gf:.2f}%', color=MUTED, fontsize=10)\n    ax.legend(fontsize=8)\n    ax.grid(axis='y', alpha=0.3)\n\nplt.tight_layout()\nplt.show()\n\nprint(f'\\nSummary')\nprint(f'-------')\nprint(f'Gender gap  before: {gap_sex_b:.2f}%   after: {gap_sex_f:.2f}%   {gap_change(gap_sex_b, gap_sex_f)}')\nprint(f'Race gap    before: {gap_race_b:.2f}%   after: {gap_race_f:.2f}%   {gap_change(gap_race_b, gap_race_f)}')\nprint(f'Age gap     before: {gap_age_b:.2f}%   after: {gap_age_f:.2f}%   {gap_change(gap_age_b, gap_age_f)}')\nprint()\nprint('Note: gender gap increased slightly after proxy removal.')\nprint('Proxy variables did not carry strong gender signal \u2014 their')\nprint('removal shifted prediction patterns in a way that widened')\nprint('the gender gap by ~0.02pp. Race and age gaps reduced substantially.')"
  },
  {
   "cell_type": "markdown",
   "id": "9be9912d",
   "metadata": {},
   "source": "## Key Insight\n\nRemoving `race`, `gender`, and `age` alone is not enough. Four proxy variables reconstruct protected-class signal even after the demographic columns are gone.\n\n`payer_code` is the most structurally loaded: Medicaid status encodes poverty, and poverty is racially stratified by decades of structural inequality in employment and insurance access. `discharge_disposition_id` encodes whether a patient can access a skilled nursing facility \u2014 a function of insurance coverage and geographic proximity, not clinical trajectory. Where a patient goes after discharge depends on what their insurer will cover and whether there is a facility nearby: a model that learns \"this patient goes home \u2192 higher readmission risk\" is encoding a resource gap as a patient-level risk factor. `medical_specialty` access is determined upstream by insurance and geography. And `number_inpatient` carries racial signal because AfricanAmerican patients average 0.70 prior hospitalisations vs 0.48 for Asian patients \u2014 a gap that reflects underinvestment in preventive care in minority communities, not higher individual clinical risk.\n\nOne result is honest and worth noting: the gender gap increased slightly (from 0.02% to 0.04%) after proxy removal. The proxy variables removed did not carry significant gender signal \u2014 their removal shifted the model's prediction landscape in a way that slightly widened the male/female gap. This is a real outcome of this specific mitigation, not an artefact. Race and age gaps reduced substantially: 25% and 68% respectively.\n\n**The fix:** Drop everything that encodes structural inequality as individual risk. Retain only the clinical record of this admission \u2014 diagnosis codes, lab and procedure counts, glucose control, medication management, and how the patient arrived. These are the signals a clinician would use to assess readmission risk without reference to who the patient is demographically.\n\n---\n\n*Part of the [Fair Code project](https://github.com/yakew7/Fair-Code) by [@thefaircodeproject](https://instagram.com/thefaircodeproject)*"
  }
 ]
}