Credit Risk Engine

End-to-end retail credit risk on 2.26 million Lending Club loans: WoE/logistic PD scorecard with an XGBoost challenger, LGD and EAD models, IFRS 9 expected credit loss with staging and scenarios, Basel IRB capital, and an auto-generated model validation report.

The engine builds the models a retail bank's credit risk team runs, and validates them the way a model validation team would:

• PD: a points-based scorecard (weight of evidence + logistic regression, built from scratch), validated out of time and against a gradient-boosting challenger.
• LGD from realised recoveries, and EAD from amortisation schedules and default timing.
• IFRS 9 ECL on the 583,000 loans still open at the data snapshot, with 3-stage allocation, lifetime PD term structures and probability-weighted scenarios.
• Basel IRB risk-weighted assets and capital.
• A 15-section validation report where every number comes from the run, plus a Streamlit dashboard that scores new applicants live.

---

Contents

1. Key results
2. Findings
3. Data and sample design
4. Architecture
5. PD scorecard
6. Validation
7. LGD and EAD
8. IFRS 9 expected credit loss
9. Basel IRB capital
10. Quickstart
11. Configuration
12. Outputs
13. Dashboard
14. Tests
15. Performance
16. Project structure
17. Limitations and next steps
18. References

---

Key results

Data snapshot March 2019 (the last Lending Club release). 36-month loans.

PD models (out-of-time test: 283,026 loans issued in 2015, default rate 14.9%)

Model	Gini	95% CI	AUC	KS
WoE logistic scorecard (champion, 9 characteristics)	0.371	0.365–0.376	0.685	0.271
XGBoost challenger	0.389		0.695	0.285
Scorecard without Lending Club grade / rate	0.303		0.651

• The challenger's gain is +0.018 Gini. It is statistically significant (DeLong z = 19.4) because the sample is large, but small in business terms.
• Development-to-OOT Gini rises (0.341 to 0.371), so the model shows no overfitting.
• Score PSI between train and OOT is 0.003, which is stable.

Portfolio: IFRS 9 ECL and Basel IRB (582,796 open loans, $4.64bn outstanding)

Stage	Loans	EAD	ECL	Coverage	Mean current PD
1 (performing)	512,561	$4,196mn	$150.3mn	3.6%	13.0%
2 (significant increase in credit risk)	57,754	$337mn	$19.6mn	5.8%	19.7%
3 (credit-impaired)	12,481	$104mn	$92.8mn	88.9%	23.1%
Total	582,796	$4,638mn	$262.6mn	5.66%

Probability-weighted ECL (base 50% / upside 20% / downside 30%)	$285.0mn (+8.5% vs base)
Downside scenario (PD x1.50, LGD +2.3pp)	$364.4mn
IRB risk-weighted assets	$5.98bn (average risk weight 129%)
IRB capital (8% of RWA)	$478.6mn
Mean realised LGD / downturn LGD	89.2% / 91.5%

---

Findings

1. Lending Club's own grade is the strongest predictor. Sub-grade has an IV of 0.31 and is the top SHAP feature in the challenger. Dropping grade, sub-grade, rate and instalment costs 0.068 Gini, so an independent model built only from bureau and application data reaches Gini 0.30.
2. The raw PD understates the out-of-time default rate by 15%. The 2015 vintage defaulted at 14.9% against a mean PD of 12.9%. Jeffreys tests flag PD underestimation in 7 of 10 master-scale grades. Rank order still holds: observed default rates rise monotonically across grades. The fix is re-anchoring the level, not refitting. IFRS 9 PDs are anchored to the latest vintage (point-in-time), and IRB PDs to the long-run vintage average (through-the-cycle). Both use an intercept shift that leaves Gini unchanged.
3. Structural missingness can smuggle time into a model. mort_acc was not collected before 2012 (100% missing in 2007–2011, 0% from 2013), and pub_rec_bankruptcies was mostly missing in 2007–2008. Left alone, the MISSING bin becomes a proxy for issue year: a characteristic stability index of 1.31 for mort_acc. The engine detects such fields and gives the MISSING bin a neutral WoE, which brings the CSI to 0.006.
4. A 5% minimum bin size hides the riskiest borrowers. It pooled sub-grades D4 to G5 into one 24%-bad-rate bin, while G5 loans default at 55%. Categorical characteristics now use a 1% floor (still at least 2,300 loans per bin).
5. Origination data cannot detect deterioration. Conditional on sub-grade, it takes about 188 points of FICO drop to double the odds, so a quantitative SICR test (PD at least 2x origination) flags only 108 loans. Stage 2 is therefore driven by a qualitative bureau trigger (FICO down 60+ points: 50,370 loans) and delinquency (7,276). The Stage 2 PD-overlay sensitivity (+9.3% ECL) shows the cost if Stage 2 PDs are understated.
6. Unsecured consumer loans recover almost nothing. Median LGD is 91.5%, and 56% of workouts lose more than 90%. Borrower characteristics explain almost none of it: out-of-sample R² is 0.005 for a fractional logit, and it beats grade-segment means by only 1.2% MAE. Segment LGD plus a downturn add-on is the right choice.
7. Defaults come early. 33% of defaults happen in the first 12 months on book and 77% in the first 24, so the 12-month vs lifetime distinction matters most for young loans.

---

Data and sample design

Sample	Loans	Defaults	Default rate	Issue years	Use
Train	236,597	31,130	13.2%	2007–2014	Fit binning, scorecard, challenger
Validation	101,399	13,342	13.2%	2007–2014	Early stopping, in-time check (stratified 30%)
Out-of-time	283,026	42,132	14.9%	2015	Independent test, never used in development
Live book	582,796			2015–2018	IFRS 9 ECL and IRB capital

• Only 36-month loans. Every loan issued up to 2015 had matured by the 2019 snapshot, so each outcome is observed. Including unresolved recent loans would bias the sample toward early defaulters (right-censoring).
• Default = Charged Off / Default. Lending Club charges off at about 120–150 days past due, later than Basel's 90 days.
• Leakage guard. PD features must come from an explicit allow-list of application-time fields. Any post-origination column (payments, recoveries, last FICO, outstanding balance, status) raises an error. A unit test enforces this.

---

Architecture

flowchart LR
    A[Kaggle: Lending Club<br/>2.26M loans] --> B[data.py<br/>parse, sample design,<br/>leakage guard]
    B --> C[binning.py<br/>WoE / IV]
    C --> D[scorecard.py<br/>logit, points, master scale]
    B --> E[challenger.py<br/>XGBoost + TreeSHAP]
    D & E --> F[metrics.py<br/>Gini, KS, DeLong, HL,<br/>Jeffreys, PSI/CSI]
    B --> G[lgd.py<br/>realised LGD,<br/>fractional logit, downturn]
    B --> H[ead.py<br/>amortisation,<br/>default timing]
    D & G & H --> I[ecl.py<br/>staging, term structure,<br/>scenarios]
    I --> J[capital.py<br/>IRB other retail]
    F & I & J --> K[report.py<br/>validation report]
    K --> L[Streamlit dashboard]

Loading

---

PD scorecard

Weight of evidence and information value (implemented from scratch in binning.py):

$$\text{WoE}_i = \ln\frac{\text{good}_i / \text{good}_{tot}}{\text{bad}_i / \text{bad}_{tot}}, \qquad \text{IV} = \sum_i (%\text{good}_i - %\text{bad}_i),\text{WoE}_i$$

• Numeric features: 20 quantile bins, then adjacent bins are merged until each holds at least 5% of the sample and the default rate is monotonic.
• Categorical features: levels are ordered by default rate, and levels below 1% are pooled with their nearest-risk neighbour. Missing values get their own bin (neutral for structurally missing fields).

Selection and fit:

1. Drop features with IV < 0.02, and flag IV > 0.5 as possible leakage.
2. Remove features with WoE correlation above 0.7 against a stronger feature.
3. Fit a logistic regression with backward elimination of wrong-sign coefficients and p > 0.05.
4. Report VIFs for the final features.

Final model: sub-grade, FICO, annual income, instalment-to-income, DTI, mortgage accounts, recent inquiries, home ownership, purpose.

Points scaling: 600 points at 50:1 good:bad odds, and 20 points double the odds.

$$\text{score} = \underbrace{600 - \tfrac{20}{\ln 2}\ln 50}_{\text{offset}} + \tfrac{20}{\ln 2},\ln\tfrac{1-PD}{PD}, \qquad \text{points}_{j,b} = -\Big(\beta_j,\text{WoE}_{j,b} + \tfrac{\beta_0}{k}\Big)\tfrac{20}{\ln 2} + \tfrac{\text{offset}}{k}$$

The points per characteristic add up exactly to the score (a unit test checks this), so each decision comes with reason codes.

Challenger: XGBoost with native categorical splits, early-stopped on validation AUC (253 trees), explained with exact TreeSHAP.

---

Validation

Area	Tests
Discrimination	AUC, Gini with bootstrap CI, KS, CAP curve; train vs validation vs out-of-time
Champion vs challenger	DeLong test for correlated AUCs
Calibration	Decile table, Hosmer–Lemeshow, per-grade one-sided binomial and Jeffreys tests (ECB IRB validation)
Master scale	10 PD grades; default-rate monotonicity
Stability	Score PSI train vs OOT and by issue year (including the live book); CSI per characteristic
Sensitivity	Scorecard refit without Lending Club's pricing features

---

LGD and EAD

Realised LGD for defaults with a complete workout (last payment at least 12 months before the snapshot): 83,342 loans.

$$\text{LGD} = 1 - \frac{\text{recoveries} - \text{collection fees}}{\text{funded} - \text{principal repaid}}$$

LGD model (2015 defaults, out of sample)	MAE	RMSE	R²	Spearman
Fractional logit (GLM binomial, logit link)	0.0795	0.1257	0.005	0.101
Grade-segment mean	0.0805	0.1261	-0.001	0.061
Portfolio mean	0.0805	0.1261	0.000

Downturn LGD is the worst annual mean among years with enough defaults (91.5% in 2013 vs the long-run 89.2%).

EAD: these are amortising term loans with no undrawn limit, so no credit conversion factor is needed. EAD is the outstanding principal, which then follows the contractual schedule $B_k = P(1+r)^k - A\frac{(1+r)^k-1}{r}$.

Default timing: the empirical distribution of month-on-book at default converts lifetime PD into a monthly term structure.

---

IFRS 9 expected credit loss

Reporting-date PD. Start from the origination scorecard PD, re-anchored to the point-in-time level. Then update the odds for the FICO change since origination, using the continuous FICO slope conditional on the other characteristics:

$$\text{logit},PD_{now} = \text{logit},PD_{orig} + \beta_{FICO},(\text{FICO}_{now} - \text{FICO}_{orig})$$

Staging.

• Stage 3: 31+ days late or in default.
• Stage 2: in grace or 16–30 days late, or SICR. SICR means PD_now ≥ 2 × PD_orig, or FICO down at least 60 points.
• Stage 1: everything else.

Term structure and ECL. With $F$ the cumulative default-timing curve and $m$ the current month on book, the conditional marginal PD in future month $t$ is

$$q_t = \frac{PD \cdot f(m+t)}{1 - PD \cdot F(m)}, \qquad \text{ECL} = \sum_{t} q_t \cdot \text{LGD} \cdot \text{EAD}_t \cdot (1+r)^{-t}$$

The sum runs over the next 12 months for Stage 1 and the remaining life for Stage 2. Stage 3 uses PD = 1. Discounting is at the loan's effective interest rate.

Scenarios. PD scalars come from vintage history: upside = best vintage / average (0.77), downside = worst / average (1.50). The downside also adds the downturn LGD add-on. The scenarios are probability-weighted 20/50/30.

Sensitivities:

Case	ECL	vs base
PD x1.2	$298.9mn	+13.8%
LGD +10pp	$291.4mn	+11.0%
Stage 2 PD overlay x2	$287.1mn	+9.3%
SICR: FICO drop 40 points (Stage 2 share 16.2%)	$265.2mn	+1.0%
SICR: PD ratio only, no FICO trigger (Stage 2 share 1.3%)	$259.9mn	-1.0%
No PD re-anchoring	$238.6mn	-9.1%

---

Basel IRB capital

This uses the "other retail" risk-weight function (BCBS CRE31), with through-the-cycle 12-month PD (floor 5bp) and downturn LGD (floor 30%):

$$R = 0.03,\tfrac{1-e^{-35PD}}{1-e^{-35}} + 0.16\left(1-\tfrac{1-e^{-35PD}}{1-e^{-35}}\right), \qquad K = \text{LGD}\left[N!\left(\tfrac{G(PD) + \sqrt{R},G(0.999)}{\sqrt{1-R}}\right) - PD\right]$$

RWA = 12.5 x K x EAD. For defaulted loans, K = max(0, downturn LGD − best-estimate LGD). The unit test checks PD 1% / LGD 45%, which gives K = 3.66% and a risk weight of 45.8%.

Stage	EAD	RWA	Risk weight	Regulatory EL	IFRS 9 ECL
1	$4,196mn	$5,484mn	131%	$182.2mn	$150.3mn
2	$337mn	$469mn	139%	$20.2mn	$19.6mn
3	$104mn	$30mn	28%	$92.8mn	$92.8mn
Total	$4,638mn	$5,982mn	129%	$295.2mn	$262.6mn

Risk weights above 100% are expected for unsecured consumer credit with PDs around 13% and LGDs around 90%. Regulatory EL ($295mn) exceeds the IFRS 9 provisions ($263mn). Under IRB, that shortfall is deducted from CET1 capital.

---

Quickstart

git clone https://github.com/US30/credit-risk-engine.git
cd credit-risk-engine
conda env create -f environment.yml
conda activate credit-risk

Kaggle credentials (the data is downloaded automatically on the first run):

1. Go to kaggle.com → Settings → API → Create New Token.
2. Put the credential in ~/.kaggle/kaggle.json. Either the legacy {"username", "key"} JSON or the newer bare API token works. Setting KAGGLE_API_TOKEN also works.

python run_pipeline.py                 # first run downloads 392 MB and caches data/loans.parquet
streamlit run app/streamlit_app.py     # dashboard
pytest -q                              # 26 tests, synthetic data, no network

Options:

python run_pipeline.py --synthetic     # offline run on simulated loans (about 5 s)
python run_pipeline.py --quick         # fewer bootstrap draws and boosting rounds
python run_pipeline.py --no-grade      # PD model without Lending Club's grade, rate and instalment
python run_pipeline.py --refresh       # re-download and re-parse

---

Configuration

All settings live in config.yaml:

Section	Key settings
sample	term (36), development years (≤ 2014), out-of-time year (2015), validation share
binning	fine bins (20), minimum numeric bin share (5%), minimum categorical share (1%)
scorecard	IV thresholds, max correlation, max p-value, PDO / base score / base odds, master-scale PD edges
challenger	XGBoost depth, learning rate, early stopping, subsampling
lgd	workout completeness window (12 months)
ecl	SICR PD ratio (2x), FICO trigger (60 points), stage status mapping, scenario weights
capital	PD floor (5bp), LGD floor (30%)

---

Outputs

File	Contents
model_validation_report.md + figures/	15-section validation report
models.joblib	Fitted scorecard, challenger, LGD model, PD anchors
points_table.csv, scorecard_coefficients.csv	The scorecard itself
feature_selection.csv	IV per characteristic and why each was kept or dropped
discrimination_*.csv, calibration_oot.csv, grade_tests_oot.csv	Discrimination and calibration
psi_by_year.csv, csi.csv	Stability
oot_predictions.parquet	Out-of-time PDs from both models
lgd_evaluation.csv, default_timing.csv	LGD and EAD
ecl_by_stage.csv, ecl_by_grade.csv, ecl_scenarios.csv, ecl_sensitivity.csv	ECL
live_book_ecl.parquet	Per-loan stage, PDs, LGD and ECL
irb_capital.csv, pd_anchors.csv	Capital and PD calibration anchors
run_metadata.json	Timings, machine, headline numbers

---

Dashboard

streamlit run app/streamlit_app.py opens seven tabs:

• Overview: headline Gini, KS, ECL, weighted ECL, RWA; samples; vintages.
• Score an applicant: enter application data to get the score, PD (as fitted and point-in-time), master-scale grade and points per characteristic.
• PD performance: ROC for both models, calibration, discrimination tables, master scale, PD anchors, SHAP.
• Stability: PSI by issue year, CSI.
• LGD & EAD: LGD model comparison, default timing.
• ECL & capital: coverage and PD distribution by stage, grade breakdown, scenarios, sensitivities, IRB.
• Report: the full validation report with a download button.

---

Tests

pytest -q runs 26 tests on synthetic data (no network needed):

Area	Checks
Sample design	Leakage guard rejects post-origination and unknown columns; train/validation disjoint; OOT year isolated; stratified default rates
Binning	WoE/IV hand calculation (IV = 0.2335); monotonic bins with ≥ 5% share; missing and unseen categories; structural-missing detection and neutral WoE
Scorecard	+20 points doubles the odds; points sum exactly to the score; all coefficients negative; FICO slope sign
Metrics	Gini/KS match sklearn/scipy; DeLong SE matches bootstrap; Hosmer–Lemeshow detects a 1.5x miscalibration; PSI = 0 on identical data and 0.0405 on a hand example; binomial/Jeffreys flag an underestimated grade
LGD / EAD	LGD hand value; GLM mean reproduces sample mean; amortisation closed form and zero balance at maturity; timing curve is a distribution
ECL	Marginal PDs sum to lifetime PD for new loans and to the conditional remainder for seasoned loans; lifetime ≥ 12-month ECL; stage rules; SICR by PD ratio and FICO trigger; re-anchoring hits the target and keeps rank order; scenario scalar raises ECL
Capital	IRB correlation and K for PD 1%/LGD 45% (K = 3.66%, RW 45.8%); PD/LGD floors; defaulted-exposure capital

---

Performance

Machine	First run (download + parse)	Later runs	Peak memory
Apple M3 MacBook Air, 8 GB	~2.5 min	~40 s	~2 GB

The work is CPU-only. XGBoost uses the hist method, and no GPU is needed.

---

Project structure

credit-risk-engine/
├── config.yaml
├── environment.yml
├── run_pipeline.py
├── creditrisk/
│   ├── data.py         # download, parsing, sample design, leakage guard, synthetic data
│   ├── binning.py      # WoE / IV binning
│   ├── scorecard.py    # selection, logistic fit, points, master scale, FICO slope
│   ├── challenger.py   # XGBoost + TreeSHAP
│   ├── metrics.py      # Gini, KS, DeLong, HL, binomial/Jeffreys, PSI/CSI
│   ├── lgd.py          # realised LGD, fractional logit, downturn LGD
│   ├── ead.py          # amortisation, default timing, EAD ratio
│   ├── ecl.py          # staging, PD re-anchoring, term structure, ECL, scenarios
│   ├── capital.py      # Basel IRB other retail
│   ├── plots.py, report.py, pipeline.py
├── app/streamlit_app.py
├── tests/
└── assets/             # README figures

---

Limitations and next steps

Limitations:

• Approved applicants only: no reject inference.
• Charge-off default definition, not 90 days past due.
• 36-month loans only.
• No prepayment in the lifetime term structure. This overstates Stage 2 lifetime ECL.
• Default timing pooled across risk levels. High-risk loans default earlier, so their 12-month PDs are understated.
• The only behavioural signal is FICO, so Stage 2 relies on a qualitative trigger.
• Scenario scalars come from vintage history, not a macroeconomic model.
• Recoveries include debt sales at a discount.

Next steps:

• A behavioural PD built from payment history.
• A competing-risks (default vs prepayment) survival model for the term structure.
• A macro-linked PD (unemployment, GDP) for the IFRS 9 scenarios.
• Reject inference.
• Monotone-constrained XGBoost as a more explainable challenger.

---

References

• Siddiqi, N. (2017). Intelligent Credit Scoring. Wiley.
• DeLong, E., DeLong, D. & Clarke-Pearson, D. (1988). Comparing the areas under two or more correlated ROC curves. Biometrics.
• Papke, L. & Wooldridge, J. (1996). Econometric methods for fractional response variables. Journal of Applied Econometrics.
• IASB (2014). IFRS 9 Financial Instruments, section 5.5 (impairment).
• BCBS (2019). CRE31: IRB approach, risk weight functions.
• ECB (2019). Instructions for reporting the validation results of internal models (Jeffreys test).
• Lending Club loan data, 2007–2018Q4 (Kaggle, wordsforthewise/lending-club).