This paper derives the exact chi-square distributions and benchmark values for Jensen–Shannon Divergence (JSD) and Kullback–Leibler Divergence (KLD), and benchmarks them against PSI in monitoring distribution shifts of credit default probabilities generated by Merton, Merton with Jump, and Stochastic Volatility with Jump models. Key findings: JSD shows superior Type I error control near 5% but sacrifices power at small samples; KLD offers higher sensitivity; PSI tends to have higher power in larger samples. The work provides practical guidance on binning, sample-size requirements (min(n,m) ≥ 400), and when to use permutation/bootstrap corrections for sparse data, with empirical validation showing consistent discrimination of PD distributions across models
速读内容
Divergence measures and their asymptotic behavior
JSD and KLD are shown to follow chi-square distributions under the null, with JSD offering better Type I error control (close to 5%) but reduced power at small samples (e.g., 27% power for JSD vs 32% for PSI and KLD at n=m=200)
PSI, KLD, and JSD can be used for monitoring PD distributions from structural credit risk models; JSD is noted as more robust to zero-binning and mixture models, while KLD is more suited for Bayesian comparisons
Empirical validation and practical guidelines
Figure 1 confirms empirical distributions align well with theoretical χ2 distributions for PSI, KLD, and JSD (n=m=500, B=10) with small sup distance between empirical and theoretical CDFs; R^2 for QQ plots > 0.99 across measures
Thresholds and sample-size guidance:
min(n, m) ≥ 400 recommended for reliable chi-square approximations; when min(n, m) < 400, JSD is preferred due to conservative Type I error control
Bin construction: ensure enough expected counts per bin (n·pi ≥ 10 and m·qi ≥ 10 is a conservative rule)
Application to credit risk models (Merton family)
PD distributions from Merton, Merton with Jump, and SVJ differ in meaningful ways; despite similar time trajectories, distributional differences remain detectable by divergence measures, with PSI showing the largest differences across models
Critical values and empirical results on model comparisons
Tables 9–11 report divergence benchmarks (PSI: 0.0991, KLD: 0.0495, JSD: 0.0123) at 5% significance using B=10 bins and 1,000-obligor portfolios; all measures reject the null of identical PD distributions in the presented comparisons, with PSI generally yielding larger absolute divergence values than KLD and JSD
Summary of practical findings
JSD is more conservative in detecting distributional differences, potentially reducing false alarms; KLD and PSI are more sensitive, especially in larger samples; differences among the measures diminish with very large samples; the choice of measure should reflect the cost of false alarms versus detection sensitivity
For monitoring PD distributions under credit risk, the framework provides explicit binning and sample-size guidelines, and points toward small-sample corrections or bootstrap-based calibrations as future work
总体而言,该研究系统性地建立了JSD与KLD在信用风险监控中的统计性质、基准值和实证可用性,提供了较为清晰的实践指引(在 bin 构造、样本量、以及在小样本情境下的替代方法方面),同时也清晰界定了外部数据验证与小样本检验的未来研究方向。对于金融机构在日常模型监控与风控框架设计中,本文的结论可作为选择分布差异度量的理论与实践参考,特别是在需要平衡误警成本与检出敏感性时的工具选型与阈值设定。