Deep learning consistently outperforms simpler models on complex prediction tasks, yet regulated industries like financial services are often reluctant to adopt it — largely because "black box" models are hard to explain to regulators, auditors, and customers. What would it take to close that trust gap?
- Cleaned and engineered features on a 2.26 million row Lending Club loan dataset down to 1.2 million completed loans and 86 usable features
- Built and compared a TensorFlow neural network against Random Forest, Logistic Regression, and XGBoost baselines
- Applied LIME and SHAP to make the deep learning model’s predictions interpretable at both the global and individual-prediction level
Showed that explainability tooling can translate deep learning outputs into the kind of case-by-case justification regulated industries require — evidence that the barrier to adoption is less about raw model capability and more about making decisions auditable.
- Python, Pandas
- TensorFlow, XGBoost, Scikit-learn
- LIME, SHAP
Background
Regulated sectors such as banking and insurance operate under model risk management rules that demand a clear account of why a model made a given decision — something traditional deep learning struggles to provide out of the box. This dissertation used real-world credit risk data to ask whether modern explainability techniques can bridge that gap without giving up the predictive power that makes deep learning worth adopting in the first place.
What I learned
Most of the hard work happened before any model was trained: filtering out loans that hadn’t reached a final outcome, dropping fields that wouldn’t be available at the point of application, and deciding how to treat sparse or high-cardinality columns without leaking information or destroying signal. That data discipline mattered as much as the choice of algorithm, and it shaped how I think about deploying models into environments where every decision needs to be defensible.