From Jupyter Notebook to Live API: Building a Production-Grade ML System for Hospital Readmission Prediction
The article documents a complete end-to-end ML engineering pipeline for hospital readmission prediction, emphasizing deployment, monitoring, and production readiness over model performance alone. Key technical decisions include using XGBoost with `scale_pos_weight` for class imbalance (11% positive rate), MLflow for experiment tracking and model registry, FastAPI for serving, Docker for containerization, and Evidently AI for drift monitoring. The most significant learning was that real ML engine
Analysis
TL;DR
- The article documents a complete end-to-end ML engineering pipeline for hospital readmission prediction, emphasizing deployment, monitoring, and production readiness over model performance alone.
- Key technical decisions include using XGBoost with
scale_pos_weightfor class imbalance (11% positive rate), MLflow for experiment tracking and model registry, FastAPI for serving, Docker for containerization, and Evidently AI for drift monitoring. - The most significant learning was that real ML engineering challenges are predominantly configuration and deployment issues (file paths, missing dependencies, registry metadata) rather than modeling problems.
- The system achieves an AUC of 0.682, with the author acknowledging that dropping high-cardinality diagnosis columns was a deliberate trade-off to focus on the engineering pipeline.
- The full stack includes: data preprocessing → XGBoost training with MLflow tracking → FastAPI serving with SQLite logging → Docker containerization → Render deployment → Evidently AI monitoring → Streamlit frontend.
Why It Matters
This article provides a rare, honest account of the deployment and operational challenges that dominate real-world ML engineering—areas rarely covered in tutorials that stop at model training. For AI practitioners, it demonstrates that a production-grade system requires careful attention to data validation, experiment tracking, container configuration, and continuous monitoring, not just model accuracy. The troubleshooting narrative (especially the Docker/MLflow registry issues) serves as a practical guide for anyone shipping ML systems beyond notebooks.
Technical Details
- Dataset & Preprocessing: UCI Diabetes 130-US Hospitals dataset with 101,766 patient encounters; handled
?-encoded missing values, converted age ranges to midpoints, binarized 23 medication columns, and created a binary 30-day readmission target with stratified 70/15/15 splitting. - Modeling: XGBoost classifier with
scale_pos_weight ≈ 7.96to address class imbalance; three experiments tracked via MLflow with AUC as the primary metric; champion model (Run 2, AUC 0.682) registered in MLflow Model Registry with a "champion" alias for version-agnostic serving. - Serving Layer: FastAPI with Pydantic validation,
/healthendpoint,/predictand/predict_batchroutes, and SQLite-based prediction logging; model loaded directly from MLflow registry viamodels:/readmission-model@champion. - Deployment & Containerization: Docker containerization revealed three critical issues—Windows-specific
pywin32inrequirements.txt, MLflow registry metadata (mlflow.db) not copied into the image, and a missing slash in the SQLite URI (sqlite:////app/mlflow.db); deployed on Render with free-tier cold-start behavior (30–60s wake time). - Monitoring: Evidently AI used for data drift detection with a two-report validation strategy (baseline confirming no drift on held-out test data, and simulated drift confirming sensitivity by artificially aging patients and extending hospital stays).
Industry Insight
- The "notebook-to-production" gap remains the single biggest bottleneck in ML adoption; investing in robust pipelines (MLflow, FastAPI, Docker, monitoring) yields more business value than marginal AUC improvements on a model that never gets used.
- Configuration debugging (file paths, environment variables, dependency snapshots) accounts for the majority of deployment failures—practitioners should prioritize understanding containerization and registry mechanics over additional model tuning.
- For healthcare ML systems, the class imbalance problem is best addressed through loss-function weighting rather than synthetic data generation when dealing with mixed binary/categorical features, as SMOTE can produce clinically implausible samples.
Disclaimer: The above content is generated by AI and is for reference only.