How much outreach can the team handle?
Explore the saved evaluation on 1,409 held-out IBM Telco customers. Choose the highest-risk share of the customer base to see the trade-off between a more concentrated list and reaching more churners.
Selecting the highest-risk 10% captures 25.4% of churners; 67.4% of that selected group churned in the test data.
These are precomputed test-set results, not live customer predictions or measured retention improvements. The interactive prediction service is available below. The full comparison table remains available without JavaScript.
The operating decision
IBM Telco is a familiar benchmark. I used it to work on the engineering around a model: a separate prediction API, per-customer explanations, prediction logging and automated deployment. Keeping the Streamlit interface separate from inference means the same service can support another client without rebuilding the model logic.
- Streamlit collects inputs
- FastAPI on Cloud Run runs XGBoost and SHAP
- BigQuery records predictions
- GitHub Actions tests and deploys changes
The problem
A retention team cannot contact every customer. A useful model should help decide who to contact first and explain the signals behind each score. Accuracy alone does not answer that operational question.
Data and approach
I used the IBM Telco dataset of 7,043 customers. The workflow cleaned whitespace in TotalCharges, converted it to numeric and encoded categorical fields into 30 model features. I used a stratified 80/20 split, training-only scaling and SMOTE, then compared Logistic Regression, Random Forest and XGBoost with cross-validation.
How I built the solution
I served the tuned XGBoost model behind a FastAPI service on Cloud Run. A Streamlit client sends customer inputs to the API. Each prediction includes the five leading SHAP contributions. Prediction logging goes to BigQuery, while GitHub Actions runs tests, builds the container and deploys it. This separates the interface from inference and provides records for monitoring.
How decisions were made
The training workflow explores 108 XGBoost parameter configurations. For business relevance, I also measured precision, captured churners and lift at different outreach capacities. SHAP explains the model’s associations for each customer; it does not establish why a customer will leave or which intervention will work.
What worked
On the held-out test set, my model achieved ROC-AUC 0.808, F1 0.5914 and accuracy 0.7637. Among the highest-risk 10% of test customers, 67.4% were actual churners, capturing 25.4% of all churners. Expanding contact capacity to 30% captured 62.6%. The ranking provides a more useful operational view than the headline accuracy alone.
What remains unresolved
The remaining limitations are uncalibrated probabilities, no temporal validation, no automated drift alerts and a retained gender feature that needs fairness assessment before real use. A random stratified test split is weaker evidence for performance on future customers than a time-based holdout.
Effect and next decisions
The demonstrated effect is offline prioritisation quality, not customers retained or revenue saved. A live pilot would need to compare outreach strategies and measure actual retention outcomes. Calibration, temporal validation, fairness checks and alerting should precede claims of production effectiveness.
Ranking quality on the held-out test set
| Highest-risk group | Precision | Churners captured | Lift |
|---|---|---|---|
| Top 5% | 71.4% | 13.4% | 2.69× |
| Top 10% | 67.4% | 25.4% | 2.54× |
| Top 20% | 62.1% | 46.8% | 2.34× |
| Top 30% | 55.3% | 62.6% | 2.08× |
Offline results on the held-out IBM Telco test set. Lift is relative to untargeted selection, not a measured increase in retention.
Try a live prediction
Enter customer details to inspect a churn prediction and the features influencing it. Use example data when exploring the demo. The external demo may take a moment to wake up.
Open interactive demo