Your Churn Model Is an Ex Who Thinks You're Still Together
Most churn models get trained once, shipped, and then ghosted. Then a price change lands, or a competitor shows up, or the product pivots, and the whole reason customers leave quietly rewrites itself. The model has no clue. It keeps answering with total confidence based on a world that stopped existing months ago.
And the worst part is how slow you find out. Accuracy does not crash. It erodes. By the time a dashboard looks wrong enough for someone to notice, you have been making bad calls for three weeks.
churnfm watches the shape of its own predictions with the Population Stability Index and retrains the second that shape shifts, instead of waiting for a human to feel like something is off.
A Loop, Not a Model
The classifier was never the interesting part. The loop wrapped around it is.
• Drift detection: it compares the distribution of predictions now against a reference window, and catches the relationship shifting before accuracy visibly falls off a cliff.
• Auto retraining: the moment PSI crosses the line, it retrains on recent data and resets its baseline.
• Recent window, not everything: it retrains on a sliding window on purpose. Dump in all your old data and you drag the dead relationship along with you.
• Bring your own model: the default is a stdlib logistic regression, but anything with fit and predict_proba drops into the same monitor.
The Instinct That Was Wrong
My gut said retrain on all the history. More data, safer model, right? Wrong. Mixing the old world and the new one keeps the drift alarm screaming forever, and the model never settles into what is actually true now. The recent-window fix looks obvious written down. It only became obvious after I benchmarked both and watched the naive version thrash.
I also scored it on precision-at-k instead of a fixed cutoff, because churn is a single-digit-percent event and a fixed probability threshold is basically noise when the thing you care about is that rare.
The Number
On a subscription stream with a drift injected halfway through (price-sensitivity suddenly becomes the thing that matters), the static model stalls at 80% post-drift precision. The version that notices and retrains itself climbs back to 89%.
6 tests green in CI, with the example and the full drift benchmark running on every push. No API keys.