When Were Your Models Last Updated?

Ask a sales leader when the lead score was last retrained and you often get a pause. Ask who owns the churn model and sometimes the answer is someone who has since left the company.

That is the question worth putting on the table. Not "is the model good?" but "how old is it?" A model is a snapshot of your customers on the day it was trained. Every day after that, it is describing a slightly older version of your business.

Every model has an age

Researchers from Harvard Medical School, MIT, and other institutions gave this a name in a 2022 Scientific Reports study: AI aging. They defined a model's age as the time passed since it was last trained on then-current data.^[1]

They tested four standard model types, from regularized linear regression to gradient boosting and neural networks, on 32 datasets from four industries. Across the 128 model and dataset pairs, they observed quality degrading over time in 91% of cases.^[1] They also deliberately chose datasets with no abrupt changes in the outcome being predicted. The models aged anyway.^[1]

Two findings matter for an executive:

• Decline is not always gradual. In one case, a model began to degrade after performing accurately for a year.^[1] A clean track record does not tell you how much runway is left.
• Averages can hide it. Some models held a reasonable error on average while the spread of their errors grew, which the authors warn can create the illusion of accurate performance.^[1]

Their practical advice was plain: once a model is in production, measure its error as often as possible and alert its users when the error crosses an acceptable threshold.^[1]

A model is a snapshot of your customers. The longer you go without a new one, the older the picture.

Yearly retraining was a constraint, not a strategy

Many companies still retrain once a year, or when something breaks. That habit made sense when compute was scarce and data arrived in monthly extracts.

Google Cloud's MLOps guidance describes the fully manual setup as level 0 and says it assumes a team manages a few models that don't change frequently, with a new model version deployed only a couple of times per year.^[2] The same guidance opens by listing what is now available to anyone applying ML, including large datasets and inexpensive on-demand compute resources.^[2]

The cost curve backs that up. Stanford's 2025 AI Index reports that at the hardware level, costs have declined by 30% annually, while energy efficiency has improved by 40% each year.^[3]

Bandwidth has changed too. Fivetran, a common tool for loading CRM and billing data into a warehouse, syncs every six hours by default and offers schedules as frequent as every 5 minutes, or every minute on its top plans.^[4] Your newest customer behavior can land in the warehouse the same day it happens.

In our experience, retraining a churn or lead model on a few years of CRM and billing history is a scheduled job on tools most companies already pay for, not a project. (How to set that up: Production ML Without Massive New Infrastructure.) The limit today is usually attention, not compute.

Check what your model is still learning from

Age is one problem. The training window is another.

If a model was built in 2022 on three years of history and never refreshed, much of what it knows about your buyers comes from 2020 and 2021. Ask whether those customers still look like the ones you sell to now. Your reps and CSMs can usually tell you within a minute.

Wider is not automatically better. The Scientific Reports authors note that adding more history risks including outdated data, and that the real challenge is balancing "recent enough" against "large enough."^[1] For most lead and churn models, we test a few windows on recent periods the model never saw and keep the one that holds up best.

How often is often enough

There is no single right cadence. Google's Rules of Machine Learning frame it as a question: how much does performance degrade if you have a model that is a day old? A week old? A quarter old?^[5]

Google's MLOps guidance lists several reasons to kick off a retrain: on a schedule (daily, weekly, or monthly), when new training data becomes available, when performance degrades noticeably, or when the inputs shift significantly. It adds that the right frequency depends on how quickly the data patterns change and how expensive it is to retrain.^[2]

For commercial models, we tie the schedule to two things:

• How fast outcomes arrive. A lead model learns from meetings booked and deals closed, which pile up every week. A churn model on annual contracts only learns at renewal. You can retrain only on outcomes you have actually seen.
• How fast your market moves. A pricing change, a new segment, or a new competitor is a reason to retrain early rather than wait for the calendar.

A reasonable starting point for many mid-market teams: score nightly, retrain monthly, and review the schedule each quarter against how much accuracy slips between retrains.

A retrain is a candidate, not a replacement

Retraining on a schedule does not mean pushing every new model live. Google's MLOps guidance says a newly trained model should be compared against the current production model and promoted only if it performs better. It also says to check that performance is consistent across segments, using a churn model that improves overall but varies widely by customer region as its example.^[2]

Google's Rules of Machine Learning add two habits worth copying. Run sanity checks on held-out data before a model goes live.^[5] And if a model is trained on data through January 5th, test it on data from January 6th and after, because that better reflects what it will face in production.^[5]

The guidance also recommends keeping a pointer to the previous trained model so you can roll back.^[2] If the new version disappoints, the old one returns the same day.

Put the date next to the score

The simplest fix costs almost nothing. Wherever a score appears in the CRM, add the date the model was last trained next to it.

Reps and CSMs will start asking about it once that date gets old. So will you. It turns "is the model still good?" from a gut feeling into a field anyone can read.

What to ask for

1. A model inventory listing every model in use, its owner, and the date it was last trained.
2. A training window you have tested, not just inherited.
3. A retraining schedule tied to how fast outcomes arrive and how fast your market moves.
4. A comparison rule: a new model goes live only if it beats the current one on recent data, across segments.
5. An error alert with a named owner, checked every scoring cycle.
6. A "last trained" date on every score your team sees.

If you want lead and churn models that stay current with your customers instead of describing last year's, talk to Gamify Data. We start with the data you already collect and set a retraining rhythm your team can keep.

Sources

1. Daniel Vela, Andrew Sharp, Richard Zhang, Trang Nguyen, An Hoang, and Oleg S. Pianykh, "Temporal quality degradation in AI models," Scientific Reports, vol. 12, July 8, 2022. https://doi.org/10.1038/s41598-022-15245-z
2. Google Cloud, "MLOps: Continuous delivery and automation pipelines in machine learning," Cloud Architecture Center, last updated August 28, 2024. https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
3. Stanford Institute for Human-Centered Artificial Intelligence, "The 2025 AI Index Report," Stanford HAI, 2025, accessed October 19, 2026. https://hai.stanford.edu/ai-index/2025-ai-index-report
4. Fivetran, "Sync Overview," Fivetran documentation, accessed October 19, 2026. https://fivetran.com/docs/core-concepts/syncoverview
5. Martin Zinkevich, "Rules of Machine Learning: Best Practices for ML Engineering" (Rules #8, #9, and #33), Google for Developers, last updated August 25, 2025. https://developers.google.com/machine-learning/guides/rules-of-ml