Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Foundation · Core
A versioned catalog of trained models with their metadata, stage, and lineage, so you know exactly what is deployed and can roll back. The control plane for shipping models safely.
In short
A model registry is a catalog of every trained model version, with its metrics, the data and code that produced it, and a label saying which version is live. Serving asks the registry for the live version, so promotion and rollback become a quick, recorded label change.
In short
A model registry is a catalog of every trained model version, with its metrics, the data and code that produced it, and a label saying which version is live. Serving asks the registry for the live version, so promotion and rollback become a quick, recorded label change.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Model Registry across a run of lessons, not one page. These 3 alone are about 188 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
How a model registry becomes the single source of truth for every model version, its metrics, its lineage, and which one is serving real users right now.
What does a model registry record, and how does 'load the production model' actually work? I ran MLflow 3.16.1 locally with three versions of the features chapter's model. Loading models:/shop-buyers@champion gave predictions identical to the trained model, to the bit. A rollback changed 1 of 320 rows in 1 of 18 files. Moving the alias to a model that takes other columns raised no error and scored test AP 0.1731, below all 20 random orders. The default save refused the model, and deleting a version left its alias behind.
When a new model misbehaves, the team goes back to the one that worked. I rebuilt last month's model five ways from its written record. With every note kept, the rebuild gave the same answer on all 6,000 rows it had served, and so did the saved model file loaded back. With the seed not written down, 19 guesses agreed on 0.920 to 0.970 of rows, none exactly, and the check on the training data stayed quiet. Changed data made the record's fingerprint fire, and one wrong rebuild even scored higher than the real model. What to write down, what to keep, and which check sees what.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Model Registry and Versioning: Knowing What Is Actually Livepay once,
yours for life
A model registry is the system of record for trained models, in the way a git repository is the system of record for code. It stores every version a team has produced, not just the latest one, and it always knows which version is serving users. Without one, models tend to live as loose files with names like model_final_v2.pkl in a shared bucket.
The registry exists to answer three questions quickly during an incident. Which version is serving users right now? What produced it: which data, which code, which settings? And how do I get back to the last version that worked? If you cannot answer the first question with certainty, you cannot debug or undo anything.
The registry is not where the model runs. It sits between training and serving. Training writes new numbered versions into it, and serving reads the live version out of it. Serving asks for a stage by name, such as models:/churn/Production, and the registry resolves that name to whichever version is live at that moment.
A registered version is much more than a weights file with a number. It has an identity, a model name plus an integer version that never changes; a retrain makes v9 rather than overwriting v8. It records the evaluation metrics on held-out data, the hyperparameters and random seed, and lineage keys pointing to the exact data snapshot, the code commit and the training run.
It also holds a pointer to the artifact, the serialized model file, and often a model card: a short document with the owner, the intended use and known weaknesses. For high-stakes models such as credit scoring, a promotion should not pass until that card is filled in.
Registries split these two kinds of data on purpose. The small records, meaning versions, stages, metrics and approvals, go into a transactional database such as Postgres, because they are queried constantly. The heavy files, such as a 400 megabyte weights file, go into blob storage such as S3. Each version's row carries the address of its file, and serving fetches the file only when it loads the model.
The key idea is that versions are immutable and stage labels are movable. Once v7 is written, it never changes. Labels such as Production, Staging and Archived are pointers, one of each per model, that you move between versions. In MLflow's classic scheme, a version starts at None, moves to Staging to be tested against the current live model, and reaches Production only if it passes. Promoting a new version moves the old one to Archived, where it stays intact.
Once you see labels as pointers, promotion and rollback become the same small operation. Promotion moves the Production pointer from v8 to v9. Rollback moves it from v8 back to v7. Nothing about the versions changes, so neither action can lose data, and because serving asks for the pointer by name, the next model load follows it with no code change.
Newer MLflow versions have deprecated the fixed stage names in favour of aliases, such as @champion, which you assign to a version yourself. The idea is the same: a movable name pointing at a fixed version. Cloud registries in SageMaker and Vertex AI use their own terms for the same pattern.
Say the live churn model is v8. Training registers v9 and moves it to Staging. An evaluation job scores both on the same held-out data using AUC, a score from 0.5 (random) to 1.0 (perfect) for how well a model ranks positive cases above negative ones. In the example, v9 scores 0.931 and v8 scores 0.910. A better score is necessary but not enough: the gate also requires a complete model card and sign-off from a reviewer. Only then does the Production label move, and the score, the approver and the time are written to an audit trail.
Now a newly promoted version quietly lowers fraud recall on live traffic that the held-out data did not represent, and monitoring pages the on-call engineer. Because the previous version is still in the registry, the engineer moves the Production label back to it, serving reloads, and the bad version is tagged so nobody promotes it again. Without a registry, the same fix means searching for an old file and hoping it still loads with today's library versions.
Lineage makes the past answerable. Because each version records its data snapshot, code commit, parameters and seed, you can trace a live prediction, such as a loan denial, back to exactly what produced it, and rebuild that version from the same inputs. In MLflow, the flow is to log the model and its tags in a run, register it under a name, and have serving load it by name rather than by file path.
For one person shipping one model once a quarter, a well-named file is honestly fine at the start. The registry becomes necessary when several people ship several models on a regular schedule; without it, 'which model is live?' becomes a hunt within weeks.
In regulated areas such as credit, health or insurance, a registry is close to required. You must show what data trained a model and who approved it, and the audit trail and lineage do exactly that. It also makes comparisons cheap: comparing v5 with v12 is one query, not a rerun of old models whose metrics were lost.
The main cost is discipline. A registry is another service to run and another database to back up, and its guarantees disappear the moment someone copies a model file into production by hand 'just this once'. It only works as the source of truth if every model reaches production through it.
MLflow Model Registry
The open-source registry in MLflow, created at Databricks, stores numbered model versions with lineage and lets serving load a model by name and stage or alias.
Amazon SageMaker Model Registry
Groups model versions into model package groups and records an approval status for each version before deployment.
Vertex AI Model Registry
Google Cloud's registry keeps versions of a model, supports version aliases, and deploys a chosen version to an endpoint.
It is a catalog of every trained model version with its metrics, parameters, lineage and a label saying which version is live. Training writes versions into it and serving reads the live one out, so everyone agrees on what is running.
Experiment tracking records every training run, including the many you throw away. A registry holds the versions you intend to deploy, with stages or aliases, approvals and a rollback history. In MLflow, a tracked run's model is registered as a numbered version when it is worth keeping.
Old versions are never deleted, only relabelled. To roll back, you move the Production label or alias back to the last good version, and serving loads it on its next reload. It is a small, recorded change rather than a search for an old file.
Yes. Newer MLflow versions deprecate the fixed Staging and Production stages and recommend model version aliases, such as @champion, plus tags. Aliases work the same way: a movable name that points to one fixed version.
Not on day one if one person ships it rarely. It becomes necessary once several people ship models on a schedule, or when you must prove which data trained a model and who approved it.
A Model Registry, Hands On: An Alias Is a Pointer, and a Rollback Moved One Row
What does a model registry record, and how does 'load the production model' actually work? I ran MLflow 3.16.1 locally with three versions of the features chapter's model. Loading models:/shop-buyers@champion gave predictions identical to the trained model, to the bit. A rollback changed 1 of 320 rows in 1 of 18 files. Moving the alias to a model that takes other columns raised no error and scored test AP 0.1731, below all 20 random orders. The default save refused the model, and deleting a version left its alias behind.
Lineage and Rollback: Rebuilding Last Month's Model Exactly, and What Breaks When You Cannot
When a new model misbehaves, the team goes back to the one that worked. I rebuilt last month's model five ways from its written record. With every note kept, the rebuild gave the same answer on all 6,000 rows it had served, and so did the saved model file loaded back. With the seed not written down, 19 guesses agreed on 0.920 to 0.970 of rows, none exactly, and the check on the training data stayed quiet. Changed data made the record's fingerprint fire, and one wrong rebuild even scored higher than the real model. What to write down, what to keep, and which check sees what.
Model Drift
When a model's accuracy decays over time because the live data has moved away from what it was trained on. Detected by monitoring inputs and outputs, and fixed by retraining.
Model Serving
Exposing a trained model behind an API that answers predictions within a latency budget, with batching, autoscaling, and versioning. Where most of the production cost and the request-path complexity lives.
Feature Store
A single source of truth for features, so training and serving compute them from one definition and cannot drift apart. It ends the train-serve skew bug and lets teams reuse features across models.
Train-Serve Skew
When a feature is computed one way in training and a slightly different way in production, so every prediction is subtly wrong. Nothing crashes, accuracy just leaks away. Feature stores exist to prevent it.
Feature Engineering
Turning raw data into the signals a model actually learns from. Often the highest-leverage work in a classical ML system, and where a feature store keeps definitions consistent.
Data Lakehouse
A design that keeps data as open files, usually Parquet, in cheap object storage and adds a log on top that makes changes all-or-nothing, checks column types and keeps old versions readable.
This page explains the idea. The "Model Registry and Versioning: Knowing What Is Actually Live" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Model Registry and Versioning: Knowing What Is Actually Live lessoncourse 2
Take models from notebook to production, with labs on real models.
Asked about Model Registry in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (180 diagrams in all). A PDF you keep, junior to staff level.
The ML Engineering and MLOps pack is being written and is not on sale yet.
India: UPI, cards or netbanking through Razorpay. Elsewhere: PayPal. Download at once from My packs.
