MLOps Interview Questions and Answers
Here are 27 MLOps interview questions, with short answers in simple English. They cover the whole path a model takes to production. Where our course measured something, you see the number and the lesson.
How to answer an MLOps question
Name the stage you are talking about: data, training, serving or monitoring. Then say what can go wrong there and how you would catch it. MLOps interviews reward people who think about failure.
1. MLOps basics
Start with the idea. An interviewer wants to hear that you know why machine learning is harder to run than normal software.
What is MLOps?
MLOps means the practices and tools for putting machine learning models into production and keeping them working. It covers data, training, deployment, monitoring and retraining.
It is like DevOps for software, with one big difference. A model's behaviour depends on data, and data keeps changing.
Why do ML models that work in a notebook fail in production?
Usually not because of the model. Common causes are different data in production and features (model inputs) calculated differently. Others are missing libraries, slow responses, and nobody noticing when quality drops.
A notebook proves an idea once. Production needs the same result every day, for real users.
What do you need to version in ML, compared with normal software?
Normal software versions its code. ML must version three things: the code, the data the model was trained on, and the model file itself.
Without all three, you cannot rebuild a model or explain why it behaves as it does.
2. Packaging and serving
Here the focus is turning a trained model into a reliable service.
Why is a saved model file not enough to deploy?
The model file needs the exact library versions and the code that prepares inputs. On another machine, a small version difference can break it or quietly change results.
So teams package the model, its code and its libraries together, usually in a Docker container image. That is a sealed bundle that runs the same on any machine.
What is the difference between online and batch serving?
Online serving answers one request at a time, in milliseconds, through an API. An example is a fraud check during a payment. Batch serving scores many rows on a schedule, such as every night.
Batch is cheaper and simpler. Use online only when the answer is needed right away.
How do you roll out a new model safely?
In shadow mode, the new model sees real traffic, but its answers are only logged and compared, not used. In a canary release, a small share of users get the new model first.
Watch the metrics, then increase the share slowly. Keep the old model ready, so going back is fast.
Why look at p99 latency, not the average?
p99 latency means: 99% of requests finish within this time. The slowest 1% of requests are the ones users notice and complain about.
A good average can hide a very slow tail, for example from large inputs or a busy server.
3. Pipelines, registries and versions
Retraining by hand does not scale. These questions are about making it automatic and repeatable.
What is a training pipeline?
A training pipeline is a series of steps that run in order. It gets data, checks it, prepares features, trains, evaluates and registers the model. Tools like Airflow or Kubeflow run it.
It can start on a schedule or when new data arrives. A new model is released only if it passes its checks. It must also do at least as well as the current one.
When should a model be retrained?
On a schedule, such as every week, or when a signal says the model got worse. Many teams use both: a slow schedule as a floor, and a signal on top.
Then compare the new model, the challenger, with the live one, the champion, on the same recent data before switching.
What is a model registry?
A model registry is a central list of every model version. For each version, it stores its metrics, the data and code that made it, and whether it is live.
When something goes wrong, it tells you exactly which model is serving and how to roll back.
What is experiment tracking, and why does it matter?
Experiment tracking records every training run: the code version, data version, settings, random seed and results. MLflow is a common tool for this.
Without it, you cannot say which run produced the best model, or train it again the same way.
How do you version training data?
Tools like DVC or LakeFS store versions of datasets, a bit like Git does for code. Delta Lake and Iceberg store big tables as files. They let you read a table as it was at an earlier time.
Then each model can point to the exact data version it was trained on.
What is a data contract?
A data contract is an agreement about the shape of data a team sends: its columns, types and allowed values. The producing team must not break it without warning.
Say the sending team renames a column. The contract stops that change from silently breaking every model that uses the data.
4. Data quality and features
Most production problems in ML start in the data. So interviewers ask about it often.
What is a feature store, and what problem does it solve?
A feature store keeps the definitions and values of model inputs, called features, in one place. It serves them for training and for live predictions from the same definition.
That prevents training and serving skew, where a feature is calculated differently in the two places.
How do you stop bad data from reaching a model?
Check data at the start of the pipeline: types, ranges, missing values and row counts. Tools like Great Expectations and Pandera do this.
If a check fails, stop the pipeline instead of training on bad data, the way a fuse cuts power.
What is data leakage, and how do you prevent it?
Leakage is when information the model should not have gets into training. Examples are test data, or data from the future. The model then looks great in testing and fails live.
Split the data first. Then do every preparation step using only the training part. For example, fit the scaler on training data, then apply it to test data. For data that changes over time, test on the newest rows.
What we measured: Choosing columns before the split made pure noise look 92.7% accurate. On data put away at the start, it scored 49.8%, the same as guessing.
How many wrong labels can a model survive?
It depends on the model and on the kind of mistake. Random mistakes are often outvoted by correct examples. Mistakes that follow a pattern, like two classes often swapped, do much more damage.
Check a sample of labels by hand, and look for rows where a model strongly disagrees with the label.
What we measured: On 1,797 handwritten digits, with 30% of training labels wrong, a random forest scored 96.4% when the mistakes were random. With look-alike digits swapped, it scored 82.3%.
How do you handle a rare class, such as fraud?
Do not trust accuracy, because a model that never says "fraud" can still look accurate. Use precision and recall. Precision: when it says fraud, how often is it right? Recall: how much of the fraud does it catch?
Teams often copy or invent extra rare rows, called oversampling or SMOTE. First, try moving the decision threshold: the score above which the model says fraud.
What we measured: On handwritten digits, over 100 runs, a plain model with a tuned threshold beat every rebalanced model left at 0.5. With very few rare rows, choose the threshold by cross-validation instead.
When do you need streaming data instead of batch?
Use streaming when a feature must be fresh within seconds. An example is how many payments a card made in the last minute. Use batch when hourly or daily data is good enough.
Streaming costs more to build and run, so choose it only when fresh data changes the result.
5. Monitoring and drift
A model can get worse without any error message. This part is about noticing.
What is the difference between data drift and concept drift?
Data drift means the inputs change, such as users getting younger. Concept drift means the link between inputs and the right answer changes, such as fraudsters using a new trick.
Prediction drift means the model's outputs change. Each one can happen without the others, so watch all three.
How is data drift measured?
Compare the live data with the training data, one feature at a time. PSI (population stability index) and the KS (Kolmogorov-Smirnov) test both score how different two sets of values are. A bigger score means more change.
Set alert limits carefully. Too low, and harmless changes ring the alarm. Too high, and real drift slips through.
Does a drift alarm mean the model got worse?
Not always. A drift alarm says the data changed. It does not say the model's errors went up. Some changes do not matter to the model, and some harmful ones do not trigger the alarm.
Measure real errors when labels arrive. Treat a drift alarm as a reason to check the model's recent predictions.
What we measured: We sent five kinds of new data to a house-price model, 100 batches each. For busy districts, the PSI alarm rang on every batch, but the model was not hurt. When we raised every price 20% in a simulation, error rose 64% and PSI never rang.
How do you monitor a model when true answers arrive late?
For fraud, the true answer may come weeks later. Meanwhile, watch the signals you have now. These include input drift, prediction drift, how often people override the model, and predictions near the threshold.
When the labels finally arrive, measure real accuracy and compare.
6. Scaling and cost
GPUs are expensive. Expect a question on keeping serving fast without a huge bill.
How do you reduce the cost of serving models on GPUs?
Group requests into batches, so each GPU does more work at once. Use a smaller or quantized model, which stores numbers with fewer bits. Cache repeated answers.
Scale the number of servers up and down with traffic. Use spot machines, cheaper cloud servers that can be taken back at short notice, for work that can be interrupted.
What should you autoscale model servers on?
CPU use is often a poor signal for GPU models. Scale on things closer to the work, such as requests waiting in the queue or GPU use.
A new GPU server can take minutes to load a large model. So scale up before the queue gets long.
7. MLOps for LLMs
Many teams now run large language models too. The ideas carry over, with a few new parts.
How is running an LLM app different from classic MLOps?
You usually do not train the model yourself. Instead, you manage prompts, retrieved documents and tool calls. Quality is harder to measure, because answers are free text.
So evals, tracing each request, and watching token costs become the main work. Evals are repeatable tests of answer quality.
How do you test a change to an LLM app before release?
Run a fixed set of test inputs before and after the change, and compare scores. Use code checks where you can, and an LLM judge that you have checked against people.
Run the same eval a few times first, to see how much the score moves on its own.
Learn it properly, not just the answers
Every answer on this page comes from our AI Engineering course: 112 lessons on RAG, evals, agents, serving, security and MLOps. Many of them are built around a real experiment. You learn why the answer is right, which is what an interviewer checks with the second question. 10 lessons are free to read, with no card needed.