A data scientist finishes a fraud model on a Friday. She sends the platform team a single file, model.pkl, and a message: "It's ready, just load it and serve it." On Monday the platform engineer loads that file on a server and it explodes with an error nobody understands. AttributeError. Something about a numpy version. The model that hit 96 percent accuracy on her laptop will not even open on the machine that is supposed to run it.

You have heard "works on my machine" from backend engineers. For machine learning it is worse, and it is worse for a specific reason. A normal web service is mostly code, and code is fairly forgiving about which exact library version runs it. A trained model is a frozen snapshot of numbers that was produced by an exact stack: this Python, this version of PyTorch, this numpy, this CUDA driver, on this operating system. Change any one of those and the model can refuse to load, or load and quietly return wrong answers, which is the more dangerous outcome because nothing throws an error.

Packaging is the discipline of making the model run the same way on every machine, forever, by shipping the machine along with the model.

This is the first real piece of the car we started building in Topic 1. Before you can serve a model, scale it, or monitor it, you have to be able to move it off the laptop without it falling apart. That is what this lesson is about.

The default handoff in machine learning is a pickle: joblib.dump(model, "model.pkl") or torch.save. It feels like saving the model. It is not enough, for three concrete reasons.

It is version-locked. A pickle does not store the model in some neutral format. It stores references to the exact Python classes and library versions that existed when you saved it. Load model.pkl under a different scikit-learn or PyTorch and you get either a hard crash or, worse, a silent mismatch where the object loads but behaves differently.

It captures no dependencies. The file itself says nothing about which Python interpreter, which numpy, which CUDA produced it. The person receiving it has to guess the environment, and guessing wrong is the whole failure mode above.

It is a security hole. Unpickling executes arbitrary code by design. Loading a .pkl you did not create can run anything on your server. The Python docs say it plainly: never unpickle data from an untrusted source. That alone makes bare pickles a poor way to move models between teams.

So people add a requirements.txt. That helps, but it only pins the Python packages. It does not pin the operating system, the Python interpreter version, the CUDA driver, or native libraries like libgomp that native code links against. The fix is to stop shipping a file and start shipping an environment.

Packaging a model correctly means putting three things into one sealed, immutable unit:

1. The model artifact itself: the trained weights, as model.pt, model.onnx, or model.pkl.
2. The exact pinned dependencies: not torch, but torch==2.3.1; not numpy, but numpy==1.26.4. Down to the patch version, from a locked file.
3. The inference code: the code that loads the model, runs the same preprocessing used in training, calls predict, and returns a result over HTTP.

That third one trips up more teams than you would think. If your preprocessing (the tokenizer, the scaler, the feature encoding) lives only in the training notebook and does not travel with the model, the serving code will feed the model differently shaped inputs than training did. The model does not crash. It just returns worse answers, and you find out weeks later from a drop in a business metric. Shipping the preprocessing code inside the same unit as the weights is what prevents this.

The technology that seals all three together is the container, and the standard is Docker. A container image bundles the OS, the interpreter, the pinned libraries, the weights, and the code into one immutable artifact with a content hash. Here is what actually lives inside that image.

A Dockerfile is the recipe that builds the image. For a model service it is short and readable. The order of the lines matters more than it looks, and we will see why in a moment.

# Pin the base. Never python:latest, or the floor moves under you.
FROM python:3.11-slim

# System libraries some ML wheels link against at runtime.
RUN apt-get update && apt-get install -y --no-install-recommends \
    libgomp1 && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Install deps FIRST, from a fully pinned lockfile, and as its own layer.
# This layer is cached and reused on every build where deps did not change.
COPY requirements.lock .
RUN pip install --no-cache-dir -r requirements.lock

# Copy the model weights and the serving code LAST, because they
# change most often. Only these thin top layers rebuild on a code edit.
COPY model.pt .
COPY serve.py .

EXPOSE 8000
# One process, listening on a port, answering prediction requests.
CMD ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8000"]

Two habits in that file are worth burning into memory. First, requirements.lock is fully pinned (torch==2.3.1, not torch>=2.3), because a range means two builds a month apart can install different versions and produce two different environments from the same file. Second, dependencies are installed before the model and code are copied. Docker builds the image as a stack of cached layers, so putting the rarely changing, expensive install step near the bottom means a one-line code change rebuilds only the thin top layers in seconds. Get the order backwards and every tiny edit reinstalls torch.

Here is the payoff that makes all this pinning worth the trouble. In Topic 1 we said the thing that makes MLOps hard is that the data moves on its own. There is a second, quieter source of wrong predictions that has nothing to do with drift: the training environment and the serving environment silently disagree.

Suppose you trained with numpy 1.26 and served with numpy 2.0. A default changed between those versions, so a preprocessing step rounds slightly differently. The model still loads. It still returns numbers. Those numbers are just a little wrong, on every single request, forever, and no alarm fires. This is called train/serve skew, and pinned dependencies are the cure.

The workflow that guarantees parity is simple:

• After training, freeze the exact environment: pip freeze > requirements.lock. This writes every package at its exact installed version.
• Build the serving image from that same requirements.lock.
• Now the serving container installs byte-identical libraries to the ones that trained the model.

The lockfile is the contract between the person who trained the model and the machine that serves it. It says: recreate my exact environment, or do not run. That is train/serve parity, and it is the reason a pinned requirements file is not bureaucracy, it is correctness.

Open a naive deep learning image and it is often 5 to 10 GB. A plain backend image is 100 to 300 MB. The gap is almost entirely one thing: the GPU stack. CUDA plus cuDNN can be 2 to 4 GB, and a full PyTorch install adds a couple more. Size is not cosmetic. An 8 GB image is slow to push, slow to pull onto a new node, and slow to autoscale, which directly hurts you during a traffic spike when new pods need to come up fast.

There are four moves that shrink these images, and they stack:

The multi-stage build is the one worth explaining, because it is pure profit. You use one stage to install and compile everything (which needs compilers, caches, and dev headers), then you copy only the finished packages and the model into a second, clean slim stage. The compilers and build junk never reach the final image. Watch the size fall as you apply all four moves.

Writing Dockerfiles by hand for every model is repetitive and easy to get subtly wrong. BentoML is an ML-native packaging tool that does this for you. The flow has four steps: save the model into a versioned store, define a Service that describes the API, run bentoml build to snapshot everything into a Bento, and let BentoML generate the Docker image.

import bentoml
import numpy as np
from pydantic import BaseModel

# 1. After training, save the model into BentoML's versioned store.
#    BentoML records its version, framework, and input/output signature.
saved = bentoml.pytorch.save_model("fraud_model", trained_model)

class Transaction(BaseModel):
    amount: float
    features: list[float]

# 2. Define the Service: which model to load and what the API looks like.
@bentoml.service(
    resources={"cpu": "2"},
    traffic={"timeout": 10},
)
class FraudService:
    # BentoML loads this exact model version into the running container.
    model_ref = bentoml.models.get("fraud_model:latest")

    def __init__(self) -> None:
        self.model = bentoml.pytorch.load_model(self.model_ref)

    @bentoml.api
    def predict(self, txn: Transaction) -> dict:
        x = np.array([txn.features], dtype=np.float32)
        score = float(self.model(x).item())
        return {"fraud_score": score, "flagged": score > 0.8}

With that file written, two commands finish the job. bentoml build snapshots the model reference, the code, and the pinned dependencies into a Bento, a self-describing bundle. bentoml containerize fraud_service:latest reads that Bento and produces a standard Docker image, Dockerfile generated for you, with request batching, health checks, and OpenAPI docs already wired in. You never hand-write the Dockerfile, and every model on the team gets packaged the same consistent way.

These are not competing choices so much as increasing levels of rigor. The question is how far up you need to go.

A useful rule: the moment a model leaves the machine it was trained on, it needs a container. And if your team ships more than a couple of models, a model-native tool like BentoML pays for itself by making every package look the same, so the platform team is not debugging a different bespoke Dockerfile for every model.

The mistake to avoid is over-packaging a tiny model. Wrapping a 2 MB scikit-learn classifier in an 8 GB GPU image is waste on every axis: storage, pull time, cost, and startup . Match the packaging to the model.

This is standard practice at every company running machine learning at scale, and the pattern is remarkably consistent.

Uber's Michelangelo standardized packaging so that hundreds of teams did not each invent their own way to containerize a model. A model trained on the platform came out the other side as a deployable, versioned artifact with its dependencies sealed in, ready to serve. Standardizing the package was a big part of what let so many teams ship without a platform engineer babysitting each one.

Hugging Face distributes models with a config and tokenizer bundled alongside the weights precisely because weights alone are not runnable. The preprocessing (the tokenizer) has to travel with the model, or the same text turns into different token ids and the model sees inputs it was never trained on. That is the train/serve parity idea, shipped by default.

DoorDash and Spotify both describe serving stacks where models are containerized and the exact dependency set is pinned per model, so a model that trained cleanly cannot fail at serve time because of a version mismatch. The container is the boundary that makes the model a normal deployable unit, indistinguishable from any other service to the infrastructure that runs it.

The through-line is the same everywhere. A model is not deployable until its environment is nailed down and sealed. Once you can put a model in a box that runs identically anywhere, everything downstream in this track (serving it under load, scaling it, monitoring it) becomes possible. The box comes first.