MLOps

MLOps is the set of practices and infrastructure for getting a trained machine learning model into production and keeping it working reliably once it's there — versioning, deployment, monitoring, and retraining. It picks up where model development leaves off: training a good model is only useful if it can be deployed safely, observed in production, and kept current as the world it's making predictions about changes.

Key components

• Versioning — tracking not just the model artifact but the exact data and code that produced it, so any deployed model can be traced back to a reproducible training run.
• Staged deployment pipeline — rolling a new model out to a small slice of traffic first (a canary or shadow deployment), checking it behaves as expected, and expanding gradually, rather than a single all-at-once release to every user.
• Model performance monitoring — tracking prediction quality in production (accuracy, calibration, drift from training-time behavior) as its own signal, separate from system uptime or latency.
• Retraining trigger or cadence — a defined condition or schedule for retraining the model (a performance-drop threshold, a fixed interval, a data-drift alert), rather than deploying it once and leaving it running indefinitely.

Why monitoring model performance differs from monitoring system health

A model can be perfectly "up" — serving requests, low latency, no errors — while its predictions have quietly gotten worse, because the real-world data it's now seeing has drifted from the data it was trained on. Standard infrastructure monitoring (CPU, latency, error rate) has nothing to say about this; a request that gets a confidently wrong answer looks identical, at the infra level, to one that gets a right answer. Catching this requires monitoring built specifically for model behavior: tracking prediction distributions, comparing live input data to training data, and where possible measuring actual outcomes against predictions after the fact.

Common pitfalls

• Only system/infra health monitored — dashboards track uptime and latency while prediction quality silently degrades from data drift, and nobody notices until a downstream metric or a customer complaint makes the degradation impossible to ignore.
• No versioning tying a model to its data and code — a production issue traces back to "some model deployed sometime last quarter," with no way to reproduce exactly what was trained, on what data, or debug what went wrong.
• No staged rollout — a new model goes to 100% of traffic immediately, so a regression reaches every user before anyone notices, instead of being caught in a small canary slice first.
• Retraining left to happen "whenever" — with no defined trigger or cadence, a model quietly keeps serving on stale training data long after the population it predicts on has shifted.
• No rollback path — a bad deployment has no quick way back to the previous known-good model, turning a monitoring alert into an extended incident.
• Monitoring built only for the model that's live today — the pipeline isn't set up to compare a new candidate model against the current one before full rollout, so regressions are found in production instead of in a staged comparison.

Learn more

• Machine Learning Model Card for documenting a model's intended use and limitations, complementary to actually operating it in production.
• Data Pipeline for the data-movement and transformation processes that feed both training and live inference.
• Configuration Management for the broader discipline of tracking exactly what configuration produced a given running system.
• On-Call Rotation for who actually responds when a model performance or rollout alert fires.