Immutable Infrastructure
An approach where infrastructure components are never modified after deployment; instead, updates are made by replacing entire instances with new ones built from a common base image.
Immutable infrastructure treats servers like cattle, not pets. Instead of SSH-ing into a server to update software or change configuration, you build a new machine image with the desired state, deploy it as a replacement, and destroy the old one. No server is ever modified in place after it starts running.
This approach eliminates configuration drift, where servers that were initially identical gradually diverge through ad-hoc changes. It makes deployments reproducible because the same image always produces the same behavior. Rollbacks are fast because you simply redeploy the previous image. And security is improved because servers are regularly replaced with freshly built instances.
For AI serving infrastructure, immutable infrastructure ensures that model serving environments are consistent and reproducible. When deploying a new model version, you build a new container image containing the model weights, dependencies, and serving code, then replace all running instances. This guarantees that every instance serves exactly the same model version, eliminating the risk of partially updated fleets.
Related Terms
A/B Testing
A controlled experiment comparing two or more variants to determine which performs better on a defined metric, using statistical methods to ensure reliable results.
Feature Flag
A software mechanism that enables or disables features at runtime without deploying new code, used for gradual rollouts, A/B testing, and targeting specific user segments.
MLOps
The set of practices combining machine learning, DevOps, and data engineering to reliably deploy, monitor, and maintain ML models in production.
Model Serving
The infrastructure and systems that host trained ML models and handle inference requests in production, optimizing for latency, throughput, and cost.
Semantic Search
Search that understands the meaning and intent behind a query rather than just matching keywords, typically powered by embedding-based similarity comparison.
CI/CD (Continuous Integration / Continuous Deployment)
An automated software practice where code changes are continuously integrated into a shared repository, tested, and deployed to production, reducing manual intervention and accelerating delivery cycles.