gRPC
A high-performance, open-source RPC framework that uses Protocol Buffers for serialization and HTTP/2 for transport, enabling efficient service-to-service communication with strong typing.
gRPC (Google Remote Procedure Call) is designed for fast, reliable communication between services. It uses Protocol Buffers (protobuf) to define service contracts and serialize data in a compact binary format, making it significantly faster than JSON-based REST APIs for internal service communication.
HTTP/2 transport provides multiplexing (multiple requests over a single connection), header compression, and bidirectional streaming. These features make gRPC ideal for microservice architectures where services make thousands of inter-service calls per second. Latency reductions of 2-10x over REST are typical for high-throughput internal APIs.
For AI infrastructure, gRPC is commonly used for model serving APIs (TensorFlow Serving, Triton Inference Server), feature store access, and real-time prediction pipelines where latency matters. The strong typing from protobuf definitions ensures that service contracts are explicit and breaking changes are caught at compile time rather than runtime.
Related Terms
A/B Testing
A controlled experiment comparing two or more variants to determine which performs better on a defined metric, using statistical methods to ensure reliable results.
Feature Flag
A software mechanism that enables or disables features at runtime without deploying new code, used for gradual rollouts, A/B testing, and targeting specific user segments.
MLOps
The set of practices combining machine learning, DevOps, and data engineering to reliably deploy, monitor, and maintain ML models in production.
Model Serving
The infrastructure and systems that host trained ML models and handle inference requests in production, optimizing for latency, throughput, and cost.
Semantic Search
Search that understands the meaning and intent behind a query rather than just matching keywords, typically powered by embedding-based similarity comparison.
CI/CD (Continuous Integration / Continuous Deployment)
An automated software practice where code changes are continuously integrated into a shared repository, tested, and deployed to production, reducing manual intervention and accelerating delivery cycles.