WebSocket
A communication protocol providing full-duplex, persistent connections between client and server over a single TCP connection, enabling real-time bidirectional data exchange.
WebSockets upgrade an initial HTTP connection into a persistent, two-way channel. Unlike HTTP's request-response model where the client must initiate every interaction, WebSockets allow both the server and client to send messages at any time without the overhead of establishing new connections.
This protocol is essential for real-time features: chat applications, live dashboards, collaborative editing, streaming AI responses, and push notifications. The persistent connection eliminates the latency of repeated HTTP handshakes and the overhead of polling, making it far more efficient for high-frequency updates.
For AI products, WebSockets power streaming LLM responses where tokens are sent to the client as they are generated, creating the typewriter effect users expect. They also enable real-time collaboration features, live model inference dashboards, and instant notification systems. The main engineering considerations are connection management at scale, reconnection handling, and load balancer configuration for long-lived connections.
Related Terms
A/B Testing
A controlled experiment comparing two or more variants to determine which performs better on a defined metric, using statistical methods to ensure reliable results.
Feature Flag
A software mechanism that enables or disables features at runtime without deploying new code, used for gradual rollouts, A/B testing, and targeting specific user segments.
MLOps
The set of practices combining machine learning, DevOps, and data engineering to reliably deploy, monitor, and maintain ML models in production.
Model Serving
The infrastructure and systems that host trained ML models and handle inference requests in production, optimizing for latency, throughput, and cost.
Semantic Search
Search that understands the meaning and intent behind a query rather than just matching keywords, typically powered by embedding-based similarity comparison.
CI/CD (Continuous Integration / Continuous Deployment)
An automated software practice where code changes are continuously integrated into a shared repository, tested, and deployed to production, reducing manual intervention and accelerating delivery cycles.