loading
All projects
2026Low-latency voice reasoning with sub-300ms responseActively maintained
Conversational Voice AI Agent
A real-time bidirectional conversational AI agent engineered with GPT-4.1, WebSocket audio streaming, and ElevenLabs TTS synthesis.
<300mslatency
Full DuplexWebSocket streaming
Traditional conversational AI agents suffer from jarring latency pauses between user speech and generated voice response. This project eliminates that friction by building an ultra-low latency, bidirectional voice agent pipeline operating with sub-300ms latency.
The architecture uses full-duplex WebSockets to stream raw audio chunks from the client to a Python/FastAPI backend. Incoming audio is processed by streaming speech-to-text with voice activity detection (VAD) and interrupt handling, allowing users to naturally interrupt the agent mid-sentence.
Reasoning is powered by GPT-4.1 with structured system prompts and token streaming directly into ElevenLabs WebSocket synthesis endpoints. Sentences and clauses are synthesized incrementally as tokens generate, enabling instant audio playback with natural cadence and multilingual adaptability.
highlights
- Sub-300ms end-to-end turnaround from user utterance to streaming audio response
- Duplex WebSocket streaming architecture with Voice Activity Detection (VAD) and barge-in interruption
- Chunked token-to-speech pipelining: audio synthesis begins on the first generated clause
- Multilingual prompt engineering ensuring consistent persona, pacing, and tone across topics
- GPT-4.1
- ElevenLabs API
- WebSocket
- LangChain
- Python
- FastAPI
- AsyncIO