Developer Guide · 10

Realtime Voice (S2S WebSocket)

10.1 S2S module (legacy path)

app/voice/router.py — WebSocket /api/voice/ws proxies OpenAI Realtime API.

text
Browser ──WebSocket──→ FastAPI ──WebSocket──→ wss://api.openai.com/v1/realtime
  • Tool injection on session.update; JWT in query; S2S_API_KEY / S2S_BASE_URL

Separate process plugins/voice/ + Core routes app/routes/voice_live.py:

text
Browser ─WebRTC→ LiveKit Server ← Worker(jellyfish_voice)

Worker ─HTTP/SSE─→ Core /api/voice/live/{session,delegate,llm/v1/chat/completions}
                      └── delegates via chat._stream_agent(yolo=True), shared thread_id
  • Bridge token: app/voice/live_token.py
  • Config: users/{uid}/voice_agent_config.json; UI Settings → Voice
  • Deploy: docker-compose.voice.yml; env LIVEKIT_*, VOICE_BRIDGE_SECRET, FISH_*, DASHSCOPE_API_KEY