Pick WebSockets when streaming PCM audio between two backend servers in a datacenter (such as a Twilio/SIP telephony bridge). Pick direct client-to-API media streaming with short-lived ephemeral session secrets—the architecture used by GPT Realtime Assistant on iPhone, iPad, and Mac—when running on real-world cellular and Wi-Fi networks where low jitter, acoustic echo control, and keeping user audio off middleman application servers matter.
| Networking Property | Direct Client WebRTC + Ephemeral Secret | Backend-Proxied WebSocket Audio | Direct Client WebSocket (Raw Key) |
|---|---|---|---|
| Transport layer | UDP-based SRTP media + SCTP data channel for events | TCP client → app server → TCP OpenAI WebSocket | TCP WebSocket from device |
| Cellular / Wi-Fi packet-loss behavior | Graceful — adaptive jitter buffer avoids head-of-line stalls | TCP head-of-line blocking can cause bursty audio stalls | TCP head-of-line blocking on spotty mobile links |
| Extra server hop latency | Zero — audio flows straight between Apple device and OpenAI | Adds middleman server relay latency + bandwidth cost | Zero relay hop |
| API key security on mobile client | Safe — backend mints short-lived ephemeral session token only | Safe — master API key stays on backend proxy | Unsafe unless user manually supplies own BYOK key |
| Middleman audio exposure | App developer's backend never touches raw audio frames | Middleman proxy receives every raw PCM audio chunk | No middleman proxy |
| Acoustic echo & audio device routing | Integrates cleanly with native iOS/macOS voice I/O | Requires manual PCM chunking, resampling & echo handling | Requires manual PCM buffer management |
Why TCP Head-of-Line Blocking Hurts Mobile Voice
WebSockets run over TCP, which guarantees strict byte ordering: if a single packet drops while your train enters a tunnel or your office Wi-Fi roams between access points, the OS holds back every subsequent audio frame until the missing packet is retransmitted—creating an awkward silence followed by robotic fast-forwarded speech.
UDP-based real-time media tracks (WebRTC) were designed specifically for live human conversation, smoothing brief jitter while keeping conversational turn-taking crisp.
The Privacy Win of Ephemeral Session Secrets
If an app proxies WebSocket audio through its own backend just to hide its OpenAI API key, that vendor's server has your raw room audio in memory. By issuing short-lived session secrets instead, GPT Realtime Assistant lets your Apple device talk straight to OpenAI's Realtime endpoint—so Arte Soft's backend handles only credit balances and receipts, never your voice or screenshots.
| Connection Stage | What Happens Under the Hood | Privacy & Performance Impact |
|---|---|---|
| 1. Session handshake | App requests a short-lived session secret from `assistant-api.wonek.com` | Master OpenAI credentials never ship inside the iOS/Mac binary |
| 2. Direct Realtime stream | Your iPhone, iPad, or Mac streams directly to OpenAI's Realtime API | Zero middleman audio proxy delay; Arte Soft never receives your audio |
| 3. Live transcript & events | Transcript deltas and Brief/Balanced/Detailed answers render live on screen | Immediate visual feedback alongside spoken turns |
| 4. Session End & settlement | Stream closes; unused reserved credits refund; transcript saves locally | Transcripts, Tasks & attachments stay 100% on your device |
| 5. Offline Mac alternative | Local Whisper Large v3 Turbo + Qwen 3 8B on Apple Silicon | Zero network sockets opened during session |
Why iOS Foreground Rules Apply to Live Realtime Streams
Live bidirectional sessions maintain an active real-time socket and microphone session while you watch or interact with the workspace. On iPhone and iPad, keep GPT Realtime Assistant open on screen during your session—locking the phone or switching away cleanly ends the active connection so you never burn credits unintentionally in your pocket.
| Profile | Best Choice | Why It Wins |
|---|---|---|
| Using Realtime voice AI right now on iPhone, iPad & Mac | GPT Realtime Assistant ($9.99/mo) | Turnkey ephemeral-secret architecture + 58 languages + 17 templates |
| Building a PSTN phone-call bot on Twilio / Vonage | Server-to-Server WebSockets | Datacenter fiber has near-zero packet loss and easy SIP PCM bridging |
| Building a custom iOS/browser voice widget from scratch | WebRTC + Ephemeral Session Token API | OpenAI's recommended pattern for untrusted client devices |
| Air-gapped transcription & Q&A with zero network packets | GPT Realtime Assistant Offline Mac Mode | Runs entirely in Apple Silicon unified memory |
Frequently Asked Questions
Does Arte Soft's server record or relay my voice audio?
No. Arte Soft's backend (`assistant-api.wonek.com`) only mints short-lived session secrets and tracks credit balances; your audio and visual context stream directly from your device to OpenAI's Realtime API.
Why did my live session stop when I locked my iPhone?
GPT Realtime Assistant runs live sessions strictly while open on screen on iOS and iPadOS. Ending on lock prevents accidental pocket audio streaming and unexpected credit drain.
Can I avoid network streaming altogether on Mac?
Yes. Apple Silicon Mac subscribers can run Whisper Large v3 Turbo and Qwen 3 8B locally offline with zero network traffic.
Related Comparisons & Session Guide
Managed AI Credits vs Bring Your Own OpenAI API Key (BYOK)
Compare managed AI credit subscriptions (GPT Realtime Assistant) vs Bring-Your-Own-Key (BYOK) OpenAI Realtime apps on cost, key security, setup, and Mac perks.
GPT Realtime vs GPT Realtime Mini Audio Models Compared
Compare OpenAI GPT Realtime vs GPT Realtime Mini audio models—plus offline Mac Whisper + Qwen 3 8B—on nuance, speed, translation, and credit cost.
