Google has expanded Gemini Live with direct camera streaming and real-time screen awareness. Users can point their phone camera at physical objects, code, or diagrams and converse with the assistant naturally with sub-300-millisecond latency.

The underlying Gemini 3.8 Live architecture processes simultaneous audio and visual inputs in a continuous stream without turn-taking delays. Google has also opened developer WebSocket endpoints across Google AI Studio and Vertex AI for custom interactive agent applications.

The rollout sets a new standard for hands-free developer workflows, hardware diagnostics and interactive tutoring. Teams building multimodal voice agents can now bypass complex pipeline chaining of separate speech-to-text and vision models.

What this means

  • Gemini Live now understands live camera video streams and computer screens in real time.
  • End-to-end speech and vision latency is reduced to sub-300 milliseconds.
  • Developer WebSocket endpoints are available in Google AI Studio and Vertex AI.

Read the announcement

We explained the update in plain English. Use the original source below for the full announcement and any later changes.

Read the original source