The audio visual interface to work with agents

Talk to your coding agent. It answers with something you can point at.

Wispr Flow made talking to your computer faster than typing. Saral makes the answer faster than reading: your agent replies with diagrams, interactive models and a voice, instead of a wall of text.

Download for Mac · this week See it work ↓

Saral

The same question

Left: 400 words. Right: a diagram you can point at.

Chat

Where do the 1.8 seconds go in our voice assistant?

Where the time goes

Your assistant answers in about 1.8 seconds, measured from the moment the user stops speaking to the moment the speaker plays the first sound. That budget is spent in four sequential stages, and nothing in the current design overlaps.

The four stages

Speech-to-text, about 320 ms. The recognizer needs a short window of trailing audio before it will emit a final transcript. Streaming partials are available earlier, but the pipeline ignores them.

LLM first token, about 900 ms. This is half of the budget. The request is sent only after the final transcript arrives, so the model has done no work at all while the user was talking. Prompt length, model size and the provider's queue all sit inside this number.

Text-to-speech, about 400 ms. The synthesizer waits for the first LLM token, then produces the first chunk of audio.

Playback, about 180 ms. Buffering and device latency before the first sample is audible.

Where the slack is

The 900 ms LLM stage is the only one large enough to matter, and it is the only one that is idle while the user is still speaking. Speech-to-text already streams partial transcripts; if the model subscribes to those partials, the first token can be in flight before the sentence ends. In practice this recovers roughly 600 ms, bringing the total to about 1.2 seconds.

The tradeoff

Starting early means sometimes starting on an incomplete sentence. If the user pauses mid-thought, the model may answer half a question, which is worse than answering slowly. The usual guard is an end-of-turn detector: hold the answer until there has been a short window of silence, typically 200 to 300 ms, before committing it to speech. On a genuine pause you pay that window; the rest of the time you pay nothing.

Suggestions

Instrument each stage separately before changing anything. Then move the model onto partial transcripts behind a flag, add the silence guard, and measure both first-sound latency and the rate of answers committed on incomplete sentences.

Scroll. Read. Rebuild the picture in your head.
Saral

Where the 1.8 seconds go

900 of the 1,800 ms are the LLM, and it is idle while you talk.

A turn runs four stages end to end and nothing overlaps. Speech-to-text emits a final transcript only after a short window of trailing audio, and the LLM subscribes to that transcript, so it does no work at all while you are still speaking.

turn_pipeline.py:41-88 · stt_stream.py:12-30

Where do you think most of it goes?

You said text-to-speech. It is the LLM: 900 of 1,800 ms.

Text-to-speech takes 400 ms and starts the moment the first token lands. The LLM has not begun at all while you are talking, and that is where the slack is.

turn_pipeline.py:41-88 · stt_stream.py:12-30

leaves out: network time, barge-in

The LLM waits because the request is built from the final transcript. A partial transcript is available about 600 ms earlier and is good enough to start on, as long as an end-of-turn guard holds the answer until the sentence looks finished.

The LLM idles while you finish your sentence. Start it on the partial transcript and you save about 600 milliseconds.

turn_pipeline.py:41-88 · end_of_turn.py:1-44

“Where do the 1.8 seconds go?”

Point at the slow part. Ask why. Every block names the file it came from.

How it works

Follow one change from “something's slow” to shipped.

A voice assistant answers in 1.8 seconds. Here is how a PM, an engineer and their agent take it to 1.2 seconds, with Saral answering at each step.

PM

Saral

Where the 1.8 seconds go

900 of the 1,800 ms are the LLM, and it is idle while you talk.

A turn runs four stages end to end and nothing overlaps. Speech-to-text emits a final transcript only after a short window of trailing audio, and the LLM subscribes to that transcript.

turn_pipeline.py:41-88 · stt_stream.py:12-30

Where do you think most of it goes?

“Where do the 1.8 seconds go?”

Every island above is the kind of thing Saral draws for you. They compute; they are not pictures.

Runs on your Mac.

Local speech.

Listening and speaking run on your machine. Fast, and nothing leaves it.

Your subscription.

Uses the Claude or Codex plan you already pay for. No new API bill.

Your sessions.

Attaches to the Claude Code and Codex sessions you already have open.

Stop reading the answer.

Download for Mac · this week