Repository navigation
fix(duplex): keep late transcripts on the audio they describe - #7264
Conversation
The adapter attached queued transcript fragments only inside the frame handler while the gate was open. A fragment arriving after the last accepted frame stayed queued: the next utterance took it as its first fragment and anchored on it, or the unclaimed timeout emitted it under a new id 3 s later. _close_burst now attaches due fragments before closing. A fragment whose span sits on audio the last anchored burst already played is emitted at once as a text-only generation, so it can no longer anchor the next burst. Fixes #7227
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
_open_burst resolved a pending generate_reply for any burst that carried a message. A text-only generation for words the model already spoke could claim the request, and the speech that followed was then emitted as a turn of the model's own. _open_burst takes the kind of burst it opens: speech, text or tool. Only speech is the reply asked for.
| break | ||
| late += 1 | ||
| if late: | ||
| burst = self._open_burst(kind="text") |
There was a problem hiding this comment.
🟡 Late transcripts follow newer replies
During generate_reply, _open_burst(kind="text") queues late text behind the pending requested handle. _scheduling_task keeps that handle current until its speech finishes. The old transcript therefore displays after the newer reply.
Learn more
A requested reply owns the scheduler's current speech handle before it calls RealtimeSession.generate_reply in _realtime_reply_task. A non-user-initiated generation creates another normal speech handle in _on_generation_created. The scheduler cannot authorize that queued handle until the requested handle receives and finishes its speech. Marking historical text as non-user-initiated preserves reply correlation, but routes it through this blocked queue.
Example: A request is waiting when " there." arrives for the previous "Hello" burst. The model then speaks "How can I help?" as the requested reply. The room receives "How can I help?" before the older " there." transcript.
Recommended fix: Preserve the pending reply for the next speech while delivering historical text before that reply. Add an explicit historical-text distinction to the generation contract and handle it without placing a normal speech handle behind the current requested handle, or provide equivalent scheduler ordering.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
will fix it separately
Problem:
DuplexRealtimeAdapterattaches queued transcript fragments only inside the frame handler, while the gate is open, so a fragment that arrives after the last accepted frame stays queued. The next utterance takes it as its first fragment and sets its time anchor from it, or the 3 s unclaimed timeout emits it under a new id with an error log.Fix:
_close_burstattaches due fragments before it closes the burst, so words that arrive between the last frame and the close still join it. A fragment whose span, by the last anchored burst's anchor, sits on audio that burst already played is emitted at once as a text-only generation under a fresh id, the shape the unclaimed path already produces, so it can no longer anchor the next burst.Fixes #7227. Supersedes #7244.
Context for reviewing and coding agents