Realtime API reference

Dialt Realtime API

AsyncAPI 3.1.0 · API version 1.0.0

Use the Realtime API for live voice and text conversations. Connect to wss://api.dialt.com/v1/realtime and send a start message with your session key. Wait for ready before sending audio or text. The browser and Python SDKs handle this connection sequence for you. For auth probes, post-session feedback and client failure reports, open a separate connection and send auth, feedback or client_error as the first frame instead of start. These requests return ok or error and end the connection without a ready event.

Need exact payload schemas or validation constraints? Download the machine-readable AsyncAPI contract.

Operations

sendClientMessages

send · Send the start frame, microphone audio, and client controls

requestReply, start, inputText, setVoice, continueWithAgent, toolDeferred, toolResult, toolProgress, toolPartialResult, toolInteractionUpdate, toolCancelClient, playbackStopped, playbackReport, microphoneAudio, webrtcOffer, webrtcCandidate, webrtcIceComplete, interrupt, setTools, setToolChoice, triggerReply, injectContext, playbackPaused, resolveToolPermission, setInstructions, wrapUp, inputAudioDone

receiveServerMessages

receive · Receive lifecycle events, assistant audio, transcripts, and tool calls

replyAck, responseDone, interruptAck, conversationSnapshot, conversationItem, speechStarted, speechStopped, ready, silentMic, turn, asr, utterance, injectContextAck, handoffAck, done, sessionEndRequested, interrupted, canceled, toolDeferredAck, toolDeferredResume, toolCall, toolCancelServer, toolJobDeliveryFailed, toolJobNarration, working, toolInteractionUpdateAck, voiceChanged, error, assistantAudio, webrtcIce, webrtcAnswer, webrtcBye, textDelta, permissionPending, permissionApproved, permissionRefused, sessionClosingSoon, lowCreditWarning, playbackPause, playbackResume, permissionExpired, permissionResolution, policyFlag, policyError, policyCheck, policySettled, asrCorrection

sendHandoffAgent

send · Request one atomic agent handoff at a safe boundary

continueWithAgent

receiveHandoffAck

receive · Receive the correlated handoff acknowledgement

handoffAck

sendOneShotRequest

send · Send one authenticated auth, feedback or client-error request as the first frame on a fresh connection, instead of start

auth, feedback, clientError

receiveOneShotResponse

receive · Receive ok or error for a one-shot request; this connection does not become a live conversation

ok, error

Messages

Client and server frames

MessageTitleSummary
requestReplyrequestReplyRequires request_reply_v1. FIFO capacity 16, lifetime receipt budget 256. Same ID and payload recovers latest state, including after resume. No implicit user message or replay under a new ID.
replyAckreplyAck
responseDoneresponseDoneEmitted once at logical response teardown across all segments, with status completed, cancelled or failed. May precede the final done segment fence. This is generation completion, not proof of playback; preserve queued audio and wait for each segment's done fence.
interruptAckinterruptAck
conversationSnapshotconversationSnapshotAfter ready on fresh connection and resume. Atomically replaces the read-only retained window, at most 512 items and 512 KiB compact UTF-8 JSON.
conversationItemconversationItem
speechStartedspeechStarted
speechStoppedspeechStopped
startstartOpens a conversation. api_key may be omitted when the WebSocket handshake supplied Authorization: Bearer <key>. Voice mode requires audio; text mode omits it and requires the WebSocket transport.
inputTextinputTextCommit a user turn in text mode. Dialt emits the ordinary asr, turn, text_delta, utterance, and done events without audio frames.
setVoicesetVoiceSelect a current stock voice key from GET /v1/voices for the next reply. Empty, unknown, retired or unavailable keys emit invalid_voice and leave the current voice unchanged. In voice sessions, changing the listening route or language hints also emits invalid_voice and requires a new call. Text sessions may change language. A voice event confirms every accepted selection, including the active key.
setInstructionssetInstructionsReplace the session instructions (start.mode.instructions) mid-session; applies from the next reply and re-stamps prompt_config. new_speaker: true remains available for an independent speaker change and folds the call so far into one prior-agent transcript. With new_speaker the frame is refused while a reply is in flight (error code instructions_busy, retryable): send it after done.
handoffAgentcontinueWithAgentAtomically continue the conversation with an initialized replacement agent at a safe boundary. One continuation may be pending. The broker sends queued, then applied or rejected within 10 seconds. It does not retry, survive reconnect, or start specialist speech. The former handoff_agent type remains a compatibility alias.
toolResulttoolResult
toolDeferredtoolDeferredDetach an eligible call from its voice turn while preserving managed job identity.
toolProgresstoolProgress
toolPartialResulttoolPartialResultDeliver a structured segment of an in-flight tool's eventual result without resolving the call.
toolInteractionUpdatetoolInteractionUpdateHost-side close of an open interaction (docs/tool-orchestration.md, Partial results and interactions): the decision was resolved out-of-band, cancelled, or superseded by newer intent -- without completing the parent call. Stops the ask's queued or actively-speaking narration and tells the model the decision is closed. Always answered by toolInteractionUpdateAck; late/duplicate/unknown updates are rejected deterministically.
toolCancelClienttoolCancelClient
playbackStoppedplaybackStopped
readyreadyAuthoritative initial session state, including selected voice and negotiated audio format, not merely an echo of start. Keep resume_token private and replace it after a successful resume; later acknowledged configuration changes update the initial state.
silentMicsilentMic
turnturn
asrasr
injectContextAckinjectContextAckAuthoritative accepted/rejected acknowledgement for a correlated inject_context request.
handoffAckhandoffAckCorrelated handoff acknowledgement. Rejection leaves the active configuration unchanged.
utteranceutterance
donedonePer-segment output fence identified by turn_id. It does not settle the logical response: response_done reports logical teardown and may arrive before the final done. Neither event proves client playback.
sessionEndRequestedsessionEndRequested
interruptedinterrupted
canceledcanceled
textDeltatextDeltaStreamed assistant text, sent as the model produces it (ahead of audio and the final utterance frame).
permissionPendingpermissionPendingA permission-gated tool call is held for the configured caller or application approval source.
permissionApprovedpermissionApprovedA pending permission was approved and the admitted execution path returned, including accepted deferral. Emitted after tool_call, not as a pre-execution receipt. Handle the tool call and return its result without waiting for this event. This does not establish business success. The id matches the held tool call.
permissionRefusedpermissionRefusedA pending permission was declined; the held tool call did not run.
toolCalltoolCall
toolDeferredAcktoolDeferredAck
toolDeferredResumetoolDeferredResume
toolCancelServertoolCancelServer
toolJobDeliveryFailedtoolJobDeliveryFailedA queued background-job completion voiceover could not be delivered. The turn still closes with done; job_ids lists jobs whose spoken delivery is terminally failed (their results remain in conversation context and can be asked about), requeued lists jobs re-queued for one more proactive spoken attempt (a newer user turn may supersede it, as with any queued completion).
toolJobNarrationtoolJobNarrationNarration-lifecycle ack for a queued tool_partial_result `interaction` (docs/tool-orchestration.md, Partial results and interactions). Scoped to interaction-kind jobs only -- a plain completion has no such wire-visible lifecycle.
workingworkingDialt is blocking on a tool result with nothing audible (the silence after the spoken bridge, until the final reply's first audio), or that wait has ended.
toolInteractionUpdateAcktoolInteractionUpdateAckDeterministic answer to every toolInteractionUpdate. applied=false carries a stable reason: invalid_interaction_id, invalid_state, unknown_interaction (never seen, or from before a reconnect -- interaction state does not survive a resume), or already_closed (first close wins: cancel dismissed it, its call resolved, a newer ask replaced it, or a duplicate of this very frame -- a user turn superseding the ask's narration deliberately does NOT close the decision).
voiceChangedvoiceChanged
errorerror
webrtcIcewebrtcIceExperimental. Sent over the signaling WebSocket right after a start frame carrying transport.kind webrtc. Build the RTCPeerConnection with these ice_servers, then send webrtcOffer.
webrtcOfferwebrtcOfferExperimental. Client's SDP offer, sent over the signaling WebSocket once the RTCPeerConnection is built with webrtcIce's servers.
webrtcCandidatewebrtcCandidateExperimental. One client ICE candidate sent after the offer when webrtcIce advertises trickle_ice.
webrtcIceCompletewebrtcIceCompleteExperimental. Marks the end of client candidate gathering after all webrtcCandidate frames.
webrtcAnswerwebrtcAnswerExperimental. Server's SDP answer to webrtcOffer. The call then lives on the peer connection; the signaling socket may drop afterwards without affecting it.
webrtcByebyeSent on the webrtc "control" data channel just before the server closes the peer connection — the webrtc transport's counterpart of a WebSocket close frame/code.
microphoneAudiomicrophoneAudio
assistantAudioassistantAudio
interruptinterruptLegacy uncorrelated stop remains supported. Targeted interruption requires targeted_interrupt_v1, operation_id and exactly one response_id or target_operation_id. A stale target cannot stop a newer response. WebRTC does not support targeted interruption.
setToolssetToolsMid-session tool swap: agents that change capability by call phase no longer need a reconnect, which would destroy the conversation.
setToolChoicesetToolChoiceReplace the effective tool_choice mid-session; applies from the next reply. one_shot: true reverts to the previous choice after the next caller or explicitly host-requested reply consumes the new one; autonomous jobs do not consume it. An invalid value is rejected with an `error` frame (code invalid_tool_choice) and changes nothing. A mid-session setTools resets tool_choice to "auto"; re-apply it after swapping tools.
triggerReplytriggerReplyClient-triggered reply. Silently ignored if a reply is already in flight — do not stack these.
injectContextinjectContextText/context injection with authoritative delivery acknowledgement. Supply message_id to correlate inject_context_ack and typed-turn asr events.
sessionClosingSoonsessionClosingSoonWarns the client shortly before the server closes the connection due to the session time limit (or scoped credential expiry) — gives the client a chance to wrap up, or to reconnect with start.resume_token once closed.
lowCreditWarninglowCreditWarningAPI-key (paid) sessions only. Warns the client once when the account's remaining balance is running low, so the call doesn't just cut off mid-conversation when credit runs out.
playbackReportplaybackReport
playbackPauseplaybackPausePause and retain the unplayed PCM for this reply. Sent only to WebSocket clients advertising playback_pause_v1. Acknowledge the matching hold ID with playback_paused.
playbackResumeplaybackResumeResume the retained PCM for the matching hold ID and turn ID. Ignore stale IDs. A hard interruption or cancellation discards the held audio instead.
playbackPausedplaybackPausedAcknowledges a provisional pause with retained duration and hardware output tail. Repeated acknowledgements may report a buffer limit error; they never authorize resuming a different hold.
permissionExpiredpermissionExpiredA pending request expired without execution. Reset, tool replacement and disconnect also invalidate pending ids.
resolveToolPermissionresolveToolPermissionSubmit an application decision as the authenticated session host. Only application-source requests accept this command; the application authenticates its reviewer. Arguments and actor claims cannot be supplied.
permissionResolutionpermissionResolutionAcknowledges an application decision. A duplicate, invalid, expired or wrong-source decision cannot execute an action. Completion uses permission_approved/refused plus normal tool-result delivery.
policyFlagpolicyFlag
policyErrorpolicyError
policyCheckpolicyCheck
policySettledpolicySettled
wrapUpwrapUpRequest a graceful host-initiated sign-off in voice or text mode, including when mode.end_call is disabled. An active reply receives a bounded grace before the sign-off. No dedicated acknowledgement is sent; continue handling reply events and normal closure, and drain local playback. Sending this frame does not confirm the farewell was heard.
inputAudioDoneinputAudioDoneCommit accumulated voice input when start.mode.turn_detection is client. Send after the audio for that turn. Ignored when client turn detection is not active; not a text-mode control and not an acknowledgement of transcription or reply completion.
asrCorrectionasrCorrectionReplaces the text of an earlier voice transcript. When turn_id is present, update that turn rather than appending a new user turn. turn_id can be omitted; do not fabricate a correlation. Optional utterance_id, revision and policy_version describe the policy transcript revision.
authauthCheck a credential on a fresh WebSocket connection. Send as the first frame instead of start, with api_key or a Bearer handshake header. Returns ok on success or error on failure and the connection ends. Does not open a conversation, emit ready, or prove credit or session capacity is available.
feedbackfeedbackSubmit feedback as the first frame on a fresh authenticated WebSocket instead of start. Requires an owned recording and either rating or nonblank text. Success returns ok and the connection ends. Re-submitting feedback for the same session replaces the earlier feedback. Does not consume a conversation slot and remains available while the server drains.
clientErrorclientErrorReport a client-side failure as the first frame on a fresh authenticated WebSocket instead of start. Omit session_id when failure predates session creation; when present it must name an owned recording. Success returns ok and the connection ends, without consuming a conversation slot. detail and context are diagnostic log content: do not send transcripts, replies, credentials or user conversation text.
okokSuccessful acknowledgement for a fresh-connection auth, feedback or client_error request. The connection ends after this response; it is not the ready event for a conversation.

Payload schemas and exact constraints remain available in the machine-readable AsyncAPI contract.

Field reference

Message payloads

Expand a message to see its exact fields - name, type, whether it's required, and its description.

requestReplyrequestReply - Requires request_reply_v1. FIFO capacity 16, lifetime receipt budget 256. Same ID and payload recovers latest state, including after resume. No implicit user message or replay under a new ID.
FieldTypeRequiredDescription
type"request_reply"Yes
operation_idstringYes
instructionsstringNo
replyAckreplyAck
FieldTypeRequiredDescription
type"reply_ack"Yes
operation_idstringYes
acceptedbooleanYes
status"queued" | "started" | "completed" | "cancelled" | "failed" | "rejected"Yes
response_idstringNo
reasonstringNo
responseDoneresponseDone - Emitted once at logical response teardown across all segments, with status completed, cancelled or failed. May precede the final done segment fence. This is generation completion, not proof of playback; preserve queued audio and wait for each segment's done fence.
FieldTypeRequiredDescription
type"response_done"Yes
response_idstringYes
status"completed" | "cancelled" | "failed"Yes
interruptAckinterruptAck
FieldTypeRequiredDescription
type"interrupt_ack"Yes
operation_idstringYes
acceptedbooleanYes
status"interrupted" | "cancelled" | "stale" | "rejected"Yes
response_idstringNo
target_operation_idstringNo
reasonstringNo
conversationSnapshotconversationSnapshot - After ready on fresh connection and resume. Atomically replaces the read-only retained window, at most 512 items and 512 KiB compact UTF-8 JSON.
FieldTypeRequiredDescription
type"conversation_snapshot"Yes
itemsarray<ConversationItem>Yes
first_indexintegerYes
conversationItemconversationItem
FieldTypeRequiredDescription
type"conversation_item"Yes
itemConversationItemYes
indexintegerYes
first_indexintegerYes
response_idstringNo
turn_idstringNo
speechStartedspeechStarted
FieldTypeRequiredDescription
type"speech_started"Yes
speech_idstringYes
timestamp_msnumberYes
speechStoppedspeechStopped
FieldTypeRequiredDescription
type"speech_stopped"Yes
speech_idstringYes
timestamp_msnumberYes
reasonstringNo
startstart - Opens a conversation. api_key may be omitted when the WebSocket handshake supplied Authorization: Bearer <key>. Voice mode requires audio; text mode omits it and requires the WebSocket transport.
FieldTypeRequiredDescription
type"start"Yes
session_idstringYes
initial_historyarray<ImportedConversationItem>NoCompleted ordered history, atomically validated before ready. At most 128 KiB compact UTF-8 JSON, 16 KiB per item. Unique item/call IDs; every call has exactly one later result. Nonempty history is forbidden on resume. Import never executes tools or requests a reply.
participantsarray<object>NoInternal Dialt-organization Owner pilot, not production-qualified. Requires voice, mode.multispeaker=true, WebSocket and server turn detection. Unique IDs identify isolated source lines, not authenticated people. Fixed session roster, inherited unchanged on resume; never saved on a Design.
api_keystringNoPersistent or scoped key. Optional when supplied in the WebSocket Authorization header.
audioAudioConfigNo
clientClientInfoNo
modeModeYes
transportTransportConfigNo
resume_tokenstringNoA token from a previous session's ready frame. When it matches a still-live, unexpired stash from that session's disconnect, this session resumes its conversation context (and voice/tools/instructions, where this start frame doesn't already override them) instead of starting empty. An expired, unknown, consumed, or cross-account token is rejected with a non-retryable resume_failed error; callers may then deliberately start fresh without the token. Single-use: each resume consumes the stash and this session's own ready frame carries a new token.
inputTextinputText - Commit a user turn in text mode. Dialt emits the ordinary asr, turn, text_delta, utterance, and done events without audio frames.
FieldTypeRequiredDescription
type"input_text"Yes
textstringYesOne committed user turn. Accepted only when mode.modality is text.
setVoicesetVoice - Select a current stock voice key from GET /v1/voices for the next reply. Empty, unknown, retired or unavailable keys emit invalid_voice and leave the current voice unchanged. In voice sessions, changing the listening route or language hints also emits invalid_voice and requires a new call. Text sessions may change language. A voice event confirms every accepted selection, including the active key.
FieldTypeRequiredDescription
type"set_voice"Yes
voicestringYes
setInstructionssetInstructions - Replace the session instructions (start.mode.instructions) mid-session; applies from the next reply and re-stamps prompt_config. new_speaker: true remains available for an independent speaker change and folds the call so far into one prior-agent transcript. With new_speaker the frame is refused while a reply is in flight (error code instructions_busy, retryable): send it after done.
FieldTypeRequiredDescription
type"set_instructions"Yes
instructionsstringYesReplacement instructions, same layer as start's mode.instructions; capped like start. Must be non-empty: blank text is rejected with an error frame (code invalid_instructions) and changes nothing.
new_speakerbooleanNoA different agent takes the call from here: fold the conversation so far into a prior-agent transcript.
handoffAgentcontinueWithAgent - Atomically continue the conversation with an initialized replacement agent at a safe boundary. One continuation may be pending. The broker sends queued, then applied or rejected within 10 seconds. It does not retry, survive reconnect, or start specialist speech. The former handoff_agent type remains a compatibility alias.
FieldTypeRequiredDescription
type"continue_with_agent" | "handoff_agent"Yes
operation_idstringYesClient-generated identifier echoed by handoff_ack.
instructionsstringYesReplacement agent instructions: who the new agent is and what it does. Must be non-empty; blank text is rejected with a handoff_ack whose status is rejected.
toolsarray<ToolDefinition>YesReplacement client tool manifest.
voicestringNoOptional replacement stock voice key from GET /v1/voices.
contextstringNoOptional specialist context added with the handoff.
tool_choicetool_choiceNoOptional generation constraint validated against the replacement tools and applied with the agent.
tool_choice_one_shotbooleanNoWhen true, tool_choice constrains only the incoming agent's first caller- or host-requested reply, then restores auto. Requires tool_choice.
toolResulttoolResult
FieldTypeRequiredDescription
type"tool_result"Yes
idstringYes
contentanyYes
outcome"succeeded" | "failed" | "cancelled" | "timed_out" | "unknown"NoTerminal real-world outcome. Send it whenever the host can establish it: succeeded with verified=true lets the assistant confirm the action (send it when the host holds proof, such as a booking confirmation ID); unknown makes the assistant say it cannot confirm, which suits queued or fire-and-forget actions; failed makes it say the action did not go through. Missing or invalid values normalize to unknown. Guidance: https://dialt.com/docs/api/websocket/#tool-result-outcome
verifiedbooleanNoTrue when the host has proof the action completed, such as a confirmation ID. May be true only with outcome=succeeded; this pair lets the assistant confirm the action.
continue_withobjectNoAtomically end this tool turn and continue the conversation with the replacement agent. The broker validates the whole result and continuation before mutation, suppresses any not-yet-audible old-agent recorded bridge or answer, requests exactly one new-agent reply, and reports the outcome with handoff_ack.
handoffcontinue_withNoDeprecated compatibility alias for continue_with. A frame cannot contain both fields.
toolDeferredtoolDeferred - Detach an eligible call from its voice turn while preserving managed job identity.
FieldTypeRequiredDescription
type"tool_deferred"Yes
idstringYes
handlestringYes
status_labelstringNo
toolProgresstoolProgress
FieldTypeRequiredDescription
type"tool_progress"Yes
idstringYes
notestringYes
toolPartialResulttoolPartialResult - Deliver a structured segment of an in-flight tool's eventual result without resolving the call.
FieldTypeRequiredDescription
type"tool_partial_result"Yes
idstringYes
contentanyYesStructured partial result content. Capped at 2 KiB by default; at most 8 partials are accepted per call. Does not resolve the call.
replybooleanNoAsk Dialt to narrate this milestone immediately. The partial always lands in context, but the proactive reply is skipped if another reply is already in flight. Ignored when `interaction` is given.
interactionobjectNoMarks this partial as needing a user decision rather than a routine milestone. Unlike reply:true alone, never silently dropped when busy -- falls back to the same completion-job queue tool completions use, preempting them, with a far more persistent delivery retry. See tool_job_narration for its lifecycle ack.
toolInteractionUpdatetoolInteractionUpdate - Host-side close of an open interaction (docs/tool-orchestration.md, Partial results and interactions): the decision was resolved out-of-band, cancelled, or superseded by newer intent -- without completing the parent call. Stops the ask's queued or actively-speaking narration and tells the model the decision is closed. Always answered by toolInteractionUpdateAck; late/duplicate/unknown updates are rejected deterministically.
FieldTypeRequiredDescription
type"tool_interaction_update"Yes
idstringYesThe parent call id (or deferred handle) the interaction was raised on.
interaction_idstringYesThe interaction's stable identity: the host-chosen interaction.id from the raising tool_partial_result, or the broker-derived id echoed in tool_job_narration's interaction_ids.
state"resolved" | "cancelled" | "superseded"Yesresolved: the decision was made out-of-band (e.g. in the host's own UI) and the tool is continuing. cancelled: the decision is no longer needed. superseded: replaced by newer intent (typically the user's latest instruction); raise a fresh interaction (new id) if a decision is still needed.
notestringNoOptional short human-readable reason, surfaced to the model as context (e.g. "approved in the IDE").
toolCancelClienttoolCancelClient
FieldTypeRequiredDescription
type"tool_cancel"Yes
idstringYes
playbackStoppedplaybackStopped
FieldTypeRequiredDescription
response_idstringNoTarget discarded playback even after generation completed. Required for targeted reporting; use discarded_ms. WebRTC owns accounting and sends no duplicate client report.
type"client_event"Yes
event"playback_stopped"Yes
remaining_msnumberNo
discarded_msnumberNo
barge_seqintegerNoOptional interruption sequence; the server uses the current sequence when omitted.
readyready - Authoritative initial session state, including selected voice and negotiated audio format, not merely an echo of start. Keep resume_token private and replace it after a successful resume; later acknowledged configuration changes update the initial state.
FieldTypeRequiredDescription
type"ready"Yes
capabilitiesarray<string>NoSupported conversation_v1, request_reply_v1, targeted_interrupt_v1 and speech_activity_v1. Text has no activity; WebRTC has no targeted interruption.
active_speechnull | SpeechActivityNoNull on fresh ready and resumed ready.
modality"voice" | "text"No
voicestringYes
namestringYes
admissionobjectNoPresent for Spot-originated calls, including pinned resumes. Terms remain fixed for the conversation; resume restores these terms without a new fallback decision.
voicesarray<Voice>Yes
audioobjectNoThe negotiated downlink format for binary assistantAudio frames, echoing start.audio's output_encoding/output_sr (or their defaults, pcm16 at 16000, when omitted). Assert on this instead of assuming an encoding.
resume_tokenstringYesPass this back as start.resume_token on a reconnect to resume this session's conversation context if the connection drops. Short-lived (150 seconds by default), single-use and account-bound; the server stash survives broker restarts and broker-instance transfers.
session_uuidstringNoServer-owned logical session identifier. Identical across successful resumes and broker-instance transfers, and new for a fresh dialogue even when the client-selected session_id is reused. Distinct from session_id, which is client-chosen for correlation; this is not a credential.
silentMicsilentMic
FieldTypeRequiredDescription
type"silent_mic"Yes
reason"digital_silence" | "near_silent"Yes
duration_msnumberYes
peakintegerYesHighest absolute PCM16 sample observed in the detection window.
turnturn
FieldTypeRequiredDescription
type"turn"Yes
response_idstringNoOpaque logical response ID shared across bridge, tool wait and final segments.
turn_idstringYesCompatibility segment ID; response_id identifies the full logical response.
asrasr
FieldTypeRequiredDescription
participant_idParticipantIdNo
type"asr"Yes
textstringYes
turn_idstringYesId of the turn this transcript belongs to.
message_idstringNoClient-generated correlation id for an injected text turn. Omitted for voice input.
input_source"voice" | "text"NoWhether the final transcript came from microphone speech or injected text.
utterance_idstringNo
revisionintegerNo
policy_versionintegerNo
injectContextAckinjectContextAck - Authoritative accepted/rejected acknowledgement for a correlated inject_context request.
FieldTypeRequiredDescription
type"inject_context_ack"Yes
message_idstringYesThe client-generated id from inject_context.
acceptedbooleanYesAuthoritative broker decision. False means the injection was not delivered.
role"user" | "context"No
input_source"text" | "context"No
turn_idstringNoStable server turn id for an accepted typed user turn.
reply_startedbooleanNoFor context with reply=true, whether the requested reply also claimed the floor.
retryablebooleanNoFor a rejection, whether retrying later is safe.
detailstringNoHuman-readable rejection reason.
queuedbooleanNoTrue when accepted context requested a reply that is waiting for the caller/assistant floor. Waiting updates coalesce. The acknowledgement records the original acceptance outcome.
handoffAckhandoffAck - Correlated handoff acknowledgement. Rejection leaves the active configuration unchanged.
FieldTypeRequiredDescription
type"handoff_ack"Yes
operation_idstringYesThe operation_id from continue_with_agent, tool_result.continue_with, or their deprecated handoff aliases.
status"queued" | "applied" | "rejected"Yes
acceptedbooleanYesTrue for queued or applied; false for rejected.
detailstringNoHuman-readable rejection detail when status is rejected.
utteranceutterance
FieldTypeRequiredDescription
type"utterance"Yes
response_idstringNo
textstringYes
turn_idstringYesId of the turn this assistant utterance belongs to.
barge_seqintegerNo
correctedbooleanNoTrue when this frame revises the earlier interrupted utterance with the same turn_id.
utterance_idstringNo
revisionintegerNo
policy_versionintegerNo
donedone - Per-segment output fence identified by turn_id. It does not settle the logical response: response_done reports logical teardown and may arrive before the final done. Neither event proves client playback.
FieldTypeRequiredDescription
type"done"Yes
response_idstringNo
turn_idstringYesId of the turn that just completed.
sessionEndRequestedsessionEndRequested
FieldTypeRequiredDescription
type"session_end_requested"Yes
farewellstringNoThe farewell argument the agent passed to end_call, present whether or not it was spoken: it is not spoken when the reply text of that turn ended a sentence. Its presence does not confirm playback or that the caller heard it. Absent when the end was not requested through the tool.
interruptedinterrupted
FieldTypeRequiredDescription
type"interrupted"Yes
response_idstringNo
turn_idstringYesId of the barged turn.
clearbooleanNo
barge_seqintegerNo
canceledcanceled
FieldTypeRequiredDescription
type"canceled"Yes
response_idstringNo
turn_idstringYesId of the eager speculative turn that was retracted.
textDeltatextDelta - Streamed assistant text, sent as the model produces it (ahead of audio and the final utterance frame).
FieldTypeRequiredDescription
type"text_delta"Yes
response_idstringNo
turn_idstringYesId of the turn this text belongs to.
deltastringYesAn incremental slice of assistant text, as the model produces it. Concatenate deltas for a given turn_id to reconstruct the streamed text; the final text still arrives as utterance.
welcomebooleanNoTrue when this delta belongs to the session greeting.
permissionPendingpermissionPending - A permission-gated tool call is held for the configured caller or application approval source.
FieldTypeRequiredDescription
type"permission_pending"Yes
idstringYesSession-scoped approval id. External approvers return this id through resolve_tool_permission.
toolstringYesName of the held tool.
argumentsobjectYesExact held arguments for review. A decision cannot change them.
source"caller" | "application"YesRequired authorization source.
status_labelstringYesHuman-readable action label.
expires_atnumberYesExpiry as Unix time in seconds; a repeated request does not extend it.
permissionApprovedpermissionApproved - A pending permission was approved and the admitted execution path returned, including accepted deferral. Emitted after tool_call, not as a pre-execution receipt. Handle the tool call and return its result without waiting for this event. This does not establish business success. The id matches the held tool call.
FieldTypeRequiredDescription
type"permission_approved"Yes
idstringYesThe approval id that was resolved.
toolstringYes
permissionRefusedpermissionRefused - A pending permission was declined; the held tool call did not run.
FieldTypeRequiredDescription
type"permission_refused"Yes
idstringYesThe approval id that was resolved.
toolstringYes
toolCalltoolCall
FieldTypeRequiredDescription
type"tool_call"Yes
response_idstringNo
idstringYes
namestringYes
argsobjectYes
toolDeferredAcktoolDeferredAck
FieldTypeRequiredDescription
type"tool_deferred_ack"Yes
acceptedbooleanYes
idstringYes
handlestringNo
job_idstringNo
status_labelstringNo
deferred_timeoutnumberNo
reasonstringNoRejection or replay reason. wait_expired means the first defer arrived after the ordinary observation deadline; existing deferred handles remain usable.
toolDeferredResumetoolDeferredResume
FieldTypeRequiredDescription
type"tool_deferred_resume"Yes
idstringYes
handlestringYes
namestringYes
job_idstringYes
status_labelstringYes
toolCancelServertoolCancelServer
FieldTypeRequiredDescription
type"tool_cancel"Yes
idstringYes
toolJobDeliveryFailedtoolJobDeliveryFailed - A queued background-job completion voiceover could not be delivered. The turn still closes with done; job_ids lists jobs whose spoken delivery is terminally failed (their results remain in conversation context and can be asked about), requeued lists jobs re-queued for one more proactive spoken attempt (a newer user turn may supersede it, as with any queued completion).
FieldTypeRequiredDescription
type"tool_job_delivery_failed"Yes
job_idsarray<string>YesJobs whose spoken delivery is terminally failed.
requeuedarray<string>YesJobs re-queued for one more proactive spoken attempt. A newer user turn may supersede the retry, as with any queued completion.
detail"queue_deadline_exceeded" | "voiceover_inaudible" | "voiceover_markup_suppressed" | "voiceover_transport_failed"YesStable failure code. Any other internal cause is reported as voiceover_transport_failed.
toolJobNarrationtoolJobNarration - Narration-lifecycle ack for a queued tool_partial_result `interaction` (docs/tool-orchestration.md, Partial results and interactions). Scoped to interaction-kind jobs only -- a plain completion has no such wire-visible lifecycle.
FieldTypeRequiredDescription
type"tool_job_narration"Yes
state"queued" | "started" | "superseded" | "cancelled" | "failed" | "resolved"Yesqueued: enqueued, floor busy -- always bounded: a deadline watchdog re-attempts delivery every INTERACTION_QUEUE_DEADLINE_S (10s default, up to INTERACTION_QUEUE_DEADLINE_ROUNDS rounds per delivery cycle), and an ask whose own blocked turn holds the floor yields that turn and is voiced immediately. started: now being spoken. superseded: a real user turn or barge preempted it before anything was heard (a fresh interaction can be raised again if still needed), a newer interaction on the same call replaced it, or the host closed it as superseded via tool_interaction_update. cancelled: cancel_pending_job, the host's tool_cancel of the call behind it, or a host tool_interaction_update state=cancelled dismissed it first. failed: a terminal give-up -- retried past the delivery bound, or the queue deadline expired with the floor unclaimable; telemetry-only (nothing was spoken -- fall back to your own UI for the decision). resolved: the decision no longer awaits the voice channel -- the interaction's own call sent its terminal result, or the host closed it as resolved via tool_interaction_update. A requeued ask (claimed but unheard) emits a fresh queued, so every delivery cycle ends in an observable state change.
job_idsarray<string>YesThe interaction job(s) this state applies to. Multiple pending interactions are coalesced into one turn, so more than one id may share a `started` event.
interaction_idsarray<string>YesStable interaction identities this state applies to (host-chosen `interaction.id`, or the broker-derived "<call_id>#i<n>"), aligned with the affected subset of job_ids. How a host learns a broker-derived id: it appears on the first queued/started ack for the ask.
kindstringYesAlways "interaction" today -- present so a future kind could share this ack without a new message type.
workingworking - Dialt is blocking on a tool result with nothing audible (the silence after the spoken bridge, until the final reply's first audio), or that wait has ended.
FieldTypeRequiredDescription
type"working"Yes
activebooleanYesTrue while a tool-calling turn waits for its result. False when the answer begins or the turn ends. Pure state signal; the SDK adds no background audio.
toolInteractionUpdateAcktoolInteractionUpdateAck - Deterministic answer to every toolInteractionUpdate. applied=false carries a stable reason: invalid_interaction_id, invalid_state, unknown_interaction (never seen, or from before a reconnect -- interaction state does not survive a resume), or already_closed (first close wins: cancel dismissed it, its call resolved, a newer ask replaced it, or a duplicate of this very frame -- a user turn superseding the ask's narration deliberately does NOT close the decision).
FieldTypeRequiredDescription
type"tool_interaction_update_ack"Yes
idstring | nullYesThe canonical server call id the update resolved to, or null if the call reference was unknown.
interaction_idstring | nullYes
statestring | nullYesEcho of the requested state.
appliedbooleanYes
reason"invalid_interaction_id" | "invalid_state" | "unknown_interaction" | "already_closed" | NoneYesnull when applied; otherwise the stable rejection reason.
voiceChangedvoiceChanged
FieldTypeRequiredDescription
type"voice"Yes
voicestringYes
namestringYes
errorerror
FieldTypeRequiredDescription
type"error"Yes
detailstringYesHuman-readable detail. May keep changing freely — do not match against it; use code instead.
codestringYesStable machine-readable error code (e.g. unauthorized, server_busy, invalid_tools, missing_instructions, invalid_instructions). Safe to match on for retry/handling logic.
retryablebooleanYesWhether retrying the same request is expected to succeed. Defaults per-code (e.g. server_busy/server_restarting are retryable; unauthorized/invalid_start_frame/missing_instructions are not) and is always present.
retry_after_secondsnumberNoOptional advisory delay for an explicit new attempt. spot_unavailable may provide 5 seconds, while retryable remains false. spot_unavailable and spot_voice_unavailable are terminal for automatic reconnect.
webrtcIcewebrtcIce - Experimental. Sent over the signaling WebSocket right after a start frame carrying transport.kind webrtc. Build the RTCPeerConnection with these ice_servers, then send webrtcOffer.
FieldTypeRequiredDescription
type"webrtc_ice"Yes
ice_serversarray<IceServer>YesSTUN plus (when configured) short-lived TURN credentials. Do not build the RTCPeerConnection until this arrives — gathering ICE earlier would miss TURN.
trickle_icebooleanNoWhen true, send webrtcOffer immediately, followed by webrtcCandidate frames and webrtcIceComplete. When absent or false, send a fully gathered offer.
webrtcOfferwebrtcOffer - Experimental. Client's SDP offer, sent over the signaling WebSocket once the RTCPeerConnection is built with webrtcIce's servers.
FieldTypeRequiredDescription
type"webrtc_offer"Yes
sdpstringYesThe client peer connection's local SDP offer, created after webrtcIce's servers are configured.
webrtcCandidatewebrtcCandidate - Experimental. One client ICE candidate sent after the offer when webrtcIce advertises trickle_ice.
FieldTypeRequiredDescription
type"webrtc_candidate"Yes
candidateIceCandidateYes
webrtcIceCompletewebrtcIceComplete - Experimental. Marks the end of client candidate gathering after all webrtcCandidate frames.
FieldTypeRequiredDescription
type"webrtc_ice_complete"Yes
webrtcAnswerwebrtcAnswer - Experimental. Server's SDP answer to webrtcOffer. The call then lives on the peer connection; the signaling socket may drop afterwards without affecting it.
FieldTypeRequiredDescription
type"webrtc_answer"Yes
sdpstringYesThe server peer connection's local SDP answer.
ice_serversarray<IceServer>No
webrtcByebye - Sent on the webrtc "control" data channel just before the server closes the peer connection — the webrtc transport's counterpart of a WebSocket close frame/code.
FieldTypeRequiredDescription
type"bye"Yes
codeintegerYesClose code from the same vocabulary as a WebSocket close code.
reasonstringNo
microphoneAudiomicrophoneAudio

PCM16 little-endian mono microphone audio at audio.sr, sent in binary WebSocket frames. Keep frames flowing for the whole session, silence included: the server measures turn-end silence in received audio, so a client that sends only while the user speaks leaves the turn open.

assistantAudioassistantAudio

Mono assistant audio in the output_encoding and output_sr selected in ready.audio, received in binary WebSocket frames.

interruptinterrupt - Legacy uncorrelated stop remains supported. Targeted interruption requires targeted_interrupt_v1, operation_id and exactly one response_id or target_operation_id. A stale target cannot stop a newer response. WebRTC does not support targeted interruption.
FieldTypeRequiredDescription
operation_idstringNo
response_idstringNo
target_operation_idstringNo
type"interrupt" | "cancel"YesEither spelling stops the in-flight reply the same way.
setToolssetTools - Mid-session tool swap: agents that change capability by call phase no longer need a reconnect, which would destroy the conversation.
FieldTypeRequiredDescription
type"set_tools"Yes
toolsarray<ToolDefinition>YesReplacement tool manifest, same shape as start's mode.tools.
setToolChoicesetToolChoice - Replace the effective tool_choice mid-session; applies from the next reply. one_shot: true reverts to the previous choice after the next caller or explicitly host-requested reply consumes the new one; autonomous jobs do not consume it. An invalid value is rejected with an `error` frame (code invalid_tool_choice) and changes nothing. A mid-session setTools resets tool_choice to "auto"; re-apply it after swapping tools.
FieldTypeRequiredDescription
type"set_tool_choice"Yes
tool_choicetool_choiceYes
one_shotbooleanNo
triggerReplytriggerReply - Client-triggered reply. Silently ignored if a reply is already in flight — do not stack these.
FieldTypeRequiredDescription
type"trigger_reply"Yes
promptstringNoOptional nudge text for the reply. Omit to make Dialt speak first with no real user turn.
injectContextinjectContext - Text/context injection with authoritative delivery acknowledgement. Supply message_id to correlate inject_context_ack and typed-turn asr events.
FieldTypeRequiredDescription
type"inject_context"Yes
message_idstringNoClient-generated correlation id echoed by inject_context_ack and, for role=user, the canonical final asr event. New SDKs always send this field.
textstringYesThe text to inject. Required, 1–2000 characters.
role"user" | "context"Yes"user": treated exactly like a real spoken user turn arriving as text instead of voice — visible in the transcript, answered the same way a turn.end is. "context": silent background information (a CRM lookup, a RAG snippet) appended to the model's context; never shown to the client as something the user said.
replybooleanNoWhether this injection requests a reply. Context is appended immediately; a busy context reply is queued until idle. User-role injections can be rejected while busy. With message_id, rejection is an inject_context_ack with accepted=false; legacy uncorrelated calls receive an inject_rejected error frame.
sessionClosingSoonsessionClosingSoon - Warns the client shortly before the server closes the connection due to the session time limit (or scoped credential expiry) — gives the client a chance to wrap up, or to reconnect with start.resume_token once closed.
FieldTypeRequiredDescription
type"session_closing_soon"Yes
seconds_remainingnumberYesApproximate seconds until the server closes this connection: the session hit its maximum duration, a scoped API credential is expiring, or (text sessions, reason="idle") the connection has been idle for configured text idle timeout with nothing in flight. Sent once, shortly before the close; any client frame cancels an idle close.
reason"idle"NoPresent only for a text session's idle close. Absent for the session time limit and credential expiry.
lowCreditWarninglowCreditWarning - API-key (paid) sessions only. Warns the client once when the account's remaining balance is running low, so the call doesn't just cut off mid-conversation when credit runs out.
FieldTypeRequiredDescription
type"low_credit_warning"Yes
seconds_remainingnumberYesApproximate remaining talk time at the current balance. Sent once, the first time it drops under the warning threshold — read-only, never adjusts the balance itself.
playbackReportplaybackReport
FieldTypeRequiredDescription
type"client_event"Yes
event"playback_report"Yes
underrunsintegerYesTimes this reply's audio queue drained at the speaker before the next chunk arrived.
starved_msnumberNoTotal audible starvation across those underruns.
max_gap_msnumberNo
event_age_msnumberNoAge of the browser MessageEvent when the SDK handled done. This indicates browser event-queue delay; it is not a raw network-arrival timestamp.
turn_idstringNo
playbackPauseplaybackPause - Pause and retain the unplayed PCM for this reply. Sent only to WebSocket clients advertising playback_pause_v1. Acknowledge the matching hold ID with playback_paused.
FieldTypeRequiredDescription
type"playback_pause"Yes
hold_idintegerYes
turn_idstringYes
playbackResumeplaybackResume - Resume the retained PCM for the matching hold ID and turn ID. Ignore stale IDs. A hard interruption or cancellation discards the held audio instead.
FieldTypeRequiredDescription
type"playback_resume"Yes
hold_idintegerYes
turn_idstringYes
playbackPausedplaybackPaused - Acknowledges a provisional pause with retained duration and hardware output tail. Repeated acknowledgements may report a buffer limit error; they never authorize resuming a different hold.
FieldTypeRequiredDescription
type"client_event"Yes
hold_idintegerYes
turn_idstringYes
event"playback_paused"Yes
remaining_msnumberYesEstimated hardware output tail after scheduled playback is stopped. The broker waits this long before resuming.
retained_msnumberYesUnplayed audio retained for this hold. More than 3000 ms abandons recovery.
errorstringNo
permissionExpiredpermissionExpired - A pending request expired without execution. Reset, tool replacement and disconnect also invalidate pending ids.
FieldTypeRequiredDescription
type"permission_expired"Yes
idstringYesThe approval id that was resolved.
toolstringYes
resolveToolPermissionresolveToolPermission - Submit an application decision as the authenticated session host. Only application-source requests accept this command; the application authenticates its reviewer. Arguments and actor claims cannot be supplied.
FieldTypeRequiredDescription
type"resolve_tool_permission"Yes
idstringYesThe pending application approval id, scoped to this authenticated session.
decision"approve" | "decline"Yes
permissionResolutionpermissionResolution - Acknowledges an application decision. A duplicate, invalid, expired or wrong-source decision cannot execute an action. Completion uses permission_approved/refused plus normal tool-result delivery.
FieldTypeRequiredDescription
type"permission_resolution"Yes
idstring | nullYes
decisionstring | nullYes
acceptedbooleanYesThe decision was accepted for asynchronous resolution, not proof the tool succeeded.
reason"invalid_permission_resolution" | "unknown_or_expired_approval_id" | "permission_source_mismatch" | "resolution_in_progress"No
policyFlagpolicyFlag
FieldTypeRequiredDescription
type"policy_flag"Yes
rulestringYesThe rule id from mode.policy.
action"next_turn"Yes
evidencestringYesThe judge's one-sentence quote of what triggered the rule.
deliveredbooleanYesGuidance was appended to context; not proof that the agent followed it.
reply_startedFalseYes
occurrence_idstringNo
utterance_idstringNo
revisionintegerNo
status"raised" | "updated" | "retracted"No
policyErrorpolicyError
FieldTypeRequiredDescription
type"policy_error"Yes
check_idinteger | stringYes
reason"timeout" | "provider_error" | "checker_failed"Yes
continuesbooleanYes
policyCheckpolicyCheck
FieldTypeRequiredDescription
type"policy_check"Yes
check_idstringYes
policy_versionintegerYes
status"completed"Yes
policySettledpolicySettled
FieldTypeRequiredDescription
type"policy_settled"Yes
policy_versionintegerYes
checked_versionintegerYes
healthybooleanYes
wrapUpwrapUp - Request a graceful host-initiated sign-off in voice or text mode, including when mode.end_call is disabled. An active reply receives a bounded grace before the sign-off. No dedicated acknowledgement is sent; continue handling reply events and normal closure, and drain local playback. Sending this frame does not confirm the farewell was heard.
FieldTypeRequiredDescription
type"wrap_up"Yes
reasonstringNoReason included in the wrap-up context. Omitted or empty values use time_limit; strings are truncated to 40 characters.
inputAudioDoneinputAudioDone - Commit accumulated voice input when start.mode.turn_detection is client. Send after the audio for that turn. Ignored when client turn detection is not active; not a text-mode control and not an acknowledgement of transcription or reply completion.
FieldTypeRequiredDescription
type"client_event"Yes
event"input_audio_done"Yes
asrCorrectionasrCorrection - Replaces the text of an earlier voice transcript. When turn_id is present, update that turn rather than appending a new user turn. turn_id can be omitted; do not fabricate a correlation. Optional utterance_id, revision and policy_version describe the policy transcript revision.
FieldTypeRequiredDescription
participant_idParticipantIdNo
type"asr_correction"Yes
textstringYes
turn_idstringNoId of the turn this transcript belongs to.
utterance_idstringNo
revisionintegerNo
policy_versionintegerNo
authauth - Check a credential on a fresh WebSocket connection. Send as the first frame instead of start, with api_key or a Bearer handshake header. Returns ok on success or error on failure and the connection ends. Does not open a conversation, emit ready, or prove credit or session capacity is available.
FieldTypeRequiredDescription
type"auth"Yes
api_keystringNoPersistent or scoped key. Optional when supplied in the WebSocket Authorization header.
feedbackfeedback - Submit feedback as the first frame on a fresh authenticated WebSocket instead of start. Requires an owned recording and either rating or nonblank text. Success returns ok and the connection ends. Re-submitting feedback for the same session replaces the earlier feedback. Does not consume a conversation slot and remains available while the server drains.
FieldTypeRequiredDescription
type"feedback"Yes
api_keystringNoPersistent or scoped key. Optional when supplied in the WebSocket Authorization header.
session_idstringYesClient session_id of a recording owned by the authenticated account; not the server session_uuid.
rating"up" | "down"No
textstringNoFeedback comment; trimmed and truncated to 2000 characters. Required if rating is omitted.
device"mac" | "iphone" | "ipad" | "windows" | "android" | "linux" | "other"No
browser"chrome" | "safari" | "firefox" | "edge" | "other"No
clientErrorclientError - Report a client-side failure as the first frame on a fresh authenticated WebSocket instead of start. Omit session_id when failure predates session creation; when present it must name an owned recording. Success returns ok and the connection ends, without consuming a conversation slot. detail and context are diagnostic log content: do not send transcripts, replies, credentials or user conversation text.
FieldTypeRequiredDescription
type"client_error"Yes
api_keystringNoPersistent or scoped key. Optional when supplied in the WebSocket Authorization header.
session_idstringNoClient session_id of a recording owned by the authenticated account; not the server session_uuid.
detailstringYesNonblank failure diagnostic; trimmed and truncated to 500 characters.
contextstringNoOptional diagnostic context; trimmed and truncated to 200 characters.
okok - Successful acknowledgement for a fresh-connection auth, feedback or client_error request. The connection ends after this response; it is not the ready event for a conversation.
FieldTypeRequiredDescription
type"ok"Yes