Bidirectional Streaming Speech Recognition WebSocket API
Supports streaming audio input with real-time recognition results, delivering text output as you speak. With low recognition latency, it is suitable for low-latency scenarios such as real-time meeting captions, live broadcast subtitles, and intelligent outbound calls.
WebSocket wss://ai-tokenhub.com/api/v3/asr/stream?model=doubao-seedasr-2-0-duration
Request Parameters
API Overview
| Item | Value |
|---|---|
| Method | WebSocket |
| URL | wss://ai-tokenhub.com/api/v3/asr/stream?model=doubao-seedasr-2-0-duration |
| Authentication | Bearer Token (Authorization header) |
| Transport | Bidirectional streaming |
| Model name | doubao-seedasr-2-0-duration (specified via URL query parameter model) |
Request Headers
Authorization string Required
Format: Bearer <token>, where <token> is the API key obtained from the TokenHub platform.
URL Parameters
model string Required
Model identifier, fixed value: doubao-seedasr-2-0-duration
Binary Protocol
WebSocket messages use a custom binary protocol. Each message consists of a header and a body.
Message Header Structure (4 bytes)
| Byte | Bits | Description |
|---|---|---|
| Byte 1 | All | Fixed value 0x11 |
| Byte 2 | High 4 bits | Message type (mt): 0x01=full request, 0x02=audio frame, 0x09=server event, 0x0F=error response |
| Byte 2 | Low 4 bits | Flags: bit0=has sequence, bit1=last packet, bit2=has timestamp |
| Byte 3 | High 4 bits | Sequence number (ser) |
| Byte 3 | Low 4 bits | Compression (comp): 0x00=none, 0x01=gzip |
| Byte 4 | All | Reserved, fixed 0x00 |
Client-Sent Messages
After connection is established, the client sends in order: full request → audio frames → end frame.
1. Full Request (mt=0x01)
The first message contains complete configuration parameters. The message body is a JSON string.
audio object Required
Audio configuration.
format string Required
Specifies the audio format. Options: wav / mp3 / ogg / pcm / spx / amr / aac / m4a
codec string
Specifies the audio codec format, default is raw (pcm). Options: raw / opus
rate int
Specifies the audio sample rate, default value is 16000
bits int
Specifies the audio sample bit depth, default value is 16
channel int
Specifies the number of audio channels, default value is 1. Options: 1 (mono) / 2 (stereo)
request object
Recognition request parameters.
model_name string Required
Specifies the model name, currently only supports bigmodel
enable_nonstream bool
Enable second-pass recognition mode, default is false. When enabled, on top of bidirectional streaming real-time recognition, each utterance is additionally re-recognized using a non-streaming model to improve final utterance accuracy.
Behavior when enabled:
- Automatically enables Voice Activity Detection (VAD) for utterance segmentation, default 800ms silence threshold, adjustable via
end_window_size - Each time VAD detects the end of an utterance, the non-streaming model re-recognizes that utterance
- Only non-streaming model second-pass results carry the
"definite": truefield
enable_speaker_info bool
Enable speaker diarization, default is false. When enabled, show_utterances must also be enabled to obtain speaker diarization results.
enable_itn bool
Enable conversion of speech recognition results to standard written format, default is true. When enabled, the system automatically converts spoken numbers, amounts, dates, etc. into Arabic numerals and symbols.
enable_ddc bool
Enable semantic smoothing, default is false. When enabled, the system removes or corrects disfluencies such as filler words, modal particles, and semantic repetitions.
output_zh_variant string
Output recognition results in Traditional Chinese. Options:
traditional: Simplified → Traditional (Mainland)tw: Simplified → Taiwanese orthographyhk: Simplified → Hong Kong Traditional
show_utterances bool
Enable output of utterance, word segmentation, speaker, and speech pause information, default is false
show_speech_rate bool
Enable utterance info with speech rate, default is false. When enabled, the system returns speech rate info in utterance additions, in tokens/s.
show_volume bool
Enable utterance info with volume, default is false. When enabled, the system returns volume info in utterance additions, in dB.
enable_lid bool
Enable Chinese/English/dialect recognition, default is false. Supports: Mandarin, English, Shanghainese, Minnan dialect, Sichuan dialect, Shaanxi dialect, Cantonese.
When enabled, the system returns language/scene labels in additions.
enable_emotion_detection bool
Enable emotion detection, default is false. When enabled, the system returns corresponding emotion labels in utterance additions. Supported labels: angry / happy / neutral / sad / surprise
enable_gender_detection bool
Enable gender detection, default is false. When enabled, the system returns gender labels (male/female) in utterance additions
enable_age_detection bool
Enable age detection, default is false. When enabled, the system returns the speaker's age in utterance additions, as a string float.
result_type string
Specifies how recognition results are returned, default is full. Options:
full: Full result returnsingle: Incremental result return, i.e. does not return previous utterance results
enable_accelerate_text bool
Enable first-character return acceleration, default is false. Enabling this will try to accelerate first-character return but may reduce first-character accuracy.
accelerate_score int
Specifies first-character acceleration rate, default is 0 (no acceleration). Higher values mean faster first-character output. This parameter requires enable_accelerate_text to be enabled.
vad_segment_duration int
Specifies the maximum silence threshold for semantic utterance segmentation, default is 3000, in ms. When silence exceeds this value, the recognition result is segmented here.
end_window_size int
Specifies the VAD silence stop threshold, default is 800, in ms. Range: [300, 5000], recommended: [800, 1000]
force_to_speech_time int
Sets the duration at the start of the audio stream to force treat as speech, default is 0, in ms. Recommended: 1000
sensitive_words_filter string
Enable sensitive word filtering. When enabled, sensitive words in recognition results can be masked or replaced.
Example:
"sensitive_words_filter":"{\"system_reserved_filter\":true,\"filter_with_empty\":[\"sensitive\"],\"filter_with_signed\":[\"sensitive\"]}"system_reserved_filter bool
Enable system built-in sensitive word library, default is false. When enabled, matched system sensitive words are replaced with *
filter_with_empty string
Set custom sensitive word list to be replaced with empty string
filter_with_signed string
Set custom sensitive word list to be replaced with *
enable_poi_fc bool
Enable POI Function Call, default is false. When enabled, professional map domain recommendation word service can be called to assist recognition. Requires enable_nonstream to be true.
enable_music_fc bool
Enable Music Function Call, default is false. When enabled, for words difficult to recognize, professional music domain recommendation word service can be called to assist. Requires enable_nonstream to be true.
corpus object
Configure context dictionary. Custom hotwords, replacement words, and context info can be configured to improve recognition accuracy in specific contexts.
boosting_table_name string
Hotword table name
boosting_table_id string
Hotword table ID. If boosting_table_name and boosting_table_id correspond to different tables, boosting_table_id takes precedence.
correct_table_name string
Replacement word table name. When configured, specific words recognized by the model can be replaced with target words
correct_table_id string
Replacement word table ID. If correct_table_name and correct_table_id correspond to different tables, correct_table_id takes precedence.
regex_correct_table_name string
Regex replacement word table name. Compared to exact-match replacement words, regex replacement words are suitable for batch format conversion, fuzzy pattern matching, etc.
regex_correct_table_id string
Regex replacement word table ID
context string
Context feature. Pass auxiliary context information in the recognition request to help the model improve recognition accuracy combined with context.
contextmust be serialized as a JSON string
Example:
{
"hotwords": [
{ "word": "custom hotword A" },
{ "word": "custom hotword B" }
],
"context_type": "dialog_ctx",
"context_data": [
{ "speaker": "user", "text": "Can you help me look up information?" },
{ "speaker": "bot", "text": "Of course, what information would you like to look up?" }
]
}hotwords array: Hotword list passed directly to improve recognition of specified words
- word
string: Hotword content
context_type string: Context type, currently only supports dialog_ctx
context_data array: Context data list for passing historical dialogue and other context info
- text
string: Historical dialogue text - image_url
string: Image URL for visual context (only supported whenenable_nonstreamis enabled, max 1 image, ≤500KB, supports jpeg/jpg/png)
2. Audio Frame (mt=0x02)
Continuously send audio data, each message contains one audio frame. Message body structure:
| Offset | Length | Description |
|---|---|---|
| 0 | 4 bytes | Frame sequence number (big-endian signed int), starting from 2 |
| 4 | 4 bytes | Audio data length (big-endian unsigned int) |
| 8 | N bytes | Audio binary data |
End signal: send an empty audio frame with a negative sequence number (absolute value equals total frames sent) to indicate end of audio stream.
Response Parameters
The response is a WebSocket binary message stream. The client must continuously receive and parse each message packet.
Response Processing Logic
- Establish WebSocket connection
- Send full request configuration (mt=0x01)
- Loop sending audio frames (mt=0x02)
- Simultaneously receive recognition results from the server
- After audio is finished, send an end frame (negative sequence empty frame)
- Close the connection after receiving the last result packet (is_last_package=true)
Response Field Description
code int
Request status code. 0 means recognition succeeded, non-zero means recognition failed
event int
Session event type identifier
is_last_package bool
Whether this is the last response packet. true means all recognition results have been returned
payload_sequence int
Response data packet sequence number
payload_size int
Byte size of response data payload
payload_msg object
Response data body, containing audio info and recognition results
audio_info object
Audio information
duration int
Audio duration in milliseconds (ms)
result object
Recognition result, returned after successful recognition
additions object
Additional information
log_id string: Server-side logid for troubleshooting
text string
Audio recognition result text, returned after successful recognition
utterances array
Speech utterance info. Returned when show_utterances is set to true and recognition is successful.
definite bool: Whether the current utterance result is final. true means the utterance will no longer change
additions object: Additional information
- fixed_prefix_result
string: Fixed prefix recognition result - source
string: Utterance result source - speaker_id
string: Speaker ID. Returned after enabling speaker diarizationenable_speaker_info=true
text string: Utterance text content
end_time int: Utterance end timestamp (milliseconds)
start_time int: Utterance start timestamp (milliseconds)
words array: Word-level timestamp list
- text
string: Word text - start_time
int: Word start time (milliseconds) - end_time
int: Word end time (milliseconds)