Voice agent, capture, and sound activation
Process model
S97 starts voice-agent.sh -mode=master. The wrapper constrains glibc allocation (MALLOC_ARENA_MAX=1, MALLOC_MMAP_THRESHOLD_=16384) and execs the stripped AArch64 PIE.
Observed: two processes run normally:
voice-agent -mode=mastervoice-agent -mode=app -watchdog
The application process also listens on TCP 80 and 8081 and UDP 5353. The master/app split isolates the large recognition/media runtime from the supervisor process.
Audio stack
PulseAudio 16.1 is started by S50 with exit-on-idle disabled. voice-agent links both ALSA and PulseAudio and contains alsa_capture, pulse_capture, async_pulse_capture, alsa_playback, and pulse_playback implementations.
Relevant built-in device strings include:
pulse/16000/1pulse:mics/16000/1pulse:alsa_playback
The main keyword/capture path is therefore 16 kHz mono. Media decoding uses FFmpeg libraries plus Opus and FLAC; local prompts live under /usr/share/voice-agent.
okcall uses separate Pulse endpoints call and call_src; see calls and stereo.
Local keyword spotter
Decompiled: activation is local, not a continuous cloud search for the wake phrase. KeywordSpotter wraps kws_mini::Spotter and TensorFlow Lite. /usr/share/voice-agent/model.bin is 1,224,560 bytes and begins with the FlatBuffers/TFLite identifier TFL3 at offset 4.
Initialization in voice-agent:0x1949a0 sets:
| Parameter | Value |
|---|---|
| Input sample rate | 16,000 Hz |
| Input fragment | 1,280 samples (80 ms) |
| Analysis window | 640 samples (40 ms) |
| Hop | 160 samples (10 ms) |
| FFT size | 2,048 |
| Filterbank count | 40 |
The loader memory-maps the model, verifies TFL3, constructs the interpreter, and registers audio operations including MFCC and AudioSpectrogram. Missing model configuration raises No kws model!.
Model schema
firmware/manifest/kws-model.json, generated by inspect_kws_model.py, records the complete graph summary without weights. The model is MLIR Converted., schema version 3:
| Tensor | Shape/type | Role recovered from Spotter |
|---|---|---|
serving_default_args_0:0 |
[1, 8, 40], float32 |
Eight frames of 40 features |
StatefulPartitionedCall:0 |
[1, 8, 1, 2], float32 |
pos_neg_probs: negative/keyword confidence |
StatefulPartitionedCall:2 |
[1, 8, 1, 2], float32 |
int_unint_probs: secondary intentional/unintentional score |
StatefulPartitionedCall:1 |
[1, 8, 1, 1], float32 |
kw_loc: keyword location used for context offset |
The binary's request keyword is the literal marusya. No human-readable class labels are embedded in model metadata; the class roles come from output consumers, not from guessing based on the filename.
Thresholds and activation
Both the positive/negative and secondary thresholds must pass. Defaults recovered from static data:
| Flag | Quiet/default | Playback mode |
|---|---|---|
kws_threshold / kws_threshold_playback |
0.994 | 0.995 |
kws_sp_threshold / kws_sp_threshold_playback |
0.2 | NaN in this build |
voice-agent:0x1307b0 switches these values when playback threshold mode is toggled. The NaN is the actual serialized default, not a decompiler placeholder; it should not be replaced with an invented numeric threshold.
Engine states are IDLE, WAITING_FOR_KEYWORD, PHRASE_IN_PROGRESS, and PHRASE_RESULT_PENDING. Touch and voice activation use different prompts: 03_listening_start_touch.flac and 03_listening_start_voice.flac. keyword_sound controls the audible cue, and false wakeups are explicitly logged.
Reference signal and statistics
/usr/share/voice-agent/reference.pcm is 646,400 bytes of float PCM. The spotter reads it, computes an MD5, and retains the resulting context. This is separate from the identically sized ADSSS setup reference; reuse of the same waveform does not make the two subsystems interchangeable.
SpotterDataCollector and kws_skip collect/upload spotter statistics and selected contexts. spotter_stats_period_ms defaults to 250 ms; spotter_stats_sent_period_ms defaults to 1,800,000 ms (30 minutes). These are separate from the 16 kHz cloud phrase stream and from telemetryd's on-demand audio samples.
Cloud phrase recognition
After local activation, audio is streamed with gRPC:
/vk.voice.grpc.front.v1.service.Phrase/Speech/vk.voice.grpc.front.v1.service.Phrase/GetResult/vk.voice.grpc.front.v1.service.ClientProbe/AuthReady
The binary contains protobuf types for init, options, audio chunks, end-of-speech, abort, recovery, keyword confirmation, partial text, commands, and final results. Authentication uses the provisioned Marusya session rather than the one-time setup xToken.
The local spotter therefore gates capture; VK servers perform general speech recognition and return commands/media instructions.
Microphone mute
Physical/software mute is handled by toggleMicMute and SoftMicMute at voice-agent:0x2bf7f0. GPIO 446 is an active-low microphone-enable output: the agent writes 0 to mute and 1 to unmute, exporting it and setting direction out if necessary. Thus this is a hardware capture gate shared by voice and call paths, not merely an application flag. The live state was 0/out in the captured muted state. mic_on.json, mic_off.json, 05_mic_on_to_mic_off.flac, and 06_mic_off_to_mic_on.flac provide feedback.