audio: microwakeword: add mww component by singalsu · Pull Request #11135 · thesofproject/sof

singalsu · 2026-08-27T13:30:17Z

No description provided.

Copilot

🟡 Changes recommended

There are correctness issues that will break integration (notably the KPB UUID mismatch with the UUID registry) and should be fixed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces a new microWakeWord (MWW) keyword-spotting component and integrates it into SOF’s IPC4/Zephyr-based build, including new topology2 pipelines (KPB-based Wake-on-Voice branches), rimage manifests, and supporting LLEXT/AMS/DP-scheduler plumbing needed to run TFLite Micro–based C++ extensions.

Changes:

Add the microwakeword audio component (C/C++ + Kconfig/CMake + rimage TOML) and offline training/tuning scripts.
Add topology2 widget/pipeline templates and platform overlays to enable MWW+KPB Wake-on-Voice capture branches on HDA and SDW.
Extend DP-scheduler integration (optional DP→DP binding), AMS payload handling, and LLEXT build/export behavior for C++ multi-TU libraries.

File summaries

File	Description
zephyr/Kconfig	Broadens IPC4 pipeline2.0 + DP scheduler defaults to ACE/CAVS; adds DP→DP bind Kconfig.
zephyr/CMakeLists.txt	Adds `-Wl,-Bsymbolic-functions` for LLEXT shared builds to avoid unresolved local cross-TU calls.
uuid-registry.txt	Registers UUID for new `mww` module.
tools/topology/topology2/sof-hda-generic.conf	Adds optional HDA mic MWW/KPB capture include and required class includes.
tools/topology/topology2/platform/intel/sdw-jack-mww-kpb.conf	New SDW jack MWW/KPB WoV capture branch overlay topology.
tools/topology/topology2/platform/intel/sdw-dmic-mww-kpb.conf	New SDW DMIC MWW/KPB WoV capture branch overlay topology.
tools/topology/topology2/platform/intel/hda-mic-mww-kpb.conf	New HDA analog MWW/KPB WoV capture branch overlay topology.
tools/topology/topology2/platform/intel/dmic1-mfcc.conf	Removes redundant MFCC include (comment-only change).
tools/topology/topology2/include/pipelines/cavs/src-kpb-be.conf	New reusable SRC→KPB backend pipeline class for WoV branching.
tools/topology/topology2/include/pipelines/cavs/host-gateway-src-mfcc-mww-capture.conf	Adds a detection pipeline template combining SRC+MFCC+MWW.
tools/topology/topology2/include/pipelines/cavs/host-gateway-micsel-mfcc-mww-capture.conf	Adds a stereo-compatible detection pipeline template using micsel+MFCC+MWW.
tools/topology/topology2/include/components/mww.conf	Adds topology2 widget class definition for `mww`.
tools/topology/topology2/include/components/mfcc/mel80.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/mel80_compress.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/mel80_compress_dtx.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/mel40.conf	Adds new 40-bin MFCC config blob.
tools/topology/topology2/include/components/mfcc/mel40_compress.conf	Adds new compressed-output 40-bin MFCC config blob.
tools/topology/topology2/include/components/mfcc/mel40_10ms.conf	Adds new 40-bin, 10ms-hop MFCC config blob for MWW.
tools/topology/topology2/include/components/mfcc/mel40_10ms_compress.conf	Adds compressed-output 40-bin, 10ms-hop MFCC config blob.
tools/topology/topology2/include/components/mfcc/default.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/ceps13_compress_dtx.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/kpb.conf	Updates KPB UUID (currently inconsistent with uuid registry; see comment).
tools/topology/topology2/include/common/common_definitions.conf	Adds feature flags for enabling MWW/KPB overlays.
tools/topology/topology2/include/common/abi.conf	Adds ABI manifest blob definition.
tools/topology/topology2/include/bench/mfccmel40_10ms_s32.conf	Adds MFCC mel40_10ms benchmark include (S32).
tools/topology/topology2/include/bench/mfccmel40_10ms_s24.conf	Adds MFCC mel40_10ms benchmark include (S24).
tools/topology/topology2/include/bench/mfccmel40_10ms_s16.conf	Adds MFCC mel40_10ms benchmark include (S16).
tools/topology/topology2/include/bench/mfcc_controls_playback.conf	Adds bench parameter key for mel40_10ms MFCC blob.
tools/topology/topology2/include/bench/mfcc_controls_capture.conf	Adds bench parameter key for mel40_10ms MFCC blob.
tools/topology/topology2/development/tplg-targets.cmake	Adds development topology targets enabling MWW/KPB branches for HDA + SDW.
tools/topology/topology2/development/tplg-targets-bench.cmake	Adds bench target/params for mel40_10ms MFCC benchmark.
tools/topology/topology2/cavs-sdw.conf	Adds optional includes and keys for SDW MWW/KPB overlays + required class includes.
tools/topology/topology2/cavs-benchmark-hda.conf	Adds benchmark configs for mfccmel40_10ms variants.
tools/rimage/config/wcl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/tgl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/tgl-h.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/ptl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/mtl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/lnl.toml.h	Includes MWW module TOML when enabled.
src/platform/intel/cavs/platform.c	Initializes DP scheduler when enabled.
src/library_manager/llext_manager_dram.c	Ensures VMA cleanup on restore failure.
src/lib/cpp_new_export.cpp	Exports C++ allocation/runtime symbols for LLEXT modules.
src/lib/CMakeLists.txt	Builds the new C++ symbol export shim when CONFIG_CPP is enabled.
src/lib/ams.c	Adds inline payload data copy into AMS slots; exports ams_send().
src/ipc/ipc4/helper.c	Adds optional DP→DP binding support and DP ring buffer attachment changes.
src/ipc/ipc4/ams_helpers.c	Exports AMS helper functions for external users/modules.
src/include/sof/lib_manager.h	Extends lib manager module struct with export segment + VMA tracking.
src/include/sof/audio/mfcc/mfcc_vad.h	Tunes MFCC VAD constants (noise rise alpha + threshold).
src/audio/module_adapter/module_adapter.c	Propagates DP domain from extended init; minor sync constant fix.
src/audio/microwakeword/tune/sof_mww_verify.py	Adds streaming verification script for quantized MWW models.
src/audio/microwakeword/tune/sof_mww_train_pipeline.sh	Adds end-to-end dataset→features→train→verify pipeline runner.
src/audio/microwakeword/tune/sof_mww_prepare_silence_unknown.sh	Adds script to prepare silence/unknown classes from Speech Commands v2.
src/audio/microwakeword/tune/sof_mww_plot_mtrace.py	Adds mtrace visualization tool for MWW diagnostics.
src/audio/microwakeword/tune/sof_mww_generate_keyword_dataset.sh	Adds multi-speaker synthetic keyword dataset generator (has duplicated helper; see comment).
src/audio/microwakeword/tune/sof_mww_generate_keyword_dataset_from_dir.sh	Adds real-speech ingestion + augmentation dataset builder.
src/audio/microwakeword/tune/sof_mww_dataset.py	Adds feature loader + augmentation utilities for training.
src/audio/microwakeword/tune/sof_mfcc_extract_features.sh	Adds batch feature extraction via sof-testbench4 + topology2 bench tplg.
src/audio/microwakeword/tune/README.md	Documents offline MWW training/tuning toolchain and workflows.
src/audio/microwakeword/README.md	Documents MWW component architecture, dataflow, and deployment notes.
src/audio/microwakeword/mww.toml	Adds rimage module manifest entry for MWW.
src/audio/microwakeword/mww_model.h	Adds C API surface for the MWW TFLM inference wrapper.
src/audio/microwakeword/mww_model.cc	Implements TFLM interpreter setup + streaming inference wrapper.
src/audio/microwakeword/mww_model_data.h	Adds generated model-data header declaration.
src/audio/microwakeword/llext/llext.toml.h	Adds LLEXT-specific TOML wrapper for MWW module.
src/audio/microwakeword/llext/CMakeLists.txt	Adds LLEXT build for the MWW module and its private TFLM library.
src/audio/microwakeword/llext-wrap.c	Adds LLEXT portability stubs and PIC-safe math overrides.
src/audio/microwakeword/Kconfig	Adds Kconfig options for MWW component and debug/model-loading modes.
src/audio/microwakeword/CMakeLists.txt	Adds static + LLEXT build logic for MWW and its private deps.
src/audio/mfcc/tune/setup_mfcc.m	Adds MFCC export profiles for mel40 (20ms) and mel40_10ms (+ compress variants).
src/audio/mfcc/mfcc_common.c	Clarifies VAD input comment.
src/audio/Kconfig	Includes microwakeword Kconfig in audio menu.
src/audio/CMakeLists.txt	Adds microwakeword subdir to audio build when enabled.
src/audio/buffers/ring_buffer.c	Adds vregion refcount release for DP→DP bind case.
src/audio/buffers/audio_buffer.c	Adds DP→DP dual-secondary-buffer sync support and updates DP→DP commentary.
scripts/xtensa-build-zephyr.py	Installs symlinks by UUID name for library binaries.
scripts/tensorflow-clone.sh	Refactors dependency clone script to fetch + checkout pinned commits under workspace parent dir.
scripts/llext_offset_calc.py	Handles ELF with no allocated sections by returning size 0.
app/llext_relocatable.conf	Enables export-by-SLID for LLEXT relocatable builds.
app/boards/intel_adsp/Kconfig.defconfig	Enables DP→DP bind by default.
app/boards/intel_adsp_cavs25.conf	Enables MWW (static), increases heap sizes, and adds related settings for cavs2.5.
app/boards/intel_adsp_ace30_ptl.conf	Enables MWW as LLEXT, adjusts library base address, heap sizing, and LLEXT heap.

Review details

Files reviewed: 86/88 changed files
Comments generated: 4
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

🟡 Changes recommended

The review found concrete correctness issues in new/changed runtime code paths (heap free mismatch in ring buffer cleanup, unsafe AMS memcpy_s handling, and incorrect syscall stub signatures/errno semantics) that should be fixed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 85/87 changed files
Comments generated: 4
Review effort level: Lite

Copilot

🟡 Changes recommended

It introduces an unsafe free path in ring_buffer_free() that can corrupt module allocation lifetime (use-after-free/double-free risk).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/audio/buffers/ring_buffer.c:113

ring_buffer_free() must not free the module allocation context. alloc is owned by the module (allocated with sof_heap_alloc() in module_adapter_mem_alloc() and freed in module_adapter_mem_free()). Freeing it here via rfree() can cause use-after-free/double-free when the module later tears down, and rfree() is also not the matching deallocator for sof_heap_alloc().

	if (alloc && alloc->vreg) {
		if (!vregion_put(alloc->vreg))
			rfree(alloc);
	}

Files reviewed: 85/87 changed files
Comments generated: 1
Review effort level: Lite

Copilot

🟡 Changes recommended

The DP-to-DP ring buffer teardown path currently risks incorrect/freeing of shared allocation context and AMS slot message pointer rebinding relies on an invalid cross-core pointer value.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/audio/buffers/ring_buffer.c:113

ring_buffer_free() drops the vregion refcount and then frees the shared mod_alloc_ctx with rfree(). This ctx is allocated with sof_heap_alloc() (see module_adapter_mem_alloc()) and is also freed via sof_heap_free() in other vregion_put() call sites (e.g. comp_buffer_free()), so using rfree() (and freeing it here at all) risks allocator mismatch or double-free when DP-to-DP bindings are torn down.

	if (alloc && alloc->vreg) {
		if (!vregion_put(alloc->vreg))
			rfree(alloc);
	}

Files reviewed: 86/87 changed files
Comments generated: 1
Review effort level: Lite

 	msg = shared_c->slots[slot].u.msg;
+	if (msg.message && msg.message_length > 0) {
+		if (msg.message_length <= sizeof(msg_buf)) {
+			if (memcpy_s(msg_buf, sizeof(msg_buf),
+				     (__sparse_force void *)(shared_c->slots[slot].u.msg_raw + sizeof(msg)),
+				     msg.message_length) != 0) {
+				ams_release(shared_c);
+				return -EINVAL;
+			}
+			msg.message = msg_buf;
+		} else {
+			msg.message = (__sparse_force uint8_t *)(shared_c->slots[slot].u.msg_raw + sizeof(msg));
+		}
+	}


Copilot

Copilot review overview

🟡 Changes recommended

One or more issues must be addressed before approval.

Review effort: Lite
Findings: 13 · 2

Open (15)

Remove unconditional Clang flag from global build options · New
Require selector sink readiness before draining · New
Prevent PCAN from bypassing normal MFCC DCT output · New
Add IPC4 and module-adapter dependencies for MWW · New
Include Zephyr kernel API for debug timing calls · New
Align PCAN hop layout with topology selection · New
Set MWW module capacity to one sink · New
Match PCAN log scale shift with training configuration · New
Enable PCAN for the cavs25 MWW topology · New
Declare one output pin for routed MWW sink · New
Declare the routed MWW output pin · New
Match MWW formats to configured channel count · New
ams_process_slot() decides whether to rebind msg.message based on the saved pointer value…
Preserve legacy Mel spectrogram log defaults · New
Quantize PCAN features using model parameters · New

Resolved since last review (1)

ring_buffer_free() frees the module allocation context with rfree(), but mod_alloc_ctx is allocated…

+if(CMAKE_C_COMPILER_ID MATCHES "Clang")
+	add_compile_options(-mllvm --text-section-literals=false)
+endif()


+#if CONFIG_COMP_MFCC_PCAN
+	if (state->pcan.enable_pcan) {
+		int8_t *out8 = (int8_t *)state->out_stage;
+
+		for (k = 0; k < num_ceps; k++) {
+			/* Map PCAN log-scaled output (~0..666) to int8 [-128..127] */
+			int32_t val = (int32_t)state->mel_linear[k];


+	depends on SOF_STAGING
+	depends on CPP
+	depends on STD_CPP17


+
+#include <stdarg.h>
+#include <stdio.h>
+#include <zephyr/sys/printk.h>


+			stack_bytes_requirement 8192
+			heap_bytes_requirement "$[(24 * 1024)]"
+			num_input_audio_formats 1
+			num_output_pins 0


+			num_input_pins	1
+			num_output_pins	0
+			num_input_audio_formats 1
+			num_output_audio_formats 1


+			Object.Base.input_audio_format [
+				{
+					in_channels 		2
+					in_bit_depth 		32
+					in_valid_bit_depth 	32
+					in_rate 		16000
+					ibs 			$MWW_INPUT_BYTES


 	cfg.vtln_warp = 1.0;
 	cfg.window_type = 'hann';
-	cfg.mel_log = 'log10';
+	cfg.mel_log = 'log'; % Set to 'db' for librosa, set to 'log10' for matlab


+#if CONFIG_COMP_MWW_PCAN
+		/* PCAN mode: MFCC has already normalized and quantized Mel energies
+		 * to int8_t (Q1.7). Copy directly into the feature buffer slice.
+		 */
+		memcpy(slice, hop_src + sizeof(struct mfcc_data_header), MWW_FEATURE_SIZE);


This patch enables the modules needed for keyword detect in a static build: KPB, MFCC, MWW, Tensorflow, etc. The impacted platforms are cavs25 and ace30. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Keep keyword detection active while suppressing MFCC feature traffic to the host when D0i3 entry is allowed. Resume the host output when power gating is prevented again. When a keyword is detected in D0i3, latch the MWW host output on until D0i0 so the next MFCC hop wakes the host through PCM103. Mark both the WoV capture and detector PCMs as D0i3-compatible, and avoid dereferencing an absent KPB host sink when detection runs without the WoV capture stream. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

Allow MWW detection modules to run without a firmware output sink. The kernel ignores routes involving virtual widgets, so the detector must be source-only while the virtual route connects the DAPM graph to the KPB host copier. Update all MWW topology variants to use virtual.detect_sink and remove the dedicated detect PCM. Keep the alternate SRC pipeline consistent with the same source-only behavior. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Use the DAI-side DMIC micsel output directly as the KPB and MWW input for the ADL MWW topology. Remove the redundant detector-side micsel and configure the detector for mono MFCC frames. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Consolidate and refactor all microWakeWord (MWW) and Key Phrase Buffer (KPB) Wake-on-Voice topology configurations to use a shared mono 16 kHz pipeline architecture across SoundWire DMIC, SoundWire jack, PCH DMIC, and CAVS NoCodec capture paths. Changes: - Add mww-kpb-fe.conf: shared frontend containing the host-gateway-capture pipeline (100), mww-detect pipeline (103), WoV PCM (102), and common drain routes. - Add micsel-src-kpb-be.conf: pipeline class that downmixes stereo DAI capture to mono via micsel before SRC and KPB buffering. - Add sdw-mww-kpb.conf: SoundWire MWW KPB backend template including micsel-src-kpb-be and mww-kpb-fe; used by both DMIC and jack paths. - Refactor sdw-dmic-mww-kpb.conf and sdw-jack-mww-kpb.conf as thin wrappers that define only the endpoint module-copier ID and stream name. - Refactor dmic-mww-kpb.conf to define only its DAI-side backend route and include mww-kpb-fe.conf. - Refactor cavs-nocodec-mww-kpb.conf to use micsel-src-kpb-be and mww-kpb-fe, removing redundant pipeline/route/PCM definitions. - Move micsel to between the DAI module-copier and SRC on SoundWire and NoCodec paths (was before the MFCC detection pipeline). Configure KPB, host copier, and WoV PCM for single-channel mono operation at 16 kHz. Connect KPB directly to MFCC with a 640-byte hop buffer. HDA topologies are left unchanged as mono capture is not supported on HDA. - Rename the TGL NoCodec MWW topology target from sof-tgl-nocodec-mww to sof-tgl-nocodec-mww-pcan-kpb for consistency with other PCAN KPB topologies; set MWW_PCAN=true explicitly in the target arguments. - Remove sof-arl-cs42l43-l0-cs35l56-l23-mww-kpb and sof-mtl-rt713-l0-rt1316-l12-mww-kpb from development topology targets; both platforms lack sufficient DSP RAM for the MWW and KPB pipelines. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

When connecting a pipeline with CONFIG_ZEPHYR_DP_SCHEDULER enabled, ipc4_comp_connect() checks if either source or sink is in the DP domain (src_is_dp || sink_is_dp). Previously, it unconditionally called comp_mod() on both source and sink to retrieve module private data (mpd.in_buff_size and mpd.out_buff_size) for ring_buffer_create(). However, components that are not module adapters (such as SOF_COMP_KPB) do not have dev->mod set, returning NULL from comp_mod(). When binding KPB directly to a DP module (such as MFCC), dereferencing src_module_data->mpd resulted in an EXCCAUSE 0x0000000d (LoadStorePIFDataErrorCause) synchronous PIF data error DSP panic at helper.c:956. Only retrieve module private buffer sizes when the corresponding endpoint is both in the DP domain and is a module adapter. Otherwise, fall back safely to ibs / obs. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

A pipeline containing only DP-domain modules (e.g. mww-detect with mfcc + mww) never had pipeline_comp_ll_task_init() called during pipeline_comp_prepare(), because the call was guarded by: if (current->ipc_config.proc_domain == COMP_PROCESSING_DOMAIN_LL) As a result, p->pipe_task and p->trigger_task remained NULL. When the host triggered such a pipeline to RUNNING, pipeline_schedule_triggered() called schedule_task(p->trigger_task, 0, 0) on a NULL pointer (producing a 'failed to schedule trigger task' log), then called pipeline_schedule_copy() which dereferenced p->pipe_task via task_is_active(), causing a LoadStorePIFDataErrorCause DSP panic. Fix by removing the LL-only guard in pipeline_comp_prepare() so that pipeline_comp_ll_task_init() is called unconditionally for every pipeline component. Every pipeline needs the LL trigger task to handle IPC trigger commands and the pipe task to drive inter-component ring buffer synchronisation, regardless of whether its modules run in LL or DP domain. Add defensive guards to make the code robust against future regressions: - task_is_active(): return false for NULL task pointer. - pipeline_schedule_copy(): early return with an error log if pipe_task is NULL to prevent a silent crash. - pipeline_schedule_triggered(): check trigger_task != NULL before calling schedule_task() in all IPC4 branches. - pipeline_task_init(): guard p->sched_comp != NULL before dereferencing it when computing task->registrable. - pipeline_comp_ll_task_free(): NULL out trigger_task and pipe_task after freeing to prevent use-after-free. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Address valid concerns raised by the GitHub Copilot review bot on PR thesofproject#11135. Items already fixed in later commits (HTTP->HTTPS, memcpy_s return check, ams.c message pointer, llext-wrap.c ssize_t stubs, shell script globs and duplicate function) are not repeated here. CMakeLists.txt: Remove the global Clang --text-section-literals=false flag which was applied to every Clang build, including targets unrelated to MWW. Older Clang versions reject this option, breaking unrelated targets. The MWW component's own CMakeLists.txt already guards it with check_cxx_compiler_flag() scoped to the MWW target only. src/audio/microwakeword/Kconfig: Add 'depends on IPC_MAJOR_4'. mww.c calls ipc4_d0ix_is_allowed() which is an IPC4-only API; selecting COMP_MWW for an IPC3 build results in an unresolved symbol at link time. src/audio/microwakeword/mww.c: Add #include <zephyr/kernel.h> under the CONFIG_COMP_MWW_DEBUG_TRACE guard. The debug path calls k_cycle_get_32() and k_cyc_to_us_near32() which are declared in that header; without it the build fails when DEBUG_TRACE is enabled. Add a compile-time #error guard that enforces CONFIG_COMP_MWW_PCAN and CONFIG_COMP_MFCC_PCAN are always set together. If MFCC produces int32 Q9.23 hops (PCAN off) while MWW expects int8 hops (PCAN on), every subsequent hop header and feature frame is desynchronized and inference results are garbage. The topology MWW_PCAN flag must also match this Kconfig pair. src/audio/mfcc/mfcc_common.c: In PCAN mode mfcc_prepare_output() indexes mel_linear[] by k < num_ceps, but the array has only melfb.mel_bins entries. If num_ceps > mel_bins (misconfiguration) the loop reads out of bounds. Add an early return guard using the correct field name melfb.mel_bins. src/audio/kpb.c: kpb_init_draining() already guards host_sink, but the draining path later unconditionally dereferences kpb->sel_sink when pausing the selector widget. With IPC4 late-binding sel_sink can still be NULL when a drain request arrives on another core. Add sel_sink to the is_sink_ready check so draining is deferred until both sinks are present and active. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Update the NHLT configuration and DMIC DAI copier input formats for the Alder Lake microWakeWord (MWW) PCAN KPB topology. Specifically: - In dai-copier-micsel-eqiir-kpb-be, restrict the DMIC DAI copier input audio formats to 32-bit (16 kHz, 4 channels), removing 16-bit and 24-in-32 formats to match the NHLT configuration. - In tplg-targets.cmake, update NHLT_BIN to nhlt-sof-adl-max98357a-rt5682-mww-pcan-kpb.bin for the sof-adl-max98357a-rt5682-mww-pcan-kpb target. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Copy the static build keyword detect Kconfig configuration from ACE 3.0 (intel_adsp_ace30_ptl) to ACE 4.0 (intel_adsp_ace40_nvl and nvls). This enables the static modules needed for keyword detect and feature processing (MFCC, PCAN, MWW, AMS, DRC, MULTIBAND_DRC, S24_3LE), disables loadable modules, adjusts EDF stack and heap/malloc arena sizes, sets IDC timeout, enables telemetry, disables UAOL, and retains platform-specific settings for Nova Lake. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Under Zephyr, synchronized draining mode reschedules the draining task on the shared EDF workqueue every 500 us with k_sched_lock(). This tight rescheduling loop starves other tasks on the primary core's EDF queue, notably IPC message processing, leading to IPC timeouts and system hangs when host power-management commands arrive during draining. Switch to unlimited draining mode by default during prepare. This allows KPB to empty buffered audio to the host sink in continuous chunks without repeated workqueue rescheduling, keeping the EDF queue responsive. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

Add WoV pipeline for SDW DMIC. Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

cavs-sdw also supports PCH DMIC. Add the WoV support for the PCH DMIC. Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

Add PCH DMIC WoV feature topology. Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

This patch updates the high intel model. The verification report is below: ====================================================================== microWakeWord PCAN Streaming Evaluation Report (Threshold: 0.85, Consec: 3) ====================================================================== Class Role Files Detected Rate Mean Peak ------------------------------------------------------------------ ambient Negative 1900 0 0.0% 0.002 unknown Negative 5000 23 0.5% 0.022 hi_intel Positive 4660 4657 99.9% 0.996 ------------------------------------------------------------------ >>> Confusion Matrix: Actual Positive Actual Negative Predicted Positive: TP = 4657 FP = 23 Predicted Negative: FN = 3 TN = 6877 >>> Performance Summary Metrics: - Accuracy : 99.78% - Precision : 99.51% - Recall / True Positive Rate : 99.94% (4657/4660) - False Positive / Alarm Rate : 0.33% (23/6900) - F1-Score : 0.9972 ====================================================================== Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Add sof_text_to_speech.py offline utility to synthesize phrases passed on the command line to a 48 kHz mono 16-bit WAV file using Piper TTS. The script defaults to the LibriTTS-R PyTorch checkpoint and also supports single-speaker ONNX voice models with optional speaker and speed adjustments. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

MFCC configured the PCAN gain control with zero input correction bits while the log scale stage used three. Google's microfrontend uses log2(fft_size) - 6 for both, which compensates the 1/N scale of its fixed-point FFT. The mismatch made the SOF PCAN features about 15 int8 steps lower on speech than the TFLM microfrontend features the microWakeWord models are trained with, and the models failed to detect keywords on the DSP. SOF compensates the FFT length in the Mel filter bank and scales the linear Mel magnitude to Google's 512-point range, so the correction is 3 for every FFT size. Add PCAN_INPUT_CORRECTION_BITS and use it for both stages. Also limit the linear Mel magnitude to 32 bits instead of 16, as the microfrontend filter bank output is not clamped to 16 bits. With the change the testbench features of recorded keywords differ from the microfrontend features by about 1.5 int8 steps on average. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The PCAN trainer computed its features with the TensorFlow microfrontend, which differs from the SOF MFCC output by a few int8 steps on average and is evaluated cold-start, while the firmware sees a continuously running stream. Run the SOF MFCC component in sof-testbench4 instead, repeating each clip for at least two seconds so the PCAN noise estimate is warm. The TensorFlow path is kept with --frontend tf. Real detector input on the DUT peaks around -25 dBFS, well below the -10 dBFS peak-normalized training clips. Peak-normalize every clip to a random level between -45 and -6 dBFS before feature extraction and use each keyword clip twice at independent levels. The ineffective feature-domain gain offset is removed. Also pin the model input quantization to the firmware int8 PCAN mapping (scale 666/256, zero point -128) instead of the calibrated range, which was a few percent off. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The streaming evaluation and the babble check required three consecutive inferences at or above 0.85. The firmware now triggers when the mean of the last four probabilities reaches 0.5, and the model output alternates between inferences, so the old rule reported many real detections as misses. Use the firmware rule in the report. The --consecutive-steps option is replaced by --avg-len and the default threshold becomes 0.5. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Playing the same keyword several times on the DUT gave very different scores. Offline, delaying a real capture by a few milliseconds moved the averaged probability between about 25% and 98%, so the model is sensitive to where the keyword falls against the 10 ms hop grid and the 30 ms inference stride. Training placed every positive window at 3-hop offsets and never shifted the audio by less than a hop, so only one phase was ever seen. Delay every clip by a random 0..159 samples before feature extraction and place the positive windows at one offset per phase of the 3-hop stride. The streaming evaluation also delays each clip by a random 0..479 samples so the report covers all phases. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The streaming model looks back about 149 hops (1.5 s), but training used 100-hop windows and filled the missing context with zero feature padding, while each clip was warmed up with copies of itself. On the device the model state holds real room background instead. The same synthetic keywords that triggered 85-98% as isolated clips triggered only 10-52% when streamed after silence or recorded room background, which also made the score depend strongly on input timing. Embed every clip in 2 s of continuous background taken from the ambient classes, at 5..35 dB SNR and a random 0..29 ms phase, and extract features from the whole stream. Training windows are 160 hops and are cut from that stream, so the full receptive field sees real context. The streaming evaluation embeds clips the same way and counts detections from the clip start. Note that the window length change requires retraining all models. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

====================================================================== microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50) ====================================================================== Class Role Files Detected Rate Mean Peak ------------------------------------------------------------------ ambient Negative 1900 3 0.2% 0.003 unknown Negative 5000 17 0.3% 0.006 strawberry Positive 3000 3000 100.0% 0.996 ------------------------------------------------------------------ >>> Confusion Matrix: Actual Positive Actual Negative Predicted Positive: TP = 3000 FP = 20 Predicted Negative: FN = 0 TN = 6880 >>> Performance Summary Metrics: - Accuracy : 99.80% - Precision : 99.34% - Recall / True Positive Rate : 100.00% (3000/3000) - False Positive / Alarm Rate : 0.29% (20/6900) - F1-Score : 0.9967 ====================================================================== ====================================================================== microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50) ====================================================================== Class Role Files Detected Rate Mean Peak ------------------------------------------------------------------ ambient Negative 1900 3 0.2% 0.008 unknown Negative 5000 27 0.5% 0.011 banana Positive 3000 2981 99.4% 0.992 ------------------------------------------------------------------ >>> Confusion Matrix: Actual Positive Actual Negative Predicted Positive: TP = 2981 FP = 30 Predicted Negative: FN = 19 TN = 6870 >>> Performance Summary Metrics: - Accuracy : 99.51% - Precision : 99.00% - Recall / True Positive Rate : 99.37% (2981/3000) - False Positive / Alarm Rate : 0.43% (30/6900) - F1-Score : 0.9918 ====================================================================== ====================================================================== microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50) ====================================================================== Class Role Files Detected Rate Mean Peak ------------------------------------------------------------------ ambient Negative 1900 0 0.0% 0.001 unknown Negative 5000 36 0.7% 0.010 orange Positive 3000 2995 99.8% 0.995 ------------------------------------------------------------------ >>> Confusion Matrix: Actual Positive Actual Negative Predicted Positive: TP = 2995 FP = 36 Predicted Negative: FN = 5 TN = 6864 >>> Performance Summary Metrics: - Accuracy : 99.59% - Precision : 98.81% - Recall / True Positive Rate : 99.83% (2995/3000) - False Positive / Alarm Rate : 0.52% (36/6900) - F1-Score : 0.9932 ====================================================================== Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Many generated multi-speaker keywords were slurred or noisy, for example "banana" often sounded like "panana", and the resulting banana model did not detect real speech at all. Pass lower noise_scale (0.45, 0.55, 0.667) and noise_scale_w (0.6, 0.8) values to piper-sample-generator so the consonants are more distinct, and cycle the text through "kw", "kw." and "kw!" over the speaking-rate loops for stress and intonation variety. Also mix the in-clip noise 6 dB quieter by default (6 to 31 dB SNR); the PCAN trainer now adds its own continuous background around every clip. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

====================================================================== microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50) ====================================================================== Class Role Files Detected Rate Mean Peak ------------------------------------------------------------------ ambient Negative 1900 5 0.3% 0.008 unknown Negative 5000 27 0.5% 0.010 hi_intel Positive 4980 4973 99.9% 0.995 ------------------------------------------------------------------ >>> Confusion Matrix: Actual Positive Actual Negative Predicted Positive: TP = 4973 FP = 32 Predicted Negative: FN = 7 TN = 6868 >>> Performance Summary Metrics: - Accuracy : 99.67% - Precision : 99.36% - Recall / True Positive Rate : 99.86% (4973/4980) - False Positive / Alarm Rate : 0.46% (32/6900) - F1-Score : 0.9961 ====================================================================== Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

singalsu force-pushed the mww_mfcc_kpb_development branch from 2151ccb to 28a11a0 Compare August 27, 2026 16:51

singalsu commented Aug 28, 2026

View reviewed changes

Comment thread app/boards/intel_adsp_ace15_mtpm.conf

singalsu force-pushed the mww_mfcc_kpb_development branch from 28a11a0 to 2674e49 Compare August 28, 2026 16:37

lyakh reviewed Aug 31, 2026

View reviewed changes

Comment thread scripts/llext_link_helper.py Outdated

singalsu force-pushed the mww_mfcc_kpb_development branch from 2674e49 to c036f90 Compare September 3, 2026 14:18

singalsu requested a lite review from Copilot September 3, 2026 14:20

Copilot started reviewing on behalf of singalsu September 3, 2026 14:21

View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

Comment thread src/audio/microwakeword/tune/sof_mww_prepare_silence_unknown.sh

Comment thread tools/topology/topology2/include/components/kpb.conf

Comment thread src/lib/ams.c Outdated

Comment thread src/audio/microwakeword/tune/sof_mww_generate_keyword_dataset.sh Outdated

singalsu force-pushed the mww_mfcc_kpb_development branch from c036f90 to 56a1ad9 Compare September 3, 2026 16:21

singalsu requested a lite review from Copilot September 3, 2026 16:23

Copilot started reviewing on behalf of singalsu September 3, 2026 16:26

View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

Comment thread src/audio/buffers/ring_buffer.c

Comment thread src/audio/microwakeword/llext-wrap.c Outdated

Comment thread src/lib/ams.c Outdated

Comment thread src/audio/microwakeword/tune/sof_mww_plot_mtrace.py Outdated

singalsu force-pushed the mww_mfcc_kpb_development branch from 56a1ad9 to f518867 Compare September 4, 2026 07:30

singalsu requested a lite review from Copilot September 4, 2026 07:32

Copilot started reviewing on behalf of singalsu September 4, 2026 07:33

View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

Comment thread src/audio/microwakeword/tune/sof_mww_prepare_silence_unknown.sh Outdated

singalsu force-pushed the mww_mfcc_kpb_development branch from f518867 to 35f972b Compare September 4, 2026 16:18

singalsu requested a lite review from Copilot September 4, 2026 16:19

Copilot started reviewing on behalf of singalsu September 4, 2026 16:21

View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

singalsu force-pushed the mww_mfcc_kpb_development branch from a6eed3b to d36213c Compare September 11, 2026 15:47

singalsu force-pushed the mww_mfcc_kpb_development branch 4 times, most recently from f4282c5 to c87661e Compare September 25, 2026 12:51

singalsu requested a lite review from Copilot September 28, 2026 09:25

Copilot started reviewing on behalf of singalsu September 28, 2026 09:25

View session

Copilot AI reviewed Sep 28, 2026

View reviewed changes

lyakh mentioned this pull request Sep 30, 2026

data-blob: add a syscall to call comp_data_blob_set() from userspace #11251

Merged

singalsu force-pushed the mww_mfcc_kpb_development branch from 9972bbf to d8efb6c Compare September 30, 2026 11:33

singalsu and others added 19 commits October 12, 2026 17:23

[DNM] boards: Enable modules for static build keyword detect

1c74605

This patch enables the modules needed for keyword detect in a static build: KPB, MFCC, MWW, Tensorflow, etc. The impacted platforms are cavs25 and ace30. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

tools: rimage: add to nvl include of mww.toml

b83273b

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

audio: microwakeword: tune: remove stale glitch recording check

87e27c0

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

audio: microwakeword: add models for three fruits

e017b53

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

topology2: add CHANNEL_MAP_DUAL_MONO definition

b9a23e2

Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

topology: add SDW WoV feature topology

3e4866f

Add WoV pipeline for SDW DMIC. Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

topology: cavs-sdw: add PCH DMIC WoV support

c9bad43

cavs-sdw also supports PCH DMIC. Add the WoV support for the PCH DMIC. Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

topology: add PCH DMIC WoV feature topology

3fce589

Add PCH DMIC WoV feature topology. Signed-off-by: Bard Liao <yung-chuan.liao@linux.intel.com>

singalsu force-pushed the mww_mfcc_kpb_development branch from 4034cd2 to 7b0129f Compare October 12, 2026 14:47

singalsu added 10 commits October 13, 2026 11:47

[DNM] app: boards: set model for control for ace3.0

3c51d01

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Navigation Menu

Repository navigation

audio: microwakeword: add mww component - #11135

audio: microwakeword: add mww component#11135
singalsu wants to merge 46 commits into
thesofproject:mainfrom
singalsu:mww_mfcc_kpb_development

singalsu commented Aug 27, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

singalsu commented Aug 27, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants