Guest debugging: add support for watchpoints. by cfallin · Pull Request #14775 · bytecodealliance/wasmtime

cfallin · 2026-10-15T19:30:08Z

With guest debugging enabled, every linear memory now has a 1-to-1 shadow memory. This shadow stores, for every byte, a byte of flags. The one currently defined flag is a "write watchpoint" flag, which causes a guest-debug event to be emitted whenever the corresponding memory byte is written to.

Compiled guest code's store instructions, memory.fill, memory.copy, and memory.init will all hit a write watchpoint. Right now, external (host-API) writes to memories do not. Shared memories, and the linear memories underlying GC heaps, are not watched.

Shadow memories, like ordinary linear memories, are defined by a VM struct that contains a base pointer; when a memory is imported by another instance within a store, it gets a pointer to this VM struct as well. This single base pointer location allows the shadow memory to grow when the corresponding linear memory does.

Memory::debug_watch in the host API watches or unwatches a byte range, and a write to a watched byte raises DebugEvent::Watchpoint with the memory, address, length, and (for stores) value, before the write happens.

The debugger crate reports these as DebugRunResult::Watchpoint, and its WIT API now has memory.add-watchpoint/remove-watchpoint methods and a watchpoint event.

Supported is added in the gdbstub adapter for these watchpoints as well.

Fixes bytecodealliance#12188. With guest debugging enabled, every linear memory now has a 1-to-1 shadow memory. This shadow stores, for every byte, a byte of flags. The one currently defined flag is a "write watchpoint" flag, which causes a guest-debug event to be emitted whenever the corresponding memory byte is written to. Compiled guest code's store instructions, `memory.fill`, `memory.copy`, and `memory.init` will all hit a write watchpoint. Right now, external (host-API) writes to memories do not. Shared memories, and the linear memories underlying GC heaps, are not watched. Shadow memories, like ordinary linear memories, are defined by a VM struct that contains a base pointer; when a memory is imported by another instance within a store, it gets a pointer to this VM struct as well. This single base pointer location allows the shadow memory to grow when the corresponding linear memory does. `Memory::debug_watch` in the host API watches or unwatches a byte range, and a write to a watched byte raises `DebugEvent::Watchpoint` with the memory, address, length, and (for stores) value, before the write happens. The debugger crate reports these as `DebugRunResult::Watchpoint`, and its WIT API now has `memory.add-watchpoint`/`remove-watchpoint` methods and a `watchpoint` event.

Implement GDB remote protocol write watchpoints (`Z2`/`z2`) in the gdbstub component; read and access watchpoints are rejected. Wasmtime stops before a watched write, but gdb and LLDB expect watchpoints to trigger after it and then compare old and new values, so the stub steps over the write before reporting the stop, at the first watched address the write covers. The debuggee watches bytes, not ranges, so the stub tracks its client's (possibly overlapping) watchpoints and re-watches the bytes that the remaining ones cover when one is removed.

github-actions · 2026-10-15T21:46:18Z

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

If you added a new Config method, you wrote extensive documentation for
it.

Details

Our documentation should be of the following form:

Short, simple summary sentence.

More details. These details can be multiple paragraphs. There should be
information about not just the method, but its parameters and results as
well.

Is this method fallible? If so, when can it return an error?

Can this method panic? If so, when does it panic?

# Example

Optional example here.

If you added a new Config method, or modified an existing one, you
ensured that this configuration is exercised by the fuzz targets.

Details

For example, if you expose a new strategy for allocating the next instance
slot inside the pooling allocator, you should ensure that at least one of our
fuzz targets exercises that new strategy.

Often, all that is required of you is to ensure that there is a knob for this
configuration option in wasmtime_fuzzing::Config (or one
of its nested structs).

Rarely, this may require authoring a new fuzz target to specifically test this
configuration. See our docs on fuzzing for more details.
If you are enabling a configuration option by default, make sure that it
has been fuzzed for at least two weeks before turning it on by default.

Details

To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

fitzgen

Overall, looks reasonable to me, but I'm a little unsure about the bounds checking for shadow memories. Can you verify that out of bounds wasm stores and bulk memory libcalls don't result in out of bounds loads from the shadow memory, regardless whether we have guard pages configured or not? Probably good to have a test for this too. Also, double checking: do we turn debugging on randomly in the fuzzer config? Would want to make sure that this also gets enabled randomly through that in our fuzzing.

fitzgen · 2026-10-16T19:58:44Z

+    /// The base of this linear memory's shadow bytes, when stores check them
+    /// for watchpoints (`Tunables::memory_watchpoints`).
+    pub shadow: Option<VmctxLoadChain>,


How does bounds checking work for these shadow memories? Are they guaranteed to have the same guards and movable/immovable base pointers as normal memories? Seems like a good thing to document here.

The basic idea is that we always probe the main memory first, even on a store; so we cannot reach the shadow memory access until we've validated that the access is in-bounds (and then the shadow memory is grown along with the memory it shadows so it's always the same size).

The libcalls that also can hit watchpoints (for bulk ops) do explicit bounds-checks before accessing.

I'll add a comment!

fitzgen · 2026-10-16T20:01:51Z

+            let low = builder.ins().load(I64, shadow_flags, shadow_addr, 0);
+            let high = builder.ins().load(I64, shadow_flags, shadow_addr, 8);


Why two explicit 64-bit loads instead of an i128 load?

We do a brif on the result, so I think the goal here was to try to avoid a brif on an i128 (instead OR'ing the halves together); but we have a lowering case for that so I should just use it. Will fix.

fitzgen · 2026-10-16T20:04:07Z

+    fn to_u64(builder: &mut FunctionBuilder<'_>, value: ir::Value) -> ir::Value {
+        if builder.func.dfg.value_type(value) == I64 {
+            value
+        } else {
+            builder.ins().uextend(I64, value)
+        }
+    }


Maybe assert that the given value's type is an integer and is less than or equal to 64 bits wide?

alexcrichton

At a high level, could this integrate with the normal memory allocator? It feels a bit odd to forcibly use malloc to allocate these memories and also perhaps a performance hazard waiting to happen. Fuzzing-wise I'm also not certain of this where we more-or-less rely on the fuzzer not seeing the memory allocation of linear memories because otherwise large reservations would immediately OOM.

I'm also personally always wary of adding more paths to do more things related to memory -- it's a pretty security/correctness critical part so I'd favor reuse of existing primitives over the creation of new ones.

cfallin · 2026-10-19T18:33:27Z

(Getting back to this now after the weekend, haven't addressed Nick's comments yet, sorry)

At a high level, could this integrate with the normal memory allocator?

I can definitely explore this direction more; my own early thoughts on that, though, were that it might break many more invariants (the defined-memories index space would no longer correspond to Wasm's memory index space above the imports).

perhaps a performance hazard waiting to happen

My thoughts were (IMHO at least): it's not a concern for debugging-enabled configs: such a config is already going to be pretty slow (due to the instrumentation), moreso with a "watchpoint barrier" at every store. I would, for example, definitely not want to try to integrate watchpoints with the pooling allocator if it takes explicit extra work to do that, because such a config feels like an antipattern to me: one should not need to optimize instantiation time when one is doing interactive debugging with wasmtime-cli and expects more-or-less one instantiation for the process lifetime. So it's additional configation testing surface that doesn't actually serve a user need (again IMHO).

Happy to be convinced otherwise though! And I'm very sympathetic to "don't add special cases" so maybe it's actually easier to just ride along with the configured allocator. I'll play with it some more.

alexcrichton · 2026-10-19T19:30:00Z

I was thinking more like what you have now where a memory points to another memory and it's got a special area in the VMContext for shadow memories -- I'd agree that changing the meaning of defined memories would probably wreak havoc everywhere. My rough and probably naive idea is that the normal instance allocator could be used to allocate shadow linear memories instead of malloc. That'd then be stored within the "main" linear memory as the shadow memory is today, and the main advantage would be that it'd integrate with all the other existing subsystems/etc we have by default.

My main worry is that by adding a new thing on the side we'll end up needing features for that over time and/or have bugs about how its subtly different from the "other thing". For example having to be so careful here about validating bounds checks already has me a bit worried, although I don't see any concrete issue doing a load first and then looking at the shadow memory (we can say we actually fix the store-across-two-pages-but-the-second-is-invalid problem!). If it's too onerous to implement this though it's probably not worth pursuing as it's all scoped ideally to debug-mode-only anyway.

cfallin added 2 commits October 15, 2026 12:26

cfallin requested review from a team as code owners October 15, 2026 19:30

cfallin requested review from alexcrichton and removed request for a team October 15, 2026 19:30

github-actions Bot added wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime labels Oct 15, 2026

fitzgen reviewed Oct 16, 2026

View reviewed changes

alexcrichton reviewed Oct 19, 2026

View reviewed changes

Navigation Menu

Repository navigation

Guest debugging: add support for watchpoints. - #14775

Guest debugging: add support for watchpoints.#14775
cfallin wants to merge 2 commits into
bytecodealliance:mainfrom
cfallin:the-watchpoints-are-watching-you

cfallin commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

fitzgen left a comment

Uh oh!

fitzgen Oct 16, 2026

Uh oh!

cfallin Oct 16, 2026

Uh oh!

fitzgen Oct 16, 2026

Uh oh!

cfallin Oct 16, 2026

Uh oh!

fitzgen Oct 16, 2026

Uh oh!

alexcrichton left a comment

Uh oh!

cfallin commented Oct 19, 2026

Uh oh!

alexcrichton commented Oct 19, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		let low = builder.ins().load(I64, shadow_flags, shadow_addr, 0);
		let high = builder.ins().load(I64, shadow_flags, shadow_addr, 8);

Conversation

cfallin commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Label Messager: wasmtime:config

Uh oh!

fitzgen left a comment

Choose a reason for hiding this comment

Uh oh!

fitzgen Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

cfallin Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

fitzgen Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

cfallin Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

fitzgen Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

cfallin commented Oct 19, 2026

Uh oh!

alexcrichton commented Oct 19, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants