Export GPU device memory (free/total) for FFI consumers' memory checks #9

Release triage — 2026-10-05

Priority: P2 · not a release blocker. Without it llamadart skips the image-model memory check on non-Metal GPUs, which is the documented current behavior; nothing regresses. Schedule after the release; needs a runtime release, then llamadart adoption (leehack/llamadart#802).

llamadart's image-model memory check needs the free and total memory of the GPU a model loads on. The runtime exports only the SD_API symbols of stable-diffusion.h, and none of them reports device memory: sd_list_devices prints only name<TAB>description per device, and ggml's ggml_backend_dev_memory is not exported.

Without it, llamadart cannot budget a Vulkan GPU correctly (leehack/llamadart#802):

• Host RAM is the wrong figure on a discrete GPU. On an NVIDIA L4 (24 GB) with 32 GB of RAM, host RSS stayed under 2 GB for every model, so comparing the estimate with MemAvailable would wrongly refuse FLUX.1-schnell Q8 or SD 3.5 Large Turbo Q8 on a 16 GB host, and would allow FLUX on a host with plenty of RAM and an 8 GB GPU, which then depends on auto-fit spilling.
• With vae_conv_direct, llamadart's estimate (weights x 1.25 + overhead) tracked measured peak VRAM within about 0 to 15% at 1024x1024, so a device-memory figure is all that is missing.

For now llamadart skips the check on non-Metal GPUs (Metal uses recommendedMaxWorkingSetSize, read through the Objective-C runtime from Dart).

Request

Export a way to read device memory, outside the upstream sources and on the export allowlist, for example either of:

1. size_t sd_native_list_devices_with_memory(char* buffer, size_t buffer_size) printing name<TAB>description<TAB>free_bytes<TAB>total_bytes per device (ggml_backend_dev_memory), same buffer contract as sd_list_devices; or
2. bool sd_native_device_memory(const char* device_name, size_t* free_bytes, size_t* total_bytes).

Notes:

• It must work before any context exists (the check runs before new_sd_ctx), loading dynamic backends the way sd_list_devices does.
• Report what ggml reports; for Vulkan, ggml uses VK_EXT_memory_budget when available, so free reflects other processes' use.
• Unified-memory devices (Apple, integrated GPUs) should report their figure too; llamadart will keep its Metal path but could use this as a cross-check.

After a release with the export, llamadart will budget Vulkan devices against free_bytes and document it.