Skip to content

Allocate Win32 thread server buffers on first use - #6125

Merged
martin-frbg merged 1 commit into
OpenMathLib:developfrom
montjeffrey:win32-lazy-thread-buffer
Oct 14, 2026
Merged

martin-frbg merged 1 commit into
OpenMathLib:developfrom
montjeffrey:win32-lazy-thread-buffer

Conversation

@montjeffrey

Copy link
Copy Markdown
Contributor

Refs #5587.

Problem

On Windows, blas_thread_server allocates its GEMM buffer as the first statement of every worker thread. The Windows allocator uses VirtualAlloc(MEM_RESERVE | MEM_COMMIT), so each idle worker adds BUFFER_SIZE to the commit charge of any process that merely loads the library, before a single BLAS call. With the 32 MB buffer the numpy wheels use, a 20-thread CPU commits ~608 MB on import numpy. Processes that load OpenBLAS transitively and never call BLAS pay it in full, once per process.

Change

Allocate the per-thread buffer the first time a queued job needs it (sa == NULL), which is the only place it is used, and free it at shutdown only if it was allocated. The pthreads server (blas_server.c) already allocates lazily on first work; this brings the Win32 server in line.

Measurements

Windows 11, 20 logical CPUs, MinGW-w64 gcc 15, TARGET=HASWELL BUFFERSIZE=20 NUM_THREADS=24 BUILD_WITHOUT_LAPACK=ON, same tree with and without the patch. Private bytes (PROCESS_MEMORY_COUNTERS_EX.PrivateUsage):

Point develop this PR
After load, no BLAS call 610.6 MB 1.5 MB
After a small single-threaded dgemm 642.7 MB 33.6 MB
After a 4-thread dgemm (n=1024) 642.8 MB 129.8 MB
After all 20 threads have run d/s/zgemm 643.0 MB 643.0 MB
After 20 more threaded dgemm 643.0 MB 643.0 MB

Once every worker has done work the footprint is unchanged; only workers that never run a job stop paying for a buffer.

Testing

  • ctest: 16/16 pass with and without the patch.
  • dgemm (n=1024), sgemm (n=1024) and zgemm (n=512) with 4 and with all threads match a naive reference exactly.
  • 8 concurrent caller threads x 40 threaded dgemm each from a cold start (so worker buffers are first allocated while other callers are mid-flight): 0 mismatches over 3 runs.
  • No new compiler warnings in blas_server_win32.c.

Not tested locally: MSVC / clang-cl builds and DYNAMIC_ARCH.

blas_thread_server allocated its GEMM buffer as the first statement of
every worker thread. On Windows that allocation is VirtualAlloc with
MEM_RESERVE | MEM_COMMIT, so each idle worker added BUFFER_SIZE to the
commit charge of any process that merely loads the library: with the
32 MB buffer used by the numpy wheels, 19 workers on a 20-thread CPU
committed 608 MB before a single BLAS call.

Allocate the buffer the first time a queued job needs it (sa == NULL),
as the pthreads server already does, and only free it if it was
allocated.

Measured on a 20-thread Windows 11 machine (MinGW-w64 gcc 15, HASWELL,
BUFFERSIZE=20, NUM_THREADS=24), private bytes:
  after load, no BLAS call:        610.6 MB -> 1.5 MB
  after a 4-thread dgemm:          642.8 MB -> 129.8 MB
  after all 20 threads have run:   643.0 MB -> 643.0 MB (unchanged)
ctest passes 16/16, dgemm/sgemm/zgemm results match a naive reference,
and 8 concurrent callers x 40 threaded dgemm from a cold start show no
mismatches.

Refs OpenMathLib#5587
@montjeffrey montjeffrey mentioned this pull request Oct 13, 2026
@martin-frbg martin-frbg linked an issue Oct 14, 2026 that may be closed by this pull request
@martin-frbg

Copy link
Copy Markdown
Collaborator

Thank you very much

@martin-frbg
martin-frbg merged commit 504237f into OpenMathLib:develop Oct 14, 2026
145 of 151 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 GB at the start

2 participants