Repository navigation
2 GB at the start #5587
Description
Just entering the main() in a program linking to "openblas" under MSYS2 consumes 2 GB of RAM.
VMMap shows it as 16 blocks 128 MB each. Are these allocations necessary? Can they be postponed to when I really need something from OpenBLAS?
Setting these variables reduces the memory to 600 MB. It looks like 128 MB is allocated per thread in DLLMain
set OPENBLAS_NUM_THREADS=4
set GOTO_NUM_THREADS=4
set OMP_NUM_THREADS=4
This does not help either:
openblas_set_num_threads(1);
The original issue was against libllama: ggml-org/llama.cpp#18024
Activity
martin-frbg commentedon Dec 29, 2025
That is due to the "GEMM buffer" and associated management structures needed for data exchange in multithreaded operation - to improve performance, this is created as a static array on startup, instead of calling malloc later.
The size of this buffer directly relates to the maximum matrix dimensions the library will be able to handle - if you are sure that you will not work with matrix sizes above 30000, you can reduce it at compile time by passing BUFFERSIZE=20 (or lower) to make or CMake (see Makefile.rule, the 20 translates to 32<<20 i.e. 32MB instead of the default 128MB corresponding to 32<<22)
tim-lebedkov commentedon Dec 30, 2025
Do you agree that this behavior is bad or should I elaborate further?
martin-frbg commentedon Dec 30, 2025
I consider it a feature, and it has been that way since the early days of GotoBLAS, 20+ years.
What has changed is the diversity of applications, and the expectation to find a pre-installed library instead of tailoring it to one's particular needs when building from source.
tim-lebedkov commentedon Dec 30, 2025
I agree that this is a feature, but a bad one, the one that should be removed. A well behaving library should not initialize itself via DLLMain. It should have a function init() and the corresponding done().
As an example, my application consumes about 100 MB. It is a UI. If and when the user decides to talk to AI, I would like to initialize libllama and openblas and compute the answers. And if the users closes the "Chat with AI" window, I would want to free the used memory.
Your argument about a static array and performance is not valid as OpenBLAS calls VirtualAlloc 16 times and this could easily be done in a init(). Obviously, everything what is done in DLLMain should be done in init().
tim-lebedkov commentedon Jan 4, 2026
While you are still thinking, here is an idea how to do this and keep the backwards compatibility: check GetProcAddress("LIBOPENBLAS_AUTOINIT") in DLLMain and skip the initialization if the function returns a non-null value. The main program could then export a function named "LIBOPENBLAS_AUTOINIT" and prevent the auto-initialization. I have not tested this though.
jschueller commentedon Jul 2, 2026
hello, what about this patch ?
threads.patch.txt
martin-frbg commentedon Jul 2, 2026
Sorry, your attachment is unreadable - all I see is an xml'ified error message about range being invalid for the resource.
Also note that this problem may be related to #4665 - there was a somewhat optimistic attempt at reorganizing the GEMM buffer code, but it (probably) led to spurious allocation of a per-thread buffer that goes unused in subsequent operations. Reverting that part of the original PR may already be sufficient to solve this
montjeffrey commentedon Oct 13, 2026
I opened #6125 for this. It makes the Win32 thread server allocate each worker's buffer on first use instead of at thread start, which is what the pthreads server (
blas_server.c) already does.The startup commit comes from
blas_thread_serverinblas_server_win32.ccallingblas_memory_alloc(2)as its first statement, and the Windows allocator commits the memory immediately. Measured withBUFFERSIZE=20(the numpy wheel setting) on a 20-thread machine, private bytes go from 610.6 MB to 1.5 MB after load with no BLAS call. Footprint is unchanged once every thread has done work. Details and test results are in the PR.Allocate Win32 thread server buffers on first use