[AlexaCampusano]

Related read Storage Strategies for Collectionsin Dynamically Typed Languages

What are we trying to do?

This PR is a proposal to add fixnums stride information into the array object, so heap arrays have more information about the data it's storing and we can optimize it.

Prior Art: #1081

In this PR I'm targetting all fixnum arrays, not just frozen ones. This PR affects these common paths:

• At first mutation, arrays collect the stride for the fixnum (whether it's 8bit, 16bit, 32bit or VALUE), it also collects the signedness of the value being appended, both get added to the Array flags. We then proceed to narrow the VALUE to its raw representation for that value. This happens the moment the Array gets a heap buffer (on the embedded->heap spill).
• Fixnum Array literals, when the bytecode compiler sees an all literal fixnum array, it builds a hidden frozen template array and embeds it in the iseq of duparray. Now it narrows that template if it's all fixnums, and at runtime duparray copies the narrow buffer into a new narrow array instead of handing out a shared view of the template. The literal owns its own (small) buffer, and the template is collected with its iseq. There are concerns I have with this approach, that I'd like to discuss.
• Mutations keep the array narrow, if a value doesn't fit the current width we promote W8 → W16 → W32 in place. The first non-fixnum (or a fixnum past 32 bits, or a sign change for now) widens the array back to VALUEs, once, and it stays wide.
• Copies stay narrow. dup, clone, a[i, n], first(n), last(n), pop(n), replace, sort_by! copy the narrow buffer rather than sharing it, so neither the source nor the copy gets widened by being read from.
• Reading an element decodes the raw integer back into a VALUE on the fly, so looking at a narrow array never changes its representation. C extensions are the exception: they read arrays through the public RARRAY_PTR family, which has to return a buffer of actual VALUEs. When an extension asks for that pointer on a narrow array, the array is widened first, once, and stays wide. Extensions keep working unchanged; they just lose the memory saving for the arrays they touch.
• freeze no longer does the narrowing. By the time an array is frozen it's already narrow from how it was built, so freeze just trims the unused capacity. The one case it still narrows is an array that was filled by C code through a raw pointer, since that path never went through our write logic.
• Both JITs already called into C for most array work, and C is stride-aware, so they mostly didn't need changes. The exceptions are the handful of places where a JIT reads or writes the buffer directly in generated code. Those now check the stride first and fall back to the C path for narrow arrays, so compiled code never touches a narrow buffer as if it held VALUEs.

Some numbers!!!!

More narrowed paths exercised vs the previous #1081 work.

 Memory: master -> patch   (base 8fdf4342b5; patch = e8ed7bed71 + rb_ary_widen export)
     Array/total = ObjectSpace.memsize_of_all after full GC, including hidden literal templates
     (dump_all over every ARRAY); RSS = mean per-iteration after warmup

     == Narrow path exercised ==============================================

     blurhash
       1 array x 124,848 elems (pixel buffer, make_shareable)
       Array       1168.0 KiB  ->      311.9 KiB   -73.3%
       total          3.3 MiB  ->        2.4 MiB   -25.7%
       RSS           20.4 MiB  ->       19.5 MiB    -4.2%

     mail
       Ragel parser tables, unfrozen literals (hidden templates 3038 KiB -> 75 KiB)
       Array       3289.7 KiB  ->     1026.4 KiB   -68.8%
       total          8.8 MiB  ->        6.5 MiB   -25.6%
       RSS           74.6 MiB  ->       73.2 MiB    -1.8%

     erubi-rails
       same mail-gem tables via ActionMailer (hidden templates 3172 KiB -> 86 KiB)
       Array       3747.6 KiB  ->     1383.1 KiB   -63.1%
       total         12.3 MiB  ->        9.9 MiB   -18.9%
       RSS          117.6 MiB  ->      118.2 MiB    +0.5%

     railsbench
       same mail-gem tables (hidden templates 3201 KiB -> 104 KiB)
       Array       4149.8 KiB  ->     1784.4 KiB   -57.0%
       total         15.8 MiB  ->       13.5 MiB   -14.7%
       RSS          124.6 MiB  ->      125.1 MiB    +0.5%

     lobsters
       same mail-gem tables (hidden templates 3919 KiB -> 133 KiB)
       Array       5848.7 KiB  ->     2958.3 KiB   -49.4%
       total         34.2 MiB  ->       31.4 MiB    -8.2%
       RSS          374.8 MiB  ->      341.4 MiB    -8.9%

     chunky-png
       Array        599.6 KiB  ->      427.0 KiB   -28.8%
       total          4.4 MiB  ->        4.3 MiB    -4.2%
       RSS           99.5 MiB  ->      100.9 MiB    +1.4%

     rubocop
       Array       2743.0 KiB  ->     2167.3 KiB   -21.0%
       total         13.5 MiB  ->       12.9 MiB    -4.3%
       RSS          109.6 MiB  ->      115.2 MiB    +5.2%

     lee
       Array        310.1 KiB  ->      267.8 KiB   -13.6%
       total          4.1 MiB  ->        4.1 MiB    -1.2%
       RSS           44.3 MiB  ->       41.5 MiB    -6.3%

     optcarrot
       emulator tables, Array.new(n, 0) / [0] * n, mutable
       Array      30834.5 KiB  ->    28995.0 KiB    -6.0%
       total         31.2 MiB  ->       29.4 MiB    -5.8%
       RSS           62.2 MiB  ->       53.6 MiB   -13.8%

     sudoku
       Array        280.3 KiB  ->      260.4 KiB    -7.1%
       total          2.3 MiB  ->        2.3 MiB    -1.4%
       RSS           21.5 MiB  ->       21.2 MiB    -1.5%

     liquid-compile   Array -9.2%   total -3.5%   RSS -2.7%
     knucleotide      Array -9.1%   total +0.1%   RSS  0.0%
     graphql          Array -5.2%   total -0.8%   RSS -0.4%
     nbody            Array -4.9%   total -0.9%   RSS -0.6%
     sequel           Array -4.9%   total -0.1%   RSS +0.2%
     protoboeuf       Array -4.4%   total -0.4%   RSS +0.1%
     liquid-c         Array -4.1%   total -0.9%   RSS -3.2%

     == Narrow path not exercised (Array within -3%, total within -1%) ========

     ruby-lsp, nqueens, ruby-json, liquid-il, binarytrees, liquid-render,
     activerecord, psych-load, fannkuchredux, matmul, rubyboy, hexapdf, shipit

     No benchmark grew.  RSS deltas under ~3% on processes over 100 MiB are noise.

[AlexaCampusano]

I'm concerned about this part here, I'm currently talking to the llm about it.

I don't see why we can't also share the root's buffer if it's narrowed, we should be able to do CoW for this as well.

The shared view would just need to know the root's stride to read it. I might have to change a few things to make narrow arrays shareable, mostly around how a view stops sharing without touching the root's buffer. That way we avoid this, having to create new buffers when we read an array partially or when evaluating array literals and shared templates, and those stay O(1) like they are today in the normal path.

[AlexaCampusano]

Added changes in 7d5b74b to support doing this.

Basically the change is:

Everything inside array.c and the VM works at the element width: reads decode, writes encode, slices share, GC skips. Nothing internal ever needs the buffer as VALUEs. Everything is stride aware.

When something external asks for the buffer as a VALUE * (RARRAY_CONST_PTR, RARRAY_PTR, the public RARRAY_AREF, i.e. C extensions), the array has to produce one, and what happens depends on what the array is:

• Owns its buffer, mutable: widened in place. It stays wide after that.
• A view: unshares into its own wide buffer. The root and sibling views are untouched.
• Frozen: never changes. It gets a wide copy it keeps alongside the narrow buffer, so views into it and other Ractors reading it are unaffected.

[AlexaCampusano]

So far I've been avoiding embedded arrays. I'm only narrowing when an array gets a heap buffer (the embed→heap spill, or the first write into a buffer that was allocated up front), because the stride can only change the size of the malloc'ed buffer. The object itself is allocated in a slot picked from its capacity, and I can't change that, those are fixed.

Buuut, I was thinking, and maybe this is obvious, but I've had it in my mind as well: the slot payload is just bytes. We could narrow the embedded elements too and fit much more in the slot the GC already gave that Array, and delay having to malloc at all because more packed values fit embedded. A 40-byte slot has 24 bytes of payload: 3 VALUEs today, but the same bytes could hold 24 8bit integers or 12 16bit ones.

Is there a reason not to do that?

[AlexaCampusano]

It could also be a follow up if we think it will add complexity and needs discussion