Skip to content

The type of a huge integer literal no longer costs a division by a power of ten as large as it - #432

Closed
enumeratio wants to merge 1 commit into
cortex-js:mainfrom
enumeratio:big-integer-type
Closed

enumeratio wants to merge 1 commit into
cortex-js:mainfrom
enumeratio:big-integer-type

Conversation

@enumeratio

Copy link
Copy Markdown
Contributor

Fix #431.

  • The .type of an integer literal past the double range: literalEnclosureType now returns no enclosure as soon as the literal's double value is ±Infinity, before any bignum work. Both bounds would have had to be finite doubles, so the type is unchanged (integer<1..>, integer<..-1>).
  • bigintDigits past the pow10 cache limit (100 000 digits): it now estimates log10(n) from the top hex digits and the hex length, and makes one exact n >= 10^k comparison only when the estimate is within its rounding margin of an integer. Below the limit nothing changes.
  • bigRatioToFloat, which computes .re: it reads bit lengths from the hex form, and when the numerator is wider it shifts only the numerator. The quotient and the sticky bit (which decides rounding) are the same as before.

ce.number(7n * 10n ** BigInt(d) + 12345n).type, per new literal (ms, median of 7):

digits before after toString(16)
25 000 0.41 0.05 0.01
100 000 6.26 0.13 0.06
200 000 21.0 0.28 0.14
1 000 000 193 1.3 0.61

New tests check:

  • bigintDigits against toString().length around 10^k on both sides of the cache limit, and for random values;
  • rationalAsFloat at exact ties and just above them, with a wide numerator;
  • the type of a 400-digit and a 200 000-digit literal.

… past the double range; count the digits of a huge bigint from a log10 estimate with one exact check; shift only the numerator in bigRatioToFloat. Fix cortex-js#431
@arnog

arnog commented Oct 14, 2026

Copy link
Copy Markdown
Member

Landed on main as de19cf1 with your commit's content, credited as co-author. Two small edits on the way in: the CHANGELOG entry moved to a new [Unreleased] section (0.153.0 had shipped), and the four rationalAsFloat tie tests now use 2^200 denominators so the quotients stay finite (near 2^953; at 2^1053 the expected value was Infinity and the tests passed without the sticky bit). Thanks for the fix and the careful error analysis.

@arnog arnog closed this Oct 14, 2026
arnog added a commit that referenced this pull request Oct 14, 2026
… large integers (merge PR #432)

Co-authored-by: enumeratio <enumeratio@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

The type of a large integer literal takes ~20 ms at 200k digits

2 participants