[ulfarsson]

Why

With a property as input, auto_bisc kept every good and every bad permutation up to its current length as lists of Perm objects. At length 10 the bad ones are nearly all of S_10, which take 0.9 to 1.8 GB as lists, depending on the Python. Runs on sets whose learning reaches length 10 grew past 8 GB. Three such sets, from Dong and Xu's paper on fixed and pixed points, still stopped at an 8 GB cap.

What

• PropertyBitmaps (in permuta.bisc) holds a property's values on the permutations, length by length, in the smaller of two forms:
  • a bitmap, (n! + 7) // 8 bytes (S_10's is 454 KB). Bit i of length n is the label of Perm.unrank(i, n), so a permutation's bit is its rank() less the number of shorter permutations. This is the order in which Perm.of_length(n) yields them.
  • the sorted indices of the good permutations, 8 bytes each in an array("Q"), when fewer than one permutation in 64 of the length is good, so that they take less than the bitmap, but for the few percent an array allocates ahead. Three permutations of length 14 take 24 bytes, where the bitmap would take 10.9 GB. Past length 20, where an index can need more than 8 bytes, the indices are kept in a list, so there is no longest length. There a length's bitmap cannot be built, and its bad permutations, more than sys.maxsize, have no len(), though they can be iterated over and tested with in.
  • The form is chosen length by length, by its count of good permutations, since a property's share of them can rise with the length. Every way in chooses it. add_length(n) labels a length with the property, keeping the indices while they are few. Once there are more than a length keeps as indices, it sets them in the bitmap and labels the rest of the length in it, a bit for each good permutation, so that its peak is two bitmaps.
  • It can be made from a property, from given bitmaps, or from both. A given bitmap must be bytes-like and of its length's size, with no bit set past the length's permutations, and it is turned into indices when they are the smaller.
  • bitmaps (length to bitmap) is a read-only Mapping. It gives a length kept as a bitmap as it is kept, as bytes. It builds the bitmap of a length kept as indices each time it is read, so items(), values() or dict() build every such length's bitmap, 10.9 GB for length 14; it builds none to answer in, to iterate over the lengths, or for len(). good_counts gives the number of good permutations of each length. No int is made of a whole bitmap: its bits are counted a chunk at a time.
• Every input becomes one. bisc and auto_bisc take what they took before, and turn each input into a PropertyBitmaps at the start. From there, one code path reads the good and the bad permutations through its views.
  • A list's permutations are the good ones, and a length with none listed is empty.
  • A property labels each length bisc or auto_bisc reads: in auto_bisc, those up to 8 at the start, and each longer one when it is first needed.
  • A dictionary given to bisc, or any other Mapping, is read for the lengths bisc reads.
  • For a tuple of two dictionaries, or a stored set, the good and the bad permutations of each length must not overlap and must together be all of that length's permutations. Otherwise a ValueError says how many are in both and how many in neither. A permutation given twice on one side counts once. Every stored set in permuta/resources/bisc passes.
  • A PropertyBitmaps labels the lengths it lacks with its property, and keeps them for a later run.
• Its views, good and bad, are each a Mapping from lengths to Collections of permutations.
  • Iterating over a length yields its permutations in lexicographic order. For a length kept as indices, the good permutations are unranked from them, and the bad ones are generated with the good indices skipped in step. From a bitmap, when at most one in 16 is wanted, the view unranks them; otherwise it generates the length and tests each bit. Each of these is a small generator of its own.
  • len() and bool() read the count kept with the labels. in takes any sequence that is a permutation of the length, and reads its bit, or looks for its index by binary search.
• Typing: bisc.py no longer starts with # type: ignore, so mypy checks it, and its new code is typed.
  • bisc_subfunctions.py keeps its errors ignored, but now with # mypy: ignore-errors, so that mypy can see its functions.
  • Typing bisc_subfunctions.py is left to a pull request of its own.
  • bisc and auto_bisc have return types, and permuta/bisc/__init__.py is checked too.

PropertyBitmaps is exported from permuta.bisc. The README's BiSC section describes it, with an example that runs as a doctest.

Results

What bisc and auto_bisc print and return is byte for byte the same as on develop, on CPython 3.11 and PyPy 3.11, for every input kind compared:

• auto_bisc given three properties, one of them growing past length 9;
• auto_bisc given a list, a shuffled list, a tuple of dictionaries (with its lists in order and reversed), and two stored sets;
• bisc given a list, a shuffled list, a property (with m <= n and with m > n), and a dictionary (with its lists in order and reversed);
• bisc given three permutations of length 12, and three of length 14, with every length up to them kept as indices;
• bisc given one permutation of length 21, and one of length 25.

Malformed input is treated differently:

• A permutation listed twice counts once. With (0, 1) listed twice among the increasing permutations, develop's lists took length 2 as complete and found nothing; now bisc finds 10.
• A tuple or stored set whose good and bad permutations overlap, or miss one, raises a ValueError. Before, it was taken as it was.
• A permutation under another length's key raises a ValueError that names it.
• A Perm whose values are not those of a permutation of its length, such as one written from 1, raises a ValueError that names it. Before, it was mined as it was given, which made no sense of it: for Av(231) written from 1, bisc(..., 3, 6) forbade all six patterns of length 3.
• A tuple whose bad dictionary stops before length 7 stops auto_bisc at once, with the message develop printed at the end of its run: "You need to add longer permutations to the dictionaries".

The permutations are read in lexicographic order. So with a dictionary in another order, a message may name a different first permutation, and a pattern's shadings may come in another order. Three permutations of length 13, not given in lexicographic order, give the same patterns and shadings as on develop, in the order develop gives for the same three sorted. For the same reason, a list's order no longer changes the time either.

Speed and memory

Input	CPython 3.11, develop	CPython 3.11, #209	PyPy 3.11, develop	PyPy 3.11, #209	CPython, tracemalloc's peak: develop, #209
bisc(stack-sortable, 3, 8)	0.49 s, 41 MB	0.50 s, 40 MB	0.66 s, 100 MB	0.70 s, 103 MB	0.6 MB, 0.2 MB
bisc(list, 3, 8)	0.50 s, 41 MB	0.50 s, 41 MB	0.64 s, 98 MB	0.66 s, 100 MB	0.6 MB, 0.8 MB
bisc(dict, 3, 8)	0.50 s, 41 MB	0.51 s, 42 MB	0.64 s, 98 MB	0.66 s, 100 MB	0.6 MB, 0.8 MB
bisc, 20 permutations of length 10	0.67 s, 41 MB	0.68 s, 42 MB	0.46 s, 90 MB	0.47 s, 90 MB	1.2 MB, 1.2 MB
bisc, 20 permutations of length 11	1.15 s, 42 MB	1.14 s, 42 MB	0.58 s, 92 MB	0.58 s, 92 MB	1.4 MB, 1.4 MB
bisc, 3 permutations of length 12	4.12 s, 42 MB	4.15 s, 42 MB	1.20 s, 100 MB	1.19 s, 102 MB	1.4 MB, 1.4 MB
bisc, 3 permutations of length 13	2.99 s, 42 MB	3.04 s, 42 MB	1.05 s, 99 MB	0.98 s, 97 MB	1.3 MB, 1.4 MB
bisc, 3 permutations of length 14	2.54 s, 42 MB	2.53 s, 42 MB	0.79 s, 90 MB	0.88 s, 95 MB	1.4 MB, 1.4 MB
auto_bisc(stack-sortable)	0.60 s, 55 MB	0.63 s, 41 MB	1.25 s, 132 MB	1.28 s, 119 MB	13.8 MB, 0.3 MB
auto_bisc(West-2-stack-sortable)	0.89 s, 55 MB	0.93 s, 41 MB	2.03 s, 150 MB	1.75 s, 113 MB	13.8 MB, 0.2 MB
auto_bisc(smooth)	0.85 s, 55 MB	0.89 s, 41 MB	1.31 s, 128 MB	1.10 s, 113 MB	13.9 MB, 0.3 MB
auto_bisc(list to length 8)	2.87 s, 55 MB	0.63 s, 42 MB	2.00 s, 125 MB	1.10 s, 115 MB	13.8 MB, 0.9 MB
auto_bisc(tuple to length 8)	0.89 s, 55 MB	1.02 s, 55 MB	1.88 s, 138 MB	1.78 s, 136 MB	13.8 MB, 13.9 MB
auto_bisc("smooth")	0.59 s, 62 MB	0.72 s, 62 MB	1.37 s, 144 MB	1.49 s, 142 MB	19.2 MB, 19.2 MB
auto_bisc("SimSun")	0.84 s, 62 MB	0.96 s, 62 MB	1.39 s, 130 MB	1.35 s, 136 MB	19.1 MB, 19.1 MB
auto_bisc, a property growing to length 10	2.94 s, 172 MB	2.78 s, 41 MB	1.44 s, 196 MB	1.41 s, 112 MB	125.1 MB, 0.5 MB

Each figure is a median of runs, each run in its own process, with develop's and this branch's interleaved, on an idle Mac at normal priority: ten runs on PyPy 3.11 (8.0.1), five on CPython 3.11.11. The seconds are wall time, which the CPU time matched, and the memory is the peak resident. The last column is tracemalloc's peak on CPython, from a run of its own. CPython 3.14 was not run. The figures are for b746f89, run with -I; the commits after it change the time only by the check of a listed permutation's values (below).

• Against develop. On CPython every input is within 5% of develop, but for the tuple and the stored sets below, and two that are faster. On PyPy every input is within about 10%, with auto_bisc(smooth), auto_bisc(West-2-stack-sortable) and a list given to auto_bisc faster.
• PyPy's slow mode. PyPy spends about 0.8 s of these one- to two-second runs tracing and compiling (its jit-summary), and the same code can run in a slower mode from one run or session to the next, and with -I or without it. So its figures move more than CPython's. In ten runs, auto_bisc(smooth) took 1.09 to 1.37 s here, and 1.19 to 1.35 s on develop.
• The cost. Every permutation given in a list, a dictionary, a tuple or a stored set is ranked, and its values checked, to make its bit or its index; a tuple and a stored set are also checked for overlap. On CPython that costs about 15% for the tuple to length 8 and for "SimSun", and 22% for "smooth". For a list or dictionary given to bisc, it shows only where the mining is short. For the Baxter permutations to length 9 with m = 3, which leave mine nothing to do, the review measured 0.006 s on develop and 0.14 s on b746f89, on CPython. With m = 4, develop and this branch are within the runs' noise of each other. Henning accepted these costs (2026-10-24).
• The check of the values, added after b746f89, costs about a tenth of rank()'s time on CPython and a third of it on PyPy. Against b746f89, in five runs each at background priority, it added about 5% to the tuple to length 8 on CPython, 1% to "smooth", and an eighth to the 0.65 s of the Baxter permutations with m = 3. On PyPy it added a third to those (0.27 s against 0.20 s), and the medians of the other cases were 0.79 to 1.07 of b746f89's, within PyPy's noise.
• Faster. A list given to auto_bisc takes a fifth of develop's time on CPython and a half on PyPy, since develop took its complement with perm not in a list.
• Memory. tracemalloc's peak falls where develop kept lists: from 125 MB to 0.5 MB for a property growing to length 10, and from 13.8 MB to 0.3 MB for auto_bisc(stack-sortable). A tuple or a stored set keeps its input's own lists, so its peak is develop's.
• A short list of long permutations runs as on develop, in the time and memory of the mining, with each length up to them kept as indices.

Tests

• tests/bisc/test_property_bitmaps.py covers:

  • the bit order, against Perm.unrank and Perm.rank;
  • the views, against lists and as a Mapping of Collections, both when they generate and when they unrank;
  • round trips through bitmaps, and the refusal of malformed ones;
  • each input kind of bisc and auto_bisc, with a spy on mine() for the bitmaps it is given;
  • a list with a length missing and with a duplicate, and a property with m > n;
  • the ValueError for a tuple, and every stored set;
  • auto_bisc growing past the labeled lengths.

• tests/bisc/test_bisc_inputs.py covers how bisc and auto_bisc take each kind of input:

  • the default n of each, m > n, and m or n zero;
  • the empty set, and report=True;
  • what auto_bisc prints for each input kind, and where it stops;
  • growth that would label a length it was given;
  • where a view starts to unrank.

  Its expected values are the answers from before this change, or follow from the mathematics.

• tests/bisc/test_sparse_storage.py covers the two forms of a length's labels:

  • both forms against brute force, for every view, the count and the bitmap, and for every way in (a list or a mapping, shuffled and with permutations given twice, a tuple, bitmaps and a property), at lengths 0 to 8. The threshold is set to one in 2 as well as the default one in 64, and the good sets have no permutation, all, as many as the threshold, one fewer, and a random number on each side of it;
  • add_length switching to a bitmap in the middle of a length, or staying with indices one short of the threshold, or at the very end of a length; dropping its indices once it has the bitmap (CPython only); a property that raises in it, before or after the switch, leaving the length as it was; and labels replaced from one form to the other;
  • a bitmap given as a bytearray or a memoryview, of any format, shape or stride, and changed afterwards; bitmaps giving bytes, the same object for a length kept as a bitmap, and a round trip through it;
  • a permutation listed many times, and a tuple's overlap counted in every pair of forms, lists of indices included; the first and the last permutation of a length kept as indices, a view of one iterated twice or given to bisc, and a view with nothing to yield, which reads no labels;
  • no bitmap built for a length kept as indices unless its bitmap is asked for, and none sized at all for a length of 14 or for the lengths past n, checked through the code's structure, so that this runs on PyPy too;
  • the last permutation of lengths 20, 21, 25, 100 and 200, an index of length 21 above 2**63, and bisc on one permutation of length 21 or 25, against develop's answer; a length past 20 counted, iterated, tested with in and true, and its bitmap refused; the lengths past n kept as empty lists past 20, with m = 25; an empty length 25; and the bad view of length 20 read a few permutations at a time;
  • sequences that are not permutations, refused in both forms, through bisc and through a PropertyBitmaps: ones whose rank is another permutation's, such as (0, 0, 0) and ones written from 1, and ones whose rank is outside their length's;
  • three permutations of length 14 giving develop's answer, with tracemalloc's peak under 1 MB (CPython only). A fault that would size a bitmap past length 10 fails it at once, rather than allocating one.

• The existing tests pass unchanged, and so do all 70 examples in the README.

Review

Reviewed before submission, over nine rounds, by an automated reviewer. The last round changed only the README, and its checks ran on it; the rest ran on the commit before it, whose code and tests are the same:

• flake8, mypy (which now reads permuta/bisc/bisc.py and permuta/bisc/__init__.py), pylint and black, and Permuta's tests on CPython 3.14 (695 passed) and PyPy 3.11 (693 passed, and the two CPython-only tests skipped). Tilings' tests, which do not import permuta.bisc, passed against this branch on both in an earlier round (3719 each).
• A comparison with develop on CPython 3.14 and PyPy 3.11 for every input kind, permutations of lengths 21 to 30 among them: what bisc and auto_bisc return and print is the same, except for the malformed inputs listed above.
• A brute-force check of both forms and their views, at four thresholds and with the list form forced at short lengths: 900 PropertyBitmaps each, built every way in, lengths 0 to 8, on both Pythons.
• Mutation testing of the new and changed functions, and faults made by hand in each new branch: every fault that changes a result fails a test.
• Timing against develop, interleaved, on both Pythons, with PyPy's runs paired by hash seed. On CPython 3.14 a property given to auto_bisc took 6 to 7% longer than on develop, and the inputs that are ranked cost what is stated above. On PyPy 3.11 every input took develop's time or less, but for those that are ranked; auto_bisc(smooth) took 1.36 s against develop's 1.60 s, over 40 paired runs.

---

AI assistance: this change was written with an AI coding agent, and reviewed before submission, over nine rounds, by an automated reviewer (Claude Code, Opus 5.5, with Sonnet and Haiku subagents), which ran the checks above. Henning Ulfarsson checked it and opened it.