diff --git a/CHANGELOG.md b/CHANGELOG.md index df4bf4c..4ec6681 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ - A draft is filled in by `in2lambda draft question add`, `in2lambda draft part add QUESTION` and `in2lambda draft question solution QUESTION`. Each takes `--text` to copy the wording out of the frozen source, as a block id such as `b3` or as lines such as `s10:14`, or `--literal TEXT` where the source does not say it in a form the field can take, which records the field as edited and written by layer 4 rather than 3. Question and part numbers are worked out from the fields already written rather than given, so a replay arrives at the same ids. `in2lambda draft split block BLOCK AT` cuts a block the parser made one of two things into `b3a` and `b3b`, so that each half can be quoted on its own. A command writing a field that is already written, or quoting lines another field was taken from, is refused: the first naming the field, the second naming both. - `in2lambda draft field replace FIELD OLD NEW` changes the wording inside a field that is already written, for the faults only an edit can fix - a brace the OCR dropped out of some maths, which no range of the source says correctly. OLD has to occur in the field exactly once, or the command is refused saying how many times it occurs; `--regex` reads it as a regular expression and NEW as what to replace it with. The field is left quoting the lines it was taken from, at the layer that wrote it, but recorded as edited and by whoever replaced the wording, so the change can be shown against the source. - `in2lambda spec run SPEC` runs a YAML file of selectors over the frozen source: it says which blocks are questions, parts and solutions, which to ignore, what to strip off the front of each one, and which of the four filters lays the solutions out. It fills in the draft's fields with the markdown of the lines each was taken from, records the spec's name and hash in the log so a replay runs the same file, and reports every block it made nothing of. Running an edited spec over a draft it has already filled in is refused, as freezing a document that has changed is: `in2lambda source add --start-over` begins the draft again. Reading a spec needs pyyaml, which the `convert` extra now installs alongside panflute. See [the spec page](https://lambda-feedback.github.io/in2lambda/spec.html) for the selectors and layouts. +- A spec's `question`, `part`, `solution` and `ignore` now each take a list of selectors as well as one selector, written as a YAML list under the key, and a block has that role where any one of them matches it. A run that would write a field twice - a document of nothing but solutions has more solutions than the layout has questions to answer - now leaves the second block in no field and reports it, naming the field and the block that holds it, instead of refusing the whole run. In the Python API, `in2lambda.spec.Spec` holds a list of selectors per role rather than one or None, and `in2lambda.spec.fields` returns a third list, of `in2lambda.spec.Doubled`, beside its fields and its ignored blocks. - A field quoted out of a list item is now dedented as commonmark reads the item: the marker comes off the first line and as much of the same width off every line under it. So a question written `1. ` no longer carries its number, a continuation line no longer arrives indented far enough to be rendered as a code block, and a spec's `strip` is left with what pandoc does not read as a marker. Values written by `in2lambda spec run`, `in2lambda draft question add`, `in2lambda draft part add` and `in2lambda draft question solution` change accordingly; the ranges behind them still name the same source lines. - `in2lambda validate` checks a draft over as a whole and writes what it finds into it as a `report`: source blocks in no field and not marked ignore, two fields taken from the same lines, gaps in the numbering of the questions or their parts, and fields holding nothing, each at level `error`; a part, or a question written without parts, that nothing in the draft answers is reported at level `warning` instead. Each finding names the level, the field and the lines it is about, so it can be acted on without reading the draft. Finding something is not a failure and the command still exits 0; the report is replaced by the next run of the checks and dropped by the next command that changes the draft, since it describes the draft as it stood. It also checks over the set the draft describes, as a converted document is checked at export - maths delimiters, what KaTeX will not render, images the export would not carry, and the compile Lambda Feedback's PDF generator does where pandoc and xelatex are installed, with a warning saying what to install where they are not - and reports each of those against the draft field the text is written in, at level `error`, so that `in2lambda build` refuses them as it refuses anything else at that level. - `in2lambda build` writes a draft out as a Lambda Feedback set: one question per `qN.text` field, holding the parts written for it and the worked solutions, with the images those fields refer to under `media/`, as `in2lambda convert` writes a set - a field naming an image that is not beside the draft is refused saying which file is missing, since the checks read the draft and not the folder it is in, and a question's own solution written beside a solution for every part it has becomes a part of its own holding just that solution, as `convert` pairs them up. It is refused unless `in2lambda validate` has been run since the draft last changed - every command that changes one drops its report - and found nothing at level `error`, and the refusal prints those findings so they can be acted on without opening the draft. A finding at level `warning` - a part or question nothing in the draft answers - does not stop it: half the sheets there are keep their solutions in another file or have none at all, so the warning is printed and the set written all the same, rather than a solution having to be invented to quiet it. `in2lambda render` writes each question as a PDF instead, compiled as Lambda Feedback's own PDF generator compiles it, which needs pandoc and xelatex; it is gated on nothing, since looking at a draft is how what the checks found gets fixed. Both take `-o/--out`, as `convert` does. diff --git a/docs/source/spec.md b/docs/source/spec.md index a403ef6..478f3f6 100644 --- a/docs/source/spec.md +++ b/docs/source/spec.md @@ -52,6 +52,16 @@ A block is whatever the first of `ignore`, `question`, `part`, `solution` to mat That order is fixed, whatever order the keys are written in, so a spec whose selectors overlap has to tell them apart by what they match rather than by where they are in the file. +`question`, `part`, `solution` and `ignore` each take one selector, or a list of them written +under the key. A block has that role where any one of the selectors in the list matches it, so +one spec selects the questions of a document that writes them two different ways: + +```yaml +ignore: + - Header level=1 + - Para text~'^Marks' +``` + ## Selectors A selector is a block type, then any number of constraints: @@ -134,6 +144,10 @@ the solutions are, and what each of them answers. | `PartPartSolSol` | The parts come together and their solutions come after, in the same order. | | `PartsSepSol` | Every solution is at the end: the first answers the first part of the first question, and so on. | +A sheet holding more solutions than the layout has questions and parts to answer sends two of +them to the one field. The second is left in no field and reported, naming the field and the +block that holds it. The section below says the same of a document of solutions. + ## A separate solutions document Many sheets come as two files: the questions, and the solutions written separately from them. @@ -166,10 +180,16 @@ changes is what the selectors mean in a document of solutions, which is what `in the sheet itself is laid out as. A solution past the last slot is reported as being in no field, like any other block the spec -made nothing of; one landing on a question the solutions before it have answered is refused, -saying that the field - `q2.solution`, say - is already written and that no command here writes -a field twice. `in2lambda draft field replace` changes the wording of one, and `in2lambda source -add --start-over` begins the draft again. +made nothing of. So is one landing on a question the solutions before it have answered, with a +second line naming the field it would have gone in and the block that holds it: + +``` +b7 (lines 14-15) is in no field and not marked ignore. +b7 (lines 14-15) would be q2.solution, which b5 (lines 10-11) already holds. +``` + +The run writes every other field, so a spec that sends two solutions to one field still fills +the draft in and names the block to look at. ## What it writes diff --git a/in2lambda/draft/__init__.py b/in2lambda/draft/__init__.py index 87b1f1e..2318288 100644 --- a/in2lambda/draft/__init__.py +++ b/in2lambda/draft/__init__.py @@ -16,7 +16,7 @@ from typing import Any import in2lambda.spec -from in2lambda.draft.report import _order, checks, overlapping, uncovered +from in2lambda.draft.report import _order, _where, checks, overlapping, uncovered from in2lambda.source import ( SourceError, _digest, @@ -889,7 +889,7 @@ def _spec_run( (_elements(markdown, number), markdown) for number, markdown in enumerate(sources, start=1) ] - fields, ignored = in2lambda.spec.fields(spec, documents, functions) + fields, ignored, doubled = in2lambda.spec.fields(spec, documents, functions) for found in fields: record( draft, @@ -920,6 +920,14 @@ def _spec_run( # A spec writes a draft's worth of fields, so what it hands back is the other way # round: what it made nothing of, which is what is left for anyone to act on. Said # in the words `in2lambda validate` says it in, since it is the same check. - if left_out := uncovered(draft): - return "\n".join(finding["message"] for finding in left_out) + reported = [finding["message"] for finding in uncovered(draft)] + # A doubled block is one of those, and this names the field it would have gone in + # and the block that holds it, which the coverage report cannot say. + reported += [ + f"{block}{_where(ranges)} would be {key}, which {by_block}" + f"{_where(by_ranges)} already holds." + for block, ranges, key, by_block, by_ranges in doubled + ] + if reported: + return "\n".join(reported) return "Every block is in a field or ignored." diff --git a/in2lambda/spec/__init__.py b/in2lambda/spec/__init__.py index 9eff8c4..7bd1769 100644 --- a/in2lambda/spec/__init__.py +++ b/in2lambda/spec/__init__.py @@ -10,6 +10,13 @@ ignore: Header level=1 layout: PartsSepSol +A role is one selector, or a list of them written under it. A block has that role where +any of the role's selectors matches it:: + + ignore: + - Header level=1 + - Para text~'^Marks' + Where a selector cannot say it, a spec names a Python file beside it and calls functions from it: ``predicates: predicates.py`` and then ``question: Para bold_lead()``, where ``bold_lead`` takes the panflute element and says whether the block is one. The file is @@ -49,7 +56,7 @@ """What a constraint can be about: a heading's level, a block's text, its first word.""" _ROLES = ("ignore", "question", "part", "solution") -"""The selectors a spec holds, in the order a block is tried against them. +"""The roles a spec holds selectors for, in the order a block is tried against them. A block is whatever the first of them to match it says it is. The order is this one whatever order a spec writes its keys in: ignore before the rest so that a page nobody @@ -144,13 +151,17 @@ def matches( @dataclass class Spec: - """What a spec file says, once it has been read.""" + """What a spec file says, once it has been read. + + Each role holds one selector or several, and a block has that role where any of them + matches it. A role a spec leaves out holds none. + """ - question: Selector + question: list[Selector] layout: str - part: Optional[Selector] = None - solution: Optional[Selector] = None - ignore: Optional[Selector] = None + part: list[Selector] = field(default_factory=list) + solution: list[Selector] = field(default_factory=list) + ignore: list[Selector] = field(default_factory=list) strip: "list[re.Pattern[str]]" = field(default_factory=list) predicates: Optional[str] = None """The Python file its selectors call functions from, where any of them do.""" @@ -166,6 +177,17 @@ class Field(NamedTuple): """Which of the draft's frozen sources the ranges are lines of, numbered from 1.""" +class Doubled(NamedTuple): + """A block the layout sent to a field another block of the sources had filled in.""" + + block: str + ranges: list[list[int]] + key: str + by_block: str + by_ranges: list[list[int]] + """Which block the field holds, and the lines that block was taken from.""" + + def _attribute(name: str, element: Any, pf: Any) -> Optional[str]: """What a block says for one attribute, or None where it has not got one.""" if name == "level": @@ -274,6 +296,28 @@ def _selector(text: Any, line: int) -> Selector: return _clause(head.strip(), line) +def _selectors(value: Any, line: int, item_lines: list[int]) -> list[Selector]: + """The selectors of one role: the one written after it, or the list written under it. + + Args: + value: What the role says, as YAML built it. + line: Which line the role's key is written on. + item_lines: Which line each item of the value is written on, where the value is + a list, so that a refusal names the item rather than the key. + """ + if not isinstance(value, list): + return [_selector(value, line)] + if not value: + raise _refuse( + line, + "A role is a selector or a list of selectors, which an empty list is not.", + ) + return [ + _selector(item, item_lines[index] if index < len(item_lines) else line) + for index, item in enumerate(value) + ] + + def load(text: "str | bytes") -> Spec: r"""Reads a spec, given that it says what a spec says. @@ -321,6 +365,14 @@ def load(text: "str | bytes") -> Spec: for key, _ in node.value if isinstance(key, yaml.ScalarNode) } + # Where a value is a list - a role written as several selectors, a strip of several + # patterns - a refusal about one item says the line that item is on rather than the + # line the key is on, which in a list of three is two lines away from the fault. + items = { + key.value: [item.start_mark.line + 1 for item in value.value] + for key, value in node.value + if isinstance(key, yaml.ScalarNode) and isinstance(value, yaml.SequenceNode) + } # By str, because a key someone has written need not be one: `1: Header` is YAML. if unknown := sorted(set(given) - set(_KEYS), key=str): @@ -360,26 +412,38 @@ def load(text: "str | bytes") -> Spec: f"predicates names a Python file beside the spec, which {file!r} is not. " "The name has no directory in it: the file is in the spec's own folder.", ) - question = _selector(given["question"], lines["question"]) + question = _selectors( + given["question"], lines["question"], items.get("question", []) + ) rest = { - role: _optional(given, role, lines) for role in _ROLES if role != "question" + role: _optional(given, role, lines, items) + for role in _ROLES + if role != "question" } if file is None: - for role, selector in {"question": question, **rest}.items(): - if selector is not None and (called := _called(selector)): - raise _refuse( - lines[role], - f"{called[0]}() is a function, and the spec does not say which " - "Python file its functions are in. Put the file beside the spec " - "and name it with a predicates: line.", - ) + for role, selectors in {"question": question, **rest}.items(): + for selector in selectors: + if called := _called(selector): + raise _refuse( + lines[role], + f"{called[0]}() is a function, and the spec does not say which " + "Python file its functions are in. Put the file beside the spec " + "and name it with a predicates: line.", + ) + strip_lines = items.get("strip", []) return Spec( question=question, layout=layout, part=rest["part"], solution=rest["solution"], ignore=rest["ignore"], - strip=[_pattern(pattern, lines["strip"]) for pattern in strip], + strip=[ + _pattern( + pattern, + strip_lines[index] if index < len(strip_lines) else lines["strip"], + ) + for index, pattern in enumerate(strip) + ], predicates=file, ) @@ -414,25 +478,29 @@ def predicates(spec: Spec, code: bytes, name: str) -> dict[str, Callable[[Any], exec(compile(code, name, "exec"), module.__dict__) found = {} for role in _ROLES: - if (selector := getattr(spec, role)) is None: - continue - for called in _called(selector): - function = getattr(module, called, None) - if not callable(function): - raise BadSpec( - f"{name} has no function {called} in it, and the spec calls " - f"{called}(). A predicate is a function of one argument, the " - "panflute element, that says whether the block is one of those." - ) - found[called] = function + for selector in getattr(spec, role): + for called in _called(selector): + function = getattr(module, called, None) + if not callable(function): + raise BadSpec( + f"{name} has no function {called} in it, and the spec calls " + f"{called}(). A predicate is a function of one argument, the " + "panflute element, that says whether the block is one of those." + ) + found[called] = function return found def _optional( - given: dict[str, Any], name: str, lines: dict[str, int] -) -> Optional[Selector]: - """One of the selectors a spec need not have.""" - return _selector(given[name], lines[name]) if name in given else None + given: dict[str, Any], + name: str, + lines: dict[str, int], + items: dict[str, list[int]], +) -> list[Selector]: + """The selectors of a role a spec need not have, and none where it has not got it.""" + if name not in given: + return [] + return _selectors(given[name], lines[name], items.get(name, [])) def _stems(roles: list[Optional[str]]) -> list[Optional[str]]: @@ -530,7 +598,7 @@ def _answers(roles: list[Optional[str]], slots: list[list[str]]) -> list[Optiona selector matched or its ``solution`` one, is a solution, and they take the slots in order: each question's parts, or the question itself where it has none. A solution past the last slot is in no field, and one landing on a question the solutions - before it have answered is refused by `in2lambda.draft.record`, naming both. + before it have answered is reported by :func:`fields` as doubled. """ ordered = [ (number, slot) for number, question in enumerate(slots) for slot in question @@ -562,9 +630,10 @@ def _roles( ) -> list[Optional[str]]: """What the spec says each block of one source is, or None where it says nothing. - A selector matches within the source it is run over - ``after Header text=Solutions`` - is about where a block sits in its own document - so each source is decided about on - its own, whatever the sources before it hold. + A block has a role where any one of that role's selectors matches it. A selector + matches within the source it is run over - ``after Header text=Solutions`` is about + where a block sits in its own document - so each source is decided about on its own, + whatever the sources before it hold. """ found = [element for _, element in elements] return [ @@ -572,8 +641,10 @@ def _roles( ( role for role in _ROLES - if (selector := getattr(spec, role)) is not None - and selector.matches(found, index, pf, functions) + if any( + selector.matches(found, index, pf, functions) + for selector in getattr(spec, role) + ) ), None, ) @@ -585,7 +656,7 @@ def fields( spec: Spec, documents: list[tuple[list[tuple[Block, Any]], str]], functions: Optional[dict[str, Callable[[Any], Any]]] = None, -) -> tuple[list[Field], list[str]]: +) -> tuple[list[Field], list[str], list[Doubled]]: """What a spec makes of a draft's sources: its fields, and the blocks to ignore. Args: @@ -599,26 +670,37 @@ def fields( Returns: One :class:`Field` per question, part and solution the spec found, each saying - which source it came from, and the ids of the blocks to mark as ignored: what - the ``ignore`` selector matched, and the markers of a separate document of - solutions, which say which question the solutions under them answer and nothing - else. A block in neither is in neither, which is what a coverage report is about. + which source it came from; the ids of the blocks to mark as ignored, which are + what the ``ignore`` selector matched and the markers of a separate document of + solutions, saying which question the solutions under them answer and nothing + else; and one :class:`Doubled` per block the layout sent to a field an earlier + block had filled in. A doubled block is in no field, as a block in neither of + the first two lists is, which is what a coverage report is about. """ import panflute as pf roles = [_roles(spec, elements, pf, functions) for elements, _ in documents] written = [] + doubled = [] + # Which block each field was taken from. A layout can send two blocks to the one + # field - a document of nothing but solutions has more solutions than there are + # questions to answer - and the second is left in no field, so that the run reports + # it rather than being refused by `in2lambda.draft.record` and writing nothing. + holders: dict[str, tuple[str, list[list[int]]]] = {} for number, ((elements, markdown), keys) in enumerate( zip(documents, _keys(spec.layout, roles)), start=1 ): lines = markdown.splitlines() - written += [ - Field( - key, _stripped(spec, lines, block), [[block.start, block.end]], number - ) - for (block, _), key in zip(elements, keys) - if key is not None - ] + for (block, _), key in zip(elements, keys): + if key is None: + continue + ranges = [[block.start, block.end]] + if key in holders: + by_block, by_ranges = holders[key] + doubled.append(Doubled(block.id, ranges, key, by_block, by_ranges)) + continue + holders[key] = (block.id, ranges) + written.append(Field(key, _stripped(spec, lines, block), ranges, number)) ignored = [ block.id for number, ((elements, _), found) in enumerate(zip(documents, roles), start=1) @@ -627,7 +709,7 @@ def fields( # solutions under it answer, and that question's text came from the sheet. if role == "ignore" or (number > 1 and role == "question") ] - return written, ignored + return written, ignored, doubled def _stripped(spec: Spec, lines: list[str], block: Block) -> str: diff --git a/tests/fixtures/specs/README.md b/tests/fixtures/specs/README.md index 3c842b3..59cbcf3 100644 --- a/tests/fixtures/specs/README.md +++ b/tests/fixtures/specs/README.md @@ -5,7 +5,8 @@ run over it, the `expected.json` the spec should leave in the draft's `fields`, `uncovered.txt` of the blocks the command should report as being in no field. The test freezes the source, runs the spec, compares both, and then replays the draft from its log and checks the file is unchanged byte for byte - so a folder covers what a spec makes of a document and that it -can be rebuilt from what was recorded. +can be rebuilt from what was recorded. A folder whose layout sends two blocks to the same field +has a `doubled.txt` as well, naming the blocks reported as doubled; the rest have no such file. To cover another kind of document, add a folder. There is one per layout, since a layout is only a rule about which solution answers which question or part: `parts_one_sol` has one @@ -23,6 +24,12 @@ calls functions from it, because what tells its questions from the paragraph abo bold each of them starts with, which is markup rather than text. Its log entry names that file and hashes it as it does the spec, which is what makes a changed predicate refuse to replay. +`ignore_as_a_list` writes its `ignore` as two selectors under the key rather than one beside it - +the title of the sheet, and the paragraph about marks - and pins that a block either of them +matches is ignored. `solutions_only` is a document of nothing but solutions, which every layout +runs out of questions to answer: its second paragraph goes to the `q1.solution` its first +paragraph already holds, so that block is left in no field and reported as doubled. + A selector matches what pandoc parses, and a field holds the markdown of the lines it was taken from, dedented where those lines are a list item's: `numbered_questions` is the sheet whose questions are written `1. ` and run on over several lines, and its spec strips nothing but the diff --git a/tests/fixtures/specs/ignore_as_a_list/expected.json b/tests/fixtures/specs/ignore_as_a_list/expected.json new file mode 100644 index 0000000..5ab240c --- /dev/null +++ b/tests/fixtures/specs/ignore_as_a_list/expected.json @@ -0,0 +1,50 @@ +{ + "b1.ignore": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 1, + 1 + ] + ], + "value": true + }, + "b2.ignore": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 3, + 3 + ] + ], + "value": true + }, + "q1.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 7, + 7 + ] + ], + "value": "$F = \\tau A$." + }, + "q1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 5, + 5 + ] + ], + "value": "Find the drag force on the flat plate." + } +} diff --git a/tests/fixtures/specs/ignore_as_a_list/source.md b/tests/fixtures/specs/ignore_as_a_list/source.md new file mode 100644 index 0000000..8b2dd57 --- /dev/null +++ b/tests/fixtures/specs/ignore_as_a_list/source.md @@ -0,0 +1,7 @@ +# Problem sheet 4 + +Marks are shown beside each question. + +Q1. Find the drag force on the flat plate. + +Solution: $F = \tau A$. diff --git a/tests/fixtures/specs/ignore_as_a_list/spec.yaml b/tests/fixtures/specs/ignore_as_a_list/spec.yaml new file mode 100644 index 0000000..524918b --- /dev/null +++ b/tests/fixtures/specs/ignore_as_a_list/spec.yaml @@ -0,0 +1,7 @@ +question: Para text~'^Q\d+\.' +solution: Para text~'^Solution:' +strip: ['^Q\d+\. ', '^Solution: '] +ignore: + - Header level=1 + - Para text~'^Marks' +layout: PartsOneSol diff --git a/tests/fixtures/specs/ignore_as_a_list/uncovered.txt b/tests/fixtures/specs/ignore_as_a_list/uncovered.txt new file mode 100644 index 0000000..e69de29 diff --git a/tests/fixtures/specs/solutions_only/doubled.txt b/tests/fixtures/specs/solutions_only/doubled.txt new file mode 100644 index 0000000..6d0875c --- /dev/null +++ b/tests/fixtures/specs/solutions_only/doubled.txt @@ -0,0 +1 @@ +b3 diff --git a/tests/fixtures/specs/solutions_only/expected.json b/tests/fixtures/specs/solutions_only/expected.json new file mode 100644 index 0000000..c01a4f2 --- /dev/null +++ b/tests/fixtures/specs/solutions_only/expected.json @@ -0,0 +1,26 @@ +{ + "q1.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 3, + 3 + ] + ], + "value": "The load the large piston carries is $F = pA$." + }, + "q1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 1, + 1 + ] + ], + "value": "# Solutions" + } +} diff --git a/tests/fixtures/specs/solutions_only/source.md b/tests/fixtures/specs/solutions_only/source.md new file mode 100644 index 0000000..65cfd5d --- /dev/null +++ b/tests/fixtures/specs/solutions_only/source.md @@ -0,0 +1,5 @@ +# Solutions + +The load the large piston carries is $F = pA$. + +The speed of sound in air at 300 K is $c = \sqrt{\gamma R T}$. diff --git a/tests/fixtures/specs/solutions_only/spec.yaml b/tests/fixtures/specs/solutions_only/spec.yaml new file mode 100644 index 0000000..68c73d8 --- /dev/null +++ b/tests/fixtures/specs/solutions_only/spec.yaml @@ -0,0 +1,3 @@ +question: Header level=1 +solution: Para +layout: PartsOneSol diff --git a/tests/fixtures/specs/solutions_only/uncovered.txt b/tests/fixtures/specs/solutions_only/uncovered.txt new file mode 100644 index 0000000..6d0875c --- /dev/null +++ b/tests/fixtures/specs/solutions_only/uncovered.txt @@ -0,0 +1 @@ +b3 diff --git a/tests/test_spec.py b/tests/test_spec.py index 31c388a..add10a3 100644 --- a/tests/test_spec.py +++ b/tests/test_spec.py @@ -67,6 +67,13 @@ def test_a_spec_fills_in_the_fields_beside_it_and_replays( ] assert reported == (folder / "uncovered.txt").read_text().split() + # The blocks the layout sent to a field an earlier block had filled in, which a + # folder only has a file of where its document doubles one up. + twice = folder / "doubled.txt" + assert [ + line.split()[0] for line in result.output.splitlines() if "already" in line + ] == (twice.read_text().split() if twice.is_file() else []) + # Every file the run needed is named and hashed in the log, so a replay runs the # ones that ran - and a spec calling no functions names no file of them. named = {"spec": "spec.yaml"} @@ -290,6 +297,12 @@ def test_a_replay_is_refused_once_the_spec_has_gone( "line 2", "../shared.py", ), + ("question: []\nlayout: PartsOneSol\n", "line 1", "an empty list"), + ( + "question:\n - Header\n - 5\nlayout: PartsOneSol\n", + "line 3", + "which 5 is not", + ), ], ids=[ "not yaml", @@ -302,6 +315,8 @@ def test_a_replay_is_refused_once_the_spec_has_gone( "a function with no file to find it in", "predicates that is not a file name", "predicates somewhere other than beside the spec", + "a role with no selectors under it", + "a list holding something that is not a selector", ], ) def test_a_spec_that_cannot_be_read_says_which_line_to_look_at(