Report a missing solution as a warning, not a refusal

CHANGELOG.md

@@ -11,7 +11,7 @@
 - `in2lambda draft field replace FIELD OLD NEW` changes the wording inside a field that is already written, for the faults only an edit can fix - a brace the OCR dropped out of some maths, which no range of the source says correctly. OLD has to occur in the field exactly once, or the command is refused saying how many times it occurs; `--regex` reads it as a regular expression and NEW as what to replace it with. The field is left quoting the lines it was taken from, at the layer that wrote it, but recorded as edited and by whoever replaced the wording, so the change can be shown against the source.
 - `in2lambda spec run SPEC` runs a YAML file of selectors over the frozen source: it says which blocks are questions, parts and solutions, which to ignore, what to strip off the front of each one, and which of the four filters lays the solutions out. It fills in the draft's fields with the markdown of the lines each was taken from, records the spec's name and hash in the log so a replay runs the same file, and reports every block it made nothing of. Running an edited spec over a draft it has already filled in is refused, as freezing a document that has changed is: `in2lambda source add --start-over` begins the draft again. Reading a spec needs pyyaml, which the `convert` extra now installs alongside panflute. See [the spec page](https://lambda-feedback.github.io/in2lambda/spec.html) for the selectors and layouts.
 - A field quoted out of a list item is now dedented as commonmark reads the item: the marker comes off the first line and as much of the same width off every line under it. So a question written `1.  ` no longer carries its number, a continuation line no longer arrives indented far enough to be rendered as a code block, and a spec's `strip` is left with what pandoc does not read as a marker. Values written by `in2lambda spec run`, `in2lambda draft question add`, `in2lambda draft part add` and `in2lambda draft question solution` change accordingly; the ranges behind them still name the same source lines.
-- `in2lambda validate` checks a draft over as a whole and writes what it finds into it as a `report`: source blocks in no field and not marked ignore, two fields taken from the same lines, gaps in the numbering of the questions or their parts, parts nothing answers, and fields holding nothing. Each finding names the field and the lines it is about, so it can be acted on without reading the draft. Finding something is not a failure and the command still exits 0; the report is replaced by the next run of the checks and dropped by the next command that changes the draft, since it describes the draft as it stood. It also checks over the set the draft describes, as a converted document is checked at export - maths delimiters, what KaTeX will not render, images the export would not carry, and the compile Lambda Feedback's PDF generator does where pandoc and xelatex are installed, with a warning saying what to install where they are not - and reports each of those against the draft field the text is written in, so that `in2lambda build` refuses them as it refuses anything else in the report.
-- `in2lambda build` writes the draft in this directory out as a Lambda Feedback set: one question per `qN.text` field, holding the parts written for it and the worked solutions, with the images those fields refer to under `media/`, as `in2lambda convert` writes a set - a field naming an image that is not beside the draft is refused saying which file is missing, since the checks read the draft and not the folder it is in, and a question's own solution written beside a solution for every part it has becomes a part of its own holding just that solution, as `convert` pairs them up. It is refused unless `in2lambda validate` has been run since the draft last changed - every command that changes one drops its report - and found nothing, and the refusal prints what the report says so it can be acted on without opening the draft. `in2lambda render` writes each question as a PDF instead, compiled as Lambda Feedback's own PDF generator compiles it, which needs pandoc and xelatex; it is gated on nothing, since looking at a draft is how what the checks found gets fixed. Both take `-o/--out`, as `convert` does.
+- `in2lambda validate` checks a draft over as a whole and writes what it finds into it as a `report`: source blocks in no field and not marked ignore, two fields taken from the same lines, gaps in the numbering of the questions or their parts, and fields holding nothing, each at level `error`; a part, or a question written without parts, that nothing in the draft answers is reported at level `warning` instead. Each finding names the level, the field and the lines it is about, so it can be acted on without reading the draft. Finding something is not a failure and the command still exits 0; the report is replaced by the next run of the checks and dropped by the next command that changes the draft, since it describes the draft as it stood. It also checks over the set the draft describes, as a converted document is checked at export - maths delimiters, what KaTeX will not render, images the export would not carry, and the compile Lambda Feedback's PDF generator does where pandoc and xelatex are installed, with a warning saying what to install where they are not - and reports each of those against the draft field the text is written in, at level `error`, so that `in2lambda build` refuses them as it refuses anything else at that level.
+- `in2lambda build` writes the draft in this directory out as a Lambda Feedback set: one question per `qN.text` field, holding the parts written for it and the worked solutions, with the images those fields refer to under `media/`, as `in2lambda convert` writes a set - a field naming an image that is not beside the draft is refused saying which file is missing, since the checks read the draft and not the folder it is in, and a question's own solution written beside a solution for every part it has becomes a part of its own holding just that solution, as `convert` pairs them up. It is refused unless `in2lambda validate` has been run since the draft last changed - every command that changes one drops its report - and found nothing at level `error`, and the refusal prints those findings so they can be acted on without opening the draft. A finding at level `warning` - a part or question nothing in the draft answers - does not stop it: half the sheets there are keep their solutions in another file or have none at all, so the warning is printed and the set written all the same, rather than a solution having to be invented to quiet it. `in2lambda render` writes each question as a PDF instead, compiled as Lambda Feedback's own PDF generator compiles it, which needs pandoc and xelatex; it is gated on nothing, since looking at a draft is how what the checks found gets fixed. Both take `-o/--out`, as `convert` does.
 - Importing `in2lambda.katex_convert` no longer writes a file called `log` into the working directory. What it has to say about a converted expression goes to the `in2lambda.katex_convert` logger, which is silent unless the application configures logging.
 - The Python API is unchanged: `in2lambda.main.runner` and everything under `in2lambda.api` take the same arguments and return the same objects.

in2lambda/draft/export.py

@@ -5,8 +5,10 @@
 is the one place that reads the one as the other, so both what is written out and what
 is rendered for review come from the same reading of the draft.
 
-:func:`build` refuses a draft the checks have not looked at, or have something to say
-about. There is no timestamp in that: every command that changes a draft takes its
+:func:`build` refuses a draft the checks have not looked at, or have found an error in;
+what they found at level warning - a question or part nothing answers - it says and
+exports anyway, since a sheet whose solutions are in another file or nowhere is still a
+sheet. There is no timestamp in that: every command that changes a draft takes its
 report with it, so a draft holding one has been checked since it last changed, and
 `in2lambda.source.frozen` refuses one whose source has moved on underneath it.
 :func:`render` is gated on nothing, since looking at a draft is how what the checks
@@ -191,25 +193,38 @@ def build(directory: str = ".", output_dir: str = "out") -> Path:
 
     Raises:
         NotValidated: the draft has not been checked since it last changed, or the
-            checks found something. Either way what would be uploaded is not what
+            checks found an error in it. Either way what would be uploaded is not what
             anybody has looked at.
         MissingImage: a field refers to an image file that is not beside the draft.
         SourceError: the draft is missing, is not one of ours, or was written from
             markdown that has changed since.
+
+    Warns:
+        UserWarning: once per finding the checks made at level warning, which is a
+            question or part the draft has no solution for. The set is written with it.
     """
+    # Here rather than at the top of the module: `report` checks the set this writes, so
+    # it imports this, and only what reads a report - this one function - needs it back.
+    from in2lambda.draft.report import errors
+
     draft, _ = frozen(directory)
     if "report" not in draft:
         raise NotValidated(
             f"{DRAFT} has not been validated since it last changed, so what it would "
             "export is what nothing has checked. Run in2lambda validate."
         )
-    if draft["report"]:
+    if refusing := errors(draft["report"]):
         raise NotValidated(
-            "\n".join(finding["message"] for finding in draft["report"])
+            "\n".join(finding["message"] for finding in refusing)
             + f"\n{DRAFT} is not exported while its report says this. Fix what it "
             "names, or mark the blocks it is about as ignored, and run in2lambda "
             "validate again."
         )
+    for finding in draft["report"]:
+        # Said rather than refused: a sheet whose solutions are elsewhere or absent is
+        # one to export as it stands, and writing one in to quiet this would put wording
+        # into the set that no source of it says.
+        warnings.warn(finding["message"], stacklevel=2)
     exported = as_set(draft, directory)
     # The export carries every image a field refers to into media/, which is the only
     # place Lambda Feedback looks for one, so a file that is not there is not something

in2lambda/draft/report.py

@@ -13,6 +13,15 @@
 what the text of a question says is `in2lambda.validation`'s: the set the draft describes
 is exported and checked over as well, so that maths Lambda Feedback will not render is
 reported against the field it is written in rather than found after uploading.
+
+Each finding carries the level it is found at, which is what `in2lambda.draft.export`
+goes by. An error is the draft contradicting its own source or its own export - lines
+nothing accounts for, two fields quoting the same ones, a numbering with a hole in it, a
+quotation of nothing, maths that will not render - and there is no sheet those are right
+about. A warning is something that may well be right: half the sheets there are write
+their solutions in another file, or have none, so a question nothing answers is said to
+whoever is building the set rather than stopping them - inventing a solution to quiet it
+is the one thing nobody wanted.
 """
 
 import re
@@ -25,20 +34,30 @@
 from in2lambda.validation import pdf
 
 Finding = dict[str, Any]
-"""One thing a check found: ``{"check", "field", "ranges", "message"}``.
+"""One thing a check found: ``{"check", "level", "field", "ranges", "message"}``.
 
 ``check`` is which check found it - ``problem`` where it was `in2lambda.validation`,
-over the set the draft describes - ``field`` the block id or field key it is about,
-``ranges`` the lines in question as ``[[start, end], ...]``, and ``message`` a sentence
-naming all of that, so that a line of the report can be acted on by itself.
+over the set the draft describes - ``level`` :data:`ERROR` or :data:`WARNING`, ``field``
+the block id or field key it is about, ``ranges`` the lines in question as
+``[[start, end], ...]``, and ``message`` a sentence naming all of that, so that a line of
+the report can be acted on by itself.
 """
 
+ERROR = "error"
+"""A finding the draft cannot be exported over: it says something its source does not."""
+
+WARNING = "warning"
+"""A finding the export says and goes on past: it may be what the sheet really is."""
+
 _NUMBERED = re.compile(r"((?:q\d+\.p)|q)(\d+)\.text")
 """A question's or a part's text, split into what numbers it and the number."""
 
 _PART = re.compile(r"(q\d+)\.p\d+\.text")
 """A part's text, and the question it belongs to."""
 
+_QUESTION = re.compile(r"(q\d+)\.text")
+"""A question's text, and the question it is."""
+
 _UNPLACED = float("inf")
 """Where a finding about no particular line sorts: after every finding about one."""
 
@@ -120,6 +139,7 @@ def uncovered(draft: dict[str, Any]) -> list[Finding]:
             found.append(
                 {
                     "check": "uncovered",
+                    "level": ERROR,
                     "field": block["id"],
                     "ranges": free,
                     "message": f"{block['id']}{_where(free)} is in no field and not "
@@ -140,6 +160,7 @@ def _overlaps(draft: dict[str, Any]) -> list[Finding]:
     return [
         {
             "check": "overlap",
+            "level": ERROR,
             "field": key,
             "ranges": fields[key]["ranges"],
             "message": f"{key}{_where(fields[key]['ranges'])} and {other}"
@@ -164,6 +185,7 @@ def _gaps(draft: dict[str, Any]) -> list[Finding]:
     return [
         {
             "check": "gap",
+            "level": ERROR,
             "field": f"{prefix}{missing}.text",
             "ranges": [],
             "message": f"There is no {prefix}{missing}.text, though "
@@ -176,28 +198,47 @@ def _gaps(draft: dict[str, Any]) -> list[Finding]:
 
 
 def _without_solutions(draft: dict[str, Any]) -> list[Finding]:
-    """Parts that nothing in the draft answers.
+    """Parts, and questions written without any, that nothing in the draft answers.
 
     A part is answered by its own solution or by the solution of the question it belongs
-    to, since a sheet often writes one worked solution covering every part at once.
+    to, since a sheet often writes one worked solution covering every part at once. A
+    question with parts is answered through them and is not reported itself; one with
+    none is a question in its own right, and is reported where nothing answers it.
     """
     fields = draft["fields"]
     found = []
     for key in sorted(fields):
-        if (named := _PART.fullmatch(key)) is None:
-            continue
-        part = key.removesuffix(".text")
-        if f"{part}.solution" in fields or f"{named[1]}.solution" in fields:
-            continue
-        found.append(
-            {
-                "check": "no-solution",
-                "field": part,
-                "ranges": fields[key]["ranges"],
-                "message": f"{part}{_where(fields[key]['ranges'])} has no solution: "
-                f"neither {part}.solution nor {named[1]}.solution is written.",
-            }
-        )
+        if named := _PART.fullmatch(key):
+            part = key.removesuffix(".text")
+            if f"{part}.solution" in fields or f"{named[1]}.solution" in fields:
+                continue
+            found.append(
+                {
+                    "check": "no-solution",
+                    "level": WARNING,
+                    "field": part,
+                    "ranges": fields[key]["ranges"],
+                    "message": f"{part}{_where(fields[key]['ranges'])} has no solution: "
+                    f"neither {part}.solution nor {named[1]}.solution is written.",
+                }
+            )
+        elif named := _QUESTION.fullmatch(key):
+            question = named[1]
+            if f"{question}.solution" in fields or any(
+                (belongs := _PART.fullmatch(other)) and belongs[1] == question
+                for other in fields
+            ):
+                continue
+            found.append(
+                {
+                    "check": "no-solution",
+                    "level": WARNING,
+                    "field": question,
+                    "ranges": fields[key]["ranges"],
+                    "message": f"{question}{_where(fields[key]['ranges'])} has no "
+                    f"solution: {question}.solution is not written, and it has no parts.",
+                }
+            )
     return found
 
 
@@ -206,6 +247,7 @@ def _empty(draft: dict[str, Any]) -> list[Finding]:
     return [
         {
             "check": "empty",
+            "level": ERROR,
             "field": key,
             "ranges": field["ranges"],
             "message": f"{key}{_where(field['ranges'])} is empty.",
@@ -224,7 +266,9 @@ def checks(draft: dict[str, Any]) -> list[Finding]:
     Returns:
         One :data:`Finding` per thing found, earliest line first and then by what it is
         about, with the findings about no particular line last. An empty list means the
-        draft covers its source once each, with nothing missing from its numbering.
+        draft covers its source once each, with nothing missing from its numbering; a
+        list holding only warnings is one `in2lambda.draft.export.build` says and
+        exports over.
 
     Examples:
         >>> from in2lambda.draft.report import checks
@@ -253,6 +297,25 @@ def _order(finding: Finding) -> tuple[int | float, str]:
     )
 
 
+def errors(findings: list[Finding]) -> list[Finding]:
+    """The findings of a report that a draft cannot be exported over.
+
+    Args:
+        findings: A report, as :func:`checks` or :func:`validate` writes one.
+
+    Returns:
+        Those at level :data:`ERROR`, in the order they were reported. The rest are
+        warnings, which `in2lambda.draft.export.build` says and exports anyway.
+
+    Examples:
+        >>> from in2lambda.draft.report import errors
+        >>> report = [{"level": "warning"}, {"level": "error", "check": "gap"}]
+        >>> errors(report)
+        [{'level': 'error', 'check': 'gap'}]
+    """
+    return [finding for finding in findings if finding["level"] == ERROR]
+
+
 def problems(draft: dict[str, Any], directory: str = ".") -> list[Finding]:
     """What `in2lambda.validation` finds in the set the draft describes.
 
@@ -270,6 +333,8 @@ def problems(draft: dict[str, Any], directory: str = ".") -> list[Finding]:
         rather than by the question and part of the export, so that a line of it can be
         acted on with `field replace`. A problem about no one field - the set as a
         whole failing to compile - keeps the validator's own naming of where it is.
+        All of them are at level :data:`ERROR`: what Lambda Feedback will not render is
+        not something to upload.
 
     Warns:
         UserWarning: pandoc or xelatex is not installed, so the set was not compiled.
@@ -306,13 +371,15 @@ def problems(draft: dict[str, Any], directory: str = ".") -> list[Finding]:
             rest = problem.location[len(location) :]
             finding = {
                 "check": "problem",
+                "level": ERROR,
                 "field": key,
                 "ranges": ranges,
                 "message": f"{key}{_where(ranges)}{rest}: {problem.message}",
             }
         else:
             finding = {
                 "check": "problem",
+                "level": ERROR,
                 "field": "",
                 "ranges": [],
                 "message": str(problem),