Skip to content

[Bug]: == and != do not compare numeric strings as numbers, while <, >, <= and >= do #5072

Open

Description

@hnipps

Bug Description

The ordering operators compare two numeric-looking operands as numbers (_safe_compare, src/specify_cli/workflows/expressions.py:1002). == and != compare the raw values with no conversion (expressions.py:913-914). When a value is a numeric string, such as captured shell output, the results disagree. For example, "10" >= 10 and "10" <= 10 are both true, but "10" == 10 is false and "10" != 10 is true.

Steps to Reproduce

  1. From a fresh clone of spec-kit, run uv sync.
  2. Save the script below as repro.py in the repository root.
  3. Run uv run python -I repro.py.
from specify_cli.workflows.base import StepContext
from specify_cli.workflows.expressions import evaluate_condition

ctx = StepContext(steps={"count": {"output": {"stdout": "10"}}})
for op in (">=", "<=", "==", "!="):
    cond = "{{ steps.count.output.stdout " + op + " 10 }}"
    print(f"{cond} -> {evaluate_condition(cond, ctx)}")

Expected Behavior

The _safe_compare docstring (src/specify_cli/workflows/expressions.py:1003-1012) says numeric strings compare as numbers, and tests/test_workflows.py:367-369 asserts "10" > '9' numerically. The equality operators should agree with the ordering operators. If a value is both >= 10 and <= 10, then == 10 should be true and != 10 should be false.

Actual Behavior

{{ steps.count.output.stdout >= 10 }} -> True
{{ steps.count.output.stdout <= 10 }} -> True
{{ steps.count.output.stdout == 10 }} -> False
{{ steps.count.output.stdout != 10 }} -> True

Specify CLI Version

1.2.4.dev0 (main at 27d14e2)

AI Agent

Not applicable

Operating System

Linux 6.18 (Ubuntu container)

Python Version

Python 3.12.3

Additional Context

A condition such as {{ steps.count.output.stdout == 0 }} therefore behaves differently from {{ steps.count.output.stdout <= 0 and steps.count.output.stdout >= 0 }}. Applying the same both-sides-numeric conversion to == and != would make the operators consistent.

Related: #5060 proposes replacing the hand-written expression parser with a grammar so that invalid input raises. It does not mention this case.

Also reproduced on macOS (Darwin 25.6, arm64) with Python 3.14.4 and uv run python -I repro.py against 27d14e2; the output matches the above.

AI Disclosure

Found and reproduced with Claude Code (model: Claude Opus 4.8), run autonomously under human supervision. The human reporter re-ran every reproduction above before filing.

Activity

  1. mnriem commented on Oct 23, 2026

    @mnriem
    Collaborator

    The intended comparison contract is Python comparison semantics: numeric strings are not implicitly converted to numbers. Aligning the current ordering operators with that contract would be a backward-incompatible change, not a clarification or behavior-preserving bug fix.

    For the reported case, "10" == 10 returning False and "10" != 10 returning True already match Python and should remain unchanged. The discrepancy is the ordering operators implicitly coercing numeric strings. Removing that coercion would change existing conditions:

    • "10" >= 10: currently True; Python raises TypeError.
    • "10" > "9": currently True through numeric coercion; Python compares the strings lexicographically and returns False.

    Workflow authors relying on numeric comparison of captured shell output would need explicit numeric conversion. Existing ordering behavior is documented and tested, so this issue should track the compatibility and migration decision before implementation, along with updated documentation and positive and negative regression coverage. The policy direction does not authorize an immediate breaking change or establish a release commitment.

    AI disclosure: Posted and revised on behalf of @mnriem by GitHub Copilot (GPT-6.1 Sol), in interactive agent mode with human direction and approval; AI-assisted investigation and comment drafting only. No code changes.

  2. added
    triage-can-waitVerdict: valid and in-scope but deprioritized; held behind the evidence gate
    on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage-can-waitVerdict: valid and in-scope but deprioritized; held behind the evidence gate

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions