CxODEV-2125: keep the Kafka cancellation consumer running after a bad message (4.4.2)

[milansvitlica-codaxy]

Problem

KafkaConsumerBackgroundService runs its consume loop inside one try/catch. When a single message fails, the catch logs "will retry to consume after 5 seconds", waits, and ExecuteAsync returns. Nothing restarts the consumer. The worker keeps running, but it stops cancelling tasks until the process restarts.

Two kinds of message trigger this:

• The handler throws. A status event with "workflowTask": null makes HandleKafkaEvent throw a NullReferenceException on WorkflowTask.Type.
• The value fails to deserialize. Examples: invalid or empty JSON, an unknown status, or a workflowTask without name or taskReferenceName (both are Required.Always on the generated model). Consume then throws a ConsumeException.

Fix

• Catch errors per message, inside the loop. Log the error and move on to the next message.
• OperationCanceledException from the stopping token still ends the loop, so shutdown is unchanged.
• Drop the 5-second pause. It only ran once, right before the loop exited. Pausing on every bad message would delay every cancellation queued behind it.
• Add InternalsVisibleTo("ConductorSharp.Engine.Tests") to the notifier assembly, as ConductorSharp.Engine already does.
• Bump all packages 4.4.1 → 4.4.2.

Tests

New KafkaCancellationNotifierTests run the real consumer and notifier against librdkafka's in-process mock cluster (test.mock.num.brokers). No broker or Docker is needed.

Test	Before the fix	After the fix
CancelsTaskAfterUnprocessableMessage: "workflowTask": null, then a CANCELED event	❌ task not cancelled (20 s timeout)	✅ ~0.5 s
CancelsTaskAfterUnprocessableMessage: not json, then a CANCELED event	❌ task not cancelled (20 s timeout)	✅ ~0.5 s
StopAsyncEndsConsumption	✅	✅

StopAsyncEndsConsumption guards shutdown. It fails if the per-message catch also swallows OperationCanceledException; I checked this by applying that change.

Full suite, dotnet test ConductorSharp.sln: 76 passed, 0 failed. Run locally on the .NET 8 runtime (DOTNET_ROLL_FORWARD=Major); CI runs it on .NET 6.

Note for local runs on Windows with core.autocrlf=true: 30 JSON fixture tests already fail on master there (CRLF fixtures vs LF output). The run above used an LF checkout, like CI.

Backport to v3: #230 (3.10.1).

🤖 Generated with Claude Code

[milansvitlica-codaxy]

Closing without merging: not pursuing this change for now.