Skip to content

Testing

Jump to bottom
github-actions[bot] edited this page Sep 17, 2026 · 18 revisions

Testing

wolfCOSE includes comprehensive testing infrastructure for unit tests, algorithm coverage, code coverage, and failure injection testing. CI requires 100% line coverage for every wolfCOSE source file. These thresholds are validated on every push and PR to ensure coverage does not regress.

Running Tests

Release Qualification

Release candidates add the ci:release label to run the read-only Release Qualification workflow. The workflow validates version metadata, exercises the optional feature profiles, audits every required CI workflow associated with the pull request head, builds reproducible source archives, and smoke-tests both archives. A manual post-merge run qualifies the exact commit used to build the release artifacts. Like wolfProvider's label-selected jobs, the workflow starts only when the label is added; remove and re-add the label to qualify a newer commit on the same pull request. Apply the label only after the candidate version and release notes are committed.

The release-only local targets are:

make release-scenarios
make release-coverage
make cxx-check
make valgrind-check
make release-validate VERSION=X.Y.Z RELEASE_REF=HEAD
make release-artifacts VERSION=X.Y.Z RELEASE_REF=HEAD

Replace X.Y.Z with the version in include/wolfcose/version.h.

release-scenarios combines HPKE one-recipient and multi-recipient delivery, stateful HSS/LMS signing, lean LMS verification, ML-DSA signing, RFC 9783 device onboarding, and delegated PSA/EAT signing. The HPKE scenario rejects modified authentication data, modified ciphertext, and a mismatched recipient. The LMS scenario signs two messages, proves the persisted state advances, and rejects a modified signature. The PSA/EAT scenario rejects a wrong challenge, a modified token, and an unapproved software measurement.

Basic Unit Tests

make test

This runs the full test suite including:

  • CBOR encoding/decoding tests (RFC 8949 Appendix A vectors)
  • COSE Sign1/Encrypt0/Mac0 tests
  • COSE Sign/Encrypt/Mac multi-party tests
  • Interoperability tests with COSE Working Group vectors

PSA/EAT Attestation Tests

make eat-psa-test
make eat-psa-float-test
make eat-psa-min-buffers-test
make eat-psa-claim-limits-test
make eat-psa-profile-test
make eat-psa-ext-sign-test
make eat-psa-config-check
make psa-eat-lean-verify
make psa-eat-demo

eat-psa-test builds the explicit full RFC 9783 profile and runs the Appendix A Sign1 and Mac0 vectors, current and legacy profile cases, current algorithm round trips, and malformed-token negatives. The vectors were generated by TF-M's iat-verifier as documented by RFC 9783.

eat-psa-ext-sign-test reruns the full profile with WOLFCOSE_ENABLE_EXT_SIGN to exercise the delegated (external) signing path for PSA/EAT tokens.

eat-psa-min-buffers-test reruns that full profile with WOLFCOSE_MIN_BUFFERS. It proves that the PSA/EAT claim parser accepts the complete current-profile fixture even when the ordinary COSE header-map limit is reduced for constrained targets.

eat-psa-float-test recompiles the full profile with WOLFCOSE_FLOAT and executes both floating-point encoder safety checks against a tolerant decoder context.

eat-psa-claim-limits-test recompiles a focused full profile with the minimum valid WOLFCOSE_EAT_PSA_MAX_CLAIMS and WOLFCOSE_EAT_PSA_MAX_COMPONENT_CLAIMS values. It accepts exact-size maps and rejects a top-level or component-map declaration one entry over its limit.

eat-psa-profile-test runs separate current Sign1-only, current Mac0-only, legacy Sign1-only, and legacy Mac0-only binaries. It proves partial current attesters can issue and generically authenticate #tfm with their enabled envelope, while their incomplete PSA/EAT receivers reject that profile. It also proves disabled standardized claim namespaces are rejected instead of skipped, and a structurally valid token using its compiled-out envelope returns WOLFCOSE_E_UNSUPPORTED before cryptographic verification.

eat-psa-config-check proves that the default library has no PSA/EAT symbols, then compiles minimal current and legacy Sign1/Mac0, issuance-only, and invalid gate combinations. It also checks the 384, 521, and above-P-521 ECC_MIN_KEY_SZ boundaries and rejects PSA/EAT resource limits below their structural floors. psa-eat-lean-verify runs the RFC Sign1 and Mac0 vectors against a verify-only, full-#tfm receiver build. psa-eat-demo runs a complete issue, challenge verification, and software-component appraisal workflow. See PSA-EAT.

Deprecated Algorithm ID Tests

make deprecated-algs-test

Rebuilds the suite with WOLFCOSE_ENABLE_DEPRECATED_ALGS so the RFC 9053 ES256/ES384/ES512/EdDSA paths, the COSE WG example vectors, and the RFC 9783 PSA tokens run. CI runs it as a dedicated step, and the minimal-build matrix also runs the full suite with the macro enabled.

CLI Tool Tests

make tool-test

Round-trip self-tests for all 17 supported CLI algorithms. Each algorithm is tested with key generation, operation, and verification.

Experimental COSE-HPKE Tests

COSE-HPKE P0 is off by default and therefore has a dedicated opt-in test path. Build against a wolfSSL configured with --enable-hpke --enable-ecc --enable-aesgcm --enable-keygen, then acknowledge experimental draft support and enable the four operation macros:

HPKE_FLAGS="-DWOLFCOSE_EXPERIMENTAL \
  -DWOLFCOSE_ENABLE_HPKE_0_ENCRYPT \
  -DWOLFCOSE_ENABLE_HPKE_0_DECRYPT \
  -DWOLFCOSE_ENABLE_HPKE_0_KE_ENCRYPT \
  -DWOLFCOSE_ENABLE_HPKE_0_KE_DECRYPT"

make test EXTRA_CFLAGS="$HPKE_FLAGS"
make hpke-demo EXTRA_CFLAGS="$HPKE_FLAGS"
make c99-hpke-check WOLFSSL_INC=/path/to/hpke-enabled-wolfssl/include
EXPECT_HPKE=true make cmdline-test EXTRA_CFLAGS="$HPKE_FLAGS"

The command-line test performs public/private key export, rejects identical, normalized-alias, case-alias (where the filesystem supports it), and symlink-alias key destinations, preserves existing POSIX destinations, and rejects a failed paired key export without leaving a private-key file. It rejects a plaintext one byte above the configured maximum for both constructions, exercises HPKE-0 integrated encryption, two-recipient HPKE-0-KE encryption and decryption at both recipient indices, maximum-message round trips for both constructions, and a focused test -a round trip for each construction. The dedicated strict C99 target compiles every HPKE-gated library, test, tool, and example path under each one-way operation gate, both convenience gates, the complete P0 configuration, and wolfSSL's NO_ECC256 plus HAVE_ALL_CURVES configuration. It also rejects HPKE when NO_ECC_SECP or an ECC_MIN_KEY_SZ above 256 makes P-256 unavailable, or when NO_AES_128 removes the fixed suite's AEAD key size. Every enabled configuration also supplies WOLFCOSE_EXPERIMENTAL; the regular make experimental-check target verifies that an HPKE operation selected without that acknowledgement is rejected at compile time. The unit suite validates draft messages that omit the optional HPKE alg (including a second selected recipient), rejects an HPKE alg in an unprotected header, accepts an unprotected HPKE-0-KE content algorithm only when its authenticated Recipient_structure binds it, and rejects a modified content algorithm. The CI backend enables Koblitz curves to prove that a 32-byte secp256k1 key cannot masquerade as P-256 at the HPKE API boundary. It also covers missing, duplicate, wrong-type, wrong-length, and wrongly placed ek, prohibited psk_id, detached ciphertext, and cleared outputs on failed decrypts and encrypts. GitHub Actions runs the same coverage in Experimental COSE-HPKE.

Comprehensive Algorithm Tests

make comprehensive

Runs ~240 algorithm combination tests covering:

  • All signature algorithms with various payloads
  • All encryption algorithms with various key sizes
  • All MAC algorithms
  • Multi-signer and multi-recipient combinations
  • Error handling and edge cases

Scenario Examples

make scenarios

Runs real-world scenario examples:

  • Firmware signing with ML-DSA
  • Multi-party approval workflows
  • IoT fleet configuration
  • Sensor attestation
  • Group broadcast MAC

Interoperability (t_cose)

make interop-tcose \
  TCOSE_DIR=/path/to/t_cose QCBOR_DIR=/path/to/QCBOR \
  TCOSE_CRYPTO_LIB="-lcrypto"

Proves RFC 9052 wire interoperability between wolfCOSE (on wolfCrypt) and t_cose (on OpenSSL): each library produces COSE messages the other consumes, both directions, across every algorithm both implement — ES256/384/512, PS256/384/512, EdDSA (Ed25519, Ed448), HMAC 256/384/512, and AES-GCM 128/192/256. The bytes on the wire are the only interface; the two APIs are never reconciled. Each primitive class also exercises a tamper case that wolfCOSE must reject.

The same pinned harness also performs current-profile PSA/EAT Sign1 and Mac0 interoperability. t_cose verifies wolfCOSE's envelope bytes; wolfCOSE verifies the t_cose envelope and validates the RFC 9783 claims.

t_cose and QCBOR are BSD-3-Clause and are not vendored; the Interop CI job fetches them at pinned SHAs. See tests/interop/t_cose/README.md for the fixed test-key provenance.

Complete upstream t_cose suite

make tcose-upstream \
  TCOSE_DIR=/path/to/t_cose QCBOR_DIR=/path/to/QCBOR

This runs the entire pinned upstream t_cose suite, including its API-level and QCBOR-adapter tests. At the current pinned revision it runs 43 tests. It is a dependency-health gate, not wolfCOSE API coverage: t_cose's internal API tests cannot be redirected to wolfCOSE. Use interop-tcose for wolfCOSE wire-format compatibility.

Interoperability (go-cose)

make interop-go-cose

This runs live, bidirectional COSE_Sign1 interop against Veraison go-cose, pinned at v1.3.0 in tests/interop/go_cose/go.mod. The matrix covers ES256, ES384, ES512, PS256, PS384, PS512, Ed25519, ES256 with external AAD, and untagged ES256. The pinned peers predate RFC 9864, so every interop target links a wolfCOSE built with WOLFCOSE_ENABLE_DEPRECATED_ALGS and still exchanges the RFC 9053 IDs. Each implementation signs a message the other verifies, validates the payload, and rejects a modified signature. Go 1.21 or later is required. go-cose requires an embedded payload for Sign1 verification, so detached Sign1 remains covered by the wolfCOSE tests and interop-tcose. go-cose's scope makes it a Sign and Sign1 oracle; interop-tcose remains the broader Mac0 and Encrypt0 wire suite. The target also verifies RFC 9783's ES256 PSA token and decodes its standard EAT claims with Go CBOR.

Interoperability (python-cwt)

python3 -m pip install -r tests/interop/python_cwt/requirements.txt
make interop-python-cwt

This runs live recipient-message interop against python-cwt, pinned at 3.3.0 with all Python dependencies locked in requirements.txt. It exchanges A128GCM direct COSE_Encrypt, ECDH-ES plus HKDF-SHA-256 COSE_Encrypt, and HMAC-256 direct COSE_Mac messages in both directions. Each message carries external AAD, checks its payload, and must reject a modified authenticated byte. The target also verifies RFC 9783's ES256 PSA token and decodes selected EAT claims with python-cwt and cbor2.

A128KW COSE_Encrypt interoperates in both directions. wolfCOSE emits an empty protected recipient bucket with the algorithm in the unprotected header, per RFC 9053 Section 6.2.1, which python-cwt 3.3.0 both produces and accepts.

Interoperability (Rust coset)

make interop-rust-coset

This runs live, bidirectional COSE_Sign1 interop against Google coset, pinned at 0.4.2 in tests/interop/rust_coset/Cargo.lock, using RustCrypto for signature operations. It covers ES256, Ed25519, ES256 with external AAD, untagged ES256, and detached ES256. Both peers validate the protected algorithm and payload semantics, then reject a modified signature. Rust 1.81 or later is required.

COSE WG Examples vectors

make test includes a curated, fixed subset from the COSE WG Examples repository, pinned at commit 53c9d634333bb4f529d78f5980fffa2667ee2c12. It verifies ES256 COSE_Sign1 and COSE_Sign, HMAC-256 COSE_Mac0, A128GCM COSE_Encrypt0, and multi-recipient A128GCM direct, A128KW, and ECDH-ES COSE_Encrypt vectors. Every selected vector checks its cleartext and rejects a modified authentication value. The selected Mac0 vectors carry alg only in an unprotected header, so the test pins the expected algorithm on the local key instead of accepting an unconstrained algorithm from the message.

PSA attestation-token acceptance

make test includes fixed acceptance vectors from RFC 9783 Appendix A. The tests verify ES256 COSE_Sign1 and HMAC-256 COSE_Mac0 PSA attestation tokens, reject a modified authentication value, and decode the EAT/PSA claim payload with wolfCOSE CBOR APIs.

Code Coverage

Running Coverage

make coverage
make eat-psa-coverage

coverage keeps the default PSA/EAT-free library configuration. The explicit eat-psa-coverage target enables the full RFC 9783 receiver profile before collecting the PSA/EAT source coverage report.

Coverage Targets

Component Target
Every src/*.c file 100%

Coverage with Failure Injection

make coverage-force-failure
make eat-psa-coverage-force-failure

The eat-psa-* form runs the corresponding full PSA/EAT profile. Both forms enable additional coverage by testing error paths that normally require wolfCrypt internal failures.

Force Failure Testing

wolfCOSE includes a failure injection system for testing error paths that are difficult to reach through normal testing.

The WOLFCOSE_FORCE_FAILURE build flag enables controlled injection of failures at specific points in the code. This allows testing of:

  • Crypto operation failures (signature, encryption, decryption, MAC)
  • Key operation failures
  • Memory/buffer errors
  • Internal state errors

Production Builds

The force failure system compiles out completely in production builds. When WOLFCOSE_FORCE_FAILURE is not defined:

  • All failure injection code is excluded
  • wolfForceFailure_Check() always returns 0
  • No runtime overhead

CI Pipeline

wolfCOSE runs the following CI checks on every push and pull request:

Build and Test Matrix

Environment Compilers
Ubuntu (latest + 22.04) GCC 10, 11, 12, 13, 14
Ubuntu (latest + 22.04) Clang 14, 15, 16, 17, 18
macOS Xcode default

Test Stages

  1. Build: Compile library and tests
  2. Unit Tests: Run CBOR and COSE test suites
  3. Comprehensive Tests: ~240 algorithm combination tests
  4. Scenario Examples: Real-world workflow tests
  5. Tool Tests: CLI round-trip tests (17 algorithms)
  6. Experimental COSE-HPKE: Opt-in P0 unit tests, example, CLI commands, and both construction-specific self-tests

The Interop workflow intentionally keeps t_cose, go-cose, python-cwt, Rust coset, and the complete upstream t_cose suite in one matrix job per wolfSSL version. They share the same wolfSSL build; separate jobs would only duplicate that setup without providing useful parallelism.

Memory and Stack Bounds

wolfCOSE is zero-heap (no malloc/XMALLOC on any path) and bounded-stack, both enforced in CI:

  • Bounded stack: built with -fstack-usage, then scripts/check_stack_usage.sh fails the build if any wolfCOSE frame exceeds 6144 bytes or is dynamic (unbounded); -Werror=vla bans VLAs/alloca.
  • Zero heap: sources, tests, tools, and examples are grepped for allocator calls.
  • WOLFCOSE_MIN_BUFFERS: constrained-target profile that shrinks the caller working buffers (not the library frames) — see Macros.
  • Minimal Build matrix: builds and tests against single-purpose minimal wolfCrypt configs (ECC-only, EdDSA-only, AEAD-only, MAC-only, …) plus a WOLFCOSE_LEAN core build.

Lean and Post-Quantum Builds

The Lean Build workflow (.github/workflows/lean-build.yml) exercises the minimal on-device build profiles:

Job What it checks
Lean verify-only Builds + runs examples/sign1_verify_lean.c with WOLFCOSE_LEAN_VERIFY against a minimal ECC-only wolfSSL; asserts the signing API is not linked.
Lean configs compile clean Strict -Werror compile of the full, WOLFCOSE_LEAN_VERIFY, sign-only, WOLFCOSE_LEAN_MLDSA, and WOLFCOSE_LEAN_VERIFY_MLDSA configurations.
Post-quantum ML-DSA Builds wolfSSL with ML-DSA and runs make mldsa-demo (sign+verify) and make mldsa-verify (verify-only); asserts the verify-only build links no signing API.

Static Analysis

Tool Purpose
cppcheck Static code analysis
Clang Static Analyzer Data flow analysis
GCC -fanalyzer GCC's built-in analyzer
Advanced Internal Static Analysis Security Audit
In PR Opus 4.6 Diff review with wolfSSL internal review bot Security Audit

Coverity Scan

Nightly defect analysis via Coverity Scan.

Coverity Scan Build Status

Test File Structure

tests/
  test_cbor.c        # CBOR vectors (RFC 8949 Appendix A) + round-trip
  test_cose.c        # COSE Sign1/Encrypt0/Mac0/Sign/Encrypt/Mac tests
  test_interop.c     # Interoperability tests with RFC vectors
  test_main.c        # Test harness (CI exit codes)
  force_failure.c    # Failure injection implementation
  force_failure.h    # Failure injection API
  vectors/           # Test vectors from COSE Working Group

Test Categories in test_cose.c

Category Description
Sign1 Tests Single-signer signature creation and verification
Encrypt0 Tests Symmetric encryption and decryption
Mac0 Tests Symmetric MAC creation and verification
Sign Tests Multi-signer messages
Encrypt Tests Multi-recipient encryption
Mac Tests Multi-recipient MAC
Key Tests COSE_Key encoding and decoding
Error Tests Invalid inputs, tampered messages
Detached Payload Tests Messages with external payloads
External AAD Tests Additional authenticated data

Test Vectors

The tests/vectors/ directory contains test vectors from:

  • COSE Working Group examples
  • RFC 9052 examples
  • Custom edge case vectors

Vector format is typically CBOR diagnostic notation or hex dumps with expected outputs.

See Also

Documentation

Clone this wiki locally