LESSON 11

Testing, fuzzing and evidence you can show an auditor

Building a test programme that satisfies Annex I Part II (3) and produces documentation as a by-product.

9 min read video walkthrough

Evidence is the deliverable

An auditor or market surveillance authority does not watch you test. They read artefacts. Design the programme so every activity emits a dated, versioned, archived artefact tied to a firmware build identifier.

The layered programme

Static analysis in CI. Compiler warnings as errors, -Wall -Wextra -Wformat-security, plus a static analyser (clang-tidy, cppcheck, Coverity, or equivalent). Enable hardening flags and record them: -fstack-protector-strong, -D_FORTIFY_SOURCE=2, -Wl,-z,relro,-z,now, position independence where the platform allows. The build log is evidence.

Composition analysis. SBOM generated on every build, matched against NVD/OSV, results stored per build. A build with an unresolved high-severity reachable finding must not become a release candidate — encode that as a pipeline gate, because the gate itself is the evidence of "no known exploitable vulnerabilities at release".

Fuzzing. This is where the real embedded bugs are. Fuzz every parser that touches untrusted input: your BLE GATT handlers, the MQTT/CoAP/Modbus frame parsers, the OTA manifest parser, the config file reader, the USB descriptor handling. Practically, extract the parser into a host build and run libFuzzer or AFL++ with a corpus in CI; supplement with on-target fuzzing over the real transport for anything the host harness cannot model. Keep the corpus in version control and record coverage.

Hardware and platform review. A signed-off checklist per release: debug interfaces disabled, readout protection set, lifecycle state locked, secure boot enforced (not just enabled), test commands removed, default credentials absent, unused services off. Verify on a production-line sample, not on a bench build.

Third-party assessment. A penetration test by an independent party before first placing on the market and after each substantial modification, with a remediation log showing every finding closed or risk-accepted with justification.

Fuzzing in a constrained target

Three practical patterns when you cannot run a fuzzer on the device:

  1. Host harness — compile the parser for the host with sanitizers (ASan/UBSan) and fuzz there. Fastest, catches most memory-safety bugs, but misses hardware-dependent behaviour.
  2. Emulation — QEMU or Renode running the real firmware, fuzzed through the emulated peripheral. Higher fidelity, more setup.
  3. On-target with instrumentation — a debug build that reports crashes over a probe. Slow but real.

Most teams get the best return from 1 in CI plus a periodic pass of 2 or 3.

The evidence pack

For each release, archive: build manifest and toolchain versions, SBOM, dependency scan report, static analysis report, fuzzing coverage and crash summary, hardware checklist signed and dated, pentest report and remediation log, and the updated risk assessment. That pack, plus the Annex I requirements matrix, is most of your technical documentation already written.

Key takeaways

  • Design the test programme so each activity emits a dated artefact tied to a build identifier.
  • Gate releases on the dependency scan — the gate is your evidence for 'no known exploitable vulnerabilities'.
  • Fuzz every untrusted-input parser, using a host harness in CI plus periodic emulation or on-target runs.
  • Archive a per-release evidence pack; it becomes most of your technical documentation.

References

  • Annex I Part II (3)
  • Annex VII points 2-3

Check your understanding

  1. Q1The most productive fuzzing targets in an embedded product are:

  2. Q2A release pipeline should gate on:

  3. Q3Independent penetration testing should be repeated:

Sign in to track progress