Vulnerability scanning

Tobby scans every image it retrieves with Trivy, run as a pinned external binary, offline, out of process (FR-031, ADR-0008). The scan happens at retrieval, in passthrough as in mirror mode, before anything of the recipe is published anywhere: pushed to a destination, written on a medium, or offered as a cookbook entry to a zone below.

What is scanned: the content, not the declared kind

An ingredient declared ContainerImage is scanned. An ingredient declared anything else — OCIArtifact, FileSet, HelmChart — is inspected first, by its content, and scanned whenever that content is an image (ADR-0017). The declared kind never decides: otherwise an image published as an “AI model” would cross unexamined, under a blocking policy too, and run as a container on the other side.

Content	Detected as	Scanned
a manifest whose config is an image config (application/vnd.oci.image.config.v1+json, application/vnd.docker.container.image.v1+json)	image-config	yes, as an image
a manifest with a filesystem layer (OCI tar, tar+gzip, tar+zstd, non-distributable variants, Docker rootfs) whose bytes really are a tar	filesystem-layers	yes, as an image
a layer blob, or a file inside a filesystem layer, that is a docker save archive	docker-archive	yes, each image it holds
the same, but an OCI image layout packed as a tar holding an image manifest	oci-layout	yes, each image it holds
anything else — a model, the Trivy database, a chart, documents	—	no: reported out of scope

Archives are recognized by their bytes — gzip and zstd by their magic, a tar by its header checksum, docker save by its manifest.json and the files it names, a layout by oci-layout and its index — never by a file name or an extension, and without extracting anything: the inspection is streamed and bounded. Content that cannot be judged (a truncated archive, a stream beyond a bound) is reported not scanned and, under a blocking policy, refused.

A FileSet is itself an OCI image (RECIPE-SPEC §7.4), so it is scanned like one, and the files it carries are looked into for image archives. The report of such an ingredient says it was scanned detected as an image from its content, with its declared kind (TBY-SCN-023); an ingredient holding no image carries an out-of-scope report saying so (TBY-SCN-024). Inspection needs neither the binary nor the database: in the degraded mode a detected image is reported not scanned, and refused under a blocking policy, like any image.

The signed tobby-system recipe is the one exception to refusal: its scanner binary is a FileSet that is an image, and it is scanned and reported, but under an advisory policy whatever the mode, the database age included (TBY-SCN-025) — refusing the deliveries that arm the scanner would leave the zone unable to arm one. Its trust rests on its signature, as the scanner’s already did.

A multi-platform image is scanned child by child, for every child the store holds — including a child that states no platform, or unknown, since it is copied and pushed like the others. The one child skipped is a genuine attestation manifest (unknown/unknown, annotated attestation-manifest, in-toto layers only), which holds no filesystem. A child the scanner cannot read fails the scan: under a blocking policy the recipe is refused as not scannable, never pushed unexamined.

One policy, three levels

Level	Where	What it sets
Instance	scan.* (configuration)	the mode (warn, the default, or block), the threshold (default 7.0), exceptions, the database age bound
Retriever	spec.scanPolicy (RECIPE-SPEC §10.4)	a threshold for its zone, exceptions for its zone
Recipe	annotation recipe.tobby.dev/scan-threshold	a stricter threshold for that recipe only

How they combine:

• Threshold — the most specific level wins: recipe, then Retriever, then instance. But a recipe can only lower the value the two others produce: a higher annotation is ignored and the report says so (TBY-SCN-020). A third-party recipe author never relaxes a zone.
• Mode — advisory unless the instance says block, or a Retriever or a recipe poses a threshold: a threshold posed for a perimeter is blocking for that perimeter. A recipe can make its own evaluation blocking, never advisory. The report names the level that made it so (mode_level).
• Exceptions — the instance list and the Retriever list add up; a recipe carries none. An exception past its expires no longer applies and is reported as expired (TBY-SCN-021).

# Retriever excerpt — the zone refuses anything at or above 7.0, except one
# accepted CVE until the end of March.
spec:
  scanPolicy:
    threshold: 7.0
    exceptions:
      - id: CVE-2024-12345
        justification: vulnerable parser disabled in our build
        expires: "2027-03-31T00:00:00Z"

How a finding is judged

• A finding violates the policy when its CVSS base score is at or above the threshold. Several sources: the highest counts (each source contributing its most recent CVSS version).
• No CVSS score: the lower bound of its severity band — LOW 0.1, MEDIUM 4.0, HIGH 7.0, CRITICAL 9.0.
• UNKNOWN without a score: not compared, reported as unscored (TBY-SCN-022), never dropped.

What a refusal does

A refused image withholds its whole recipe — the grain of R-19: not stored as a cookbook entry, not recorded in the recipe graph (so a medium written now does not deliver it, and an earlier record is withdrawn), and nothing of it is pushed. The refused ingredient’s push item and the recipe’s item carry the code (TBY-SCN-005), a scan.block event goes to the audit log, and the other recipes of the run cross.

Under a blocking policy an image that could not be scanned is refused too — scanner missing, database absent, scan failed. No configuration path relaxes security implicitly.

What a report carries

Every scanned or inspected item of a task carries items[].scan — in the task screen and, identically, in GET /api/v1/tasks/{id}:

• the scanner and its version, the scan instant;
• the database: version, digest, build timestamp and age at scan time, and whether it was stale — in every report (R-13);
• counts by severity and every finding (identifier, severity, CVSS when known, package, installed and fixed-in versions);
• for an ingredient not declared as a container image, content: its declared kind and what was detected as an image in it (digest, file, evidence);
• the evaluated policy: verdict (pass, warn, block, not-scanned, out-of-scope), mode and the level that set it, threshold and the level that set it, violations, exceptions applied or expired, and the reasons as codes.

The degraded mode is never silent

Situation	Code	Banner	Metric
no usable scanner binary	TBY-SCN-001	danger	tobby_scan_scanner_available 0
binary does not match its pin — never executed	TBY-SCN-002	danger	tobby_scan_scanner_available 0
no database in the store	TBY-SCN-003	danger	tobby_scan_db_present 0
database older than scan.maxDbAge	TBY-SCN-004	warning	tobby_scan_db_stale 1
advisory policy	TBY-SCN-006	warning	tobby_scan_policy_blocking 0

Each is also stated in the startup log, and re-evaluated after every synchronization: a degraded instance recovers by itself once the tobby-system recipe has crossed. The Retriever screen carries a permanent indicator — policy, scanner and where it came from, database age and staleness — whether or not anything is wrong.

The scanner binary, per platform

Platform	Binary	Pin
container image (linux/amd64, linux/arm64)	bundled at /usr/libexec/tobby/trivy	the release this build pins (Trivy 0.76.2), digest verified before every execution
Linux or Windows host binary	unpacked from the tobby-system FileSet (scan.trivy.ref)	the recipe’s digest, signed by the site; the unpacked file re-hashed before every execution
macOS	the first trivy on the PATH	none — the development station; reported as unpinned
any	scan.trivy.path	scan.trivy.digest, or the built-in pin

Trivy runs with every offline flag (--offline-scan, --skip-db-update, --skip-java-db-update, --skip-check-update, --skip-version-check, --disable-telemetry), a private home, outbound proxies pointed at a closed local port, a deadline (scan.timeout) and bounded outputs, on a private copy of the image — never on the store itself.

An unpacked channel binary that no longer hashes to its digest is never run. A scan that meets it refuses with TBY-SCN-002. A refresh that meets it (at startup, or after a run that records tobby-system) unpacks the verified delivery over it again, logs a WARN record with the expected and the found digests, and writes a scan.scanner.repaired event to the audit log.

Limits of this release

• Only image archives are looked for in files. A FileSet’s other files — binaries, scripts, language packages unpacked by hand — are not vulnerability-scanned beyond what Trivy finds in the FileSet’s own filesystem. Archives are looked into one level deep (an archive inside a docker save file inside a FileSet is not), a plain root-filesystem tar is not an image archive (docker import could still make one), and the files of a declared ContainerImage are not looked into for archives.

• The Java database is not handled. Only the main vulnerability database (trivy-db, schema 2) travels and is used; Trivy runs with --skip-java-db-update and no Java index, so findings that need it (JAR artifacts identified by checksum) are not reported — or, depending on the Trivy release, the image is reported not scanned, which a blocking policy refuses. Not verified on Java images in this release.

• No re-scan at media import. FR-031 scans at retrieval: a medium is scanned where it was written, and the destination side verifies signatures and digests (FR-052, FR-054) without scanning again. An instance that retrieves inside the zone — a passthrough instance reading the zone registry — scans with the database that crossed.

• No scan in plan mode. tobby sync --dry-run (R-04) resolves, sizes and checks signatures; it does not run the scanner, so it cannot predict a vulnerability refusal.

• A withheld recipe’s bytes stay in the store. Its ingredients are fetched before they can be scanned, so a refusal leaves their content in the store — on a medium, it travels. Nothing delivers it: the recipe is not in the recipe graph nor in the media manifest, so a destination never verifies nor pushes it, and tobby store verify lists it as content no recipe reaches (TBY-INT-003); the refusal’s log line says so. A cycle with a withheld recipe prunes nothing, like any cycle with a failed recipe; the content goes once the recipe is fixed (and recorded again) or dropped from the Retriever and pruned, or with a medium started over.

• What was admitted before a CVE was known stays in the local cookbook. A recipe refused today because a fresher database knows a new CVE is withheld from the medium and the destination; but its artifact, stored by an earlier cycle in this instance’s own cookbook, is not removed — a zone reading this instance in cascade can still resolve that earlier copy. Purging what a destination already holds is out of scope (SRS §5.1).

Getting the database and the binary into a zone, air gap included, is the vulnerability database bootstrap.