Add Tool Assessment Predicate Type #496
Conversation
Signed-off-by: JosephKyser <[email protected]>
Signed-off-by: JosephKyser <[email protected]>
JosephKyser
force-pushed
the
tool_assessment
branch
from
74f411c to
89065dc
Compare
|
Bumping request for review |
adityasaky
left a comment
adityasaky
left a comment
There was a problem hiding this comment.
This is pretty cool! Thanks for proposing this predicate type. :)
|
|
||
| ## Purpose | ||
|
|
||
| **The tool assessment attestation references the results and metadata associated with tools used to assess software before or after its creation.** It's primary purpose is to provide an immutable attestation of tool assessment of a software so that it can be bundled with its provenance. This enables to mapping of build provenances to assessments on a per build basis and can reflect entire DevSecOps pipeline processes. |
There was a problem hiding this comment.
nit: I would make the specific provenance bit here part of the use cases below. Ideally, the purpose section is more general in tone and content.
|
|
||
| There are many existing predicates that describe the use of specific tool types. While these predicates are well defined, they are narrowly scoped. There should exist a predicate that is general enough to effectively attest the use of any tool that can be used if a tool type does not have a predicate type yet. The tool assessment attestation type aims to solve this. | ||
|
|
||
| - The [cyclonedx](cyclonedx.md) and [spdx](spdx.md) predicate types describe SBOM standards. |
There was a problem hiding this comment.
I wonder if this is not too far from witness's commandrun attestor. @jkjell and others may have some thoughts there. If there's overlap, maybe we can consolidate the two.
There was a problem hiding this comment.
I wonder if this is not too far from witness's commandrun attestor. @jkjell and others may have some thoughts there. If there's overlap, maybe we can consolidate the two.
I've actually looked a bit at witness and I really like the concept of command run. One of my concerns is how do I map results to an ingestible format in the attestation? The answer may be a 1-2 punch where witness attests "this is what was ran with these raw results" and tool assessment could transform the raw results into an ingestible format, but then you lose the chain of trust that witness aims to provide by allowing that second step to interpret findings.
|
|
||
| ### Control Gates | ||
|
|
||
| Control gates are an essential and increasingly prevalent requirement in many |
There was a problem hiding this comment.
Could you add an example here? Not necessarily a full attestation but a scenario where a tool is used, its invocation is attested, and what the assessors may be looking at in the attestation.
|
|
||
| ### Policy as Code Enabling via Attachment to Build Artifacts | ||
|
|
||
| Tool assessment attestations bundled with container build provenance can enable policy-as-code enforcement of containers or software on IT systems. |
|
|
||
| **config.exclusions, optional** string list | ||
|
|
||
| >> List of deviations from the profile, such as rule IDs, file names, ignores, etc. |
There was a problem hiding this comment.
Can these be resource descriptors? If we're using files to populate exclusions, it'd be good to hash them for transparency, for eg.
There was a problem hiding this comment.
That was my initial thought however I found that it does not cover the case where exclusions originate from the command line as options. I did include config.files so that any exclusions that exists in files (e.g. .semgrepignore) could be included as a resource descriptor.
| } | ||
|
|
||
| // Profile Object Type | ||
| Profile: { |
There was a problem hiding this comment.
More generally, can this be a resource descriptor?
There was a problem hiding this comment.
More generally, can this be a resource descriptor?
I think it absolutely could for many cases. Databases and file based configurations come to mind for this. The problem is when an internal configuration changes how a tool behaves.
Take semgrep as an example. It provides language specific registries you can use by adding them in the tool options. I am unsure how a case like this would be described by a resource descriptor other than maybe a URI and heavy use of the annotations field.
|
|
||
| >> List of deviations from the profile, such as rule IDs, file names, ignores, etc. | ||
|
|
||
| **config.files, optional** ResourceDescriptor list |
There was a problem hiding this comment.
As my other comments may suggest, I think I'm a bit confused by profiles vs files. Do we have files that won't be profiles or exclusions? :)
There was a problem hiding this comment.
Sure! My thought process was that the predicate had to be able to included many different configurations such as databases, config files, internal options, entire rulesets, etc. My challenge was how can I create a model that could be any of those configurations, and profiles are what I came up with. AV runs with this database? That's a profile. Fortify uses these internal ruleset to apply SAST findings? That's a profile.
Files come into play where we want to deviate from the profile. This usually comes in the form of exclusions/ignores. Some tools ingest a specific file to apply these exclusions, such as a .semgrepignores. The ultimate goal is for the predicate to have enough information to replicate (or validate) the findings where findings = tool([profiles-exclusions])
|
|
||
| **output, required** ResourceDescriptor list | ||
|
|
||
| > Artifacts associated with the result of the execution of the tool. |
There was a problem hiding this comment.
This raises an interesting question. What do we foresee as the subject of this attestation? Is it the artifact under test, i.e., that the tool operated on? It would be good to add this to the model section.
There was a problem hiding this comment.
Could the output point to another attestation? For example, the test results or vuln scan predicates could be good counterparts to this one in that model.
There was a problem hiding this comment.
I work in the realm of containers. For my use-case, I envisioned all of the subjects pointing to the image digest of the containers, but I think the artifact under test would be just as valid.
There was a problem hiding this comment.
Could the output point to another attestation? For example, the test results or vuln scan predicates could be good counterparts to this one in that model.
I suppose it could, however I think those attestations could just stand on their own merits. In a perfect world where all tool types have a corresponding predicate type, tool attestations wouldn't be needed.
2 participants
This PR ads the predicate type
tool assessment.The tool assessment predicate aims to provide a general template to attest to the results of tools ran on software while also providing configuration data in order to replicate those results, leading to open and auditable security artifacts.
Currently the attestation bundle alludes to packaging different points in the software supply chain to be grouped together, however only has predicate types for sbom, vulnerability scans, test results, etc. Other type of software supply chain artifacts like SAST, DAST, Secret Scanning, STIG, etc. are missing. Instead of exhaustively creating a predicate type for every use case, tool assessment allows the ability to cover every type of tool.
If nothing else, it allows for these non-covered tool types to be represented until more narrowly defined predicates scoped for each respective tool type is defined.