Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
# measurement/REPORT.md is in the nav as Self-verification.
if [ -d measurement ]; then cp -r measurement $BUILD/measurement; fi

for fname in README.md CHANGELOG.md CONTRIBUTING.md PRIVACY.md CNAME; do
for fname in index.md CHANGELOG.md CONTRIBUTING.md CODE_OF_CONDUCT.md PRIVACY.md CNAME; do
if [ -f "$fname" ]; then cp "$fname" "$BUILD/$fname"; fi
done

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
[![TRACE Spec](https://img.shields.io/badge/TRACE-Spec_v0.2-0ea5e9)](https://github.com/agentrust-io/trace-spec)
[![Tests](https://img.shields.io/badge/Conformance_Tests-7_modules-green)]()
[![CI](https://github.com/agentrust-io/trace-tests/actions/workflows/ci.yml/badge.svg)](https://github.com/agentrust-io/trace-tests/actions/workflows/ci.yml)
[![Discord](https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&logoColor=white&style=flat)](https://discord.gg/9JWNpH7E)
[![Discord](https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&logoColor=white&style=flat)](https://discord.gg/grgzFEHgkj)

> **Test suite v0.2.** Tracks [TRACE Spec v0.2](https://github.com/agentrust-io/trace-spec).

Expand Down
83 changes: 83 additions & 0 deletions index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
title: Verify your TRACE implementation
description: Run this suite against your implementation to find out which TRACE conformance level it actually reaches, and produce a report you can hand to an auditor, a counterparty, or an acquirer.
---

# TRACE Test Suite

Conformance tests for [TRACE](https://trace.agentrust-io.com) (Trust, Runtime Attestation, and Compliance Evidence). Run this suite against your implementation to verify it meets the specification before claiming TRACE compliance.

**Seven modules cover the full specification: envelope structure, signature algorithms, TEE runtime claims, policy binding, tool-call transcripts, SCITT transparency anchoring, and supply-chain provenance.**

!!! tip "TL;DR"
- `verify` answers a question for the person running it. `report` produces an artifact for somebody who was not there.
- `report` runs every level up to `--max-level`, because the useful answer is the highest level a record reaches, not whether it cleared the level someone happened to pick.
- The HTML report is self-contained: no scripts, no fonts, no external CSS, no badge service, nothing fetched when it is opened.
- The report is not evidence, and it says so on its face. It carries the record digest, the suite and library versions, and the command to reproduce the result.

```bash
pip install agentrust-trace-tests
trace-tests verify --record path/to/trust-record.jwt --level 1
```

## A report you can hand to someone else

```bash
trace-tests report --record trust-record.json --html report.html --json report.json --badge trace.svg
```

Use `--fail-under 1` to gate CI on a level. Without it the command always exits `0`, which is what you want when you are producing an artifact rather than enforcing a threshold. `report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards and CI.

A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log. So the report tells a reader who does not trust the sender to go and check the record instead, and gives them what they need to do it.

## Where to start

<div class="grid cards" markdown>

- __Run it__

---

Score a record, read the failures, and produce a report from the same run.

[Getting Started](docs/quickstart.md)

- __Understand the levels__

---

What each conformance level requires, and what a record has to carry to reach it.

[Conformance Levels](docs/levels.md)

- __Read the modules__

---

The seven test modules, the `TR-*` error codes they emit, and what each one checks.

[Test Modules](docs/modules.md)

- __Wire it into CI__

---

Gate a pipeline on a level, and write your own conformance tests against the suite.

[CI integration](docs/tutorials/ci-integration.md)

</div>

## Test modules

| Module | ID | Tests |
|---|---|---|
| Envelope | `TR-ENV` | EAT structure, required fields, `iat` validity |
| Signature | `TR-SIG` | ES256/ES384/EdDSA, key binding, chain |
| Runtime | `TR-RTE` | TEE platform, measurement format, RIM URI |
| Policy | `TR-POL` | Bundle hash, enforcement mode, TEE binding |
| Transcript | `TR-TXN` | Tool-call transcript hash binding (Phase 2+) |
| Transparency | `TR-ANC` | SCITT receipt URI, inclusion proof |
| Provenance | `TR-SCA` | SLSA level, builder URI, digest format |

The suite tracks [TRACE Spec v0.2](https://trace.agentrust-io.com). See [Changelog](CHANGELOG.md) for what moved between suite versions.
9 changes: 5 additions & 4 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ exclude_docs: |
schemas/
LICENSE
NOTICE
README.md
ANTITRUST.md
ADOPTERS.md
MAINTAINERS.md
SECURITY.md
CHARTER.md
CODE_OF_CONDUCT.md
pyproject.toml

theme:
Expand Down Expand Up @@ -72,7 +72,7 @@ plugins:
conformance level, before shipping them to production.
sections:
Getting started:
- README.md
- index.md
- docs/quickstart.md
- docs/levels.md
Test modules:
Expand Down Expand Up @@ -141,7 +141,7 @@ extra:
link: https://github.com/agentrust-io/trace-tests
name: trace-tests on GitHub
- icon: fontawesome/brands/discord
link: https://discord.gg/9JWNpH7E
link: https://discord.gg/grgzFEHgkj
name: Join the Discord
generator: false

Expand All @@ -152,7 +152,7 @@ extra_css:
- https://agentrust-io.com/design-system.css

nav:
- Home: README.md
- Home: index.md
- Getting Started: docs/quickstart.md
- Tutorials:
- Writing conformance tests: docs/tutorials/writing-conformance-tests.md
Expand All @@ -172,6 +172,7 @@ nav:
- Project:
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- Code of Conduct: CODE_OF_CONDUCT.md
- Privacy: PRIVACY.md

extra_javascript:
Expand Down
12 changes: 2 additions & 10 deletions overrides/main.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,6 @@
repositories, and a hardcoded path is how cMCP shipped a broken card.
#}

{% block htmltitle %}
{% if page and page.is_homepage %}
<title>Verify your TRACE implementation - {{ config.site_name }}</title>
{% else %}
{{ super() }}
{% endif %}
{% endblock %}

{% block extrahead %}
{{ super() }}
{% set page_desc = page.meta.description if page and page.meta and page.meta.description else config.site_description %}
Expand All @@ -35,7 +27,7 @@

<meta property="og:type" content="website">
<meta property="og:site_name" content="{{ config.site_name }}">
<meta property="og:title" content="{% if page and page.title and not page.is_homepage %}{{ page.title }} - {{ config.site_name }}{% else %}{{ config.site_name }}: {{ social_title }}{% endif %}">
<meta property="og:title" content="{% if page and page.title %}{{ page.title }} - {{ config.site_name }}{% else %}{{ config.site_name }}: {{ social_title }}{% endif %}">
<meta property="og:description" content="{{ page_desc }}">
<meta property="og:url" content="{{ page_url }}">
<meta property="og:image" content="{{ og_image }}">
Expand All @@ -45,7 +37,7 @@
<meta property="og:locale" content="en_US">

<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="{% if page and page.title and not page.is_homepage %}{{ page.title }} - {{ config.site_name }}{% else %}{{ config.site_name }}: {{ social_title }}{% endif %}">
<meta name="twitter:title" content="{% if page and page.title %}{{ page.title }} - {{ config.site_name }}{% else %}{{ config.site_name }}: {{ social_title }}{% endif %}">
<meta name="twitter:description" content="{{ page_desc }}">
<meta name="twitter:image" content="{{ og_image }}">

Expand Down