Quickstart

Quickstart

Get from installation to your first behavioral regression check in a few minutes.

DProvenanceKit records how an AI workflow executes, saves a known-good run as a baseline, and compares future executions against it.

The basic workflow is:

Run
 ↓
Record
 ↓
Create baseline
 ↓
Change behavior
 ↓
Compare
 ↓
Gate

---

1. Install DProvenanceKit

DProvenanceKit requires Python 3.9 or newer.

Install the core package:

pip install dprovenancekit

The core package has zero third-party runtime dependencies.

Framework integrations are optional. For example:

pip install "dprovenancekit[langchain]"

or:

pip install "dprovenancekit[openai-agents]"

---

2. See the Problem Immediately

Before changing any application code, run the built-in demo:

dprovenancekit demo

The demo creates two executions of the same example agent.

The healthy execution follows:

plan
 ↓
search
 ↓
rank
 ↓
verify
 ↓
decide

The candidate execution regresses.

It drops the verification step and introduces repeated search behavior.

Conceptually:

GOLDEN                    CANDIDATE

plan                      plan
 ↓                         ↓
search                    search
 ↓                         ↓
rank                      rank
 ↓                         ↓
verify                    search
 ↓                         ↓
decide                    decide

The final result could still appear reasonable.

The execution path is not.

DProvenanceKit detects the behavioral difference and reports the regression.

---

What the Demo Creates

The demo writes local artifacts including:

demo-traces.sqlite
demo-rules.json
demo-report.html

These give you:

• structured trace data,
• anomaly rules,
• a regression result,
• and a shareable HTML report.

You can choose another output directory with:

dprovenancekit demo --output-dir ./dprovenance-demo

---

3. Instrument a Real Workflow

Now add DProvenanceKit to your own code.

Suppose your agent has two steps that matter:

1. retrieve information,
2. verify it.

Create agent.py:

from dprovenancekit import traced, traced_run


@traced
def retrieve():
    print("Retrieving evidence...")


@traced
def verify():
    print("Verifying evidence...")


with traced_run(context_id="research-agent"):
    retrieve()
    verify()

Run it:

python agent.py

DProvenanceKit records the execution locally.

The important behavior is now represented structurally:

research-agent
      │
      ├── retrieve
      │
      └── verify

---

4. Record the Known-Good Baseline

Once you have reviewed the execution and consider it correct, pin the newest run as the baseline:

dpk record

The default candidate trace store is:

.dprovenance/traces.sqlite

The accepted baseline is stored at:

.dprovenance/baseline.sqlite

You now have a known-good behavioral reference.

Current execution
       ↓
     review
       ↓
   dpk record
       ↓
Golden baseline

---

5. Introduce a Regression

Now imagine a later code change accidentally removes verification.

Change the execution block in agent.py to:

with traced_run(context_id="research-agent"):
    retrieve()

Run it again:

python agent.py

The program still runs.

There may be no exception.

If the workflow generated natural-language output, that output might even still look correct.

But the behavior changed:

GOLDEN                 CANDIDATE

retrieve               retrieve
   ↓
verify                  ← missing

---

6. Compare the Runs

Inspect the difference:

dpk compare

compare reports the structural difference without treating the existence of a difference alone as a shell failure.

Use it when you want to investigate what changed.

Conceptually, the result is:

Golden vs Candidate

✓ retrieve
✗ verify — missing

This is the central idea behind DProvenanceKit:

A final answer can remain plausible even when a required execution step disappears.

---

7. Gate the Regression

Now enforce the comparison:

dpk gate

The gate evaluates the candidate execution against the accepted baseline.

When the candidate contains a regression that violates the configured gate:

Candidate run
      ↓
Compare with golden
      ↓
Evaluate behavioral change
      ↓
Regression?
   ┌────┴────┐
   │         │
  yes        no
   │         │
exit 1     exit 0

That exit code means the same behavioral check can participate in CI.

A process change that might otherwise be invisible can now block a pull request.

---

8. Put the Gate in CI

The same workflow can be applied during software delivery:

Developer changes agent
        ↓
Run tests / agent workflow
        ↓
Produce candidate trace
        ↓
DProvenanceKit gate
        ↓
   ┌────┴────┐
   │         │
Regression   Clean
   │         │
Fail CI     Continue

This lets teams treat important AI behavior as another testable software property alongside:

• unit tests,
• integration tests,
• type checking,
• security checks,
• API compatibility,
• and performance regressions.

---

9. The Short Version

Once DProvenanceKit is instrumented, the normal workflow is only a few commands.

Establish the baseline

python agent.py
dpk record

Test a later change

python agent.py
dpk compare

Enforce it

dpk gate

Or, in one picture:

python agent.py
      ↓
  dpk record
      ↓
 accepted baseline

      later...

python agent.py
      ↓
  dpk compare
      ↓
   inspect drift

      or

   dpk gate
      ↓
 pass / fail

---

10. No Run IDs Required

The default workflow intentionally avoids requiring you to manually manage run IDs.

By default:

Candidate runs
.dprovenance/traces.sqlite

Golden baseline
.dprovenance/baseline.sqlite

For applications containing multiple agents, use a context identifier to distinguish workflows.

For example:

with traced_run(context_id="research-agent"):
    ...

You can also override storage paths when needed using the CLI options or DPROV_DB.

---

What You Just Did

You did more than record a trace.

You turned an expected AI execution path into something that can be tested.

Expected behavior
       ↓
Recorded baseline
       ↓
Candidate behavior
       ↓
Structural comparison
       ↓
Regression decision

That is behavioral assurance.

Traditional output testing asks:

Did the answer look right?

DProvenanceKit adds:

Did the system follow the process we expected?

---

Next

Now that you have run your first behavioral gate:

• Behavioral Regression Testing — understand golden runs, structural differences, and regression analysis.
• Rules and Enforcement — define explicit requirements that should hold regardless of the baseline.
• Provenance and Audit Evidence — understand how execution history becomes reviewable evidence.
• Architecture and Integrations — see where DProvenanceKit fits alongside your existing AI stack.
• Cross-Language Conformance — learn how the Python and Swift implementations maintain shared behavioral semantics.

---

The Core Workflow

Run → Record → Compare → Gate

Or, more broadly:

Observe what happened.
Define what must happen.
Detect when behavior changes.
Gate what should not ship.
Preserve the evidence.