Skip to content

Replace content with fixed, deduplicated architecture.#8

Open
croadfeldt wants to merge 59 commits into
dcm-project:mainfrom
croadfeldt:content-update
Open

Replace content with fixed, deduplicated architecture.#8
croadfeldt wants to merge 59 commits into
dcm-project:mainfrom
croadfeldt:content-update

Conversation

@croadfeldt
Copy link
Copy Markdown

Closes PR #7 — this PR replaces the content from PR #7 with the complete, fixed architecture documentation. All review comments from PR #7 are addressed:

Issues fixed from PR #7

  • @jenniferubah, @gabriel-farache: Duplicate data-model/ and architecture/data-model/ directories → eliminated. Only architecture/data-model/ exists now (55 docs with number prefixes).
  • @gabriel-farache: Broken links to foundations, provider-contract, policy-contract, capabilities matrix → all fixed. Zero broken cross-references verified.
  • @gabriel-farache: "links broken, I think all links should be reviewed" → done. Full automated cross-reference scan, zero broken links.
  • @machacekondra: Slug duplicates (e.g., foundations.md duplicating 00-foundations.md) → removed. 53 slug duplicates eliminated.
  • @machacekondra: K8s operator SDK imperative vs declarative concern → addressed in updated dcm-operator-sdk-api.md
  • @pkliczewski: Accreditation flow diagrams, data_classification ownership, GitOps handling → all clarified in updated docs

What changed vs PR #7

  • 116 duplicate files removed (55 flat data-model copy + 53 slug dupes + 8 spec slug dupes)
  • Provider count: 11 → 12 (ITSM Provider added)
  • Policy count: 7 → 8 (ITSM Action added)
  • data_classification enum: 5 → 8 values (phi, pci, classified added)
  • All OpenAPI YAMLs synced with canonical schemas
  • Capabilities Matrix deduplicated (6 duplicate summary tables removed)
  • All Discussion Topics marked Resolved with cross-references

Content structure

content/
├── _index.md
└── docs/
    ├── architecture/
    │   ├── data-model/            ← 55 docs + _index.md (SINGLE location)
    │   └── specifications/        ← 15 specs + 4 OpenAPI YAMLs + _index.md
    ├── capabilities-matrix.md
    ├── DISCUSSION-TOPICS.md
    ├── taxonomy.md
    ├── project-overview.md
    ├── enhancements/
    ├── implementations/
    └── schemas/
static/
└── capabilities/
    └── DCM-Capabilities-Matrix.csv

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…, layers, sovereignty controls.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…e model, posture groups, compliance domain groups

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…ability.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…h level, concurrent rehydration, discovered state retention.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
… deployment bootstrap info.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…alidat, policy review, governance, grouping, relationship role validation, information providers.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…tion, commit log capacity, system initiated records, distributed hash chains.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…on metadata, operational analysis.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…nty pre-filter, audit provenance, universal groups, information providers.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…vice Providers vs producers

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…ovider. Added capabilities static content.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…ification pipeline, and final gap closure

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
… Enforcer, search index, admin api spec.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…, Provider, Policy

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…ers, fix ai prompt to remove confusion on orchestration, consolidate orchestration into policies, give examples.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…ownership to accomodate for cluster as a service.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…ocumentation / specs.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…andling depedencies better, health endpoints.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…cy run applied.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…project, added some details on who,what,why,etc..

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 20000 lines

… 15 specs, 4 OpenAPI)

Addresses all PR dcm-project#7 review comments:
- Eliminated duplicate data-model directories
- Fixed all broken cross-references
- Removed 116 duplicate files
- Updated provider count (12), policy count (8), classification enum (8)
- Synced all OpenAPI YAMLs with canonical schemas

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this file numbered while other in the folder are not?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why were all the enhancements files removed? Is it because in the index there is already a link to the enhancement repo? Or because they were scattered in other places?

→ Fetches dc-us-east-1.yaml Core Layer from GitLab
→ Fetches vm-standard service layer
→ Merges: consumer fields + core layer + service layer + provenance map
→ Publishes assembled payload to Kafka → dcm-policy-engine
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the only reason we need Kafka is for dcm-request-processor-> dcm-policy-engine communication?

Comment thread content/docs/implementations/example-01-summit-demo/architecture-mapping.md Outdated
→ dcm-provider-vm receives DCM VirtualMachine payload

7. dcm-provider-vm naturalizes and executes
→ Translates DCM VirtualMachine → AAP Job Template parameters
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wanna use AAP?

Copy link
Copy Markdown
Author

@croadfeldt croadfeldt Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is that there will be a lot of providers that implement AAP. This is an example of how a provider might be implemented.

8. dcm-provider-vm denaturalizes and callbacks
→ AAP job completes → provider polls AAP for result
→ Translates AAP job result → DCM VirtualMachine realized state
→ POST /api/v1/provider/entities/{entity_uuid}/status to dcm-api-gateway
Copy link
Copy Markdown
Contributor

@machacekondra machacekondra Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we are not interested in status updates? Or why there is so many ways of sending status?

```

**Talking points:**
- Intent State is immutable and stored in GitLab — survives the DC loss
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In above you use API to communicate with DCM, but you say the intent is stored in gitlab? So user is not using git to manage it's infra but API, which is creating a commits in git?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be cleaned up. The intent was to have incoming requests via Git. The use of git internally has been removed and consolidated.

| Level | Required Endpoints | Unlocks |
|-------|-------------------|---------|
| **Level 1** | POST /api/v1/resources, GET /health, callback on completion | Basic provisioning and decommission |
| **Level 2** | + GET /discover, GET /capacity, PATCH /api/v1/resources/{id} | Discovery, drift detection, updates |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand your architecture correctly, you are here missing the most important thing, which is something like /info. This should provide you name of the networks,storage,images,etc. Which will be used during mutation phase of the policy execution. Otherwise how can you have those fields in DCM, by hardcoding policies? IMHO that's unmaintainable.

Comment on lines +176 to +179
Network Port (no deps) ──────────────────────────┐
VM (depends on: Network Port) ────────────────▶ App Deployment complete
OCP Cluster (independent, parallel with VM) ──┘
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you define the dependencies between those? depends_on is not enough, it's says what depends on what, but it doesn't say WHY. That's what is most imporant, because based on WHY you can construct the graph. WHY VM depends on Network Port? What it needs? How do I use it? How do I specify it? Where?

├── requested.yaml
├── assembly-provenance.yaml
├── placement.yaml
└── dependencies.yaml
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How VM could have dependencies, if it's just one resource? What's content of dependencies.yaml?

├── system/
│ ├── core/
│ │ ├── datacenter-layer.yaml
│ │ └── environment-layer.yaml
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you keep those files in sync with real providers data?


---

## 3. Example 1 — VM Provision End-to-End
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please show three-tier app, so we see how the three tier app is defined and the dependecies between the app? Like backend needs IP of database at least.

…in localized repos.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…olicy modifications are checked by policies, OPA policy templating, re-enforce policy triggers based on data, removed needless complexity, internal secrets handling, addressed prior PR comments.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…/ architecture testing.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
…rns as a Service Provider.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants