Fully align filter and override configuration formats with APIOps Toolkit

[petehauge]

Summary

The apiops-cli extraction filter and override configuration formats did not fully match the APIOps Toolkit's format, preventing users from reusing their existing configuration.extractor.yaml and configuration.{env}.yaml files unchanged.

Goal: A user with an existing APIOps Toolkit setup should be able to install apiops-cli and use their existing configuration files unchanged.

Status: Fixed in PR #115 (except workspace-scoped filtering/overrides, tracked in #118 and #119).

---

Issues Found & Fixed

1. Filter Config — Filename ✅

Aspect	apiops-cli (before)	APIOps Toolkit	Fix
Filename	configuration.extract.yaml	configuration.extractor.yaml	Renamed across all templates, CI/CD workflows, and documentation

2. Filter Config — Key Names ✅

The entire naming scheme differed. The CLI used a *Names suffix convention while the Toolkit uses bare camelCase plurals.

CLI (before)	Toolkit (target)	Status
apiNames	apis	✅ Fixed — both accepted
productNames	products	✅ Fixed
backendNames	backends	✅ Fixed
namedValueNames	namedValues	✅ Fixed
loggerNames	loggers	✅ Fixed
diagnosticNames	diagnostics	✅ Fixed
tagNames	tags	✅ Fixed
policyFragmentNames	policyFragments	✅ Fixed
gatewayNames	gateways	✅ Fixed
versionSetNames	versionSets	✅ Fixed
groupNames	groups	✅ Fixed
subscriptionNames	subscriptions	✅ Fixed
workspaceNames	workspaces	✅ Fixed
(missing)	policies	✅ Added — ServicePolicy filtering

Legacy *Names keys accepted as backward-compatible aliases with deprecation warnings.

3. Filter Config — Nested API Sub-Resource Filtering ✅

The Toolkit supports nested sub-resource filtering for APIs. The CLI only supported flat string arrays.

# Toolkit format — now supported
apis:
  - apiName1
  - apiName2:
      operations: []
      diagnostics:
        - diagnosticName1
      releases:
        - releaseName1

4. Filter Config — Workspace Sub-Resource Filtering 🔄

The Toolkit supports nested filtering within workspaces (workspace → apis → operations). Parsed but not yet consumed at runtime. Tracked in #119.

5. apimServiceName in Override Config ✅

The Toolkit uses apimServiceName as a root-level field. The CLI now accepts it gracefully — logs info that CLI uses --service-name flag instead, and ignores the field.

6. Override Config — Format Gaps Found & Fixed

The original issue stated overrides were "already aligned" — this turned out to be incorrect. Several gaps existed:

Gap	Status
Typed override interfaces — CLI used NamedValueOverride, BackendOverride etc. with specific typed fields instead of generic passthrough	✅ Replaced with generic OverrideEntry { properties: Record<string, unknown> } matching Toolkit deep-merge behavior
Missing override sections — Only 5 of 14 Toolkit sections were supported	✅ All 14 sections now supported: apis, backends, diagnostics, loggers, namedValues, policies, gateways, versionSets, groups, subscriptions, products, tags, policyFragments, workspaces
Nested child overrides not supported — API child overrides (diagnostics, operations, policies, releases) were silently dropped	✅ 2-level nesting implemented (API→Diagnostics, API→Operations, API→Policy, API→Releases, Product→Policy)
3-level grandchild overrides — API→Operation→Policy overrides not supported	✅ Implemented
Policy overrides not applied during publish — Override merger was never called for policy resources	✅ applyOverrides() added to publishPolicy()
Singleton policy descriptors broken — ServicePolicy (nameParts: []), ApiPolicy (nameParts: [apiName]), ProductPolicy (nameParts: [productName]), ApiOperationPolicy (nameParts: [apiName, opName]) all failed because code assumed name was in nameParts	✅ Fixed — uses fixed singleton name "policy" derived from ARM path metadata
Recursive override parser bug — Dotted section path used for child lookup instead of bare key, breaking operations→policies parsing	✅ Fixed — separate sectionKind parameter for lookup
apiRevision/isCurrent not stripped — Toolkit strips these from API overrides before merge; CLI did not	✅ Fixed — now stripped with warning
Workspace-scoped overrides — Parsed but not applied at runtime	🔄 Tracked in #118

7. Additional Improvements

• Case-insensitive override name matching
• Duplicate override name detection with warnings
• Unknown filter/override key warnings
• Updated apiops init templates with all sections and nested examples
• Updated docs (filtering-resources.md, environment-overrides.md)
• workspaces: [] (empty array) now correctly excludes all workspaces

---

Remaining Work (separate issues)

• Workspace-scoped override properties not applied #118 — Workspace-scoped override properties not applied
• Workspace-scoped sub-resource filtering not implemented #119 — Workspace-scoped sub-resource filtering not implemented

---

Related

• PR Fully align filter and override config formats with APIOps Toolkit #115 — Implementation
• Closes the gap left by fix: align override configuration format with APIOps Toolkit #102 which aligned the list-based override structure