|
| 1 | +# Catalog Structure Analysis: v9-260216-amd64.yaml vs. Subsequent Releases |
| 2 | + |
| 3 | +## Executive Summary |
| 4 | + |
| 5 | +This analysis compares the structure of [`v9-260216-amd64.yaml`](src/mas/devops/data/catalogs/v9-260216-amd64.yaml) (baseline) against subsequent catalog files to identify properties that were added in the February 16, 2026 release but were not carried forward to later releases. |
| 6 | + |
| 7 | +**Analysis Date:** April 15, 2026 |
| 8 | +**Baseline File:** v9-260216-amd64.yaml |
| 9 | +**Comparison Files:** v9-260226, v9-260305, v9-260313, v9-260318, v9-260326, v9-260414 |
| 10 | + |
| 11 | +--- |
| 12 | + |
| 13 | +## Key Findings |
| 14 | + |
| 15 | +### 1. Missing Property: `datarefinery_version` |
| 16 | + |
| 17 | +**Status:** ❌ **MISSING in all subsequent releases** |
| 18 | + |
| 19 | +**Baseline (v9-260216):** |
| 20 | +```yaml |
| 21 | +# Line 42 |
| 22 | +datarefinery_version: 11.0.0+20250513.203727.232 |
| 23 | +``` |
| 24 | +
|
| 25 | +**Context from baseline file (lines 38-42):** |
| 26 | +```yaml |
| 27 | +# TODO: Why is this here, but commented out? |
| 28 | +# datarefinery_build: +20240517.202103.146 |
| 29 | + |
| 30 | +# I have added this as a guess as to the actual version used, we are currently using the wsl_version, but that does not exist for datarefinery |
| 31 | +# See: https://ibm-mas.slack.com/archives/C02PUHKQB5L/p1770849370378689 |
| 32 | +datarefinery_version: 11.0.0+20250513.203727.232 |
| 33 | +``` |
| 34 | +
|
| 35 | +**Impact:** This property was explicitly added to address a known issue where `wsl_version` was being used for Data Refinery, but Data Refinery doesn't have a WSL version. The Slack reference indicates this was a deliberate fix. |
| 36 | + |
| 37 | +**Subsequent files:** All files from v9-260226 onwards only have the commented-out `datarefinery_build` line and do NOT include the `datarefinery_version` property. |
| 38 | + |
| 39 | +--- |
| 40 | + |
| 41 | +## Detailed Comparison |
| 42 | + |
| 43 | +### Properties Present in v9-260216 (Baseline) |
| 44 | + |
| 45 | +The baseline file contains the following structure with **comprehensive documentation**: |
| 46 | + |
| 47 | +#### Cloud Pak for Data Dependencies Section |
| 48 | +```yaml |
| 49 | +# Dependencies - Cloud Pak for Data |
| 50 | +# ----------------------------------------------------------------------------- |
| 51 | +cpd_product_version_default: 5.2.0 |
| 52 | +
|
| 53 | +ibm_licensing_version: 4.2.17 |
| 54 | +common_svcs_version: 4.13.0 |
| 55 | +common_svcs_version_1: 4.11.0 |
| 56 | +cp4d_platform_version: 5.2.0+20250709.170324 |
| 57 | +ibm_zen_version: 6.2.0+20250530.152516.232 |
| 58 | +wsl_version: 11.0.0+20250521.202913.73 |
| 59 | +wsl_runtimes_version: 11.0.0+20250515.090949.21 |
| 60 | +wml_version: 11.0.0+20250530.193146.282 |
| 61 | +spark_version: 11.0.0+20250604.163055.2097 |
| 62 | +cognos_version: 28.0.0+20250515.175459.10054 |
| 63 | +
|
| 64 | +postgress_version: 5.16.0+20250827.110911.2626 |
| 65 | +ccs_build: 11.0.0+20250605.130237.468 |
| 66 | +ccs_extras_version: 11.0.0 |
| 67 | +
|
| 68 | +elasticsearch_version: 1.1.2667 |
| 69 | +opensearch_version: 1.1.2494 |
| 70 | +
|
| 71 | +datarefinery_version: 11.0.0+20250513.203727.232 # ⚠️ MISSING IN LATER FILES |
| 72 | +
|
| 73 | +events_version: 5.0.1 |
| 74 | +``` |
| 75 | + |
| 76 | +#### Db2u Dependencies Section |
| 77 | +```yaml |
| 78 | +# Dependencies - Db2u |
| 79 | +# ----------------------------------------------------------------------------- |
| 80 | +db2u_version: 7.3.1+20250821.161005.16793 |
| 81 | +db2u_extras_version: 1.0.6 |
| 82 | +db2u_filter: db2 |
| 83 | +db2_channel_default: v110509.0 |
| 84 | +``` |
| 85 | + |
| 86 | +#### Other Dependencies |
| 87 | +```yaml |
| 88 | +# Dependencies - CouchDb |
| 89 | +couchdb_version: 1.0.13 |
| 90 | +
|
| 91 | +# Dependencies - Minio |
| 92 | +minio_version: RELEASE.2025-06-13T11-33-47Z |
| 93 | +
|
| 94 | +# Dependencies - MongoDB |
| 95 | +mongo_extras_version_default: 8.0.17 |
| 96 | +mongo_extras_version_4: 4.4.21 |
| 97 | +mongo_extras_version_5: 5.0.23 |
| 98 | +mongo_extras_version_6: 6.0.12 |
| 99 | +mongo_extras_version_7: 7.0.23 |
| 100 | +mongo_extras_version_8: 8.0.17 |
| 101 | +
|
| 102 | +# Dependencies - Amlen |
| 103 | +amlen_extras_version: 1.1.3 |
| 104 | +
|
| 105 | +# Dependencies - Suite License Service |
| 106 | +sls_version: 3.12.5 |
| 107 | +
|
| 108 | +# Dependencies - Truststore Manager |
| 109 | +tsm_version: 1.7.2 |
| 110 | +
|
| 111 | +# Dependencies - Data Dictionary |
| 112 | +dd_version: 1.1.21 |
| 113 | +``` |
| 114 | + |
| 115 | +### Structural Changes in Subsequent Files |
| 116 | + |
| 117 | +Starting with **v9-260226**, the file structure was reorganized: |
| 118 | + |
| 119 | +1. **Section headers were simplified** - removed detailed subsection comments |
| 120 | +2. **Properties were reordered** - dependencies are no longer grouped by product category |
| 121 | +3. **New properties were added:** |
| 122 | + - `uds_version: 2.0.12` |
| 123 | + - `uds_extras_version: 1.5.0` |
| 124 | + - `appconnect_version: 6.2.0` |
| 125 | + |
| 126 | +4. **The `datarefinery_version` property was removed** despite the baseline having explicit documentation about why it was needed |
| 127 | + |
| 128 | +--- |
| 129 | + |
| 130 | +## Comparison Table: Property Presence |
| 131 | + |
| 132 | +| Property | v9-260216 | v9-260226+ | Notes | |
| 133 | +|----------|-----------|------------|-------| |
| 134 | +| `datarefinery_version` | ✅ Present | ❌ Missing | **Critical: Explicitly added to fix WSL version issue** | |
| 135 | +| `uds_version` | ❌ Missing | ✅ Present | New in v9-260226 | |
| 136 | +| `uds_extras_version` | ❌ Missing | ✅ Present | New in v9-260226 | |
| 137 | +| `appconnect_version` | ❌ Missing | ✅ Present | New in v9-260226 | |
| 138 | +| Section organization | Detailed | Simplified | Structure changed | |
| 139 | +| TODO comments | Multiple | Fewer | Documentation reduced | |
| 140 | + |
| 141 | +--- |
| 142 | + |
| 143 | +## Documentation Quality Comparison |
| 144 | + |
| 145 | +### v9-260216 (Baseline) - Rich Documentation |
| 146 | +The baseline file includes: |
| 147 | +- **Detailed section headers** for each dependency category |
| 148 | +- **Inline comments** explaining version choices |
| 149 | +- **TODO comments** highlighting areas needing attention |
| 150 | +- **External references** (GitHub links, Slack conversations) |
| 151 | +- **Explicit reasoning** for property additions |
| 152 | + |
| 153 | +Example: |
| 154 | +```yaml |
| 155 | +# Dependencies - Cloud Pak for Data |
| 156 | +# ----------------------------------------------------------------------------- |
| 157 | +cpd_product_version_default: 5.2.0 |
| 158 | +
|
| 159 | +ibm_licensing_version: 4.2.17 # Operator version 4.2.14 (https://github.com/IBM/cloud-pak/tree/master/repo/case/ibm-licensing) |
| 160 | +common_svcs_version: 4.13.0 # Operator version 4.13.0 (https://github.com/IBM/cloud-pak/tree/master/repo/case/ibm-cp-common-services) |
| 161 | +common_svcs_version_1: 4.11.0 # TODO: Do we really still need to mirror two different versions of common services? If so, why? |
| 162 | +``` |
| 163 | + |
| 164 | +### v9-260226+ - Simplified Documentation |
| 165 | +Later files have: |
| 166 | +- **Generic section header** ("Dependencies") |
| 167 | +- **Fewer inline comments** |
| 168 | +- **Less context** about version choices |
| 169 | +- **Minimal TODO comments** |
| 170 | + |
| 171 | +Example: |
| 172 | +```yaml |
| 173 | +# Dependencies |
| 174 | +# ----------------------------------------------------------------------------- |
| 175 | +ibm_licensing_version: 4.2.17 # Operator version 4.2.14 (https://github.com/IBM/cloud-pak/tree/master/repo/case/ibm-licensing) |
| 176 | +common_svcs_version: 4.13.0 # Operator version 4.13.0 (https://github.com/IBM/cloud-pak/tree/master/repo/case/ibm-cp-common-services) |
| 177 | +common_svcs_version_1: 4.11.0 # Additional version 4.11.0 |
| 178 | +``` |
| 179 | + |
| 180 | +--- |
| 181 | + |
| 182 | +## Recommendations |
| 183 | + |
| 184 | +### 1. Restore `datarefinery_version` Property (HIGH PRIORITY) |
| 185 | + |
| 186 | +**Action Required:** Add `datarefinery_version` back to all catalog files from v9-260226 onwards. |
| 187 | + |
| 188 | +**Rationale:** |
| 189 | +- The baseline file explicitly documents why this property was added |
| 190 | +- It addresses a known issue where WSL version was incorrectly used for Data Refinery |
| 191 | +- The Slack reference indicates this was a deliberate architectural decision |
| 192 | +- Removing it may cause Data Refinery mirroring issues |
| 193 | + |
| 194 | +**Suggested Implementation:** |
| 195 | +```yaml |
| 196 | +# After line 40 in subsequent files, add: |
| 197 | +datarefinery_version: 11.0.0+20250513.203727.232 # Data Refinery version (see v9-260216 for context) |
| 198 | +``` |
| 199 | + |
| 200 | +### 2. Restore Documentation Context (MEDIUM PRIORITY) |
| 201 | + |
| 202 | +**Action Required:** Restore the detailed section headers and TODO comments from v9-260216. |
| 203 | + |
| 204 | +**Rationale:** |
| 205 | +- The baseline file contains valuable context about version choices |
| 206 | +- TODO comments highlight areas needing attention |
| 207 | +- External references (GitHub, Slack) provide traceability |
| 208 | +- Future maintainers will benefit from this documentation |
| 209 | + |
| 210 | +**Suggested Sections to Restore:** |
| 211 | +```yaml |
| 212 | +# Dependencies - Cloud Pak for Data |
| 213 | +# ----------------------------------------------------------------------------- |
| 214 | +
|
| 215 | +# Dependencies - Db2u |
| 216 | +# ----------------------------------------------------------------------------- |
| 217 | +
|
| 218 | +# Dependencies - CouchDb |
| 219 | +# ----------------------------------------------------------------------------- |
| 220 | +
|
| 221 | +# Dependencies - Minio |
| 222 | +# ----------------------------------------------------------------------------- |
| 223 | +
|
| 224 | +# Dependencies - MongoDB |
| 225 | +# ----------------------------------------------------------------------------- |
| 226 | +
|
| 227 | +# Dependencies - Amlen |
| 228 | +# ----------------------------------------------------------------------------- |
| 229 | +
|
| 230 | +# Dependencies - Suite License Service |
| 231 | +# ----------------------------------------------------------------------------- |
| 232 | +
|
| 233 | +# Dependencies - Truststore Manager |
| 234 | +# ----------------------------------------------------------------------------- |
| 235 | +
|
| 236 | +# Dependencies - Data Dictionary |
| 237 | +# ----------------------------------------------------------------------------- |
| 238 | +``` |
| 239 | + |
| 240 | +### 3. Establish Property Tracking Process (MEDIUM PRIORITY) |
| 241 | + |
| 242 | +**Action Required:** Create a process to track property additions/removals across catalog versions. |
| 243 | + |
| 244 | +**Suggested Approach:** |
| 245 | +1. Maintain a "catalog schema" document listing all valid properties |
| 246 | +2. Require justification for property removals |
| 247 | +3. Use automated diff checking between catalog versions |
| 248 | +4. Document breaking changes in release notes |
| 249 | + |
| 250 | +### 4. Review TODO Comments (LOW PRIORITY) |
| 251 | + |
| 252 | +The baseline file contains several TODO comments that should be addressed: |
| 253 | + |
| 254 | +```yaml |
| 255 | +# Line 20: TODO: Do we really still need to mirror two different versions of common services? |
| 256 | +# Line 33: TODO: If this is only used in CPD 5.1.3, we shouldn't need this in this catalog metadata file |
| 257 | +# Line 37: TODO: Why is this here, but commented out? |
| 258 | +# Line 44: TODO: Why is this here, there is no evidence that it is being used in image mirroring? |
| 259 | +# Line 70: TODO: We probably don't need to keep carrying forward the now unsupported versions |
| 260 | +# Line 158: TODO: What is this? It almost certainly should not be here. |
| 261 | +``` |
| 262 | + |
| 263 | +--- |
| 264 | + |
| 265 | +## Impact Assessment |
| 266 | + |
| 267 | +### Critical Impact |
| 268 | +- **`datarefinery_version` removal:** May cause Data Refinery mirroring to fail or use incorrect versions |
| 269 | + |
| 270 | +### Medium Impact |
| 271 | +- **Documentation loss:** Makes maintenance harder for future developers |
| 272 | +- **Context loss:** Removes rationale for version choices |
| 273 | + |
| 274 | +### Low Impact |
| 275 | +- **Section reorganization:** Cosmetic change, but reduces readability |
| 276 | + |
| 277 | +--- |
| 278 | + |
| 279 | +## Conclusion |
| 280 | + |
| 281 | +The v9-260216-amd64.yaml file introduced important structural improvements and the critical `datarefinery_version` property. However, subsequent releases (v9-260226 onwards) did not carry forward this property, potentially reintroducing the issue it was meant to solve. |
| 282 | + |
| 283 | +**Primary Action Item:** Restore `datarefinery_version` to all catalog files from v9-260226 onwards to maintain consistency with the baseline structure and prevent potential Data Refinery mirroring issues. |
| 284 | + |
| 285 | +--- |
| 286 | + |
| 287 | +## Appendix: File Versions Analyzed |
| 288 | + |
| 289 | +| File | Date | Status | |
| 290 | +|------|------|--------| |
| 291 | +| v9-260216-amd64.yaml | Feb 16, 2026 | ✅ Baseline (correct structure) | |
| 292 | +| v9-260226-amd64.yaml | Feb 26, 2026 | ❌ Missing `datarefinery_version` | |
| 293 | +| v9-260305-amd64.yaml | Mar 5, 2026 | ❌ Missing `datarefinery_version` | |
| 294 | +| v9-260313-amd64.yaml | Mar 13, 2026 | ❌ Missing `datarefinery_version` | |
| 295 | +| v9-260318-amd64.yaml | Mar 18, 2026 | ❌ Missing `datarefinery_version` | |
| 296 | +| v9-260326-amd64.yaml | Mar 26, 2026 | ❌ Missing `datarefinery_version` | |
| 297 | +| v9-260414-amd64.yaml | Apr 14, 2026 | ❌ Missing `datarefinery_version` | |
| 298 | + |
| 299 | +--- |
| 300 | + |
| 301 | +**Generated:** April 15, 2026 |
| 302 | +**Analyst:** Bob (Planning Mode) |
0 commit comments