Skip to content

CMR-10964 Change schemas to accommodate better collection quality information metadata#2448

Open
eereiter wants to merge 5 commits into
masterfrom
CMR_10964_umm_c
Open

CMR-10964 Change schemas to accommodate better collection quality information metadata#2448
eereiter wants to merge 5 commits into
masterfrom
CMR_10964_umm_c

Conversation

@eereiter

Copy link
Copy Markdown
Contributor

Overview

What is the objective?

New UMM-C version 1.18.6 to change the quality metadata field from a string to an object that contains more detailed information.

What are the changes?

Changed UMM-C, DIF 10, ECHO 10 schemas. Added translations to all of the supported schemas. Added migration up and down for UMM-C. created and updated tests.

What areas of the application does this impact?

UMM-C, DIF 10, ECHO 10, translations, UMM-C migrations. These fields are not searched so the changes do not include search or indexer code.

Required Checklist

  • New and existing unit and int tests pass locally and remotely
  • clj-kondo has been run locally and all errors in changed files are corrected
  • I have commented my code, particularly in hard-to-understand areas
  • I have made changes to the documentation (if necessary)
  • My changes generate no new warnings

Additional Checklist

  • I have removed unnecessary/dead code and imports in files I have changed
  • I have cleaned up integration tests by doing one or more of the following:
    • migrated any are2 tests to are3 in files I have changed
    • de-duped, consolidated, removed dead int tests
    • transformed applicable int tests into unit tests
    • reduced number of system state resets by updating fixtures. Ex) (use-fixtures :each (ingest/reset-fixture {})) to be :once instead of :each

<action>changed</action>
<src>added mixed case to field with new fields with text and enum on 2026-01</src>
<since>10.2.g</since>
<note>If the sub-fields are used only the sub-fields should have text and there should be no text between the main field and sub-fields. Allow the user to add text to the main field if only the main field is used but no sub-fields should be used to define the quality.</note>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This statement is confusing


Example Usage:
* Associated_DOIs/Type = 'Other'
* Associcated_DOIs/Type = 'Other'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was this changed? Associated is spelled wrong in the new update

(m-spec/update-version c :collection "1.18.5")))

(defmethod interface/migrate-umm-version [:collection "1.18.5" "1.18.6"]
;; Converts the 1.18.5 single string into the 1.18.6 QualityType's Summary."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should you mention that QualityType Summary is a map? Such as you did with the above 1.18.16 > 1.18.5 translation

;; quality or quality flags - both validated or invalidated; 4) recognized or potential problems
;; with quality; 5) established quality control mechanisms; and 6) established quantitative
;; quality measurements.
;; Elements describing quality of the collection data. The description may include: 1) Known

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you put the list on new lines.
1)..
2)...
for easier readability

"Returns the expected DOI."
[quality]
(when (:Summary quality)
(assoc quality :Summary (string/trim(:Summary quality)))))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

space between trim and (:Summary...

[:S3CredentialsAPIEndpoint (:S3CredentialsAPIEndpoint direct-dist-info)]
[:S3CredentialsAPIDocumentationURL (:S3CredentialsAPIDocumentationURL direct-dist-info)]])]))
[:S3CredentialsAPIDocumentationURL (:S3CredentialsAPIDocumentationURL direct-dist-info)]])
(generate-quality c)])) No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline

(let [doc "<Collection><Quality><Summary>Only summary here.</Summary></Quality></Collection>"
expected {:Summary "Only summary here."}
parsed (#'echo10/parse-echo10-xml test-context doc false)]
(is (= expected (:Quality parsed))))))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR was quite large. Was there any way you could have broken it down to smaller PRs for easier (and most likely higher quality) reviews?

<Description>Some description</Description>
</Related_URL>
</DIF>"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make sure these tests actually run in our build

{:URL (str "https://cdn.earthdata.nasa.gov/umm/collection/v"
umm-spec-versioning/current-collection-version),
:Name "UMM-C"
:Version umm-spec-versioning/current-collection-version})})))) No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new line

[tag text])))))

(defn parse-quality
"Parses the 1.18.6 Quality object tree out of the existing ECHO 10 XSD configuration."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we put the version num here, that means every version update we need to update this comment. I would remove the version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants