Repository navigation
feat(descriptions): carry each series' publisher codes - #45
Merged
Merged
Conversation
A card or dataset version description can now state the publisher's
own indicator codes behind its series (World Bank IDS's
DT.DIS.BLAT.CB.CD and the like) as about.source_codes, and each drawn
value its variables declare can carry its own. A code is
{provenance, code, exact?} in Search's order and with its meaning of
exact. A list holds 1 to 50 codes and is left out past that, as
drawn.values is. A computed version and a source card take none, and
a card drawn from a computed version takes no exact code. The
components catalogue manifest carries the schema's new digest.
uniqueItems compares whole entries, so a source_codes list could name the same (provenance, code) twice when only exact differed, and could list codes in any order. check_description now refuses a list under about or on a drawn value that is not strictly ascending by (provenance, code), with the reason source_codes_not_ascending, which covers both. Two vectors pin a duplicate on a drawn value and a reversed about list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Card and dataset version descriptions can now carry the publisher's own indicator codes behind their series, so Studio's More Info can say "World Bank code DT.CUR.EURO.ZS" as Get Data already does from Search's
stat_vars.source_codes. The shape and the meaning ofexactare Search's, so the two surfaces say the same thing.Changes
description.schema.json:$defs.source_code({provenance, code, exact?}),$defs.source_codes(1 to 50, unique, sorted by provenance then code),about.source_codeson a card or a dataset version, and$defs.drawn_value, which lets eachabout.drawnvalue carry the codes its variables declare. A computed version and a source card refuseabout.source_codes, and a card drawn from a computed version refusesexactat both places ($defs.built_from_codes).structure.py:check_descriptionrefuses asource_codeslist, underaboutor on a drawn value, that is not strictly ascending by(provenance, code)(source_codes_not_ascending). JSON Schema'suniqueItemscompares whole entries, so this rule is what stops the same code appearing twice with and withoutexact.description-card-source-codes(accepted),description-card-computed-exact-code-rejected,description-source-codes-empty-rejected,description-source-codes-reversed-rejectedanddescription-drawn-value-source-codes-duplicate-rejected.catalogue/manifest.json: the schema's newreferenced_schemasdigest, which dfo-apis checks when it loads the catalogue.Notes for reviewers
A list past 50 codes is left out, as
drawn.valuesis, so a served list is always complete. A description with no codes omits the member and is unchanged on the wire.description.schema.jsonondevis not inpretty-format-json's form (the hook rewrites 160 untouched lines), so these commits skip that hook and write the additions in the file's inline style; the reformat can go in a PR of its own.structure.pyondevalready failscheck-docstring-first(its constants carry attribute docstrings), so the second commit skips that hook too.Testing:
validate_descriptions_vectors.py(124 vectors), the descriptions (142, at full coverage) and components (314) suites,validate_client_contract.py,tools/tests(624), and ruff, ruff format and ty pass. The dfo-apis composer change that fills the field passes its description, assembler, Datasets, Presentation, live-refresh and contract suites (6,097 passed) against this branch.