Replace free-form tags with a controlled vocabulary, primary category and language - #392
Merged
Conversation
`tags` is currently asked to do two jobs with opposite requirements. The website uses it to build the filter row on scverse.org/packages, which needs a small, stable, coherent vocabulary; it is also the only thing making packages findable by keyword, which needs the opposite — breadth, and a long tail nobody has to spell the same way. Serving both from one free-form field means neither works. Across 118 packages there are 259 distinct tags, 71% of them used exactly once. `single cell` and `single-cell` are separate entries, as are three spellings of spatial omics, so the filter row shows the same concept several times and pads the remainder with `python`. The website has been papering over this with a hardcoded alias map, which is invisible to the people writing the tags and silently rots as new packages arrive. Split the two jobs. `topics` is a required, closed vocabulary of 18 terms that says what a package is for, and drives the website filters. `tags` keeps its current meaning, stays free-form, and is used for search, where incoherence is harmless. Topics for the existing 118 packages were seeded from their descriptions and tags, then corrected package by package: the seeding read "annotated data matrices" as annotation, "integration testing" as integration, and missed cellxgene being a viewer. Enforcement needs no new code. `validate_registry.py` already runs `jsonschema.validate` per package, and an unknown topic produces "'foo' is not one of [...]", so contributors get the whole vocabulary at the point where they are blocked.
Member
Author
|
@grst see scverse/scverse.github.io#254 for some additional context. What is your opinion on this PR and the approach? I wonder whether we should remove free-form tags completely and only allow our controlled vocabulary. I think we could allow people that submit PRs to suggest new terms but generally nudge them to use whatever we'd already have. Thanks! |
Member
Author
|
I think this & the frontend PR are also associated with scverse/scverse.github.io#106 right? |
Contributor
|
We initially didn't control the vocabulary since we didn't know what it should contain. With >100 ecosystem packages, I think we should have a good sample of the "tag universe". Should something new be required, submitters can amend the schema in their PR. I'd just tell an LLM to collapse the currently existing topics, throw out a couple of non-descriptive ones, and enforce a controlled vocabulary in the future. |
Zethson
changed the title
topics vocabulary alongside free-form tagstopics vocabulary
…guage Follows the review: no free-form tags at all, and the vocabulary is collapsed out of the tags that already existed rather than invented. `tags` becomes a 45-term enum, folded from the 250 distinct tags in the registry, with the non-descriptive ones dropped (`python`, `single-cell`, `bioinformatics`, `utilities`, library names). `primary_category` names the one category a package is listed under. Both mirror the tutorial registry, which already solved this, so the two registries share terminology instead of inventing their own. `language` is assumed to be Python unless set, and marks the three R packages. Closes scverse/scverse.github.io#106, closes #299, closes #46.
Zethson
changed the title
topics vocabularyA JSON Schema default is documentation, not behaviour: jsonschema does not apply it, so the field stays absent from packages.json and every consumer has to reimplement the fallback. Stating it on all 118 packages costs one line each and removes that.
Zethson
commented
Aug 13, 2026
| "description": "Language a user writes code in when using the package. Assumed to be Python when omitted.", | ||
| "description": "Language a user writes code in when using the package.", | ||
| "type": "string", | ||
| "default": "Python", |
Member
Author
There was a problem hiding this comment.
We can keep this but I like things complete and explicit.
Zethson
added a commit
to scverse/scverse.github.io
that referenced
this pull request
Aug 13, 2026
scverse/ecosystem-packages#392 replaces free-form tags with a controlled vocabulary, a primary category and a language, so the normalisation this template was doing can go: the alias map, the tag counting, the `>= 3` threshold and the top-12 slice are all deleted. The chip row is now the categories actually in use, biggest first, which is stable across registry updates rather than shifting whenever a package is added. Tags filter from the cards, since 45 of them would not fit in a chip row, and non-Python packages get a language badge. Needs scverse/ecosystem-packages#392 merged and redeployed first. Until then the live packages.json has no primary_category, and the chip row renders empty.
grst
approved these changes
Aug 13, 2026
grst
left a comment
grst
left a comment
Contributor
There was a problem hiding this comment.
one small thing, otherwise LGTM
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
muskanhashim
added a commit
to ianfd/scverse-ecosystem-packages
that referenced
this pull request
Aug 17, 2026
…y, language) Update packages/singlerust/meta.yaml to conform to the schema changes in scverse#392: - map tags to (new) controlled vocabulary (scRNA-seq, preprocessing, differential expression) - add required fields: primary_category: scRNA-seq, language: Rust
This was referenced Aug 17, 2026
Open
Zethson
added a commit
to scverse/scverse.github.io
that referenced
this pull request
Aug 18, 2026
* Redesign the ecosystem package registry as a filterable card grid The registry was a two-column table of name and description, which ignored most of what packages.json actually carries. Each package is now a card showing its logo (or a monogram tile), version, license, description, tags, and a link row built from the registry data: repository, docs, tutorials, PyPI/conda/CRAN/Bioconductor and the first publication DOI. Entries in the core-* categories get a badge so they are distinguishable from community submissions. Filtering is search over name, description and tags plus a row of the twelve most common tags. Registry tags are free-form, so variants such as "single-cell", "single cell" and "singlecell" are folded together through an alias map before they are counted. Tags on a card filter by that tag as well, injecting a chip when it is not one of the top twelve. Along the way: - Wrap the remote fetch in `try`, so a registry outage degrades to a link instead of failing the build. - Match the search query as a substring instead of compiling user input into a RegExp, which threw on input such as `(`. - Collapse duplicate outbound links; many entries point documentation_home and tutorials_home at the same URL. - Drop the <p> wrapping markdownify output, which nested paragraphs. * Pin Netlify deploy previews to the Hugo version production builds with Deploy previews built with Netlify's default Hugo 0.97.2, from 2022, while production builds with `latest`, currently 0.164.0. Previews therefore did not reflect what gets published, and the `try` guard around the registry fetch failed to parse on the older version. Pin previews to 0.164.0, and drop `try`, which needs Hugo 0.141+, in favour of the plain nil check the template used before. The fallback markup stays, so an unreachable registry still renders a link rather than an empty section. * Restore the `try` guard around the registry fetch Now that deploy previews build with the same Hugo as production, the template can use `try` again: an unreachable registry falls back to a link instead of failing the build. Verified by pointing the fetch at a 404 and confirming the page still builds with the fallback markup. Note the resulting minimum Hugo version in the README. * Drop the Hugo version note from the README * Revert the registry fetch to failing the build Wrapping the fetch in `try` traded a loud failure for a silent one: a transient outage would have published the page with all 118 packages replaced by a fallback line, on a green build, until whenever the next deploy happened. Failing the build keeps the last good page live, since the deploy step in gh-pages.yml only runs after a successful build, and puts a red X in front of someone who can act on it. Degrading gracefully is worth revisiting on its own, backed by a committed snapshot of packages.json so an outage means stale data rather than an empty section, but it does not belong in a redesign. * Read the registry's controlled vocabulary instead of normalising tags scverse/ecosystem-packages#392 replaces free-form tags with a controlled vocabulary, a primary category and a language, so the normalisation this template was doing can go: the alias map, the tag counting, the `>= 3` threshold and the top-12 slice are all deleted. The chip row is now the categories actually in use, biggest first, which is stable across registry updates rather than shifting whenever a package is added. Tags filter from the cards, since 45 of them would not fit in a chip row, and non-Python packages get a language badge. Needs scverse/ecosystem-packages#392 merged and redeployed first. Until then the live packages.json has no primary_category, and the chip row renders empty. * One sentence per line in the netlify.toml and template comments * Pin the deploy workflow to the same Hugo as previews `latest` meant a Hugo release could break the build with no change in the repo, and that previews and production drifted apart — which is how previews ended up on 0.97.2 unnoticed. Both are 0.164.0 now, and moving version is a reviewable one-line change. * Stop caching the ecosystem registry JSON between builds Hugo caches remote resources forever by default, and on Netlify it puts that cache in /opt/build/cache, which survives between builds. So the deploy preview kept serving a packages.json fetched before the schema change, and the rebuild the registry triggers when packages.json changes would have been a no-op there. Production was unaffected — the Actions runner starts cold — but the same staleness bit local builds twice while developing this. * Drop the work-in-progress caveat from the ecosystem intro * Sort core packages first and tint them Core here means core-framework and core-datastructure. core-infrastructure is the scverse repos themselves — governance, the website, this registry — so pinning those to the top of a package list would be misleading. * Fix core cards being unstyled and unfilterable Prettier dropped the space inside `class="eco-card{{ if $e.core }} …"`, so the twelve core cards rendered as one bogus `eco-cardeco-card--core` class. They lost the card styling, and because the filter collects `.eco-card`, they were never in the list to hide — every search returned all twelve regardless of the query. Putting the space before the conditional is stable under prettier. Reduce the highlight to just the background wash, and keep the core badge as it was.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes scverse/scverse.github.io#106, closes #299, closes #46.
tags— 45-term enum, folded from the 250 existing tags, dropping the non-descriptive ones (python,single-cell,bioinformatics,utilities, library names). Drives filtering and search.primary_category— the one category a package is listed under, for grouping.language— now mandatory to ensure that the schema is complete and doesn't break in random places.Both vocabularies mirror the tutorial registry, which already has
primary_categoryplus a controlledtagsenum, so the two registries share terminology. New terms get added to the enum in the PR that needs them.This breaks the open PRs by design: #391, #388, #385, #382, #378, #376, #375, #373, #372, #371, #368, #367, #364, #349, #348 and #327 need their
tagsmoved onto the vocabulary and aprimary_categoryadded.