model_specs: declare every file the safetensors sources require - #550
Merged
0xShug0 merged 1 commit intoSep 15, 2026
Merged
Conversation
Six safetensors packages list fewer files than their own source needs, so they
install cleanly and then fail at load with
missing model package file '<id>': <models>/...
`files` is an explicit list rather than a filter -- the package manager iterates
it for snapshot downloads -- so a downloader has no way to discover the rest.
`optional_files` and `optional_tensors` exist for genuinely optional resources
and load through add_optional_resource_map; everything in `files` and `tensors`
is required and throws at package.cpp:283 when absent.
chatterbox_safetensors requires 9, declared 4
index_tts2_safetensors requires 20, declared 3
omnivoice_safetensors requires 7, declared 3
qwen3_tts_1_7b_base_safetensors requires 8, declared 6
seed_vc_mlx_safetensors requires 28, declared 3
supertonic_3_safetensors requires 13, declared 3
Every file added here is already published, at exactly the path the spec
declares, in the repository the package already downloads from -- so this is a
declaration fix and needs no new hosting. Existing entries keep their order and
the new ones follow in the order the source lists them.
voxcpm2_safetensors has the same shape but is deliberately left alone: it also
requires `audiovae.safetensors`, which OpenBMB/VoxCPM2 does not publish, so
completing its `files` list would not make it loadable and the gap there is
hosting rather than declaration.
Found by checking, for every package, that each `model:`-rooted path in its
format's source appears in the package's `files`. Worth having as a build-time
check -- with the caveat that it has to be format-aware, since a GGUF source
names the same config files and the GGUF embeds them, so a format-blind version
reports three quarters of the catalogue.
Owner
|
@christopherthompson81 PR merged and thanks for the consistency pass! The inconsistency is mainly because we decided to switch to GGUF as the default before implementing spec v1. Safetensors sources are no longer required and many models don't handle safetensor weights, so many of them may just be placeholders in the specs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six safetensors packages declare fewer files than their own source requires. Each installs cleanly and then fails at load:
filesis an explicit list rather than a filter — the package manager iteratespackage.filesfor snapshot kinds — so a downloader has no way to discover the rest. CONTRIBUTING's "a package should not advertise a model that the CLI/server cannot actually load" is the rule being broken.seed_vc_mlx_safetensorsindex_tts2_safetensorssupertonic_3_safetensorschatterbox_safetensorsqwen3_tts_1_7b_base_safetensorsomnivoice_safetensorsNo new hosting
Every file added here is already published, at exactly the path the spec declares, in the repository the package already downloads from. I checked all 37 missing paths against the HuggingFace API; 36 were present, and the one that was not is why
voxcpm2is left out (below).So this is a declaration fix. Existing entries keep their order and the new ones follow in the order the source lists them, so the diff is additions only.
What this costs to download
These packages get materially bigger, because the engine requires the files and they were simply never fetched. Sizes from the HuggingFace blob API:
seed_vc_mlx_safetensorsindex_tts2_safetensorschatterbox_safetensorsomnivoice_safetensorsqwen3_tts_1_7b_base_safetensorssupertonic_3_safetensorsThe comparison is a little unfair to the "before" column — those packages do not load at all today, so the smaller number buys nothing. But
seed_vcgoing 712 MB → 7.0 GB is a real change in what a user gets when they click install, and worth a deliberate decision rather than a side effect of this PR.Which raises a question I cannot answer from outside.
seed_vc's safetensors source lists the v1 stack (v1/svc,v1/whisper_bigvgan,v1/xlsr_hift) and the v2 stack, pluswhisper-small,hubert-large-ll60kandwav2vec2-xls-r-300m— all in requiredfiles/tensors, none inoptional_files/optional_tensors. If a given conversion route only ever touches one of those paths, the better fix is to move the unused ones to the optional maps rather than to declare them all here, and this PR would shrink to the genuinely-required set.I have declared exactly what the source currently marks as required, because that is what the loader enforces and what makes the packages load today. If some of those should be optional instead, say which and I will redo it that way — that is a statement about the model, not something the spec makes visible.
Two judgement calls, flagged rather than made quietly
voxcpm2_safetensorsis deliberately untouched. It has the same shape — 6 required, 4 declared — but one of the two missing files,audiovae.safetensors, is not published byOpenBMB/VoxCPM2at all. Adding onlyspecial_tokens_map.jsonwould leave the package still failing, one clue poorer. That one looks like a hosting gap rather than a declaration gap, and I did not want to paper over it.index_tts2goes 3 → 21, not 3 → 20. It already declaredbpe.model, which the safetensors source names in neitherfilesnortensors. I kept it: it ships today and is published in the repo, and removing a file a package already fetches is a different decision from adding ones it needs.The check that found it
For each package, take the source whose
formatmatches, collect everymodel:-rooted path in that source'sfilesandtensors, and assert each appears in the package'sfiles(allowing forstrip_prefix). Worth having at build time — it is the same shape as the consistency check that would have caught this class originally.Two things it has to get right, both found by getting them wrong first:
filesandtensors.optional_filesandoptional_tensorsare separate keys loaded viaadd_optional_resource_map(src/framework/model_spec/package.cpp:378,390); only the required maps throw, atpackage.cpp:283. Including the optional ones gives false positives..gguf. Run the check format-blind and it reports 173 packages — three quarters of the catalogue — every one of them false.With both accounted for it reports exactly the seven packages above and nothing else across all 237.
What is not verified
The evidence here is static: the spec says the file is required, and the repo publishes it at that path. I have not installed and loaded each patched package end to end. Happy to do that for any of them if you would rather see it before merging —
supertonicis the cheapest at ~414 MB.