Add a scheduled model-watch check - #8
Merged
Merged
Conversation
Every model ID here is a string written down once that the provider can retire underneath us, and that has now happened three times unnoticed: the chat default was shut down in March and found in July, the video default had been dead a month, and the image default had seventeen days left. Weekly, this asks the provider what it currently serves and fails when a live ID has gone or when a deprecated one is still serving as a default. On failure it opens a single issue rather than one per week. Deliberately outside the test suite: it talks to the network, so it must never be able to fail a pull request. An unreachable provider exits clean rather than being reported as a mass retirement.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three separate audits have now found the same bug in a different place: a model ID written down once, retired by the provider, noticed by us much later.
gemini-3-pro-previewwas shut down 9 March and found in July.startVideo()defaulted toveo-3.0, dead since 30 June.generateImage()defaulted to Imagen, which dies 17 August.This asks Google weekly what it actually serves, and fails when either:
@deprecated, orOn failure it opens one issue, and will not open another while that one is open.
It lives outside the test suite on purpose: it makes a network call, so it must never be able to fail a PR. An unreachable endpoint or an empty list exits clean, because our own connectivity problem is not evidence of a retirement.
It will fail on its first run, correctly:
IMAGEN_4_FASTis deprecated and still the image default. That is the real 17-day deadline, and an issue tracking it is the point.