Skip to content

Make model-watch keyless, and fix the three things it found #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MarcelloDuarte merged 1 commit into from
Jul 31, 2026
Merged

Make model-watch keyless, and fix the three things it found #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 18 additions & 32 deletions .github/model-watch.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,29 +64,22 @@
exit(2);
}

$key = getenv('GEMINI_API_KEY');
$url = 'https://generativelanguage.googleapis.com/v1beta/models?pageSize=1000'
. ($key === false || $key === '' ? '' : '&key=' . urlencode($key));

$response = @file_get_contents($url);
// Google's published model list, in plain text and with no credential. Deliberately not the
// v1beta models endpoint: that needs an API key, and a watchdog on a public repository must not
// require anyone to attach a billable key to it.
$response = @file_get_contents('https://ai.google.dev/gemini-api/docs/models.md.txt');

if ($response === false) {
fwrite(STDERR, "Could not reach the model list. Not treating an unreachable provider as a retirement.\n");
fwrite(STDERR, "Could not reach the published model list. Our connectivity is not evidence of a retirement.\n");

exit(0);
}

/** @var array{models?: list<array{name?: string}>} $decoded */
$decoded = json_decode($response, true, 512, JSON_THROW_ON_ERROR);
$served = [];

foreach ($decoded['models'] ?? [] as $model) {
// Names come back as "models/gemini-3.6-flash".
$served[] = str_replace('models/', '', (string) ($model['name'] ?? ''));
}
preg_match_all('/\b(?:gemini|imagen|veo)-[0-9][0-9a-z.\-]*/i', $response, $found);
$served = array_values(array_unique($found[0]));

if ($served === []) {
fwrite(STDERR, "The model list came back empty, which is more likely our problem than a mass retirement.\n");
if (count($served) < 5) {
fwrite(STDERR, "The published list parsed to almost nothing, which means the page format changed.\n");

exit(0);
}
Expand All @@ -109,30 +102,23 @@
echo "Already deprecated, so expected to be gone:\n " . implode("\n ", $missingDeprecated) . "\n\n";
}

// A default with a known shutdown date is a scheduled outage for every caller who does not pass a
// model. It is the exact shape of bug that has hit this package three times, so it fails too.
$deprecatedDefaults = [];

// Reported, but never a failure on its own. We already know about it, and a scheduled job that
// stays red on a known issue becomes wallpaper: the next real failure gets ignored with it.
foreach ($shipped as $name => $entry) {
if ($entry['deprecated'] && $entry['isDefault']) {
$deprecatedDefaults[] = sprintf('%s (%s)', $name, $entry['value']);
printf("Note: %s (%s) is deprecated and still used as a default. Known, and tracked separately.\n\n", $name, $entry['value']);
}
}

if ($missingLive === [] && $deprecatedDefaults === []) {
echo "Every live model ID is still served, and no default is deprecated.\n";
if ($missingLive === []) {
echo "Every live model ID still appears in Google's published list.\n";

exit(0);
}

if ($missingLive !== []) {
echo "NO LONGER SERVED, and not marked deprecated:\n " . implode("\n ", $missingLive) . "\n\n";
echo "Mark each one @deprecated with its shutdown date, and repoint anything used as a default.\n\n";
}

if ($deprecatedDefaults !== []) {
echo "DEPRECATED, AND STILL USED AS A DEFAULT:\n " . implode("\n ", $deprecatedDefaults) . "\n\n";
echo "This breaks every caller who passes no model, on the day the provider switches it off.\n";
}
// Failing only on the unacknowledged case makes this self-clearing: mark the constant
// @deprecated, and the job goes green again on the next run.
echo "NO LONGER PUBLISHED, and not marked deprecated:\n " . implode("\n ", $missingLive) . "\n\n";
echo "Mark each one @deprecated with its shutdown date, and repoint anything used as a default.\n";

exit(1);
7 changes: 3 additions & 4 deletions .github/workflows/model-watch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ name: Model watch
# underneath us. That has now happened repeatedly and silently: the chat default was shut down in
# March and nobody noticed until July, and the video default had been dead for a month.
#
# This job asks the provider what it currently serves and opens an issue when something we ship no
# longer appears. It needs no API key: the model-list endpoint is public.
# This job compares what we ship against Google's published model list and opens an issue when
# something has gone. It reads the public documentation rather than the API, so it needs no
# credential: a watchdog on a public repository must not depend on someone attaching a billable key.

on:
schedule:
Expand Down Expand Up @@ -33,8 +34,6 @@ jobs:
- name: Check models
id: check
run: php .github/model-watch.php
env:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}

- name: Open an issue when something we ship has gone
if: failure()
Expand Down
6 changes: 4 additions & 2 deletions src/GoogleProvider.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,13 +91,15 @@ class GoogleProvider implements ProviderInterface, ImageProviderInterface, Embed
/** @deprecated Shut down 9 March 2026; the alias now redirects to gemini-3.1-pro-preview. */
public const MODEL_3_0_PRO = 'gemini-3-pro-preview';
public const MODEL_3_FLASH = 'gemini-3-flash-preview';
public const MODEL_3_PRO_IMAGE = 'gemini-3-pro-image-preview';
public const MODEL_3_PRO_IMAGE = 'gemini-3-pro-image';
public const MODEL_2_5_PRO = 'gemini-2.5-pro';
public const MODEL_2_5_FLASH = 'gemini-2.5-flash';
public const MODEL_2_5_FLASH_LITE = 'gemini-2.5-flash-lite';
public const MODEL_2_0_FLASH = 'gemini-2.0-flash';
public const MODEL_2_0_FLASH_LITE = 'gemini-2.0-flash-lite';
/** @deprecated Retired; no longer published by Google. */
public const MODEL_1_5_PRO = 'gemini-1.5-pro';
/** @deprecated Retired; no longer published by Google. */
public const MODEL_1_5_FLASH = 'gemini-1.5-flash';

// Image generation. Imagen is retired as a product line; these replace it.
Expand All @@ -116,7 +118,7 @@ class GoogleProvider implements ProviderInterface, ImageProviderInterface, Embed

// Veo model aliases for video generation
public const MODEL_VEO_3_1 = 'veo-3.1-generate-preview';
public const MODEL_VEO_3_1_FAST = 'veo-3.1-fast-generate-preview';
public const MODEL_VEO_3_1_LITE = 'veo-3.1-lite-generate-preview';
/** @deprecated Shut down 30 June 2026; requests fail. Use MODEL_VEO_3_1. */
public const MODEL_VEO_3 = 'veo-3.0-generate-001';
/** @deprecated Shut down 30 June 2026; requests fail. Use MODEL_VEO_3_1. */
Expand Down
Loading