Expose safe Grok package quota status - #7004
Closed
think-back wants to merge 2968 commits into
Closed
Conversation
fix(website): refine DeepSeek landing copy
…through Final-review follow-up: Grok upstream is Responses-only, so raw chat passthrough would always fail with protocol mismatch, even when the global or per-channel pass-through switch is enabled. Extend the existing Codex guard in shouldPassThroughTextRequest to also cover APITypeGrokSubscription, mirroring the Claude-side bridge guard in claude_handler.go. Responses-mode passthrough stays unaffected.
Final-review follow-up: grokAuthHTTPDoer defaulted to http.DefaultClient, which has no Timeout, so a hung token endpoint would block the admin API goroutine indefinitely. Replace it with a dedicated 30s-timeout client; test stub injection is unaffected. Regression test pins that the default doer is not http.DefaultClient and carries a positive Timeout.
Final-review follow-up: the non-200 fallback in aggregateGrokResponsesToChat read the upstream body without any limit and inlined the full text into the client-facing error. Wrap the read in io.LimitReader(maxTokenResponseBytes) matching the refresh.go / token_exchange.go precedent, and mark the error message '(truncated)' when the cap is hit so a clipped payload cannot be mistaken for a complete JSON document.
Final-review follow-up: the comment on GrokChannelState claimed secrets live in an encrypted Channel.Key, which is not true. The credential JSON is stored in plaintext in Channel.Key, protected by DB access control exactly like every other channel key in this repo; only the PKCE verifier in GrokAuthFlow.EncryptedVerifier uses authenticated encryption. Comment-only change, no behavior touched.
…arser drift warning
…ge in prompt gallery
Keep channel 156 on the shared Flatkey asset pipeline while isolating upstream bindings by gateway, group, and credential. Constraint: reuse existing assets and asset_bindings without exposing upstream groups. Rejected: channel-id or type hard-coding | migration would not survive channel replacement. Confidence: high Scope-risk: broad Directive: keep provider selection channel-configured and credential-scoped. Tested: git diff --check Not-tested: implementation not started
Break the approved design into test-first resolver, provider, rewrite, admin, and verification slices. Constraint: implementation must reuse the existing asset library and bindings. Rejected: an unplanned second asset subsystem | would violate the approved contract. Confidence: high Scope-risk: broad Directive: keep each slice independently testable before integration. Tested: git diff --check Not-tested: implementation not started
Route explicit seedance_proxy channels through the shared Flatkey asset flow while preserving legacy provider behavior and administrator-only settings. Record the verification evidence and fail-closed configuration boundary. Constraint: upstream group identifiers, credentials, and signed source URLs remain internal; no staging promotion is authorized. Rejected: customer-managed asset groups and direct signed-URL submission | both violate the provider-neutral shared-group contract. Confidence: high Scope-risk: broad Directive: rotate the exposed API key before any staging configuration and keep the shared group operations-managed. Tested: focused Go suites, frontend channel-form tests (27 pass), frontend typecheck, diff and secret scans. Not-tested: full service suite due the documented parallel SQLite timeout; live staging acceptance.
…over 404/duplicate-slug
…ing via include array not boolean
…Compact to CLI proxy /v1/responses in GetRequestURL
…quest builds compact summary turn via BuildCompactTurn
…+ clean-room grokCompactResponseHandler (upstream Responses JSON -> OpenAI compaction item)
…-primary fix(website): keep contact sales primary when signed in
* feat(auth): add configurable registration email blacklist * fix(auth): fail closed on invalid registration blacklist regex Address review feedback on PR #754: IsEmailBlacklisted previously ignored MatchString errors and continued to the next pattern, which fails open if a pattern becomes invalid (e.g. hand-edited config or stale data). Now treats a match error as a block. * fix(auth): log invalid blacklist pattern instead of blocking registration Revert the fail-closed behavior: a broken regex pattern (e.g. from hand-edited config) should not block legitimate registrations, since that directly hurts acquisition. Log the invalid pattern via common.SysError for visibility and skip it, matching prior fail-open semantics. * fix(auth): recover from panics in registration email blacklist check Belt-and-suspenders: even an unexpected panic while evaluating blacklist patterns should not fail the registration request. Recover and treat as not-blacklisted, logging via common.SysError for visibility. --------- Co-authored-by: 1111 <a1111@1111deMac-Studio.local> Co-authored-by: Boyuan Gao <anti2moron@gmail.com>
…nabled-only endpoint - CreatePromptLibraryItem: wrap Create + explicit enabled=false fixup in one transaction so a crash between the two writes cannot leave a wrongly-enabled row (pre-Create intent capture kept: gorm default:true rewrites zero-value false on Create). - Public read path: corrupt stored JSON columns now degrade to their fallback value with a SysError log instead of 500ing the whole listing; normalizePromptLibraryPublicItem can no longer fail. - New PUT /api/prompt-library/admin/:id/enabled toggles only the enabled column (map-form single-column update), so toggling no longer races with or clobbers concurrent full-row edits.
… toggle, sidebar module registration - formDataToPayload accepts the original row and merges un-edited JSON data (output_json, extra title/summary languages, source.captured_at, extra artifact/source keys) back into the full-replace PUT payload; create path unchanged. itemToFormData falls back to top-level source_platform/source_url when source_json lacks them. - Table enabled Switch now hits PUT /:id/enabled (single-column endpoint) with a per-row in-flight guard, so toggling cannot clobber concurrent edits. - Edit dialog threads the original row into updatePromptGalleryItem. - /prompt-gallery registered in URL_TO_CONFIG_MAP + sidebar defaults (admin.prompt_gallery) so SidebarModulesAdmin filtering applies, and exposed in the system-settings sidebar modules editor with i18n copy.
…own prompt sections - dedupe_slugs switches to a used-set walk: generated -N suffixes are checked against every emitted slug, so a literal "foo-2" colliding with a generated suffix can no longer produce duplicate slugs (which would silently merge items via the upsert-by-slug import). - Unknown H2 sections containing ```text fences are now collected in PARSE_WARNINGS (reset per parse) and reported by main() through the existing failures summary + exit 1, instead of a stderr warning only. Pure link sections (Resources/Contributing) stay silent. - Real dry-run unchanged: 72 items, 0 failures, exit 0.
… toggle - import script: unknown category sections with prompt fences now abort main() with exit 1 BEFORE any image download / GCS upload / import POST, so automated retries cannot compound a half-imported state. --dry-run (no side effects) still writes items.json to aid diagnosing the drift, then exits 1 via the existing failures summary; the old late-stage warnings->failures wiring is folded into that dry-run branch so nothing double-reports. - enabled toggle: MySQL reports RowsAffected=0 for a no-change UPDATE (same enabled value within the same updated_time second), which the handler used to misreport as 404. On 0 rows it now re-checks existence via GetPromptLibraryItemById: missing -> 404, present -> 200 idempotent no-op. Model layer unchanged (returning RowsAffected stays correct; the disambiguation is controller policy). glebarez sqlite counts matched rows as affected (probed: 1), so tests pin the observable same-value-twice->200 contract and the 0-rows branch is noted as MySQL-specific.
Constraint: Preserve current registration-country, email-domain, and default-token behavior while resolving the OAuth merge. Rejected: Rebase the long-lived PR branch | A merge keeps the published branch history intact and permits a normal fast-forward push. Confidence: high Scope-risk: moderate Directive: Keep Google email identity claims transactional when extending OAuth providers. Tested: go test ./controller -run ^TestFindOrCreateGoogleUser -count=1 Not-tested: Review-requested concurrency and indexed lookup changes are implemented in following commits.
Constraint: Volatile retries must remain sendable when IndexedDB is unavailable without treating the durable queue as empty during restore.\nRejected: Collapse all outbox reads to an empty durable queue | that can overwrite newer local state with stale server state.\nConfidence: high\nScope-risk: moderate\nDirective: Scope in-flight drains by user and stream callbacks by request ownership.\nTested: focused outbox, persistence, chat request, and stream ownership tests; bun run typecheck; targeted ESLint and Prettier\nNot-tested: Browser multi-tab IndexedDB upgrade smoke test
Keep the public real-person API and persisted lifecycle stable while allowing explicitly configured TokenSpace channels to own verification and asset operations. Constraint: Channel 106 remains DoubaoVideo and TokenSpace real-person groups must not reuse the configured virtual material group. Rejected: Hardcode channel 106 inside the BytePlus client | couples the core state machine to one production channel. Confidence: high Scope-risk: moderate Directive: Keep TokenSpace credentials bound to one enabled channel key until stable key identity is persisted. Tested: Design self-review and git diff --check. Not-tested: Implementation and production real-person verification are intentionally pending design review.
Make the implementation executable in small TDD increments while preserving the existing public API, persistence model, and BytePlus lifecycle. Constraint: Channel 106 remains DoubaoVideo and TokenSpace real-person operations require one stable enabled key. Rejected: Implement verification only | assets and background reconciliation would remain bound to BytePlus. Confidence: high Scope-risk: moderate Directive: Keep the authenticated GroupId separate from the configured virtual material group in every request. Tested: Plan self-review and git diff --check. Not-tested: Production code is implemented in subsequent commits.
Bind verification calls to a provider selected from channel configuration so TokenSpace can create and poll H5 sessions without weakening native BytePlus callback validation. Constraint: TokenSpace real-person channels must have exactly one enabled API key and may only be selected for an explicitly pinned first request. Rejected: Special-case channel 106 | provider selection must remain reusable and removable. Confidence: high Scope-risk: moderate Directive: Keep TokenSpace verification polling on the profile's persisted channel and retain the 300-second H5 expiry. Tested: Focused service verification/provider suites and existing TokenSpace material tests. Not-tested: TokenSpace real-person asset CRUD and background reconciliation are implemented next.
Constraint: TokenSpace real-person groups must come only from successful liveness verification and never from the ordinary virtual material group. Rejected: A new public route or channel type | the existing API and persisted channel binding already provide the required boundary. Confidence: high Scope-risk: moderate Directive: Keep multipart upload native-only until TokenSpace exposes an equivalent protected upload path. Tested: go test -vet=off ./service -run 'Test(TokenSpace|BytePlusRealPerson|RealPersonProvider|LoadUsableRealPerson)' -count=1 -timeout=7m Not-tested: live production TokenSpace H5 completion requires deployment and the user completing the verification session
Constraint: Definitive TokenSpace rejections must replay as verification_upstream_error instead of poisoning idempotency state as outcome unknown. Rejected: TokenSpace-specific branching in the state machine | the provider-neutral error classifier already captures both upstream families. Confidence: high Scope-risk: narrow Directive: Use provider-neutral error classifiers after provider dispatch. Tested: go test ./service -run 'Test(TokenSpace|BytePlusRealPerson|RealPersonProvider|LoadUsableRealPerson)' -count=1 -timeout=7m; go test ./controller -run 'Test.*RealPerson' -count=1; go test ./model -run 'Test.*RealPerson|Test.*BytePlusAsset' -count=1 Not-tested: live upstream rejection replay before deployment
…provider feat: add TokenSpace real-person provider
Expose safe Grok account status and align CLI billing
Persist and restore Playground conversations
Store normalized user email in an indexed column and reserve Google email, subject, and user ownership through a transaction-scoped unique claim. Concurrent signup losers roll back and resolve only an exact committed winner; logged-in Google binding follows the same invariant. Constraint: Preserve legacy duplicate-email detection and support SQLite, MySQL, and PostgreSQL without process-local locks. Rejected: Add a global unique users.email constraint | Legacy duplicates must remain representable and ambiguous matches must fail closed. Confidence: high Scope-risk: moderate Directive: Every future Google bind or signup path must acquire google_oauth_claims in the same transaction as users.google_id. Tested: targeted Google/NormalizedEmail/migration tests; go vet ./oauth ./model ./controller; independent code review. Not-tested: Live MySQL/PostgreSQL concurrent execution; full controller suite has unrelated failures in unchanged Seedance/channel tests and Windows SQLite cleanup.
Constraint: Hidden-model configuration is display-only and must not change API callability, API-key selectors, or default console rankings. Rejected: Filtering the shared model-access and rankings endpoints unconditionally | it would remove callable models from administrative and key-management workflows. Confidence: high Scope-risk: moderate Directive: Keep visibility filtering behind explicit available_models or public views and preserve complete default payloads. Tested: targeted Go regression tests; go vet ./controller ./service ./setting/operation_setting; go build ./...; Console tests/typecheck/build; website tests/typecheck/build; git diff --check. Not-tested: full controller suite remains blocked by unrelated existing asset-task worker failures; full service suite did not complete, while changed service regressions pass.
fix: 仅对非 PLG 用户隐藏订阅套餐
Constraint: Real-person assets remain in byte_plus_assets while channel 106 stays DoubaoVideo with an explicit tokenspace_material provider. Rejected: Migrating real-person assets into generalized storage | broader lifecycle change than the production routing hotfix. Confidence: high Scope-risk: narrow Directive: Keep non-real legacy assets and non-Doubao channels fail-closed. Tested: targeted middleware routing, generalized-shadow fallback, provider narrowing, asset reference and resolver tests; git diff --check. Not-tested: live production real-person video generation pending deployment.
…t-routing fix: route TokenSpace real-person assets through Doubao video channels
Prevent duplicate Google accounts for verified emails
…quota Unify subscription quota and retire media credits
…sibility feat(visibility): apply hidden models to public model catalogs
Filter administratively hidden models only for Playground display, add the backend-safe Seedance 2.5 profile, and route image/video quick starts through visible compatible models. Constraint: Seedance 2.5 must match the current relay contract: 480p/720p, 4-30 seconds, no 21:9 or seed. Rejected: Reusing the Seedance 2.0 profile | it exposes unsupported 2.5 parameters. Confidence: high Scope-risk: moderate Directive: Keep /api/user/models hidden filtering opt-in so availability semantics remain unchanged for other consumers. Tested: Go operation_setting tests; targeted controller tests; 1725 frontend tests; scoped ESLint and Prettier; TypeScript typecheck; production build. Not-tested: Browser E2E against the deployed console.
…y-seedance25 feat(playground): honor hidden models and support Seedance 2.5
Parse numeric subscription tiers without float coercion and expose the two supported Grok video models through the existing Playground video flow. Constraint: Media writes remain single-submit and account status refresh remains explicit. Rejected: Treat usage percentages as paid evidence | display-only usage can authorize unknown or free accounts. Confidence: high Scope-risk: narrow Directive: Keep Grok video payload fields synchronized with the strict task DTO; never rename aspect_ratio to ratio. Tested: Grok relay/task tests; Playground 106 tests; TypeScript; ESLint; Go vet; frontend production build. Not-tested: Production Grok video generation awaits deployment and authoritative paid-tier refresh.
Keep Flatkey's existing lease-guarded billing snapshot while aligning account status with Grok2API credit fields and derived remaining balances. Constraint: Preserve existing Grok media eligibility, stale-evidence, and secret-redaction boundaries. Rejected: A new quota table or separate account service | Adds migration and duplicate lease state without improving the current channel-scoped flow. Confidence: high Scope-risk: moderate Directive: Extend the versioned sanitized snapshot only; never expose raw billing responses or credentials. Tested: Spec self-review and placeholder scan. Not-tested: Implementation behavior is intentionally pending.
Preserve Grok2API-style package balances and render them in the existing account status dialog while keeping the persisted projection strictly non-secret. Overage is retained for accounting and clamped only for display.\n\nConstraint: Keep the existing versioned snapshot, channel-scoped refresh flow, media eligibility evidence, and secret-redaction boundary backward compatible.\nRejected: Add a new quota table or expose raw upstream JSON | unnecessary schema churn and leakage risk.\nConfidence: high\nScope-risk: moderate\nDirective: Future quota fields must be added to the parser whitelist, controller projection, and frontend formatter together; never serialize credentials or raw upstream payloads.\nTested: go test ./relay/channel/groksubscription -count=1; go test ./controller -run 'TestGrok(AccountStatus|ChannelDetail|Auth)' -count=1; go test ./model -run 'TestGrok' -count=1; go build ./...; go vet ./relay/channel/groksubscription ./controller ./model; Bun Grok formatter tests; TypeScript build check; ESLint; Prettier; frontend Rsbuild build.\nNot-tested: Full controller package suite was not used because the unfiltered suite exceeded the local command window; targeted Grok controller/auth/channel tests passed.
Contributor
|
Important Review skippedToo many files! This PR contains 2815 files, which is 2515 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (185)
📒 Files selected for processing (2815)
You can disable this status message by setting the |
Author
|
Closing this duplicate: the intended review target is SolveaCX/new-api, matching the existing branch and prior PR workflow. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
go test ./relay/channel/groksubscription -count=1go test ./controller -run 'TestGrok(AccountStatus|ChannelDetail|Auth)' -count=1go test ./model -run 'TestGrok' -count=1go build ./...go vet ./relay/channel/groksubscription ./controller ./modelReview notes
Known verification gap
The unfiltered controller package suite exceeded the local command window; targeted Grok controller/auth/channel tests passed.