Skip to content

RI-8291: Add ARGREP array search endpoint#6090

Open
VaskoAtanasovRedis wants to merge 6 commits into
mainfrom
be/RI-8291/argrep-search-endpoint
Open

RI-8291: Add ARGREP array search endpoint#6090
VaskoAtanasovRedis wants to merge 6 commits into
mainfrom
be/RI-8291/argrep-search-endpoint

Conversation

@VaskoAtanasovRedis

@VaskoAtanasovRedis VaskoAtanasovRedis commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

What

First slice of the array Search vertical (RI-8221). Adds POST /array/search, backing the Redis ARGREP command.

  • SearchArrayDto: predicates (EXACT/MATCH/GLOB/RE + value), a single global AND/OR connective (no nested booleans — server limit), optional start/end (omitted ⇒ -/+), nocase, withValues (default true), limit.
  • GetArraySearchResponse{ index: string, value: RedisString | null }[] (value null when WITHVALUES is off).
  • ARGREP registered in ioredis.client.ts (required for pipelines).
  • Indexes are carried as decimal strings end to end — no Number()/parseInt. Mirrors the existing array read endpoints; no span cap (LIMIT is the backpressure).

Testing

  • 50 service unit tests (arg-building across every flag combination, flat-reply parsing incl. WITHVALUES on/off, error mapping).
  • Bruno request under Array/Search.
  • Integration test gated rte.version>=8.8 (skips in CI, runs against a local 8.8): predicate matching, global AND/OR, WITHVALUES on/off, 404 / 400 wrong-type — plus a > 2^53 index round-trip asserting the exact string survives the transport.
  • yarn lint, yarn type-check, yarn test:api pass.

Part of the RI-8221 Search vertical


Closes #RI-8291

🤖 Generated with Claude Code


Note

Low Risk
Additive browser API on existing array patterns; no auth or data-model changes beyond invoking ARGREP on user-selected keys.

Overview
Adds POST /databases/:id/array/search, exposing Redis ARGREP for array keys as the first slice of the array Search vertical (RI-8221).

The API accepts GetArraySearchDto: one or more predicates (EXACT / MATCH / GLOB / RE + value), optional global AND/OR (only sent when there are 2+ predicates), optional start/end (default -/+), nocase, withValues (defaults to true, including explicit null), and limit. Responses are { index, value } pairs with value: null when values are not requested.

ArrayService.search builds the ARGREP argv, normalizes both flat and nested Redis 8.8 reply shapes, keeps indexes as decimal strings, and maps wrong-type / ACL errors like other array reads. argrep is registered on the ioredis client for pipelines.

Coverage includes service unit tests, a Bruno request, and integration tests gated on rte.version>=8.8.

Reviewed by Cursor Bugbot for commit c5a9bf1. Bugbot is set up for automated code reviews on this repo. Configure here.

POST /array/search backs the Redis ARGREP command: predicate-based search over an array's index range with a single global AND/OR connective, NOCASE, WITHVALUES, LIMIT and -/+ range shorthands. Indexes are carried as decimal strings end to end. Mirrors the existing array read endpoints.

References: #RI-8291
Covers predicate matching, the global AND/OR connective, WITHVALUES on/off, and the 404 / 400 wrong-type error paths. Includes a >2^53 index round-trip asserting the exact string survives the transport. Gated to Redis 8.8+.

References: #RI-8291
@jit-ci

jit-ci Bot commented Jun 18, 2026

Copy link
Copy Markdown

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

@VaskoAtanasovRedis VaskoAtanasovRedis changed the title RI-8291: Add ARGREP array search endpoint (Search vertical S1) RI-8291: Add ARGREP array search endpoint Jun 18, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 862ab42f16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const rawIndex = reply[i];
if (rawIndex == null) continue;
elements.push({
index: toRequiredIndexString(rawIndex),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve exact large ARGREP indexes

ARGREP returns matching indexes as Redis integer replies (the Redis array docs show ARGREP ... WITHVALUES returning integer indexes), and ioredis documents that stringNumbers defaults to false and is required above Number.MAX_SAFE_INTEGER (Redis ARGREP, ioredis option). In production the clients are created without stringNumbers, so a match at 9007199254740993 arrives as an already-rounded JS number before this conversion, and the API serializes the wrong index string; this breaks the documented u64/exact-index contract for large sparse arrays.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed against a real 8.8 server: ARGREP returns the matched indexes as RESP integers, so ioredis rounds anything above 2^53 before our code ever sees it — a string DTO can't recover that. It's a transport-level fix (stringNumbers) shared with ARSCAN/ARLEN/ARCOUNT, so I've split it into RI-8296 and de-scoped this PR's integration test to a <=2^53 round-trip.

const {
keyName,
predicates,
combinator = ArrayCombinator.And,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve ARGREP's OR default

When callers send multiple predicates but omit combinator, this default makes the service append AND, while the Redis ARGREP docs specify that omitted AND/OR defaults to OR (docs). A request searching for values matching either of two predicates without an explicit combinator will therefore return only elements matching both, which does not match the command this endpoint wraps.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — dropped the AND default. We now send the connective only when it's provided, so the server's documented OR default applies when it's omitted.


@ApiProperty({ description: 'Pattern / value to match.', type: String })
@IsString()
@IsNotEmpty()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow empty predicate strings

ARGREP predicate arguments are arbitrary strings/patterns, and arrays can store empty string values through the create DTO, but @IsNotEmpty() rejects value: "" with a 400 before Redis sees it. That prevents valid searches such as EXACT "" for empty array elements (or intentionally empty MATCH/RE patterns).

Useful? React with 👍 / 👎.

@VaskoAtanasovRedis VaskoAtanasovRedis Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — dropped IsNotEmpty so EXACT "" can match empty array elements.

@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Code Coverage - Backend unit tests

St.
Category Percentage Covered / Total
🟢 Statements 93.01% 16192/17409
🟡 Branches 75.21% 5123/6812
🟢 Functions 87.33% 2502/2865
🟢 Lines 92.86% 15471/16661

Test suite run success

3592 tests passing in 317 suites.

Report generated by 🧪jest coverage report action from c5a9bf1

…llow empty predicate

Real Redis 8.8 returns ARGREP WITHVALUES as nested [[index, value], ...] (the docs show flat); parse both shapes like ARSCAN. Drop the invented AND default so the server's documented OR applies when the connective is omitted. Drop @isnotempty on predicate value so EXACT "" matches empty elements. The >2^53 index precision is an ioredis transport issue shared across the array commands — deferred to RI-8296; this PR's integration test asserts a <=2^53 round-trip.

References: #RI-8291

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e9efce3f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

start,
end,
nocase,
withValues = true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat null withValues as omitted

When a JSON body sends withValues: null (allowed by the DTO's @IsOptional()), the destructuring default here does not apply, so the later if (withValues) omits WITHVALUES and returns index-only results even though the endpoint documents default true. Normalize nullish values before building the ARGREP command so null behaves like an omitted optional field.

Useful? React with 👍 / 👎.

@VaskoAtanasovRedis VaskoAtanasovRedis Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real bug — fixed. IsOptional() lets a JSON null through and the destructuring default only filled undefined, so withValues: null dropped WITHVALUES. Normalized with dto.withValues ?? true and added a test.

@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Code Coverage - Integration Tests

Status Category Percentage Covered / Total
🟡 Statements 78.45% 17808/22699
🟡 Branches 61.2% 8236/13456
🟡 Functions 66.31% 2423/3654
🟡 Lines 78.04% 16751/21464

@IsOptional() lets a JSON null through and a destructuring default only fills undefined, so withValues: null wrongly omitted WITHVALUES. Normalize with ?? true.

References: #RI-8291

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3dfde06cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

criteria: ArrayGrepCriteria;

@ApiProperty({ description: 'Pattern / value to match.', type: String })
@IsString()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Accept Redis-string predicate values

When users search arrays containing non-UTF8/binary elements, this predicate field rejects the same Redis string payload shape that array writes and encoded responses use: ArrayElementDto.value is transformed/validated with RedisStringType/IsRedisString, but search predicates are limited to plain JS strings. As a result, an EXACT search using a { type: 'Buffer', data: [...] } value returned by this API gets a 400 before Redis sees it, and escaped ASCII input is sent literally instead of as the original bytes; the new endpoint should apply the same Redis-string handling to predicate values.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — aligned the predicate value with the rest of the array API: it now uses @RedisStringType() + @IsRedisString() like ArrayElementDto.value, so a value the API returns (incl. buffer-encoded) can be fed back into an EXACT search instead of 400ing.

Predicate value now uses the same RedisString handling as array writes/responses (ArrayElementDto.value), so a binary value the API returns can be searched (EXACT) and escaped input is decoded to bytes rather than sent literally.

References: #RI-8291
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant