Skip to content

feat(typed): generic, type-safe client and query builder#17

Open
mlwelles wants to merge 3 commits into
matthewmcneely:mainfrom
mlwelles:feature/typed-client
Open

feat(typed): generic, type-safe client and query builder#17
mlwelles wants to merge 3 commits into
matthewmcneely:mainfrom
mlwelles:feature/typed-client

Conversation

@mlwelles

@mlwelles mlwelles commented Jun 4, 2026

Copy link
Copy Markdown

What this adds

A generic typed layer over modusgraph.Client that binds a Go type to the
otherwise any-typed client, giving you compile-time-typed CRUD and a fluent
query builder with no per-entity code generation. It is the handwritten
substrate modusgraph-gen's generated clients compose over, and it stands on
its own wherever you want types over modusgraph.

The whole diff lives under typed/. It builds and tests green against the
current client with no changes to the root package.

The problem it solves

modusgraph.Client is value-oriented: its methods take and return any, and a
query is assembled by hand from dgman primitives and decoded into a slice you
declare at the call site. Every call site repeats the same shape — declare the
destination, build the query, decode, re-assert the type. The typed layer lifts
that shape into the type system once.

Before — untyped client

// "First person named Alice" — the type is repeated on every line,
// and the single-result case is hand-rolled.
var out []Person
q := client.Query(ctx, &Person{}).
    Filter("eq(name, $1)", "Alice").
    First(1)
if err := q.Nodes(&out); err != nil {
    return nil, err
}
var person *Person
if len(out) > 0 {
    person = &out[0]
}

After — typed client

people := typed.NewClient[Person](client) // declare the type once

person, err := people.Query(ctx).
    Filter("eq(name, $1)", "Alice").
    First() // returns *Person; nil when nothing matched

The same lift applies across the API: Nodes() returns []Person,
IterNodes() yields *Person, Get returns *Person.

Query builder

Query[T] is fluent: builder methods chain, terminals (Nodes, First,
NodesAndCount, IterNodes) execute and decode typed results.

  • Filters accumulate and AND together, each fragment parenthesized so a
    fragment containing OR keeps its precedence.
  • OrGroup ORs several sub-scopes into one parenthesized group:
    // age >= 18 AND (name == "Alice" OR name == "Bob")
    people.Query(ctx).
        Filter("ge(age, $1)", 18).
        OrGroup(
            typed.NewDetachedQuery[Person]().Filter(`eq(name, "Alice")`),
            typed.NewDetachedQuery[Person]().Filter(`eq(name, "Bob")`),
        ).Nodes()
  • WhereEdge constrains T by a scalar of a neighbour reached over an edge
    (a root filter cannot express this); resolved by a pre-pass and intersected
    with any root you set.
  • IterNodes streams arbitrarily large result sets a page at a time over a
    single read-only snapshot.
  • MultiQuery batches N same-type blocks into one round-trip.

On reusing dgman vs. reinventing it

The Or/OrGroup support builds on dgman, it does not reimplement it.
dgman exposes a single string-based Query.Filter(filter string, params ...any)
with $N positional markers and no AND/OR combinators and no root-intersection
helpers
. So any builder offering Where/Or composition must assemble the
filter string itself and hand it to dgman's Filter. This layer does exactly
that and no more:

  • the actual filtering, parameter binding, and injection-safe $N substitution
    stay in dgman (Query.Filter, parseQueryWithParams);
  • shiftPlaceholders only renumbers $N to match dgman's own convention
    when independently-written fragments are concatenated — it does not re-parse
    or re-escape values;
  • the layer adds only what dgman lacks: fragment composition (AND/OR grouping
    with correct precedence) and the type binding.

Review round — changes since first push

Addresses the automated review (cubic). Highlights:

  • Filter precedence (P1): combineAnd now parenthesizes each fragment, so
    a raw OR fragment no longer binds incorrectly when ANDed.
  • WhereEdge roots (P1): the pre-pass no longer discards a caller-set
    UID/RootFunc; it intersects via a uid() @filter instead.
  • MultiQuery reuse (P1): Add rejects the same *Query[T] under two names
    (Execute names the underlying query in place).
  • NodesAndCount tracing: now opens a span like the other terminals.
  • Tests: added a precedence regression, a WhereEdge+root intersection
    test, and a duplicate-Query guard; the filter-sequencing test now asserts
    the exact expression; the laziness check is delta-based.

Documentation

  • Package doc.go with the before/after narrative and the design rationale.
  • Runnable, compile-checked examples for the client, query builder, OrGroup,
    WhereEdge, IterNodes, and MultiQuery.
  • A verified-output example for the filter builder.

Add a generic typed layer over modusgraph.Client: typed.Client[T] with
CRUD and iterators; a fluent Query[T] builder (filters, ordering, paging,
edge traversal, IterNodes); MultiQuery for N homogeneous blocks in one
round-trip; functional options; a filter DSL (typed/filter); and ordered
result merging (typed/search).

A small no-op-by-default Tracer seam (typed.SetTracer) lets a host plug in
tracing without the typed package depending on any telemetry library.

Self-contained: builds and tests against the current client with no other
changes.
@mlwelles mlwelles requested a review from matthewmcneely as a code owner June 4, 2026 17:07

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 issues found across 16 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

Comment thread typed/multi_query.go
Comment thread typed/query.go Outdated
Comment thread typed/query.go Outdated
Comment thread typed/filter/filter_test.go Outdated
Comment thread typed/query_test.go
Addresses review feedback on the typed query builder:

- combineAnd now parenthesizes each accumulated @filter fragment. Without
  this, a fragment containing OR ANDed with another fragment rendered as
  "a OR b AND c", which dgraph parses as "a OR (b AND c)" — silently
  widening results. dgman exposes only a string Filter(), so the builder
  must compose the expression itself; this makes that composition correct.
- WhereEdge's pre-pass no longer discards a caller-set UID/RootFunc root.
  When a custom root is present, the matched UIDs are intersected via a
  uid() @filter instead of overwriting the root.
- MultiQuery.Add rejects the same *Query[T] under two names; Execute names
  the underlying query in place, so reuse would corrupt block composition.
- NodesAndCount now opens a tracing span, matching the other terminals.

Tests: add a precedence regression, a WhereEdge+UID intersection test, and
a duplicate-Query guard test; strengthen the filter-sequencing test to
assert the exact expression; make the IterNodes laziness check delta-based.

Docs: add package doc.go with a before/after narrative, runnable examples
for the client/query builder/MultiQuery, and a verified filter example.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 9 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="typed/doc.go">

<violation number="1" location="typed/doc.go:70">
P2: Global process-wide tracer is unsynchronized; concurrent SetTracer and query execution risk data races and test isolation failures</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread typed/doc.go
//
// # Tracing
//
// Every terminal opens a span through a process-wide tracer that is a no-op by

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Global process-wide tracer is unsynchronized; concurrent SetTracer and query execution risk data races and test isolation failures

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At typed/doc.go, line 70:

<comment>Global process-wide tracer is unsynchronized; concurrent SetTracer and query execution risk data races and test isolation failures</comment>

<file context>
@@ -0,0 +1,73 @@
+//
+// # Tracing
+//
+// Every terminal opens a span through a process-wide tracer that is a no-op by
+// default. Install one with SetTracer to emit spans without the typed package
+// depending on any telemetry library.
</file context>

Wrap a >120-char error string in MultiQuery.Execute and a long test helper
signature, and pre-allocate two test result slices. Resolves the Trunk
golangci-lint findings; no behavior change.

@matthewmcneely matthewmcneely left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Six subpackage files are missing the SPDX header that every other file in the repo carries: filter/filter.go, filter/fulltext.go, filter/filter_test.go, filter/fulltext_test.go, search/merge.go, search/merge_test.go.

Comment thread typed/query.go
// pre-pass resolves the UIDs of roots that satisfy every constraint, then the
// main query runs against uid(...) — keeping ordering, pagination, and result
// projection on the normal path. See
// docs/specs/2026-05-21-query-edge-filter-design.md.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you forget to commit this in this branch?

Comment thread typed/multi_query.go
}
if len(predMap) > 0 {
remapped, err := remapArrayKeys(body, predMap)
if err == nil {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This err gets swallowed. The later unmarshal usually surfaces something, but the root cause is lost.

Comment thread typed/query.go
// matchedUIDs runs the pre-pass: an @cascade query over T that keeps only
// nodes whose every WhereEdge predicate has a target matching its filter, and
// returns those nodes' UIDs.
func (qb *Query[T]) matchedUIDs() ([]string, error) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The WhereEdge pre-pass materializes every matching root UID, unbounded.

matchedUIDs runs the pre-pass with no limit and pulls all matching UIDs into the client (query.go:516-533), then embeds them as uid(uid1, …, uidN) (query.go:504-509). On a high-cardinality edge match that can exhaust memory, overflow the pre-pass gRPC response, and generate an enormous DQL string. That's a sharp edge on a branch built atop the companion WithMaxRecvMsgSize work.

It also undercuts IterNodes. A WhereEdge(...).IterNodes() chain is sold as bounded-memory streaming, but the pre-pass is eager and unbounded before paging even begins.

The idiomatic fix is a server-side var block feeding func: uid(var), which keeps the UIDs off the client entirely. At minimum, document the limitation. The referenced design doc would presumably justify the tradeoff, but it's missing (see the separate note about that).

Comment thread typed/multi_query.go
return result
}

// remapArrayKeys rewrites top-level keys in each object of a JSON array using

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MultiQuery silently mis-decodes renamed nested predicates, and the comments claiming dgman parity are inaccurate.

remapArrayKeys only remaps top-level keys (multi_query.go:181-200), but dgman's own remapPredicateKeys recurses into nested edge structs (dgman/v2@v2.2.0/query.go:78-145; remapObject recurses). Two comments overstate the parity:

  • "matching the behavior of dgman's QueryBlock.Scan path" (multi_query.go:76-78)
  • "edge fields are hydrated lazily, not in the multi-block response" (multi_query.go:183-184)

The second is the load-bearing error. MultiQuery blocks are built from conn.Query(), which applies .All(maxEdgeTraversal) (client.go:585), so the response does carry expanded edges. A type with a renamed predicate (predicate=json:) on a nested edge populates correctly through the normal Query path but silently leaves that nested field empty through MultiQuery.

Recommend either recursing like dgman, or narrowing the comments to top-level scalar predicates only and documenting the limitation loudly.

@matthewmcneely

Copy link
Copy Markdown
Owner

@mlwelles Also, can you update the README with the major feature additions of this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants