Optimize LiveJournal degree count queries

[adsharma]

What Changed Summary

• Added REL_DEGREE_TABLE logical/physical source operator.
• q07 rewrite: MATCH (u)-[:follows]->(v) RETURN count(DISTINCT u.id) now becomes REL_DEGREE_TABLE in ACTIVE_BOUND_COUNT mode.
• q06 rewrite: MATCH (u)-[:follows]->(v) RETURN u.id, count(v) AS deg ORDER BY deg DESC LIMIT 10 now becomes REL_DEGREE_TABLE in TOP_K_DEGREES mode after top-k optimization.
• Added CSR degree summary methods for native, icebug-disk/parquet CSR, and Arrow CSR rel tables.
• Kept q08 COUNT_REL_TABLE rewrite intact.

Why

LiveJournal benchmark q06 and q07 were still executing scan-based count plans. These rewrites let the planner answer the unfiltered degree/count shapes from CSR metadata instead of scanning all follows edges.

Validation

• cmake --build build/release --target lbug_shell
• LiveJournal icebug-disk benchmark smoke run with build/release/tools/shell/lbug :memory: -b -i ../live-journal-benchmark/icebug_disk/schema.cypher:
  • q06 returned the expected top 10 in ~48ms executing.
  • q07 returned 4004103 in ~0.94ms executing.
  • q08 returned 69362378 in ~0.09ms executing.
• Native in-memory smoke test for q06/q07 rewrite behavior.

[adsharma]

How does it work?

COUNT_REL_TABLE

COUNT_REL_TABLE is the fast path for plain relationship cardinality.

It replaces plans like:

  SCAN_REL_TABLE -> PROJECT f._ID -> COUNT(f._ID)

with a direct count over the relationship table.

Instead of scanning every edge row, it asks the rel table for its row count. For icebug-disk/parquet-backed rel tables this comes from table/file metadata. For native CSR tables it sums CSR lengths, with handling for committed/uncommitted rows and deletions.

It returns one row: the total relationship count.

Example:

  MATCH (a:user)-[f:follows]->(b:user)
  RETURN count(f);

REL_DEGREE_TABLE

REL_DEGREE_TABLE is the fast path for degree-derived count queries.

It has two modes.

ACTIVE_BOUND_COUNT

Used for q07:

  MATCH (u:user)-[:follows]->(v)
  RETURN count(DISTINCT u.id);

This counts how many source CSR rows have at least one outgoing edge, instead of scanning all edges and deduplicating u.id.

For icebug-disk CSR, this is:

  count rows where indptr[i + 1] > indptr[i]

TOP_K_DEGREES

Used for q06:

  MATCH (u:user)-[:follows]->(v)
  RETURN u.id, count(v) AS deg
  ORDER BY deg DESC
  LIMIT 10;

This computes each source node’s degree from CSR row lengths, keeps a top-k heap, and outputs (u.id, deg) directly.

For LiveJournal’s icebug-disk layout, this reads the CSR indptr degree summary instead of materializing ~69M follows rows, aggregating, sorting, and limiting.

Summary

• COUNT_REL_TABLE: “How many edges are in this rel table?”
• REL_DEGREE_TABLE: “How many bound nodes have edges?” or “Which bound nodes have the largest degrees?”

[aheev]

Would love to see some tests involving active writes, multiple rel tables, bwd rel count

[aheev]

shouldn't we sum for a node appearing in multiple tables?

[aheev]

what if a nodeOffset appears in both local and persistent?

[aheev]

ditto

[aheev]

this would result in crash right? same for IceDiskRel tables

[adsharma]

• Degree rows are now merged by bound node across persistent, committed in-memory, and local write-transaction
  data.
• REL_DEGREE_TABLE now sums duplicate node degrees across multiple rel tables, and active-source counts
  deduplicate nodes across tables.
• Arrow/IceDisk unsupported degree directions now return no fast-path rows instead of falling through into native
  CSR code.
• Added optimizer coverage for active writes, multiple rel tables, and backward rel count.
• Allowed the degree top-k rewrite for count(*) in the one-hop degree pattern.

[aheev]

voops tests are failing