BE-619: Split entity queries and entity counting#8897
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
Merging this PR will not alter performance
Comparing Footnotes
|
PR SummaryMedium Risk Overview GraphQL exposes Permission check in hash-api uses Reviewed by Cursor Bugbot for commit 94b2f7c. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
Pull request overview
Decouples entity result fetching from aggregation work by introducing a dedicated entity summary endpoint (summarizeEntities), and migrates API/SDK/GraphQL/frontend consumers away from inline counts/facets on queryEntities / queryEntitySubgraph. Also aligns OpenAPI schema metadata for BaseUrl / VersionedUrl with shared.json to reduce duplicate-definition/codegen warnings.
Changes:
- Adds
summarizeEntities//entities/query/summarizewith opt-in aggregation flags and removes count/facet flags from entity query endpoints. - Updates Rust store APIs, GraphQL schema/resolvers, TypeScript SDK helpers, and frontend queries to use the new summary endpoint.
- Updates integration tests/benches to match the new query + summary split; improves OpenAPI schema metadata consistency.
Reviewed changes
Copilot reviewed 46 out of 46 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/graph/integration/postgres/sorting.rs | Updates subgraph query usage after removing inline count/aggregations. |
| tests/graph/integration/postgres/partial_updates.rs | Removes deprecated include-count/aggregation flags from queries. |
| tests/graph/integration/postgres/multi_type.rs | Adds/uses summarize_entities in tests verifying aggregations. |
| tests/graph/integration/postgres/links.rs | Migrates link existence checks from count_entities to summarize_entities. |
| tests/graph/integration/postgres/lib.rs | Adjusts test EntityStore wrapper to delegate summarize_entities. |
| tests/graph/integration/postgres/entity.rs | Migrates count assertions to summarize_entities and removes inline count expectations. |
| tests/graph/integration/postgres/email_filter_protection.rs | Switches filter-protected counting helper to summarize_entities. |
| tests/graph/benches/representative_read/knowledge/entity.rs | Removes deprecated query flags in bench requests. |
| tests/graph/benches/read_scaling/knowledge/linkless/entity.rs | Removes deprecated query flags in bench requests. |
| tests/graph/benches/read_scaling/knowledge/complete/entity.rs | Removes deprecated query flags in bench requests. |
| tests/graph/benches/manual_queries/entity_queries/mod.rs | Removes include_count permutations from manual bench query generation. |
| tests/graph/benches/graph/scenario/stages/entity_queries.rs | Updates scenario bench query params after removing inline aggregations. |
| libs/@local/hashql/mir/src/interpret/suspension/mod.rs | Adjusts debug_assert_matches usage/imports. |
| libs/@local/hash-isomorphic-utils/src/graphql/type-defs/knowledge/entity.typedef.ts | Replaces countEntities GraphQL query with summarizeEntities. |
| libs/@local/hash-isomorphic-utils/src/graphql/scalar-mapping.ts | Updates scalar mappings for summarize request/response types. |
| libs/@local/hash-backend-utils/src/flows.ts | Uses summarizeEntities for total counts while querying pages independently. |
| libs/@local/graph/type-fetcher/src/store.rs | Renames count_entities to summarize_entities in store wrapper. |
| libs/@local/graph/store/src/entity/store.rs | Introduces SummarizeEntitiesParams/Response and removes count/facets from query responses. |
| libs/@local/graph/store/src/entity/mod.rs | Re-exports summarize types instead of count types. |
| libs/@local/graph/sdk/typescript/src/entity.ts | Adds summarizeEntities() helper and removes facet fields from subgraph query mapping. |
| libs/@local/graph/postgres-store/tests/deletion/validation.rs | Renames helper usage to count_entities (plural) backed by summarization. |
| libs/@local/graph/postgres-store/tests/deletion/purge.rs | Migrates deletion tests to new counting helper. |
| libs/@local/graph/postgres-store/tests/deletion/main.rs | Updates shared deletion test helper to call summarize_entities and extract count. |
| libs/@local/graph/postgres-store/tests/deletion/links.rs | Migrates deletion link tests to new counting helper. |
| libs/@local/graph/postgres-store/tests/deletion/erase.rs | Migrates erase tests to new counting helper. |
| libs/@local/graph/postgres-store/tests/deletion/drafts.rs | Migrates draft deletion tests to new counting helper. |
| libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/summary.rs | Updates summary query builder to use SummarizeEntitiesParams. |
| libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs | Removes inline query aggregations and implements summarize_entities. |
| libs/@local/graph/api/src/rest/mod.rs | Renames OpenAPI query logger variant to SummarizeEntities. |
| libs/@local/graph/api/src/rest/json_schemas/shared.json | Fixes BaseUrl description grammar. |
| libs/@local/graph/api/src/rest/entity.rs | Replaces REST /count with /summarize and returns structured summary response. |
| libs/@local/graph/api/src/rest/entity_query_request.rs | Removes deprecated include-count/aggregation options from query request parsing. |
| libs/@local/graph/api/openapi/openapi.json | Updates OpenAPI paths/schemas to include SummarizeEntities* and metadata alignment. |
| libs/@local/graph/api/openapi/models/shared.json | Fixes BaseUrl description grammar in generated models. |
| libs/@blockprotocol/type-system/rust/src/ontology/id/mod.rs | Adds title/description/maxLength metadata for URL schema types. |
| apps/hash-frontend/src/shared/notification-count-context.tsx | Switches unread notification counting to summarizeEntities. |
| apps/hash-frontend/src/shared/layout/layout-with-sidebar/sidebar/account-entities-list.tsx | Uses summary typeId counts to filter sidebar entity types. |
| apps/hash-frontend/src/shared/generate-sidebar-entities-query-variables.tsx | Generates variables for summarizeEntities instead of querying a 1-item page. |
| apps/hash-frontend/src/shared/draft-entities-count-context.tsx | Switches draft counting to summarizeEntities. |
| apps/hash-frontend/src/pages/shared/entities-visualizer/use-entities-visualizer-data.tsx | Splits visualizer data loading into subgraph query + summary count query. |
| apps/hash-frontend/src/pages/shared/entities-visualizer/shared/use-available-types.ts | Uses summary typeIds/typeTitles instead of subgraph query for facets. |
| apps/hash-frontend/src/pages/index.page/waitlisted.tsx | Uses summary count to determine whether a prospective user submission exists. |
| apps/hash-frontend/src/graphql/queries/knowledge/entity.queries.ts | Replaces countEntities GraphQL query with summarizeEntities. |
| apps/hash-api/src/graphql/resolvers/knowledge/entity/entity.ts | Replaces GraphQL resolver from countEntities to summarizeEntities. |
| apps/hash-api/src/graphql/resolvers/index.ts | Wires summarizeEntities resolver into Query root. |
| apps/hash-api/src/graph/knowledge/primitive/entity.ts | Replaces internal countEntities helper with summarizeEntities and includeCount. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8897 +/- ##
==========================================
+ Coverage 59.61% 59.63% +0.02%
==========================================
Files 1348 1348
Lines 131877 131788 -89
Branches 5944 5943 -1
==========================================
- Hits 78615 78592 -23
+ Misses 52357 52292 -65
+ Partials 905 904 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- Deduplicate the summary `hits` CTE over `edition_id` instead of the entity identity, so the temporal axis drives the count (point-in-time → per entity, unbounded → per edition), matching the former `count_entities`. - Source `typeTitles` from the edition cache's type-title array in the same aggregate (new `IsOfType`/`Title` query path), dropping the separate ontology lookup. - Rewrite the semantic filter once in `getFlowRuns` before the concurrent entity/summary requests, so the embedding lookup isn't duplicated.
- Deny on a missing/zero count in `canUserReadEntity` (was failing open: an absent `count` granted read access). - Tolerate NULL type titles in the summary decode (read `Option<String>`) instead of erroring the whole request when a type group has no title. - Drop the fabricated `definitions` field from the TS `SummarizeEntitiesResponse` (the endpoint never returns it). - Add a point-in-time summary-count assertion in `entity::update` (per-entity vs the existing per-edition unbounded count). - Remove the stale `includeCount` sweep parameter from the entity-query bench definitions so they deserialize again. - Fix the stale `Dimension` row-layout doc (now 5 columns) and initialise `count` consistently with the other requested dimensions.
`include_count: true` guarantees `count` is present, so the test helpers expect it rather than defaulting to 0 via `unwrap_or` — otherwise a regression returning `count: null` (notably where the expected count is 0) would pass unnoticed.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 17b84ea. Configure here.
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1001 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1526 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 51 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 107 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 25 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |

🌟 What is the purpose of this PR?
queryEntities/queryEntitySubgraphpreviously computed the total count and the per-web / per-type facet aggregations inline, coupling the result-set fetch to aggregation work. This splits those aggregations into a dedicatedsummarizeEntitiesendpoint, so the page, the count, and the type facets load independently.🔗 Related links
🔍 What does this change?
summarizeEntitiesendpoint (POST /entities/query/summarize) returningcountplus thewebIds/typeIds/typeTitles/ created-by aggregations, each gated by aninclude*flag.include_count/include_*aggregation flags (and thecountEntitiesendpoint) fromqueryEntities/queryEntitySubgraph.VersionedUrl/BaseUrlToSchemaoutput withshared.json(title/description/max_length) to remove duplicate-definition codegen warnings.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR: