Skip to content

Commit ac501e5

Browse files
committed
Update docs, fix code gaps, an more
1 parent a1c73a2 commit ac501e5

32 files changed

Lines changed: 441 additions & 200 deletions

README.md

Lines changed: 67 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -70,8 +70,8 @@ Runs across:
7070
+ server, browser, edge, and worker runtimes
7171
+ both local and remote data as one relational graph
7272
73-
one application-level interface in all
74-
under `80 KiB` (min+zip) in all
73+
All in under `80 KiB` (min+zip)
74+
A single interface that drops into any application
7575
7676
> One SQL interface for local, remote, and live data
7777
@@ -82,7 +82,7 @@ The big picture? **[SQL, reimagined for modern apps ↗](https://linked-ql.netli
8282
---
8383
8484
> [!IMPORTANT]
85-
> LinkedQL is backed by **over 1,000 test cases and growing**.<br>
85+
> LinkedQL is backed by **over 1,000 tests and growing**.<br>
8686
> Early usage feedback, issues and PRs help us on our path to the next 1,000 tests.<br>
8787
> See [Contributing](#-contributing)
8888
@@ -124,7 +124,7 @@ Import and use the Client for your database. LinkedQL works the same across all
124124
| PostgreSQL | `@linked-db/linked-ql/postgres` | [PostgreSQL ↗](https://linked-ql.netlify.app/docs/setup#postgresql) |
125125
| MySQL | `@linked-db/linked-ql/mysql` | [MySQL ↗](https://linked-ql.netlify.app/docs/setup#mysql) |
126126
| MariaDB | `@linked-db/linked-ql/mariadb` | [MariaDB ↗](https://linked-ql.netlify.app/docs/setup#mariadb) |
127-
| FlashQL (In-Memory) | `@linked-db/linked-ql/flashql` | [FlashQL ↗](https://linked-ql.netlify.app/docs/setup#flashql) |
127+
| FlashQL | `@linked-db/linked-ql/flashql` | [FlashQL ↗](https://linked-ql.netlify.app/docs/setup#flashql) |
128128
| EdgeClient | `@linked-db/linked-ql/edge` | [Edge / Browser ↗](https://linked-ql.netlify.app/docs/setup#edge) |
129129
| EdgeWorker | `@linked-db/linked-ql/edge-worker` | [Edge Worker ↗](https://linked-ql.netlify.app/docs/setup#edge) |
130130

@@ -254,7 +254,7 @@ await db.sync.sync(); // (FlashQL)
254254

255255
The same surface applies whether `db` is a direct PostgreSQL client, a local FlashQL engine, or an `EdgeClient`.
256256

257-
For result shapes, options, and API detail, see the [Query API docs ↗](https://linked-ql.netlify.app/docs/query-api).
257+
For result shapes, options, and API details, see the [Query API docs ↗](https://linked-ql.netlify.app/docs/query-api).
258258

259259
---
260260

@@ -322,7 +322,7 @@ for await (const row of asyncIterable) {
322322

323323
* Returns an async iterable
324324
* Lazily fetches rows on demand as you iterate
325-
* The best way to avoids materializing very large result sets in memory all at once
325+
* The best way to avoid materializing very large result sets in memory all at once
326326

327327
This is covered in [Streaming ↗](https://linked-ql.netlify.app/capabilities/streaming).
328328

@@ -375,10 +375,15 @@ See [Changefeeds (WAL) ↗](https://linked-ql.netlify.app/capabilities/changefee
375375

376376
`db.sync.sync()` is the API for sync in FlashQL.
377377

378-
You begin by creating views and pointing them to local or remote tables. You call `sync()` to execute the contract.
378+
You begin by creating views and pointing them to local or remote tables. FlashQL internally uses the sync API
379+
to fulfill the contracts.
380+
381+
At the application level, `db.sync.sync()` is primarilly called on a network reconnect event:
379382

380383
```js
381-
await db.sync.sync();
384+
window.addEventListener('online', async () => {
385+
await db.sync.sync(); // re-sync on reconnect
386+
});
382387
```
383388

384389
#### Behavior
@@ -544,9 +549,9 @@ For deeper syntax and traversal patterns, see [DeepRefs ↗](https://linked-ql.n
544549

545550
---
546551

547-
### 3. Structured Writes (Graph Mutations)
552+
### 3. Deep Writes (Graph Mutations)
548553

549-
The same relationship-aware model applies to writes.
554+
The same syntax and relationshipal model as DeepRefs can be used for writes.
550555

551556
---
552557

@@ -565,7 +570,7 @@ await db.query(`
565570

566571
* Inserts the main row
567572
* Inserts or resolves the related row
568-
* Wires the relationship automatically
573+
* Wires the relationship automatically – entirely in one statement
569574

570575
---
571576

@@ -587,12 +592,12 @@ await db.query(`
587592

588593
Traditional SQL requires:
589594

590-
* multiple statements or CTEs
595+
* multiple statements or clever CTE tricks
591596
* strict ordering of operations
592597
* manual foreign-key coordination
593598

594599
LinkedQL keeps the statement SQL-shaped, while **relationship-aware payloads desugar into the required lower-level operations**.
595-
See [Structured Writes ↗](https://linked-ql.netlify.app/capabilities/structured-writes).
600+
See [Deep Writes ↗](https://linked-ql.netlify.app/capabilities/deeprefs).
596601

597602
---
598603

@@ -601,8 +606,7 @@ See [Structured Writes ↗](https://linked-ql.netlify.app/capabilities/structure
601606
Upsert syntax varies across databases:
602607

603608
* PostgreSQL → `INSERT + ON CONFLICT`
604-
* MySQL → `INSERT + ON DUPLICATE KEY`
605-
* MariaDB → similar but not identical
609+
* MySQL/MariaDB → `INSERT + ON DUPLICATE KEY`
606610

607611
LinkedQL provides a **single, predictable form across dialects**:
608612

@@ -784,7 +788,7 @@ const upstream = new PGClient({
784788
await upstream.connect();
785789
786790
// Adapter that exposes the LinkedQL protocol over HTTP
787-
const worker = new EdgeWorker({ client: upstream });
791+
const worker = new EdgeWorker({ db: upstream, type: 'http' });
788792
789793
// Now the handler (at "/api/db") that exposes the worker:
790794
export async function POST(request) {
@@ -858,7 +862,7 @@ This is not just a browser → server abstraction.
858862
* server → server
859863
* edge → origin
860864
* worker → worker
861-
* browser (main thread) → web worker (as seen in Scenario 4 below)
865+
* browser (main thread) → web worker (as seen in Scenario 2 below)
862866

863867
---
864868

@@ -902,7 +906,7 @@ const upstream = new PGClient({ /* config */ });
902906
await upstream.connect();
903907
904908
// Automatically wires message port → LinkedQL protocol → upstream client
905-
EdgeWorker.webWorker({ client: upstream });
909+
EdgeWorker.webWorker({ db: upstream });
906910
```
907911
908912
---
@@ -919,7 +923,7 @@ For more on the runtime setup side, see [Dialects & Clients ↗](https://linked-
919923
920924
#### The Edge Protocol does the heavy lifting
921925
922-
As against just sending raw SQL over HTTP, the client (`EdgeClient`) sends structured operations that map directly to the LinkedQL client interface (e.g. `query()`, `stream()`, `subscribe()`, etc.).
926+
As against just sending raw SQL over HTTP or message port, the client (`EdgeClient`) sends structured operations that map directly to the upstream LinkedQL client interface (e.g. `query()`, `stream()`, `subscribe()`, etc.).
923927
924928
This preserves:
925929
@@ -933,10 +937,10 @@ This preserves:
933937
934938
In this scenario, we demonstrate a hybrid data architecture where the goal is to:
935939
936-
> Query remote data **as if it were local**, while controlling
940+
> Query upstream data **as if it were local**, while controlling
937941
> what stays remote, what gets cached locally, and what stays in sync.
938942
939-
The idea is straight-forward in FlashQL: you simply create a view (a database view) in your local database that mirrors the remote database.
943+
The idea is straight-forward in FlashQL: you simply create a view (a database view) in your local database that points to the upstream database as its origin.
940944
941945
```js
942946
await db.query(`
@@ -946,11 +950,28 @@ await db.query(`
946950
`);
947951
```
948952
949-
Notice the `WITH (replication_origin = ...)` specifier. That's the magic.
953+
Notice the `WITH (replication_origin = ...)` specifier. That's the part that turns a regular view into a foreign view.
954+
955+
Now, querying `public.users` on the local database will query `public.users` on the upstream database:
950956

951-
Rows in this view (`public.users`) will mirror `public.users` in the upstream database.
957+
```js
958+
await db.query(`
959+
SELECT * FROM public.users;
960+
`);
961+
```
952962

953-
But just one more thing is required for this to work:
963+
Being a regular table, it can be used just like one – e.g. in joins:
964+
965+
```js
966+
await db.query(`
967+
SELECT * FROM public.posts
968+
LEFT JOIN public.users ON posts.user_id = users.id;
969+
`);
970+
```
971+
972+
The query executes as one relational graph – but composed of both local and remote data.
973+
974+
But for this work, one thing is required:
954975

955976
> a way to connect the local FlashQL instance to the upstream database.
956977

@@ -962,7 +983,7 @@ import { EdgeClient } from '@linked-db/linked-ql/edge';
962983
963984
const db = new FlashQL({
964985
// The hook to remote
965-
async onCreateForeignClient(originUrl) {
986+
async getUpstreamClient(originUrl) {
966987
return new EdgeClient({ url: originUrl, type: 'http' });
967988
}
968989
});
@@ -972,46 +993,27 @@ await db.connect();
972993

973994
This is now a local FlashQL instance that can talk to an upstream database ondemand.
974995

975-
Above, the `onCreateForeignClient()` hook will recieve `'/api/db'` – the value of the `replication_origin` config.
976-
977-
Now, querying `public.users` on the local database will query `public.users` on the upstream database:
978-
979-
```js
980-
await db.query(`
981-
SELECT * FROM public.users;
982-
`);
983-
```
996+
Above, the `getUpstreamClient()` factory will recieve `'/api/db'` – the value of the `replication_origin` config.
984997

985-
Being a regular table, it can be used just like one – e.g. in joins:
986-
987-
```js
988-
await db.query(`
989-
SELECT * FROM public.posts
990-
LEFT JOIN public.users ON posts.user_id = users.id;
991-
`);
992-
```
993-
994-
The query executes as one relational graph – but composed of both local and remote data.
995-
996-
Given this as the base, FlashQL further lets you create other types of views with different replication modes.
998+
Given this as the base, FlashQL further lets you create different types of views for different replication behaviours.
997999

9981000
These modes determine how mirroring works; i.e. whether data stays remote, or is cached locally, or stays in sync.
9991001

10001002
| Mode | Behavior |
10011003
| :------------------------------------------------ | :-------------------------------------- |
1002-
| Basic views (the default) | This is the default idea of a view: a table that has no actual rows but just a query that executes at query-time. |
1004+
| Runtime views (the default) | This is the default idea of a view: a table that has no actual rows but just a query that executes at query-time. |
10031005
| Materialized views | These views go ahead to copy the origin data for local use and behave as local tables from that moment on. |
10041006
| Realtime views | These views are materialized views that not just copy origin data, but also stay in sync with origin data. |
10051007

1006-
* use basic views when you just want to federate remote data and don't need offline access
1008+
* use runtime views when you just want to federate remote data and don't need offline access
10071009
* use `materialized` views when you want a local copy for offline access
1008-
* use `realtime` views when the local copy should stay in sync origin data
1010+
* use `realtime` views when the local copy should stay in sync with origin data
10091011
10101012
Each mode is demonstrated below.
10111013
10121014
---
10131015
1014-
##### Mode 1: Basic Views (Pure Federation)
1016+
##### Mode 1: Runtime Views (Pure Federation)
10151017
10161018
```js
10171019
await db.query(`
@@ -1033,8 +1035,6 @@ This is **federation**:
10331035
10341036
> Querying external data as if it were part of your local schema
10351037
1036-
This is the lightest-weight mode. It gives you unification without local storage cost.
1037-
10381038
---
10391039
10401040
##### Mode 2: Materialized Views (Local Cache)
@@ -1073,7 +1073,7 @@ await db.query(`
10731073
10741074
```js
10751075
await db.query(`
1076-
CREATE REALTIME VIEW remote.posts AS
1076+
CREATE REALTIME VIEW public.posts AS
10771077
SELECT * FROM public.posts
10781078
WITH (replication_origin = '/api/db')
10791079
`);
@@ -1094,7 +1094,7 @@ This is **realtime mirroring**:
10941094
* and then catches up again when connectivity returns
10951095
10961096
Realtime views are designed to be resilient to network disconnects. All you need to do in
1097-
web app, for example, is call FlashQL's `sync.sync()` API to resume work on network reconnection:
1097+
a web app, for example, is call FlashQL's `sync.sync()` API to resume work on network reconnection:
10981098

10991099
```js
11001100
window.addEventListener('online', () => {
@@ -1104,15 +1104,23 @@ window.addEventListener('online', () => {
11041104

11051105
**`sync()`** knows how to continue from last known state.
11061106

1107+
Where still necessary, these views can be refreshed explicitly:
1108+
1109+
```js
1110+
await db.query(`
1111+
REFRESH REALTIME VIEW public.orders
1112+
`);
1113+
```
1114+
11071115
---
11081116

11091117
#### Step 5: Querying the Unified Graph
11101118

11111119
At query time, LinkedQL builds a composed execution plan:
11121120

1113-
* basic views are resolved on demand
1121+
* runtime views are computed
11141122
* `materialized` and `realtime` views are resolved locally
1115-
* results are merged into a single relational execution
1123+
* everything works as a single relational storage
11161124

11171125
```js
11181126
const result = await db.query(`
@@ -1121,10 +1129,10 @@ const result = await db.query(`
11211129
u.name,
11221130
o.total,
11231131
p.title
1124-
FROM remote.users u -- non-persistent VIEW: resolved on demand from remote DB
1125-
LEFT JOIN remote.orders o -- materialized VIEW: served locally
1132+
FROM public.users u -- non-persistent VIEW: resolved on demand from remote DB
1133+
LEFT JOIN public.orders o -- materialized VIEW: served locally
11261134
ON o.customer_id = u.id
1127-
LEFT JOIN remote.posts p -- realtime VIEW: served locally, kept in sync
1135+
LEFT JOIN public.posts p -- realtime VIEW: served locally, kept in sync
11281136
ON p.author_id = u.id
11291137
LEFT JOIN public.test t -- regular table: served locally
11301138
ON t.user_id = u.id

0 commit comments

Comments
 (0)