You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -936,162 +936,88 @@ In this scenario, we demonstrate a hybrid data architecture where the goal is to
936
936
> Query remote data **as if it were local**, while controlling
937
937
> what stays remote, what gets cached locally, and what stays in sync.
938
938
939
-
This is where LinkedQL moves beyond “client” and becomes a **data architecture layer**.
940
-
941
-
At a high level, this system works like this:
942
-
943
-
- You define **where data comes from** (local vs remote)
944
-
- You define **how it is stored** (none, cached, or synced)
945
-
- LinkedQL ensures everything behaves like a single database
946
-
947
-
The idea starts with the local database – this time, instantiated with a hook to the remote database.
939
+
The idea is straight-forward in FlashQL: you simply create a view (a database view) in your local database that mirrors the remote database.
948
940
949
941
```js
950
-
const db = new FlashQL({
951
-
// The hook to remote
952
-
async onCreateForeignClient() {
953
-
return new EdgeClient({ url: '/api/db', type: 'http' });
954
-
}
955
-
});
956
-
957
-
await db.connect();
958
-
959
-
// The queries
960
-
// which can span local and remote tables
961
942
await db.query(`
962
-
SELECT * FROM remote.users
943
+
CREATE VIEW public.users AS
944
+
SELECT * FROM public.users
945
+
WITH (replication_origin = '/api/db')
963
946
`);
964
947
```
965
948
966
-
This query may:
949
+
Notice the `WITH (replication_origin = ...)` specifier. That's the magic.
967
950
968
-
- hit a remote database,
969
-
- use a local replica,
970
-
- or combine both
951
+
Rows inthisview (`public.users`) will mirror `public.users`in the upstream database.
971
952
972
-
But it always behaves like a single SQL query.
953
+
But just one more thing is required forthis to work:
973
954
974
-
The system is introduced step by step below. Much of that is mere configuration.
955
+
> a way to connect the local FlashQL instance to the upstream database.
975
956
976
-
---
977
-
978
-
#### Step 1: Connect a Local Engine to a Remote Origin
979
-
980
-
We start with FlashQL as our local database.
981
-
982
-
Then we teach it how to reach a remote database when needed.
957
+
For this, the `EdgeClient`interface introduced above comes to play:
983
958
984
959
```js
985
960
import { FlashQL } from '@linked-db/linked-ql/flashql';
986
961
import { EdgeClient } from '@linked-db/linked-ql/edge';
987
962
988
-
// The local database
989
963
const db = new FlashQL({
990
-
991
-
// Called whenever a query references a foreign origin and needs a client
992
-
async onCreateForeignClient(origin) {
993
-
994
-
if (origin === 'primary') {
995
-
// This client will be used to reach the remote database we've designated as 'primary'
996
-
returnnewEdgeClient({
997
-
url:'/api/db',
998
-
type:'http',
999
-
});
1000
-
}
1001
-
1002
-
thrownewError(`Unknown origin: ${origin}`);
964
+
// The hook to remote
965
+
async onCreateForeignClient(originUrl) {
966
+
return new EdgeClient({ url: originUrl, type: 'http' });
1003
967
}
1004
968
});
1005
969
1006
970
await db.connect();
1007
971
```
1008
972
1009
-
Next, we create a schema – more specifically called a "namespace" in FlashQL – with the foreign origin behaviour.
973
+
This is now a local FlashQL instance that can talk to an upstream database ondemand.
1010
974
1011
-
```js
1012
-
awaitdb.query(`
1013
-
CREATE SCHEMA remote
1014
-
WITH (replication_origin = 'primary')
1015
-
`);
1016
-
```
1017
-
1018
-
This namespace will be able to incorporate data from a foreign origin tagged here as "primary". The result of `onCreateForeignClient()`
1019
-
will be the foreign client interface.
1020
-
1021
-
---
1022
-
1023
-
##### Decoding the above
1024
-
1025
-
* FlashQL is your **local database**
1026
-
* `remote` schema is a **regular schema (namespace)** that can have tables and views (`VIEWS`)
1027
-
* `replication_origin` is a setting that extends the behaviour of the namespace to include objects from a foreign origin
1028
-
* the value: `"primary"` is a custom identifier for this **remote database**. (FlashQL lets the origin details be anything: an identifier, a URL, a database connection string, etc.)
1029
-
* `EdgeClient` – the result of `onCreateForeignClient()` – is how FlashQL will talk to that remote system
1030
-
1031
-
At this point:
1032
-
1033
-
* nothing is mirrored yet
1034
-
* no data is fetched
1035
-
* we’ve only defined **how to reach the remote** from the local
975
+
Above, the `onCreateForeignClient()` hook will recieve `'/api/db'` – the value of the `replication_origin` config.
1036
976
1037
-
But the local namespace by itself is ready for use as normal. You can create tables as necessary:
977
+
Now, querying `public.users` on the local database will query `public.users` on the upstream database:
1038
978
1039
979
```js
1040
-
// FlashQL accepts multiple statements in a single call
1041
980
await db.query(`
1042
-
-- Define tables explicitly
1043
-
CREATE TABLE remote.users (
1044
-
id INT PRIMARY KEY,
1045
-
name TEXT
1046
-
);
1047
-
1048
-
-- Seed local data
1049
-
INSERT INTO remote.users (id, name)
1050
-
VALUES (1, 'Ada'), (2, 'Linus');
981
+
SELECT*FROMpublic.users;
1051
982
`);
1052
983
```
1053
984
1054
-
But the containers of foreign data will not be tables. They will be views (VIEWS).
1055
-
1056
-
Think of views as table-like objects whose contents come from other tables – whether local tables or remote tables.
1057
-
1058
-
---
1059
-
1060
-
#### Step 2: Let’s Mirror Foreign Tables Locally
1061
-
1062
-
A view created inside a namespace with `replication_origin`, like the above, automatically resolves its data from the foreign origin.
985
+
Being a regular table, it can be used just like one – e.g. in joins:
1063
986
1064
987
```js
1065
988
await db.query(`
1066
-
CREATE VIEW remote.users AS
1067
-
SELECT * FROM public.users
989
+
SELECT*FROMpublic.posts
990
+
LEFTJOINpublic.usersONposts.user_id=users.id;
1068
991
`);
1069
992
```
1070
993
1071
-
Rows in this view (`remote.users`) will mirror `public.users` in foreign origin. The persistence of this data is configurable.
994
+
The query executes as one relational graph – but composed of both local and remote data.
1072
995
1073
-
FlashQL supports **three persistence modes**. These modes determine how mirroring works; i.e. whether data stays remote, or is cached locally, or stays in sync.
996
+
Given this as the base, FlashQL further lets you create other types of views with different replication modes.
997
+
998
+
These modes determine how mirroring works; i.e. whether data stays remote, or is cached locally, or stays in sync.
| `persistence="none"`(the default) | Views set to this mode don't copy the remote data locally; they simply act as local references to remote data. These are called "non-persistent views" |
1078
-
| `persistence="materialized"` | Views set to this mode copy the origin data locally and behave as local tables from that moment on; cached data is refreshed manually. These are called "materialized views" |
1079
-
| `persistence="realtime"`| Views set to this mode copy the remote data locally and behave as local tables from that moment on; **but most notably, local data is kept in sync with origin data**. These are called "realtime views" |
1002
+
|Basic views(the default) |This is the default idea of a view: a table that has no actual rows but just a query that executes at query-time.|
1003
+
|Materialized views | These views go ahead to copy the origin data for local use and behave as local tables from that moment on.|
1004
+
|Realtime views| These views are materialized views that not just copy origin data, but also stay in sync with origin data.|
1080
1005
1081
-
* use `none`when freshness matters more than offline access
1082
-
* use `materialized` when you want a local cache you can refresh deliberately
1083
-
* use `realtime` when the local copy should stay warm automatically after initial sync
1006
+
* use basic views when you just want to federate remote data and don't need offline access
1007
+
* use `materialized` views when you want a local copy for offline access
1008
+
* use `realtime` views when the local copy should stay in sync origin data
@@ -1115,8 +1041,9 @@ This is the lightest-weight mode. It gives you unification without local storage
1115
1041
1116
1042
```js
1117
1043
await db.query(`
1118
-
CREATE MATERIALIZED VIEW remote.orders AS
1044
+
CREATE MATERIALIZED VIEW public.orders AS
1119
1045
SELECT * FROM public.orders
1046
+
WITH (replication_origin = '/api/db')
1120
1047
`);
1121
1048
```
1122
1049
@@ -1130,13 +1057,15 @@ await db.query(`
1130
1057
1131
1058
This is **materialization**:
1132
1059
1133
-
> Keeping a local snapshot of remote data for performance or offline use
1060
+
This is the mode to reach for when the data should remain queryable while offline.
1134
1061
1135
-
This is the mode to reach for when:
1062
+
These views can be refreshed explicitly:
1136
1063
1137
-
* the dataset is expensive to fetch repeatedly
1138
-
* it should remain queryable while offline
1139
-
* and "fresh on demand" is good enough
1064
+
```js
1065
+
await db.query(`
1066
+
REFRESH MATERIALIZED VIEW public.orders
1067
+
`);
1068
+
```
1140
1069
1141
1070
---
1142
1071
@@ -1145,9 +1074,8 @@ This is the mode to reach for when:
1145
1074
```js
1146
1075
await db.query(`
1147
1076
CREATE REALTIME VIEW remote.posts AS
1148
-
SELECT *
1149
-
FROM public.posts
1150
-
WHERE post_type = 'NEWS'
1077
+
SELECT * FROM public.posts
1078
+
WITH (replication_origin = '/api/db')
1151
1079
`);
1152
1080
```
1153
1081
@@ -1161,76 +1089,28 @@ await db.query(`
1161
1089
1162
1090
This is **realtime mirroring**:
1163
1091
1164
-
> A local table that tracks and syncs with the remote table over time
1165
-
1166
-
This is the richest mode:
1167
-
1168
1092
* it starts with local state
1169
1093
* keeps that state queryable even when the app is temporarily disconnected
1170
1094
* and then catches up again when connectivity returns
1171
1095
1172
-
---
1173
-
1174
-
#### Step 3: Running Sync
1175
-
1176
-
On having defined the views, you activate the synchronization via:
1177
-
1178
-
```js
1179
-
awaitdb.sync.sync();
1180
-
```
1181
-
1182
-
---
1183
-
1184
-
##### What `sync()` does
1185
-
1186
-
`sync()` is the coordination engine for "materialized" and "realtime" views.
1187
-
It is what turns definitions (VIEWS) into state (local data + subscriptions).
1188
-
1189
-
It:
1190
-
1191
-
* fetches data for all `materialized` views
1192
-
* does the same for `realtime` views and starts syncing right away – with backpressure and replay support:
1193
-
* performs catch-up if the app was offline
1194
-
* ensures local state matches expected remote state
1195
-
1196
-
---
1197
-
1198
-
##### `sync()` is:
1199
-
1200
-
* **Idempotent** → safe to call multiple times
1201
-
* **Resumable** → knows hot to continue from last known state
1202
-
* **Network-aware** → designed for reconnect flows
1203
-
1204
-
##### Typical usage:
1205
-
1206
-
First: **the initial call after defining views**:
1207
-
1208
-
```js
1209
-
awaitdb.sync.sync();
1210
-
```
1211
-
1212
-
Second: **the optional wiring to the app-level network signal:**
1096
+
Realtime views are designed to be resilient to network disconnects. All you need to do in
1097
+
web app, for example, is call FlashQL's `sync.sync()`API to resume work on network reconnection:
1213
1098
1214
1099
```js
1215
1100
window.addEventListener('online', () => {
1216
1101
db.sync.sync(); // re-sync on reconnect
1217
1102
});
1218
1103
```
1219
1104
1220
-
At that point, your local database is no longer just "configured".
1221
-
It is now hydrated, subscribed where necessary, and ready to behave like a unified relational graph.
1222
-
1223
-
> Note that the initial `db.sync.sync()` can be automatically-handled by FlashQL. Simply pass `autoSync:true` in constructor parameters:
1224
-
>
1225
-
> `newFlashQL({ autoSync:true });`
1105
+
**`sync()`** knows how to continue from last known state.
1226
1106
1227
1107
---
1228
1108
1229
1109
#### Step 5: Querying the Unified Graph
1230
1110
1231
1111
At query time, LinkedQL builds a composed execution plan:
1232
1112
1233
-
* non-persistent views are resolved on demand
1113
+
*basic views are resolved on demand
1234
1114
*`materialized` and `realtime` views are resolved locally
1235
1115
* results are merged into a single relational execution
1236
1116
@@ -1254,19 +1134,7 @@ const result = await db.query(`
1254
1134
1255
1135
---
1256
1136
1257
-
#### Resolution summary
1258
-
1259
-
* `remote.users` → fetched on demand from the remote DB
1260
-
* `remote.orders` → served from the local cache created by materialization
1261
-
* `remote.posts` → served locally, then kept hot by realtime sync
1262
-
* `public.test` → an ordinary local table with no remote involvement
1263
-
* The planner treats all of them as one relational graph even though they come from different storage modes
1264
-
1265
-
This is the key architectural promise of LinkedQL:
1266
-
1267
-
> You choose data placement and sync policy per relation, but you still query the result as one database.
1268
-
1269
-
For the FlashQL side of this model, see [Federation & Sync ↗](https://linked-ql.netlify.app/flashql/foreign-io) and [FlashQL Sync ↗](https://linked-ql.netlify.app/flashql/sync).
1137
+
For full details, see [Federation & Sync ↗](https://linked-ql.netlify.app/flashql/foreign-io) and [FlashQL Sync ↗](https://linked-ql.netlify.app/flashql/sync).
0 commit comments