Docs/no em dashes

docs/advanced/asset-health-score.mdx

@@ -10,33 +10,32 @@ import TabItem from '@theme/TabItem';
 :::info At a glance
 **Effort:** ~20–30 minutes · **You'll build:** a composite 0–100 health index from
 several signals, with a healthy/watch/critical classification · **Stack:** the SDK plus
-a little arithmetic — no model training.
+a little arithmetic, no model training.
 :::
 
 A machine rarely fails on one signal. Vibration is creeping up, the bearing runs a
-little hot, oil pressure sags at load — each is fine alone, but together they tell a
+little hot, oil pressure sags at load, each is fine alone, but together they tell a
 story. A **health score** rolls those signals into one comparable number so an operator
 can rank a whole fleet at a glance and a dashboard can show green/amber/red. It's the
 lightweight cousin of [predictive maintenance](/advanced/predictive-maintenance): no
 model to train, just a transparent, tunable index.
 
-:::tip New to machine learning?
-No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
-ideas in plain language — model, feature, training, and the algorithm itself — and use
-[Generate sample data](/advanced/generate-sample-data) for a sandbox to run this against.
-:::
-
-:::tip Need data to run this?
+:::info Needs a sandbox
 This reads several pump signals. [Generate a sandbox](/advanced/generate-sample-data)
-first — section A ingests `pump_07_bearing_temp_c`, `pump_07_oil_pressure_kpa` and a
+first, section A ingests `pump_07_bearing_temp_c`, `pump_07_oil_pressure_kpa` and a
 stand-in `pump_07_vibration_anomaly` (or run [predictive maintenance](/advanced/predictive-maintenance)
 to produce the real one).
 :::
 
+:::tip New to machine learning?
+No background needed. Skim the [gentle primer](/advanced/machine-learning-gently) for the
+ideas in plain language, model, feature, training, and the algorithm itself.
+:::
+
 ## 1. Pull the latest value of each signal
 
-Take the most recent reading (or short average) of each contributing series for the
-asset.
+Take the most recent reading of each contributing series for the asset. The seed writes
+these hourly, so ask for the latest datapoint rather than a short window that may be empty.
 
 <Tabs groupId="lang">
 <TabItem value="python" label="Python">
@@ -47,13 +46,8 @@ import intellistream_datahub_sdk, pandas as pd
 client = intellistream_datahub_sdk.DataHubClient.from_env()
 
 def latest(external_id):
-    rf = intellistream_datahub_sdk.RetrieveFilter(
-        ts=external_id,
-        start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(minutes=15),
-        end=pd.Timestamp.now(tz="UTC"),
-        aggregates=["avg"], granularity="15m")
-    pts = client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()
-    return float(pts[-1].average)
+    pts = client.timeseries.retrieve_latest_datapoints([external_id])[0].get_datapoints()
+    return float(pts[-1].value)
 
 signals = {
     "vibration": latest("pump_07_vibration_anomaly"),   # 0..~1, from the anomaly model
@@ -66,13 +60,13 @@ signals = {
 <TabItem value="java" label="Java">
 
 ```java
-// The same retrieve, per signal; the scoring arithmetic below is shown in Python.
+// Java has no latest-datapoint call: read the last day and take the newest point.
+// The scoring arithmetic below is shown in Python.
 var filter = new RetrieveFilter();
 filter.setExternalId("pump_07_bearing_temp_c");
-filter.setStart(ZonedDateTime.now().minusMinutes(15));
+filter.setStart(ZonedDateTime.now().minusHours(24));
 filter.setEnd(ZonedDateTime.now());
-filter.setAggregates(List.of("avg"));
-filter.setGranularity("15m");
+filter.setLimit(100);
 
 var request = new DataRetriever<RetrieveFilter>();
 request.setItems(List.of(filter));
@@ -84,21 +78,14 @@ double bearingTemp = Double.parseDouble(pts.get(pts.size() - 1).getValue());
 <TabItem value="rust" label="Rust">
 
 ```rust
-// The same retrieve, per signal; the scoring arithmetic below is shown in Python.
-use intellistream_datahub_sdk::generic::{DataWrapper, RetrieveFilter};
-use chrono::Utc;
-
-let filter = RetrieveFilter {
-    external_id: Some("pump_07_bearing_temp_c".into()),
-    start: Some(Utc::now() - chrono::Duration::minutes(15)),
-    end: Some(Utc::now()),
-    aggregates: Some(vec!["avg".into()]),
-    granularity: Some("15m".into()),
-    ..Default::default()
-};
-let s = api.time_series
-    .retrieve_datapoints(&DataWrapper::from(vec![filter])).await?
+// The latest datapoint, per signal; the scoring arithmetic below is shown in Python.
+use intellistream_datahub_sdk::generic::{DataWrapper, IdAndExtId};
+
+let latest = api.time_series
+    .retrieve_latest_datapoint(&DataWrapper::from(vec![
+        IdAndExtId::from_external_id("pump_07_bearing_temp_c")])).await?
     .get_items().remove(0);
+let bearing_temp = latest.datapoints.last().and_then(|p| p.value).unwrap_or(f64::NAN);
 ```
 
 </TabItem>
@@ -140,8 +127,8 @@ band = "healthy" if score >= 80 else "watch" if score >= 60 else "critical"
 
 ## 4. Publish the score and flag the bad ones
 
-Write the score back as its own series — now you can chart, rank and subscribe to asset
-health like any other signal — and raise an event when an asset drops to `critical`.
+Write the score back as its own series, now you can chart, rank and subscribe to asset
+health like any other signal, and raise an event when an asset drops to `critical`.
 
 ```python
 client.timeseries.create([intellistream_datahub_sdk.TimeSeries(
@@ -156,27 +143,32 @@ if band == "critical":
         event_time=pd.Timestamp.now(tz="UTC"),
         metadata={"asset": "pump_07", "score": str(score),
                   "worst_signal": max(parts, key=parts.get)})])
+
+# read the score back
+stored = client.timeseries.retrieve_latest_datapoints(["pump_07_health_score"])[0].get_datapoints()
+print(f"health {stored[-1].value:.1f} ({band})")
+assert abs(stored[-1].value - score) < 0.01, "the score was not written"
 ```
 
-Run it across the fleet on a schedule and you have a single ranked health view —
+Run it across the fleet on a schedule and you have a single ranked health view,
 `worst_signal` in the metadata tells maintenance *why* each asset is red.
 
 ## Where to take it further
 
 - **Feed it the model.** Swap the hand-set vibration limits for the
   [anomaly score](/advanced/predictive-maintenance) as a direct input.
-- **Trend the score.** A falling health score over days is itself a predictor — forecast
+- **Trend the score.** A falling health score over days is itself a predictor, forecast
   it like any [other series](/advanced/demand-forecasting).
 - **Roll up the graph.** Average child scores up a [site graph](/guides/model-assets-graph)
   for a line- or plant-level health number.
 
 ## Further reading
 
-- **Normalising signals to a common scale** — [Feature scaling](https://en.wikipedia.org/wiki/Feature_scaling)
-- **The ideas in plain language** — [Machine learning, gently](/advanced/machine-learning-gently)
+- **Normalising signals to a common scale**: [Feature scaling](https://en.wikipedia.org/wiki/Feature_scaling)
+- **The ideas in plain language**: [Machine learning, gently](/advanced/machine-learning-gently)
 
 ## See also
 
-- [Query & aggregate](/guides/query-and-aggregate) — pulling the contributing signals.
-- [Predictive maintenance](/advanced/predictive-maintenance) — a learned input to the score.
-- [Turn readings into events](/guides/detect-events) — flagging critical assets.
+- [Query & aggregate](/guides/query-and-aggregate): pulling the contributing signals.
+- [Predictive maintenance](/advanced/predictive-maintenance): a learned input to the score.
+- [Turn readings into events](/guides/detect-events): flagging critical assets.

docs/advanced/data-cleaning-lineage.mdx

@@ -14,19 +14,23 @@ linked in the graph, so the whole `raw → … → result` chain is traceable bo
 **Stack:** the SDK for data and graph, plus `pandas`, `numpy`.
 :::
 
-:::tip New to this?
-No data-science background needed — see the [gentle primer](/advanced/machine-learning-gently)
-for any unfamiliar term, and [generate a sandbox](/advanced/generate-sample-data) to run
-this against.
+:::info Needs a sandbox
+This reads two raw sensors that already exist. [Generate a sandbox](/advanced/generate-sample-data)
+first, section K ingests `engine_temperature_raw` and `engine_vibration_raw`, defects and all.
+:::
+
+:::tip New to machine learning?
+No data-science background needed. Skim the [gentle primer](/advanced/machine-learning-gently)
+for any unfamiliar term.
 :::
 
 **The idea in one paragraph.** Real data never goes straight from sensor to model. It's
-cleaned, resampled, turned into features, combined, scored — often a dozen steps, each
+cleaned, resampled, turned into features, combined, scored, often a dozen steps, each
 producing a new series. After a few of these, nobody remembers what came from what. Two
-questions then become impossible to answer: *"this number looks wrong — where did it come
-from?"* and *"this sensor was faulty — what did it poison?"* **Data lineage** fixes both:
-record every step as nodes in the graph — `raw → cleaning function → cleaned → feature
-function → feature → …` — and the answers become a single graph walk.
+questions then become impossible to answer: *"this number looks wrong, where did it come
+from?"* and *"this sensor was faulty, what did it poison?"* **Data lineage** fixes both:
+record every step as nodes in the graph, `raw → cleaning function → cleaned → feature
+function → feature → …`, and the answers become a single graph walk.
 
 ## 1. Run the transformations, writing each result back
 
@@ -53,7 +57,7 @@ def store(ext_id, series, name=None, unit="value"):
     client.timeseries.insert_from_lists(timestamps=s.index, values=s.to_numpy(), ts=ext_id)
 ```
 
-The pipeline — two raw sensors, cleaned, then several features extracted from each, then
+The pipeline, two raw sensors, cleaned, then several features extracted from each, then
 assembled and scored:
 
 ```python
@@ -83,15 +87,16 @@ store("engine_feature_vector", feature_vector)
 store("engine_health_score", (100 - feature_vector.abs()).clip(0, 100))
 ```
 
-Every box in the pipeline is now a stored series — but stored series alone don't tell you
+Every box in the pipeline is now a stored series, but stored series alone don't tell you
 how they *relate*. That's step 2.
 
 ## 2. Record the lineage graph
 
 Model each transformation as a **function node** and link it up: each input series is
 `processed_by` the function, which `produces` an output series. Done across the pipeline,
-this builds a branching, ten-deep lineage DAG in one call. (`resources.create` is the
-same call in Java and Rust — see [model assets as a graph](/guides/model-assets-graph).)
+this builds a branching lineage DAG of 18 nodes (8 functions and 10 series) in one call.
+(`resources.create` is the same call in Java and Rust, see
+[model assets as a graph](/guides/model-assets-graph).)
 
 ```python
 def fn(ext_id, name):
@@ -127,25 +132,29 @@ client.resources.create(functions, edges)
 ```
 
 That graph branches (each cleaned signal feeds two features), converges (four features
-into one vector), and runs ten nodes deep from raw sensor to health score.
+into one vector), and runs nine nodes deep from raw sensor to health score:
+`engine_temperature_raw` → `clean_temp_fn` → `engine_temperature_clean` → `roll_mean_fn` →
+`engine_temp_roll_mean` → `assemble_fn` → `engine_feature_vector` → `score_fn` →
+`engine_health_score`.
 
-## 3. The payoff — trace it backward
+## 3. The payoff, trace it backward
 
 Someone questions the health score. Walk the graph back from it and the entire recipe
 appears: every function and every series between the score and the **raw sensors** it
 ultimately came from. No guessing, no stale wiki page.
 
 ```python
 back = client.resources.fetch_related(external_id="engine_health_score", depth=12)
-raw_sources = [n.external_id for n in back.nodes if n.external_id.endswith("_raw")]
+raw_sources = {n.external_id for n in back.nodes if n.external_id.endswith("_raw")}
 print("this score derives from:", raw_sources)   # engine_temperature_raw, engine_vibration_raw
+assert raw_sources == {"engine_temperature_raw", "engine_vibration_raw"}, raw_sources
 ```
 
-## 4. The payoff — trace it forward (impact analysis)
+## 4. The payoff, trace it forward (impact analysis)
 
 Now the opposite, and the bigger win. The temperature sensor is found faulty. **What did
 it poison?** Walk *forward* from the raw sensor and every downstream series it touched
-lights up — while the vibration-only features stay clean. That's the precise list to
+lights up, while the vibration-only features stay clean. That's the precise list to
 recompute or quarantine.
 
 ```python
@@ -155,6 +164,7 @@ affected = [n.external_id for n in fwd.nodes
 print("affected by the bad temperature sensor:", affected)
 # engine_temperature_clean, engine_temp_roll_mean, engine_temp_roc,
 # engine_feature_vector, engine_health_score   — but NOT the vibration features
+assert "engine_health_score" in affected and "engine_vib_rms" not in affected, affected
 ```
 
 This is the same [blast-radius traversal](/guides/correlate-alarms) used for asset
@@ -164,21 +174,21 @@ damage spreads.
 ## Where to take it further
 
 - **Version the functions.** Bump `clean_temp_fn` to `clean_temp_fn_v2` and the old
-  outputs still point at the version that made them — reproducibility for free.
+  outputs still point at the version that made them, reproducibility for free.
 - **Quality events.** Emit a `data_quality` [event](/guides/detect-events) per run with
   what each step changed, for an audit trail beside the lineage.
 - **Feed the models.** Point the [forecasters](/advanced/lstm-forecasting) and
-  [detectors](/advanced/predictive-maintenance) at the cleaned/feature series — and their
+  [detectors](/advanced/predictive-maintenance) at the cleaned/feature series, and their
   outputs become new nodes on the same lineage graph.
 
 ## Further reading
 
-- **Data lineage** — [Wikipedia](https://en.wikipedia.org/wiki/Data_lineage)
-- **Directed acyclic graph (DAG)** — [Wikipedia](https://en.wikipedia.org/wiki/Directed_acyclic_graph)
-- **Outlier removal (Hampel / MAD)** — [Median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation)
+- **Data lineage**: [Wikipedia](https://en.wikipedia.org/wiki/Data_lineage)
+- **Directed acyclic graph (DAG)**: [Wikipedia](https://en.wikipedia.org/wiki/Directed_acyclic_graph)
+- **Outlier removal (Hampel / MAD)**: [Median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation)
 
 ## See also
 
-- [Model assets as a graph](/guides/model-assets-graph) — the nodes-and-relations basics.
-- [Correlate alarms with the graph](/guides/correlate-alarms) — the traversal behind both payoffs.
-- [Generate sample data](/advanced/generate-sample-data) — raw signals to run the pipeline on.
+- [Model assets as a graph](/guides/model-assets-graph): the nodes-and-relations basics.
+- [Correlate alarms with the graph](/guides/correlate-alarms): the traversal behind both payoffs.
+- [Generate sample data](/advanced/generate-sample-data): raw signals to run the pipeline on.