marilib: make the metrics probe a usable measurement instrument - #169
Open
geonnave wants to merge 18 commits into
Open
marilib: make the metrics probe a usable measurement instrument#169geonnave wants to merge 18 commits into
geonnave wants to merge 18 commits into
Conversation
edge_rx_ts_us must be written before the pending-probe match, not after: a reply that arrives past its timeout has no pending entry, but the frame is still forwarded to the cloud, where an unstamped 0 reads as a large negative round trip. Keep the stamp ahead of the match if this function is ever reordered. AI-assisted: Claude Opus 5
log_setup_parameters rewrites metrics_setup.csv on every call by design. It previously wrote once, which silently dropped every field added after construction - including schedule_name, which is only knowable when the first GATEWAY_INFO arrives. AI-assisted: Claude Opus 5
The metrics probes are unicast downlink packets competing for the same D slots as the filler traffic, so a flag that ignored them put 90% on the link when asked for 75%. The probe rate is now subtracted from the target, recomputed per packet since it grows with the node count. AI-assisted: Claude Opus 5
The interval that keeps the measurement footprint constant depends on the schedule (1.0 s on tiny, 7.9 on huge at 15%), so a period is the wrong thing to hand an operator: it makes the instrument scale with the node count unless recomputed by hand per run. --probe-load takes the invariant instead and derives the period from the schedule the gateway reports. -i still overrides, for reproducing a fixed cadence. AI-assisted: Claude Opus 5
The measured rate counts retries, so it reads above target exactly when probes are timing out - the case where the link carries more than the requested load and the latency series is being censored. Also carries the RSSI colouring on the per-node table. AI-assisted: Claude Opus 5
AI-assisted: Claude Opus 5
Same variable names as the dotbot CLI. MQTTAdapter.from_url already took the arguments and its docstring said callers thread env vars through it, but none of them did, so authenticating against a broker meant putting credentials in the URL and therefore in shell history. AI-assisted: Claude Opus 5
from_url takes is_edge positionally, so splatting the credentials with a single star landed one of them on it. They go through as keyword arguments now. The cadence derivation no longer blocks at startup with a deadline: GATEWAY_INFO arrives asynchronously over serial and can take longer than any timeout worth aborting a run over. It is applied from the main loop the moment the schedule is known, which costs nothing beforehand since the tester skips every cycle while no node has joined. AI-assisted: Claude Opus 5
Erasing a gateway meant running the two nrfjprog recovers by hand, one per core, since the script would only erase on the way to programming. AI-assisted: Claude Opus 5
Once --load began subtracting the probe rate, total offered downlink became constant by construction, so a share-derived cadence bought nothing and cost sampling resolution: 15% meant 7.9 s per node on the huge schedule. --probe-load stays for holding the footprint constant. AI-assisted: Claude Opus 5
The slotframe is the network's own clock, so one K samples every schedule at the same rate relative to its own dynamics. It also holds the probe's downlink share near-constant for free: that share is N/(K*d_down), and N/d_down is 4.13 to 5.00 across the four schedules, so it is ~4.6/K on any of them. A wall-clock cadence gave an 8x spread in share and a 10x spread in sample count over the same sweep. set_interval had been rejecting any interval above MARI_PROBE_STATS_MAX_LEN, comparing a duration against a deque length. Nothing breaks at a long interval; the bound worth having is at the short end, where the probe stream outruns downlink capacity, and that depends on the schedule and node count. The TUI reports it instead. AI-assisted: Claude Opus 5
test_state.rate was only ever set by LoadTester, which does not start when no load is generated - so the one case where the probe budget is the whole story was the case where the TUI hid it. AI-assisted: Claude Opus 5
Each node owns exactly one uplink slot per slotframe, so its budget is 1/sf_duration packets per second and the number worth seeing is how much of it is in use. Measured from received frames rather than derived from the node app's send rates: the 500 ms status packet alone is 51% of that budget on the huge schedule, and that is a firmware constant the host should not be asserting. AI-assisted: Claude Opus 5
What --load and --probe-every were set to is what was asked for; this is what the link is actually carrying, counted from frames. Downlink is measured against d_down slots per slotframe, uplink against one slot per connected node, since unassigned U slots are idle by construction and counting them would hide node saturation behind an empty schedule. AI-assisted: Claude Opus 5
log_events.csv already flushed per row; the gateway and node CSVs only at close. So a run was unreadable while in progress, and a process that died without close() left empty files behind. AI-assisted: Claude Opus 5
The gateway's schedule is a compile-time pointer, so switching it needs a rebuild; nodes do not, since they adopt what the beacon advertises. Building all of them once turns a campaign's schedule changes into flashes. main.c is restored by an EXIT trap, and the script refuses to start if the file is already modified rather than reverting someone else's edit. AI-assisted: Claude Opus 5
The refusal to run on a modified main.c blocked the normal case: the bench net id lives in that file and is never committed. The snapshot and restore already preserve whatever state the file was in, so the guard bought nothing; the restore is now verified instead, and says where the copy is if it could not be put back. AI-assisted: Claude Opus 5
AI-assisted: Claude Opus 5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preparing marilib to measure latency and PDR for a paper campaign turned up
several things that made the numbers wrong or unreadable rather than merely
inconvenient. This fixes them and adds the firmware tooling the campaign needs.
The probe stream had to stop scaling with the fleet
The probe interval was per node, so the offered probe load grew linearly with
the node count: a fixed 5 s/node is 20 probes/s at 100 nodes but 2 at 10. That
makes the instrument itself the variable under test when the fleet size or the
schedule changes.
--probe-everynow cadences in slotframes instead of seconds, so the probeshare of downlink stays roughly constant across schedules (N/(K x d_down) is
within a few percent of 4.6/K for tiny through huge).
-istill sets awall-clock interval when that is what you want. The cadence is derived lazily,
once the gateway has actually reported its schedule, because deriving it up
front meant a fixed deadline that could abort a run before the first beacon.
--loadnow means total downlink occupancy: the load generator subtractsthe measured probe rate, so passing 75 puts 75% on the link rather than 75%
plus whatever the probes were already using. Recomputed per packet, since the
probe stream grows with the node count.
Also removes an upper bound in
set_intervalthat compared a value in secondsagainst a deque length, and so silently clamped long intervals.
Late probe replies were being discarded after being counted
The edge stamped
edge_rx_ts_usonly after matching a reply against thepending-probe table, so a reply arriving after its entry had been evicted was
dropped without a timestamp - censoring exactly the slow tail that a latency
distribution is about. Stamping happens before the match now, so a late reply
still reaches the cloud carrying usable data.
The logs could not reconstruct a run
node_metrics.csvgained the raw counters (gw_tx_count,gw_rx_count,node_tx_count,node_rx_count,edge_tx_count,edge_rx_count) and theper-hop latency halves, so PDR can be computed after the fact from counter
deltas rather than trusted from a running average. This matters because a node
rejoining zeroes the gateway's counters but not its own, and only the raw pairs
let an analysis segment around that.
metrics_setup.csvrecords the run's parameters, the write-once guard on it isgone (it prevented recording a cadence derived after startup), and the metrics
CSVs are flushed at the sampling rate so a run can be analysed while it is still
going.
TUI and CLI
The TUI shows the probe budget as a share of downlink, the measured probe rate
with a retry indicator, uplink slot utilisation, and raw link saturation in both
directions - enough to see a saturating link during a run instead of at the
analysis. MQTT credentials now come from the environment.
Firmware tooling
flash.shlists only the role's own device family, reports other-family boardsseparately so a missing board is distinguishable from one plugged in under the
wrong role, and gains
--erase-onlyand abothrole.build-schedules.shbuilds one gateway net-core image per schedule. It edits asingle line to select the schedule and restores the file byte-for-byte on exit,
including through a failure, because a bench tree legitimately carries local
edits that must survive the build.
Validation
The Python changes have run continuously on the bench: 100 nodes at 2 m and
10 m, four traffic loads, and a schedule sweep across tiny/medium/big/huge, with
the counters cross-checked against the sliding-window PDR marilib reports
independently.
hatch fmt --checkandhatch testpass; firmware changes arescripts only, no compiled code is touched.