feat(gr26): persist CAN frames and trace signals back to source#139
Open
feat(gr26): persist CAN frames and trace signals back to source#139
Conversation
Adds two gr26-local tables. gr26_can stores the raw CAN frame (vehicle, node, can id, post-MQTT-header bytes, timestamp) with a composite unique key on (vehicle_id, node_id, timestamp). gr26_can_signal is a 1-N join keyed on signal_id that maps each persisted signal back to the frame it was decoded from. Both tables are gr26-only so the schema can change year-to-year without touching the shared signal table or mapache-go. HandleMessage now upserts the gr26_can row first (returning the stored ulid), persists the decoded signals, then bulk-inserts the join rows. The signal upsert in CreateSignals stops overwriting id on conflict and uses Returning so signals[i].ID reflects the actually-stored value across retransmits — the join table assumes signal ids are stable. WS publish moves to after the DB insert so subscribers see the persistent id rather than the freshly-generated-then-discarded one.
✅ Deploy Preview for gr-mapache canceled.
|
Move CreateCAN ahead of the messageStruct lookup and FillFromBytes step so unknown can ids and decode failures still produce a gr26_can row. The frame ends up in the database with no signals or join rows, which is the right shape for "what bytes did we receive that we couldn't parse" debugging — same dataset answers both "trace this signal" and "why did this signal never appear." Length and upload-key checks still bail before any persistence: a frame shorter than the MQTT envelope can't be sliced safely, and an invalid upload key means the data isn't trusted to belong to the vehicle.
Adds a nullable jsonb metadata column on gr26_can. HandleMessage now
attempts the decode first, then writes the frame in a single insert
with the outcome stamped in:
- decode succeeded -> metadata is null
- unknown can id -> {"status":"unknown_can_id"}
- decode error -> {"status":"decode_error","error":"..."}
Single write per frame instead of insert-then-update, and the upsert's
DoUpdates picks up the new column so retransmits with different
outcomes overwrite the old metadata.
Queries: "frames we couldn't decode" is now a one-liner — WHERE
metadata IS NOT NULL — without the LEFT JOIN against gr26_can_signal.
Pairs the categorical status field with a human-readable note. Renames
the prior decode_error 'error' field to 'note' so both states share a
schema:
{"status":"unknown_can_id","note":"no decoder registered for can id 0x999"}
{"status":"decode_error","note":"<err.Error()>"}
Successful decodes now write {"status":"ok"} into gr26_can.metadata
instead of leaving it null. Every frame has a populated metadata field,
so the decode-outcome filter is metadata->>'status' across the board.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gr26_cantable stores each decoded CAN frame (vehicle, node, can id, post-MQTT-header bytes, timestamp) with a composite unique key on(vehicle_id, node_id, timestamp).gr26_can_signaltable is a 1-N join keyed onsignal_idthat maps each persisted signal back to the frame it came from.HandleMessagenow upserts thegr26_canrow first, persists signals, then bulk-inserts the join rows.idon conflict and usesclause.Returningsosignals[i].IDreflects the actually-stored value across retransmits — required because the join table assumes signal ids are stable.signaltable andmapache-goare untouched so this can change year-to-year without affecting other services.