Skip to content

Publish flashable release artifacts #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PSi86 merged 2 commits into from
Jul 30, 2026
Merged

Publish flashable release artifacts #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
802d42c
Publish flashable release artifacts
PSi86 Jul 30, 2026
8c2a4ed
Collect build metadata before the build, not after
PSi86 Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 181 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
name: Build Firmware

# Compile-only counterpart to release.yml. That workflow is the only thing that
# ever built this repo, and it tags and publishes in the same run — so a broken
# build or a broken staging step surfaced in the middle of a real release.
#
# This job runs the identical build-and-stage script against the latest
# published WLED release, verifies the resulting factory images, and ships
# nothing.

on:
pull_request:
branches: [main]
push:
branches: [main]
workflow_dispatch:
inputs:
wled_ref:
description: "Optional WLED tag/ref override. Leave empty to use the latest published WLED release."
required: false
default: ""
type: string

permissions:
contents: read

concurrency:
group: build-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-latest

env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"

steps:
- name: Check out RaceLink_WLED repository
uses: actions/checkout@v6.0.2

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"

- name: Run static tests
run: python -m unittest discover -s tests -p "test_*.py"

- name: Resolve WLED source ref
id: wled_source
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
ref="$(python scripts/resolve_wled_release.py --wled-ref "${{ inputs.wled_ref }}" --print ref)"
repository="$(python scripts/resolve_wled_release.py --wled-ref "${{ inputs.wled_ref }}" --print repository)"
echo "ref=${ref}" >> "$GITHUB_OUTPUT"
echo "repository=${repository}" >> "$GITHUB_OUTPUT"

- name: Check out WLED repository
uses: actions/checkout@v6.0.2
with:
repository: ${{ steps.wled_source.outputs.repository }}
ref: ${{ steps.wled_source.outputs.ref }}
path: external/WLED
fetch-depth: 1

- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: "24"
cache: npm
cache-dependency-path: external/WLED/package-lock.json

- name: Cache PlatformIO installation
uses: actions/cache@v6
with:
path: |
~/.platformio/.cache
~/.platformio/packages
~/.platformio/platforms
key: pio-${{ runner.os }}-${{ hashFiles('external/WLED/platformio.ini') }}
restore-keys: pio-${{ runner.os }}-

- name: Install build prerequisites
run: |
python -m pip install --upgrade pip
python -m pip install -r external/WLED/requirements.txt
# Same pin as release.yml — this job rehearses the staging step.
python -m pip install "esptool>=5,<6"
npm --prefix external/WLED ci

# The same script release.yml runs, so release day differs only in the
# version string.
- name: Build and stage every shipping profile
shell: bash
run: |
bash scripts/build_and_stage_profiles.sh \
"$GITHUB_WORKSPACE" \
"$GITHUB_WORKSPACE/external/WLED" \
"0.0.0-ci" \
"${{ steps.wled_source.outputs.ref }}" \
"$GITHUB_WORKSPACE/dist"

- name: Verify the staged factory images
run: |
python - <<'PY'
import json
from pathlib import Path

manifest = json.loads(
next(Path("dist").glob("*-assets.json")).read_text(encoding="utf-8")
)
for environment in manifest["environments"]:
assets = {asset["kind"]: asset for asset in environment["assets"]}
image = (Path("dist") / assets["factory"]["file"]).read_bytes()
app_offset = environment["app_offset"]

expectations = [
(assets["bootloader"]["offset"], b"\xe9", "bootloader"),
(assets["partitions"]["offset"], b"\xaa\x50", "partition table"),
(app_offset, b"\xe9", "application"),
]
for offset, magic, label in expectations:
found = image[offset : offset + len(magic)]
if found != magic:
raise SystemExit(
f"{environment['env']}: no {label} at 0x{offset:x} "
f"(found {found.hex()}, expected {magic.hex()})"
)

published = (Path("dist") / assets["app"]["file"]).read_bytes()
if image[app_offset : app_offset + len(published)] != published:
raise SystemExit(
f"{environment['env']}: the application embedded in the factory "
"image differs from the published application image"
)
print(
f"{environment['env']}: {environment['chip']}, DEV_TYPE "
f"{environment['dev_type']}, factory image verified ({len(image)} bytes)"
)
PY

- name: Write build summary
run: |
python - <<'PY'
import json
import os
from pathlib import Path

manifest = json.loads(
next(Path("dist").glob("*-assets.json")).read_text(encoding="utf-8")
)
lines = [
"## Build Summary",
"",
f"WLED ref: `{manifest.get('wled_ref')}`",
"",
"| Environment | Chip | DEV_TYPE | Application | Factory image |",
"|---|---|---|---|---|",
]
for environment in manifest["environments"]:
assets = {asset["kind"]: asset for asset in environment["assets"]}
lines.append(
f"| `{environment['env']}` | {environment['chip']} | "
f"{environment['dev_type']} | {assets['app']['size'] / 1024:.1f} KiB | "
f"{assets['factory']['size'] / 1024:.1f} KiB |"
)
Path(os.environ["GITHUB_STEP_SUMMARY"]).write_text(
"\n".join(lines) + "\n", encoding="utf-8"
)
PY

# The rehearsed release assets, so a pull request can be flashed on
# hardware — including the factory images — without cutting a release.
- name: Upload build output
uses: actions/upload-artifact@v7
with:
name: racelink_wled_build
if-no-files-found: error
path: dist/*
113 changes: 36 additions & 77 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,10 @@ jobs:
run: |
python -m pip install --upgrade pip
python -m pip install -r external/WLED/requirements.txt
# esptool 5 renamed merge_bin to merge-bin; pin the major so the
# rename cannot break a release, and so the defaults the staging
# code relies on (flash mode/freq/size = keep) stay put.
python -m pip install "esptool>=5,<6"
npm --prefix external/WLED ci

- name: Build shipping RaceLink firmware profiles
Expand All @@ -129,88 +133,19 @@ jobs:
RELEASE_VERSION: ${{ steps.release_version.outputs.version }}
WLED_REF: ${{ steps.wled_source.outputs.ref }}
run: |
set -euo pipefail
mkdir -p dist
: > dist/built_envs.txt
# Single source of truth: SHIPPING_PROFILE_FILENAMES in
# scripts/release_profiles.py (via iter_shipping_profiles, which also
# validates each file exists). Keep this in sync by editing that tuple
# only -- do NOT hard-code the profile list here.
mapfile -t profiles < <(
python -c "from pathlib import Path; from scripts.release_profiles import iter_shipping_profiles; [print('build_profiles/' + p.name) for p in iter_shipping_profiles(Path('build_profiles'))]"
)
if [ "${#profiles[@]}" -eq 0 ]; then
echo "No shipping profiles resolved from SHIPPING_PROFILE_FILENAMES"
exit 1
fi
printf 'Shipping profiles to build:\n'; printf ' %s\n' "${profiles[@]}"

for profile in "${profiles[@]}"; do
rm -rf external/WLED/build_output/release
mkdir -p external/WLED/build_output/release

mapfile -t envs < <(
python scripts/stage_wled_profile.py stage-profile \
--repo-root "$GITHUB_WORKSPACE" \
--wled-dir "$GITHUB_WORKSPACE/external/WLED" \
--profile "$GITHUB_WORKSPACE/$profile"
)

if [ "${#envs[@]}" -eq 0 ]; then
echo "Profile $profile did not expose any envs"
exit 1
fi

for env_name in "${envs[@]}"; do
echo "$env_name" >> dist/built_envs.txt
done

args=()
for env_name in "${envs[@]}"; do
args+=("-e" "$env_name")
done

python -m platformio run \
--project-dir "$GITHUB_WORKSPACE/external/WLED" \
"${args[@]}"

python scripts/stage_wled_profile.py stage-assets \
--profile "$GITHUB_WORKSPACE/$profile" \
--release-dir "$GITHUB_WORKSPACE/external/WLED/build_output/release" \
--dist-dir "$GITHUB_WORKSPACE/dist" \
--release-version "$RELEASE_VERSION" \
--wled-ref "$WLED_REF"
done
bash scripts/build_and_stage_profiles.sh \
"$GITHUB_WORKSPACE" \
"$GITHUB_WORKSPACE/external/WLED" \
"$RELEASE_VERSION" \
"$WLED_REF" \
"$GITHUB_WORKSPACE/dist"

{
echo "built_envs<<EOF"
sort -u dist/built_envs.txt
sort -u built_envs.txt
echo "EOF"
} >> "$GITHUB_OUTPUT"

- name: Generate SHA256 manifest
run: |
python - <<'PY'
import hashlib
import json
from pathlib import Path

dist = Path("dist")
lines = []
for path in sorted(dist.glob("RaceLink_WLED-*")):
if not path.is_file():
continue
digest = hashlib.sha256(path.read_bytes()).hexdigest()
lines.append(f"{digest} {path.name}")
if not lines:
raise SystemExit("No staged release artifacts found in dist/")
version = json.loads(Path("version.json").read_text(encoding="utf-8"))["version"]
(dist / f"RaceLink_WLED-{version}-sha256.txt").write_text(
"\n".join(lines) + "\n",
encoding="utf-8",
)
PY

- name: Commit release metadata
run: |
git add version.json
Expand All @@ -229,6 +164,7 @@ jobs:
uses: actions/upload-artifact@v7
with:
name: racelink_wled_release
if-no-files-found: error
path: dist/*

- name: Publish GitHub release
Expand All @@ -237,6 +173,29 @@ jobs:
tag_name: ${{ steps.release_version.outputs.tag }}
target_commitish: ${{ inputs.target_branch }}
generate_release_notes: true
body: |
Built against WLED `${{ steps.wled_source.outputs.ref }}`. Each
asset names both versions: its own RaceLink release and the WLED
release it wraps.

### Flashing

* `…-app.bin` — application image. This is the file for WLED's
`/update` page and for the host's firmware-update dialog.
* `…-factory-usb-serial-only.bin` — bootloader, partition table,
OTA selector and application in one image, written at offset `0x0`.
**USB serial only.** Sent over OTA it is rejected in the normal
case, but with *Ignore firmware validation* ticked it is written
into the inactive slot and bricks the node until someone
re-flashes over serial. It also rewrites the partition table, so
WLED reformats its filesystem on the next boot: presets,
`cfg.json`, the master binding and the group id are all gone and
the node returns as unconfigured. A commissioning tool, never an
update tool.
* `…-bootloader.bin` / `…-partitions.bin` / `…-boot_app0.bin` — the
individual pieces, for flashing a blank chip manually.
* `…-assets.json` — machine-readable index: environment, chip,
device type, kind, flash offset, size and SHA-256 for every file.
files: dist/*

- name: Write workflow summary
Expand All @@ -251,6 +210,6 @@ jobs:
echo "- Built environments:"
while IFS= read -r env_name; do
echo " - \`${env_name}\`"
done < dist/built_envs.txt
done < built_envs.txt
echo "- Trigger: \`workflow_dispatch\`"
} >> "$GITHUB_STEP_SUMMARY"
7 changes: 7 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,10 @@ __pycache__/
.claude/
CLAUDE.md
CLAUDE.local.md

# Staged release artifacts and the scaffolding around them
# (scripts/build_and_stage_profiles.sh).
dist/
built_envs.txt
metadata.json
external/
Loading