Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,17 @@ jobs:
run: uv sync --python ${{ matrix.python }}

# ruff lint/format are Python-version-agnostic; run once, on the floor.
# PIN the ruff version: bare `uvx ruff` resolves the latest release, so a
# new ruff (0.16.0 broadened the default rule set — E501, N802, S-codes)
# would spontaneously fail CI on unchanged code. Bump this deliberately,
# fixing findings in the same PR. Keep it in step with the dev-group cap.
- name: Lint (ruff)
if: ${{ matrix.python == '3.12' }}
run: uvx ruff check .
run: uvx ruff@0.15.22 check .

- name: Format check (ruff format)
if: ${{ matrix.python == '3.12' }}
run: uvx ruff format --check .
run: uvx ruff@0.15.22 format --check .

# Coverage floor: a regression guard, not a target. Today's suite sits at
# ~79% line coverage; 75 leaves headroom for legitimately-untestable code
Expand Down
17 changes: 15 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@ The full **fetch → process → service → GUI** path runs end-to-end for **bo
- **Fetch:** `magnetics.data.fetch.toksearch` (mdsthin via the `cybele` ssh-config alias, or a
cluster-side `python -m` run orchestrated by `fetch/remote.py`) writes one HDF5 per shot to
`data/datafile/` (gitignored); read back via `magnetics.data.h5source`. The GUI can trigger a
pull from the left rail (`PullControl` → `POST /api/fetch`).
pull from the left rail (`PullControl` → `POST /api/fetch`). The `remote` backend is
location-aware: `network.on_cluster_host()` detects when the server is already running ON the
device's cluster (load-balanced submit nodes included) and fetches in-process — via the cluster
interpreter's toksearch (`remote.run_on_cluster`), else mdsthin — instead of SSHing to the host
it is running on (which hangs on an unanswerable host-key prompt).
- **Process:** `core/spectral.py` (MODESPEC) is real and pure. The SLCONTOUR quasi-stationary fit
runs end-to-end via the shim-free `core/qs_*` modules (`qs_io_data` → `qs_prep` → `qs_fit`,
adapted to nodes by `qs_bridge`; the former `_slcontour` translation, promoted into core) — real
Expand Down Expand Up @@ -66,6 +70,12 @@ The Python project **is the repo root** (a uv project, served as a webapp). `src
- `data/` — sources + `fetch/` (toksearch/mdsthin pulls, cluster orchestration); device configs
in `data/device/*.json`.
- `service/` — FastAPI; the built GUI is bundled at `service/webapp/` and served here.
- `connect.py` — the `magnetics-connect` remote-GUI launcher (server on a cluster node,
browser local, one SSH tunnel). Self-bootstraps the remote: existing `magnetics` → else
install uv → else `uvx magnetics` (targets PyPI; provisions its own Python). Deliberately
**stdlib-only and standalone** so it runs as a
bare `python3 connect.py` on gateway nodes with no magnetics install — keep it free of
package imports and of syntax newer than ~Python 3.9.

Tests in `tests/`, maintainer scripts in `scripts/`. `gui/web/` — React + Vite + TypeScript
frontend (its `dist/` is staged into `service/webapp/` for the wheel).
Expand All @@ -76,10 +86,13 @@ frontend (its `dist/` is staged into `service/webapp/` for the wheel).
tests (GA gateway / PPPL flux) are env-gated and skip unless `MAGNETICS_GA_USER` /
`MAGNETICS_FLUX_USER` is set; they are manual-only, never wired into CI.
- **Frontend:** `cd gui/web && npm run test` (vitest, one-shot; `npm run test:watch` to iterate).
- **Everything CI runs:** `uv run ruff format --check .` + `uvx ruff check .` + `uv run pytest`
- **Everything CI runs:** `uv run ruff format --check .` + `uv run ruff check .` + `uv run pytest`
+ `uv run ty check src/magnetics` (Python), and `npm run lint` + `npm run typecheck` +
`npm run test` + `npm run build` (in `gui/web/`). Run these locally before committing —
a plain `pytest` + `tsc` pass does NOT cover everything CI checks.
Use `uv run ruff` (not a bare `uvx ruff`, which resolves the *latest* release): the dev group
caps ruff at `<0.16`, matching the version CI pins, so local lint can't drift from CI when a
new ruff broadens its default rules.

## Conventions
- Physics lives in `src/magnetics/core` (pure, device-agnostic, testable); **no physics in the
Expand Down
44 changes: 43 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,12 @@ magnetics --no-browser # headless (e.g. on a server)
magnetics --data-dir PATH # where shot data lives (see below)
```

Two more console scripts come with the install:
Three more console scripts come with the install:

```sh
magnetics-fetch --shot 184927 # pull a shot to the local data dir
magnetics-service # start just the API/service (honors HOST/PORT)
magnetics-connect omega # run the server on a remote host, browse it locally
```

### Where shot data lives
Expand All @@ -47,6 +48,47 @@ usually quota'd and shot files are large) — e.g. in your shell profile:
export MAGNETICS_DATA_DIR=$SCRATCH/magnetics
```

## Remote use — run on a cluster, browse locally

The server should run where the data lives (a cluster work node), but that is
rarely where your browser lives — you're on a laptop off-site, or on a NoMachine
desktop on the site's login/gateway node with a firewall between you and the
work node. `magnetics-connect` does the standard SSH-tunnel dance in one command:
it opens one authenticated connection to the host, asks it for a free port and
its real node name, starts the server there **bound to loopback only**, carries
a `-L` port-forward on the same connection, and opens your local browser once
the server answers through the tunnel. Only HTTP crosses the wire — fetches and
fits run next to the data, plots stream back. Ctrl-C stops both ends.

```sh
magnetics-connect omega # ~/.ssh/config alias (keys, ProxyJump)
magnetics-connect me@omega.gat.com -J me@cybele.gat.com:2039
magnetics-connect omega --data-dir /cscratch/$USER/magnetics
magnetics-connect flux --remote-cmd \
'module load magnetics && magnetics --no-browser --port {port}'
```

**Zero-install on the remote.** By default the launcher bootstraps the server
for you: it runs an already-installed `magnetics` if the host has one; otherwise
it installs `uv` (the official root-free installer, into `~/.local/bin`) and
launches with `uvx magnetics`, which fetches the package **and provisions its own
Python** — so a bare cluster node with only an old system Python needs nothing
installed by hand. The first run downloads + provisions (slower; watch the
streamed `[host]` log); later runs are cached and fast.

- `--install-from SOURCE` — install from a wheel URL/path (any uv source) instead
of PyPI. This is the shim until `magnetics` is published to PyPI.
- `--remote-cmd '…{port}…'` — full override of the launch line (bypasses the
bootstrap), e.g. for an eventual `module load magnetics`.

The launcher itself is **stdlib-only and standalone** — on a gateway node with
only system `python3`, copy the single file and run it directly:

```sh
scp src/magnetics/connect.py gateway:
python3 connect.py omega
```

## Development

The Python project is the repo root (a uv project); the React GUI is in `gui/web/`.
Expand Down
9 changes: 8 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,10 @@ build-backend = "uv_build"
[dependency-groups]
dev = [
"pytest>=9.1.1",
"ruff>=0.15.18",
# Cap below 0.16: that minor broadened ruff's default rule set and would fail
# the existing tree. Raise the cap deliberately (fixing findings) in step with
# the pinned `uvx ruff@…` in .github/workflows/ci.yml.
"ruff>=0.15.18,<0.16",
# notebook tooling (examples + clean notebook commits)
"ipykernel>=7.3.0",
"jupyter>=1.1.0",
Expand All @@ -51,6 +54,10 @@ magnetics = "magnetics.cli:main"
magnetics-service = "magnetics.service.app:main"
# Fetch a shot to the local data dir (the toksearch/mdsthin CLI).
magnetics-fetch = "magnetics.data.fetch.toksearch:main"
# Run the server on a remote host (cluster) over an SSH tunnel, browse locally.
# Stdlib-only and standalone: src/magnetics/connect.py also runs as a bare
# `python3 connect.py` on hosts where magnetics is not installed.
magnetics-connect = "magnetics.connect:main"

[tool.ruff]
line-length = 100 # target version is inferred from requires-python
Expand Down
Loading
Loading