diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9532f37..dabb682 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -23,7 +23,7 @@ its own adds to them and says what it adds. ## Checking a change ```sh -task render # dry-run the whole pipeline inside the toolbox, no network +task check # render every command of the pipeline inside the toolbox, no network, and diff it against render.txt ``` The check workflow runs the same render on every pull request and compares it with the diff --git a/PULL_REQUEST_TEMPLATE.md b/PULL_REQUEST_TEMPLATE.md index f142100..549cf51 100644 --- a/PULL_REQUEST_TEMPLATE.md +++ b/PULL_REQUEST_TEMPLATE.md @@ -8,5 +8,5 @@ ## Test plan -- [ ] `task render` passes and the committed render was updated if the commands changed +- [ ] `task check` passes, and `render.txt` was updated with `task render-update` if the commands changed - [ ] Verbs that write to a bucket were run on a fork against a scratch bucket, or this change touches none diff --git a/SECURITY.md b/SECURITY.md index 6f7f390..c5d256e 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -26,7 +26,7 @@ This policy applies to every katoptra repository that has no SECURITY file of it account's token; a pull-request check gets no secret at all. Everything else on a public mirror is served as upstream serves it. Problems with the -packages themselves belong upstream, to CTAN, TeX Live, CRAN or CPAN. This +packages themselves belong upstream, to CTAN or TeX Live. This organization copies what they publish. ## Reporting diff --git a/profile/README.md b/profile/README.md index aa4596e..20b9f22 100644 --- a/profile/README.md +++ b/profile/README.md @@ -2,41 +2,55 @@ Greek for mirrors. Bytes here are closer than they appear. -Each repository is one mirror: an upstream, a bucket, and a pipeline that syncs the two -on a schedule from GitHub Actions. The mirrors are served from Cloudflare R2 and listed -with live status at [katoptra.org](https://katoptra.org/). +Each repository is one mirror: an upstream, a sink, and a pipeline that syncs the two on +a schedule from GitHub Actions. The public mirrors are served from Cloudflare R2 and +listed with live status at [katoptra.org](https://katoptra.org/). | Mirror | Upstream | Cadence | Repository | |---|---|---|---| | [ctan.katoptra.org](https://ctan.katoptra.org/) | [CTAN](https://ctan.org), all of it | hourly | [ctan](https://github.com/katoptra/ctan) | | [tlnet.katoptra.org](https://tlnet.katoptra.org/) | CTAN `systems/texlive/tlnet`, what `tlmgr` installs from | daily | [tlnet](https://github.com/katoptra/tlnet) | +Two more mirror into Proton Drive rather than onto the web. What they copy is private; +the pipelines are public and run on the same toolbox, so they are forks like the others. + +| Mirror | Upstream | Cadence | Repository | +|---|---|---|---| +| GitHub | every repository under the owners it names, one git bundle each | nightly | [github](https://github.com/katoptra/github) | +| Dropbox | one Dropbox account | nightly | [dropbox](https://github.com/katoptra/dropbox) | + More are on the way. ## Why this exists - I built these for myself. They live here so the pipelines are namespaced and share one toolbox. -- Open source in the sense that you can fork and run it. Or just use these published +- Open source in the sense that you can fork and run it. Or just use the published mirrors like I do. ## How a mirror runs -- `list` the upstream, `diff` it against the `state` the last run left behind, then - `publish` the delta and `checkpoint` the state. -- `verify` data and signatures whenever upstream provides them. -- Works in batches, sized for a GitHub Actions runner. -- Pings a healthcheck after each run if you give it one. -- Keeps its state and checkpoints in R2 or any S3-compatible bucket. Cloudflare is the - default for its versatility and free egress. -- I preferentially use 1Password for secrets, but GitHub secrets work too. +- Everything the mirrors share lives once, in [lib](https://github.com/katoptra/lib): + the toolbox that starts a run, contains it in an image, resolves its secrets, checks + it and reports it, and an engine per transport that moves the bytes. lib's README is + the manual. +- An rsync mirror lists upstream, diffs the listing against the state the last run left + in the bucket, publishes the delta in batches and checkpoints after each. It verifies + data and signatures wherever upstream provides them. +- A Proton mirror stages what its upstream holds and uploads it in one call; unchanged + files are skipped, changed ones become revisions, and Proton's version history is the + history of the mirror. +- Every run happens inside one pinned image, on a laptop and in Actions alike, and pings + a healthcheck when it finishes. Silence is the alert. +- Secrets reach a run by name from a vault. No credential, account identifier, bucket + name or endpoint is in any repository. ## Want your own? -- Fork a mirror. -- Set three vars in its Taskfile: `SOURCE` (the upstream rsync URL), `BUCKET` (your - bucket name) and `HOST` (the domain in front of it). -- Point the workflow at your vault, or your repository secret. +- Fork a mirror and follow its README's "Want your own?": the lines to change, the + bucket, the vault item, its own accounts, the first run. +- Everything the mirrors share, from making a vault and a service account to what the + bucket holds and how the workflows run, is in lib's README, once. - A full CTAN mirror costs under two dollars a month. ## Contributing