From 851a2ddcd4ebb680ddec8fa7f1526acc4c768e2f Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Thu, 13 Aug 2026 14:13:38 -0700 Subject: [PATCH 1/7] Document build arguments Updates 256lights/zb#117 Fixes #22 --- build-args.md | 95 ++++++++++++++++++++++++++++++++++++++++++++ conf.py | 2 +- index.md | 1 + lua/extensions.md | 21 ++++++++++ outputs-metafield.md | 10 +++++ 5 files changed, 128 insertions(+), 1 deletion(-) create mode 100644 build-args.md create mode 100644 outputs-metafield.md diff --git a/build-args.md b/build-args.md new file mode 100644 index 0000000..c7fef01 --- /dev/null +++ b/build-args.md @@ -0,0 +1,95 @@ +# Specifying What to Build + +The arguments to `zb build` are either a list of paths/URLs, +or, if the `--expression` flag is passed, a [Lua](lua/index.md) expression. +Here are some examples of what's possible: + +```shell +# Build what is returned from foo.lua. +zb build foo.lua +# Build what is stored in the global variable "myvar" +# in the file foo.lua. +zb build 'foo.lua#myvar' +# Build what is stored in the global variable "myvar" +# in the file foo.lua. +zb build --expression 'import("foo.lua").myvar' +# Fetch the file, evaluate it as Lua, +# and then build what it returns. +zb build https://www.example.com/foo.lua +# Fetch the zip file, unpack it, +# evaluate the file inside it called "foo.lua", +# then build what is stored in the global variable "myvar". +zb build 'https://www.example.com/archive.zip#foo.lua:myvar' +``` + +(output-values)= +## What Can Be Built + +Most commonly, the objects that you will pass to `zb build` +will be objects returned by {lua:func}`derivation`. +A {term}`derivation` specifies a {term}`builder program` to run +and its dependencies, and `zb build` will arrange to run those. + +`zb build` also accepts a few other types of objects: + +- **Strings.** Because these carry [dependency information](lua/deps.md), + `zb build` will build any derivation outputs referenced in the string + before printing the full string to standard output. +- **Tables** or anything with the `__pairs` [metamethod][Metatables and Metamethods]. + Pair keys that are not strings or numbers are ignored. + Pair values are converted via [`tostring`][] and are handled as above. +- Anything with the `__tostring` [metamethod][Metatables and Metamethods]. + The metamethod is called and the resulting string is handled as above. + If a value has both a `__pairs` metamethod and a `__tostring` metamethod, + then the `__tostring` metamethod is ignored. + +```{eval-rst} +.. index:: __outputs metatable field +``` + +zb will use a zb-specific `__outputs` [metatable][Metatables and Metamethods] field if present. +{lua:func}`outputs` can be used to obtain the value from an `__outputs` metatable field in user-defined code. + +```{include} outputs-metafield.md +``` + +(Objects returned by {lua:func}`derivation` have an `__outputs` field in their metatable, +so are not different from user-defined types.) + +[`tostring`]: https://www.lua.org/manual/5.4/manual.html#pdf-tostring +[Metatables and Metamethods]: https://www.lua.org/manual/5.4/manual.html#2.4 + +```{eval-rst} +.. index:: URL +``` + +## URL Syntax + +URL arguments to `zb build` comply with {rfc}`3986` +and can use any of the following {rfc}`schemes <3986#section-3.1>`: + +- `file` (default if absent) +- `http` +- `https` +- `gs` ([Google Cloud Storage](https://docs.cloud.google.com/storage/docs/gsutil#syntax)) + +```{eval-rst} +.. index:: URL; fragment +``` + +The {rfc}`fragment <3986#section-3.5>` of a URL to `zb build` is split into two parts +at the last colon (`:`) that appears in the fragment. +Everything before the last colon is the *archive member* +and everything after the last colon is the *key path*. +If the fragment does not contain a colon, +then the entire fragment is the key path. +The presence of an archive member instructs zb to treat the file as an archive, +extract it using the same mechanism as in {lua:func}`extract`, +then use the file inside the archive with the same name. +If the key path is empty, +then the result of evaluating a URL is the same as the result of calling {lua:func}`await` +on the result of calling {lua:func}`import` with the path to the file as its argument. +Otherwise, the key path is a slash-separated sequence of [table indexes][Variables] on the module +(e.g. a key path of `foo/bar/baz` is equivalent to `foo.bar.baz` in Lua). + +[Variables]: https://www.lua.org/manual/5.4/manual.html#3.2 diff --git a/conf.py b/conf.py index 90ae95e..79d7015 100644 --- a/conf.py +++ b/conf.py @@ -36,7 +36,7 @@ ] templates_path = ['_templates'] -exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'README.md'] +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'README.md', 'outputs-metafield.md'] primary_domain = 'lua' # -- Options for HTML output ------------------------------------------------- diff --git a/index.md b/index.md index f248f5a..231cafb 100644 --- a/index.md +++ b/index.md @@ -76,6 +76,7 @@ zb simplifies reproducible builds. Installation getting-started +build-args admin/index comparison ``` diff --git a/lua/extensions.md b/lua/extensions.md index d267ac7..4f5fc6b 100644 --- a/lua/extensions.md +++ b/lua/extensions.md @@ -136,6 +136,27 @@ The environment that the builder runs in is documented in the [Derivation Specif [Derivation Specification]: ../derivations.md +:::{function} outputs(x, system) + +Read an object's `__outputs` [metatable](https://www.lua.org/manual/5.4/manual.html#2.4) field +using the same logic that `zb build` uses to determine [what to build](project:#output-values). + +```{include} ../outputs-metafield.md +``` + +If the object does not have an `__outputs` metatable field, +then `outputs` returns its first argument. + +:param x: + The object to get outputs for. + +:param string system: + A {term}`system triple` to get outputs for. + +:rtype: any + +::: + :::{function} fetchurl{url, hash, [name], [executable]} `fetchurl` returns a derivation that downloads a URL. diff --git a/outputs-metafield.md b/outputs-metafield.md new file mode 100644 index 0000000..4068fd9 --- /dev/null +++ b/outputs-metafield.md @@ -0,0 +1,10 @@ +The value of the `__outputs` [metatable](https://www.lua.org/manual/5.4/manual.html#2.4) field can be: + +- A function. + The function will be called with the original table + and the current {term}`system triple` as its two arguments. +- A value with an `__outputs` field in its metatable. + The field will be processed recursively + (up to an implementation-defined limit). +- Any non-`nil` value, + which will be used instead of the original table. From cc16476c2d7547b718fc382fe440700256b6fc43 Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Thu, 13 Aug 2026 22:02:38 -0700 Subject: [PATCH 2/7] Add version information to output() function --- lua/extensions.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lua/extensions.md b/lua/extensions.md index 4f5fc6b..1a06339 100644 --- a/lua/extensions.md +++ b/lua/extensions.md @@ -155,6 +155,9 @@ then `outputs` returns its first argument. :rtype: any +```{version-added} 0.2 +``` + ::: :::{function} fetchurl{url, hash, [name], [executable]} From 22fd4dddc470ca8df40edd985ac57255f7456e70 Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Thu, 13 Aug 2026 22:10:15 -0700 Subject: [PATCH 3/7] Move version added info up in outputs() section --- lua/extensions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lua/extensions.md b/lua/extensions.md index 1a06339..2380a87 100644 --- a/lua/extensions.md +++ b/lua/extensions.md @@ -138,6 +138,9 @@ The environment that the builder runs in is documented in the [Derivation Specif :::{function} outputs(x, system) +```{version-added} 0.2 +``` + Read an object's `__outputs` [metatable](https://www.lua.org/manual/5.4/manual.html#2.4) field using the same logic that `zb build` uses to determine [what to build](project:#output-values). @@ -155,9 +158,6 @@ then `outputs` returns its first argument. :rtype: any -```{version-added} 0.2 -``` - ::: :::{function} fetchurl{url, hash, [name], [executable]} From e002c2c8065fddcb331089e5614b9bc7c82806f9 Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Fri, 14 Aug 2026 10:01:32 -0700 Subject: [PATCH 4/7] Simplify rules --- build-args.md | 71 +++++++++++++++++++++----------------------- lua/extensions.md | 20 +++++++------ outputs-metafield.md | 4 ++- 3 files changed, 48 insertions(+), 47 deletions(-) diff --git a/build-args.md b/build-args.md index c7fef01..274dfee 100644 --- a/build-args.md +++ b/build-args.md @@ -22,43 +22,6 @@ zb build https://www.example.com/foo.lua zb build 'https://www.example.com/archive.zip#foo.lua:myvar' ``` -(output-values)= -## What Can Be Built - -Most commonly, the objects that you will pass to `zb build` -will be objects returned by {lua:func}`derivation`. -A {term}`derivation` specifies a {term}`builder program` to run -and its dependencies, and `zb build` will arrange to run those. - -`zb build` also accepts a few other types of objects: - -- **Strings.** Because these carry [dependency information](lua/deps.md), - `zb build` will build any derivation outputs referenced in the string - before printing the full string to standard output. -- **Tables** or anything with the `__pairs` [metamethod][Metatables and Metamethods]. - Pair keys that are not strings or numbers are ignored. - Pair values are converted via [`tostring`][] and are handled as above. -- Anything with the `__tostring` [metamethod][Metatables and Metamethods]. - The metamethod is called and the resulting string is handled as above. - If a value has both a `__pairs` metamethod and a `__tostring` metamethod, - then the `__tostring` metamethod is ignored. - -```{eval-rst} -.. index:: __outputs metatable field -``` - -zb will use a zb-specific `__outputs` [metatable][Metatables and Metamethods] field if present. -{lua:func}`outputs` can be used to obtain the value from an `__outputs` metatable field in user-defined code. - -```{include} outputs-metafield.md -``` - -(Objects returned by {lua:func}`derivation` have an `__outputs` field in their metatable, -so are not different from user-defined types.) - -[`tostring`]: https://www.lua.org/manual/5.4/manual.html#pdf-tostring -[Metatables and Metamethods]: https://www.lua.org/manual/5.4/manual.html#2.4 - ```{eval-rst} .. index:: URL ``` @@ -93,3 +56,37 @@ Otherwise, the key path is a slash-separated sequence of [table indexes][Variabl (e.g. a key path of `foo/bar/baz` is equivalent to `foo.bar.baz` in Lua). [Variables]: https://www.lua.org/manual/5.4/manual.html#3.2 + +(output-eval)= +## What Can Be Built + +Once `zb build` has evaluated the Lua value from an expression or a URL, +`zb build` converts the value to a string using [`tostring`][], +unless the value is a table. +Because strings carry [dependency information](lua/deps.md), +`zb build` will build any {term}`derivation` outputs referenced in the string +before printing the full string to standard output. +If the Lua value is a table, the table will be walked using [`pairs`][]. +Pairs with keys that are not strings or numbers will be ignored. +Each value will be converted to a string using [`tostring`][]. + +```{eval-rst} +.. index:: __outputs metatable field +``` + +```{include} outputs-metafield.md +``` + +:::{seealso} + +{lua:func}`outputs` +: Built-in function to obtain a value's outputs in user-defined code. + +{lua:func}`derivation` +: Built-in function to create strings that cause zb to run a {term}`builder program`. + +::: + +[`pairs`]: https://www.lua.org/manual/5.4/manual.html#pdf-pairs +[`tostring`]: https://www.lua.org/manual/5.4/manual.html#pdf-tostring +[Metatables and Metamethods]: https://www.lua.org/manual/5.4/manual.html#2.4 diff --git a/lua/extensions.md b/lua/extensions.md index 2380a87..95719d7 100644 --- a/lua/extensions.md +++ b/lua/extensions.md @@ -136,29 +136,31 @@ The environment that the builder runs in is documented in the [Derivation Specif [Derivation Specification]: ../derivations.md -:::{function} outputs(x, system) +::::{function} outputs(x, system) ```{version-added} 0.2 ``` -Read an object's `__outputs` [metatable](https://www.lua.org/manual/5.4/manual.html#2.4) field -using the same logic that `zb build` uses to determine [what to build](project:#output-values). +Get a value's outputs +using the same logic that `zb build` uses to determine [what to build](project:#output-eval). ```{include} ../outputs-metafield.md ``` -If the object does not have an `__outputs` metatable field, -then `outputs` returns its first argument. - :param x: - The object to get outputs for. + The value to get outputs for. :param string system: A {term}`system triple` to get outputs for. -:rtype: any +:returns: + A table of output strings. + The keys will be strings that correspond to zb's name for each output. + The "default" output has an empty string key. + +:rtype: table -::: +:::: :::{function} fetchurl{url, hash, [name], [executable]} diff --git a/outputs-metafield.md b/outputs-metafield.md index 4068fd9..177dda2 100644 --- a/outputs-metafield.md +++ b/outputs-metafield.md @@ -1,4 +1,6 @@ -The value of the `__outputs` [metatable](https://www.lua.org/manual/5.4/manual.html#2.4) field can be: +If a value has a `__outputs` field in its [metatable](https://www.lua.org/manual/5.4/manual.html#2.4), +then it will override zb's built-in behavior. +The value of the `__outputs` metatable field can be: - A function. The function will be called with the original table From ffc08214a6c78bb689158a992133bf6d3585c0c6 Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Fri, 14 Aug 2026 10:26:41 -0700 Subject: [PATCH 5/7] Avoid the URL RFC reference We intentionally use slightly sloppy URLs and it's not something the user cares about. --- build-args.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/build-args.md b/build-args.md index 274dfee..cfc078e 100644 --- a/build-args.md +++ b/build-args.md @@ -28,8 +28,7 @@ zb build 'https://www.example.com/archive.zip#foo.lua:myvar' ## URL Syntax -URL arguments to `zb build` comply with {rfc}`3986` -and can use any of the following {rfc}`schemes <3986#section-3.1>`: +URL arguments to `zb build` can use any of the following {rfc}`schemes <3986#section-3.1>`: - `file` (default if absent) - `http` From 028a810632f5ca0a6cf10ba2c4f6ff717596ea18 Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Fri, 14 Aug 2026 10:31:31 -0700 Subject: [PATCH 6/7] Rework examples to show more equivalent expressions --- build-args.md | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/build-args.md b/build-args.md index cfc078e..06b4195 100644 --- a/build-args.md +++ b/build-args.md @@ -2,22 +2,32 @@ The arguments to `zb build` are either a list of paths/URLs, or, if the `--expression` flag is passed, a [Lua](lua/index.md) expression. -Here are some examples of what's possible: +Here are some examples: ```shell -# Build what is returned from foo.lua. -zb build foo.lua -# Build what is stored in the global variable "myvar" -# in the file foo.lua. -zb build 'foo.lua#myvar' +# Build what is returned from the Lua file at foo/bar.lua. +zb build foo/bar.lua +zb build --expression 'await(import("foo/bar.lua"))' + # Build what is stored in the global variable "myvar" -# in the file foo.lua. -zb build --expression 'import("foo.lua").myvar' -# Fetch the file, evaluate it as Lua, -# and then build what it returns. -zb build https://www.example.com/foo.lua -# Fetch the zip file, unpack it, -# evaluate the file inside it called "foo.lua", +# in the Lua file at foo/bar.lua. +zb build 'foo/bar.lua#myvar' +zb build --expression 'import("foo/bar.lua").myvar' + +# Build what is stored in the field "field" +# in the global table "myvar" +# in the Lua file at foo/bar.lua. +zb build 'foo/bar.lua#myvar/field' +zb build --expression 'import("foo/bar.lua").myvar.field' + +# Download the file at https://www.example.com/foo.lua, +# evaluate it as Lua, +# and then build what is stored in the global variable "myvar". +zb build 'https://www.example.com/foo.lua#myvar' + +# Download the zip file at https://www.example.com/archive.zip, +# unpack it, +# evaluate the Lua file inside it called "foo.lua", # then build what is stored in the global variable "myvar". zb build 'https://www.example.com/archive.zip#foo.lua:myvar' ``` From 35a526485a85b4ab2d45181656999e733bb72947 Mon Sep 17 00:00:00 2001 From: Roxy Light Date: Sun, 16 Aug 2026 17:12:04 -0700 Subject: [PATCH 7/7] Document `defaultOutput` function and new table value semantics --- build-args.md | 10 +++++++++- lua/extensions.md | 26 +++++++++++++++++++++++++- outputs-metafield.md | 2 +- 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/build-args.md b/build-args.md index 06b4195..5b45545 100644 --- a/build-args.md +++ b/build-args.md @@ -77,7 +77,12 @@ Because strings carry [dependency information](lua/deps.md), before printing the full string to standard output. If the Lua value is a table, the table will be walked using [`pairs`][]. Pairs with keys that are not strings or numbers will be ignored. -Each value will be converted to a string using [`tostring`][]. +The last pair with the same string key is the one that will be used. +Each value in the table is converted to its *default output* string. +The default output of any non-table value +is the result of calling [`tostring`][] with the value as its argument. +The default output of a table is the value of the key `""`, `1`, `"1"`, or `"out"`, +in descending order of preference, recursing on any table value. ```{eval-rst} .. index:: __outputs metatable field @@ -91,6 +96,9 @@ Each value will be converted to a string using [`tostring`][]. {lua:func}`outputs` : Built-in function to obtain a value's outputs in user-defined code. +{lua:func}`defaultOutput` +: Built-in function to obtain a value's default output in user-defined code. + {lua:func}`derivation` : Built-in function to create strings that cause zb to run a {term}`builder program`. diff --git a/lua/extensions.md b/lua/extensions.md index 95719d7..ba3f064 100644 --- a/lua/extensions.md +++ b/lua/extensions.md @@ -156,12 +156,36 @@ using the same logic that `zb build` uses to determine [what to build](project:# :returns: A table of output strings. The keys will be strings that correspond to zb's name for each output. - The "default" output has an empty string key. + The default output is normalized to the empty string key. :rtype: table :::: +::::{function} defaultOutput(x, system) + +```{version-added} 0.2 +``` + +Get a value's default output +using the same logic that `zb build` uses to determine [what to build](project:#output-eval). + +```{include} ../outputs-metafield.md +``` + +:param x: + The value to get the default output for. + +:param string system: + A {term}`system triple` to get the default output for. + +:returns: + The default output converted to a string. + +:rtype: string + +:::: + :::{function} fetchurl{url, hash, [name], [executable]} `fetchurl` returns a derivation that downloads a URL. diff --git a/outputs-metafield.md b/outputs-metafield.md index 177dda2..9bb2271 100644 --- a/outputs-metafield.md +++ b/outputs-metafield.md @@ -7,6 +7,6 @@ The value of the `__outputs` metatable field can be: and the current {term}`system triple` as its two arguments. - A value with an `__outputs` field in its metatable. The field will be processed recursively - (up to an implementation-defined limit). + up to an implementation-defined limit. - Any non-`nil` value, which will be used instead of the original table.