inventory-agent is built and released but nothing installs it, configures it, or
runs it. It is the only inventory component with no deployment path at all, which
means the aggregator, viewer and inspector have nothing to aggregate.
Why it is a host process, not a workload
designs/inventory.md says "The agent runs as root on each node", and lists
deployment manifests for the aggregator, viewer and inspector only. The code
agrees, and rules out a DaemonSet as currently written:
- One-shot.
Execute() (internal/inventory/agent/collect.go:104-136) collects,
writes SQLite, publishes over gRPC and returns. No loop, no interval flag. A
DaemonSet pod can only use restartPolicy: Always, so a cleanly-exiting binary
would be restarted forever.
- Root required.
collect.go:105-107 hard-fails when os.Geteuid() != 0.
- Deep host access.
/dev/ipmi0, the 0400 product_serial and product_uuid
under /sys/class/dmi/id, /proc/cpuinfo, /proc/meminfo.
--db ./inventory.db is relative to the working directory; the design calls
it a node cache.
#633 accordingly ships it as a GoReleaser binary and does not build an image for
it.
Work
1. systemd service plus timer. The one-shot, root-only shape fits a timer
exactly. Precedent for unit assets the node agent installs:
pkg/agent/phases/rootfs/assets/localdns.service, config-regeneration.service,
pkg/agent/phases/nodestart/assets/kubelet.service.
2. Binary delivery. Either through the node agent's goalstate machinery
(pkg/agent/goalstates/agentupgrade.go shows the blue-green pattern for the
agent's own binary) or baked into images/host-ubuntu2404. Whichever is chosen has
to handle upgrade alongside the rest of the node's software.
3. Collector tool packages via pkg/agent/phases/host/apt.go. Each backs a
specific collector, and a missing binary means that collector silently reports
nothing:
| Package |
Collector |
dmidecode |
cpu.go, memory.go - DMI processor and DIMM tables |
pciutils |
gpu.go - lspci enumeration |
ethtool |
network.go - NIC driver and firmware |
kmod |
disk.go, gpu.go - modinfo driver versions |
udev |
disk.go - udevadm disk properties |
lldpd |
lldp.go - lldpctl switch neighbours |
The NVIDIA tools (nvidia-smi, nvidia-imex-ctl) come from the host driver and
are version-locked to it; they cannot be installed here.
4. The collector endpoint. --collector defaults to
inventory-collector:50051 (cmd/inventory/inventory-agent/main.go:28), a cluster
Service DNS name that cannot resolve from the host. It needs a NodePort,
LoadBalancer, hostNetwork Service or an explicitly configured address, and that
address has to reach each node as config. This couples to #636, where the Service
becomes inventory-collector-<site>.
5. --db path. ./inventory.db should become something durable such as
/var/lib/unbounded/inventory.db rather than depending on the unit's working
directory.
6. Cadence and failure handling. At provisioning only, or periodically, and
with what jitter. What happens when the aggregator is unreachable: the local SQLite
cache already exists, so the question is whether the agent retries, or the next
timer firing simply re-publishes.
Open question: how to prove it works
This is where a true end-to-end test becomes possible, and it is also where it is
hardest. The collectors read DMI, IPMI, PCI, LLDP and NVLink, which return little
or nothing on a cloud VM, so a meaningful test needs real hardware - pointing at a
bare-metal site such as boulderlab, which is also the site that goes unreachable
and would need the tolerance treatment the DaemonSet deploy gate already has.
Cheaper options worth considering first: assert the unit is installed and enabled
after provisioning; assert one timer firing produces a non-empty local SQLite
database; assert a record for the node's serial appears in PostgreSQL within one
interval.
Acceptance criteria
- After a node is provisioned,
inventory-agent is installed, its unit is enabled,
and it has run at least once.
- A record identified by the node's DMI serial reaches the aggregator.
- The agent survives an unreachable aggregator without wedging the node or filling
the disk.
- The collector address is configuration, not a hardcoded cluster Service name.
inventory-agentis built and released but nothing installs it, configures it, orruns it. It is the only inventory component with no deployment path at all, which
means the aggregator, viewer and inspector have nothing to aggregate.
Why it is a host process, not a workload
designs/inventory.mdsays "The agent runs as root on each node", and listsdeployment manifests for the aggregator, viewer and inspector only. The code
agrees, and rules out a DaemonSet as currently written:
Execute()(internal/inventory/agent/collect.go:104-136) collects,writes SQLite, publishes over gRPC and returns. No loop, no interval flag. A
DaemonSet pod can only use
restartPolicy: Always, so a cleanly-exiting binarywould be restarted forever.
collect.go:105-107hard-fails whenos.Geteuid() != 0./dev/ipmi0, the 0400product_serialandproduct_uuidunder
/sys/class/dmi/id,/proc/cpuinfo,/proc/meminfo.--db ./inventory.dbis relative to the working directory; the design callsit a node cache.
#633 accordingly ships it as a GoReleaser binary and does not build an image for
it.
Work
1. systemd service plus timer. The one-shot, root-only shape fits a timer
exactly. Precedent for unit assets the node agent installs:
pkg/agent/phases/rootfs/assets/localdns.service,config-regeneration.service,pkg/agent/phases/nodestart/assets/kubelet.service.2. Binary delivery. Either through the node agent's goalstate machinery
(
pkg/agent/goalstates/agentupgrade.goshows the blue-green pattern for theagent's own binary) or baked into
images/host-ubuntu2404. Whichever is chosen hasto handle upgrade alongside the rest of the node's software.
3. Collector tool packages via
pkg/agent/phases/host/apt.go. Each backs aspecific collector, and a missing binary means that collector silently reports
nothing:
dmidecodecpu.go,memory.go- DMI processor and DIMM tablespciutilsgpu.go-lspcienumerationethtoolnetwork.go- NIC driver and firmwarekmoddisk.go,gpu.go-modinfodriver versionsudevdisk.go-udevadmdisk propertieslldpdlldp.go-lldpctlswitch neighboursThe NVIDIA tools (
nvidia-smi,nvidia-imex-ctl) come from the host driver andare version-locked to it; they cannot be installed here.
4. The collector endpoint.
--collectordefaults toinventory-collector:50051(cmd/inventory/inventory-agent/main.go:28), a clusterService DNS name that cannot resolve from the host. It needs a NodePort,
LoadBalancer, hostNetwork Service or an explicitly configured address, and that
address has to reach each node as config. This couples to #636, where the Service
becomes
inventory-collector-<site>.5.
--dbpath../inventory.dbshould become something durable such as/var/lib/unbounded/inventory.dbrather than depending on the unit's workingdirectory.
6. Cadence and failure handling. At provisioning only, or periodically, and
with what jitter. What happens when the aggregator is unreachable: the local SQLite
cache already exists, so the question is whether the agent retries, or the next
timer firing simply re-publishes.
Open question: how to prove it works
This is where a true end-to-end test becomes possible, and it is also where it is
hardest. The collectors read DMI, IPMI, PCI, LLDP and NVLink, which return little
or nothing on a cloud VM, so a meaningful test needs real hardware - pointing at a
bare-metal site such as
boulderlab, which is also the site that goes unreachableand would need the tolerance treatment the DaemonSet deploy gate already has.
Cheaper options worth considering first: assert the unit is installed and enabled
after provisioning; assert one timer firing produces a non-empty local SQLite
database; assert a record for the node's serial appears in PostgreSQL within one
interval.
Acceptance criteria
inventory-agentis installed, its unit is enabled,and it has run at least once.
the disk.