Skip to content

services/pack: build partitions as files, inject via dd - #347

Open
mwhudson wants to merge 2 commits into
canonical:mainfrom
mwhudson:no-loops-for-pack
Open

services/pack: build partitions as files, inject via dd#347
mwhudson wants to merge 2 commits into
canonical:mainfrom
mwhudson:no-loops-for-pack

Conversation

@mwhudson

Copy link
Copy Markdown
Contributor

This is 100% vibe coded, but I think it might be possible to have imagecraft not require access to loop devices at all which would be a great step towards having it be able to run in unprivileged containers (I have another vibe-coded branch that removes it in the grub case). Happy to talk about all this at some point!

Replace the losetup-based build path with a loop-free flow. For each partition in a volume, pack() now:

  1. Reads the partition's sector offset and size from sfdisk --json on the disk file (via the new diskutil.get_partition_geometry helper, which works for both GPT and MBR images).
  2. Sanity-checks the on-disk size against the structure spec.
  3. Creates a temp file sized to match the partition.
  4. Calls diskutil.format_populate_partition to mkfs/copy content into the temp file.
  5. dd's the temp file into the disk image at the right sector offset via diskutil.inject_partition_into_image.
  6. Deletes the temp file.

attach_images/detach_images/get_loop_paths and the atexit handler in ImageService are no longer called from the build path, removing the losetup requirement. The methods themselves are kept in place for now because pack/grubutil.py still uses image.attach_loopdev() for grub install; that is being addressed in a parallel Phase B effort.

This is required so imagecraft can build images inside unprivileged LXD containers, where /dev/loop-control is gated on init_user_ns CAP_SYS_ADMIN.


Replace the losetup-based build path with a loop-free flow. For each
partition in a volume, pack() now:

1. Reads the partition's sector offset and size from sfdisk --json
   on the disk file (via the new diskutil.get_partition_geometry helper,
   which works for both GPT and MBR images).
2. Sanity-checks the on-disk size against the structure spec.
3. Creates a temp file sized to match the partition.
4. Calls diskutil.format_populate_partition to mkfs/copy content into
   the temp file.
5. dd's the temp file into the disk image at the right sector offset
   via diskutil.inject_partition_into_image.
6. Deletes the temp file.

attach_images/detach_images/get_loop_paths and the atexit handler in
ImageService are no longer called from the build path, removing the
losetup requirement. The methods themselves are kept in place for now
because pack/grubutil.py still uses image.attach_loopdev() for grub
install; that is being addressed in a parallel Phase B effort.

This is required so imagecraft can build images inside unprivileged
LXD containers, where /dev/loop-control is gated on init_user_ns
CAP_SYS_ADMIN.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@steinbro
steinbro requested review from lengau and smethnani May 26, 2026 16:54

@steinbro steinbro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is clever. My main concern what the performance will be like for sizable partitions, as this strategy presumably results in multiple large file copy operations that wouldn't be necessary with a native loopback mount.

I also wonder how likely this is to work for unforeseen future exotic partition formats. At the very least, I would keep the existing loopback-based setup as an option.

@mwhudson

Copy link
Copy Markdown
Contributor Author

This is clever. My main concern what the performance will be like for sizable partitions, as this strategy presumably results in multiple large file copy operations that wouldn't be necessary with a native loopback mount.

It turns out (slightly surprisingly) that all the tools used here (mke2fs, mkdosfs, mcopy) support offsets so it ought to be possible with sufficient plumbing to write the content directly into the image file. The failure mode if we try to write more than fits into a partition will probably not be very pretty ...

I also wonder how likely this is to work for unforeseen future exotic partition formats. At the very least, I would keep the existing loopback-based setup as an option.

Yeah this is fair. I'm pretty sure all or almost all the images we ship today are ext4 and vfat only but other filesystems are more mixed on this afaict (according to claude xfs supports this as of this year, btrfs and zfs don't, etc).

Build each partition's filesystem directly inside the disk image at the
partition's offset instead of formatting a temp file and dd'ing it into
the image. mke2fs (-E offset=), mkfs.fat (--offset) and mcopy (@@offset)
all support writing at an offset, so the intermediate per-partition copy
is no longer needed.

format_populate_partition now takes an optional PartitionGeometry and
routes the offset/size through to the ext/fat helpers; the size argument
is rounded down to whole KiB so the filesystem never spills past the
partition end. The unused inject_partition_into_image (the dd path) is
removed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@mwhudson

Copy link
Copy Markdown
Contributor Author

It turns out (slightly surprisingly) that all the tools used here (mke2fs, mkdosfs, mcopy) support offsets so it ought to be possible with sufficient plumbing to write the content directly into the image file.

I had claude do this fwiw.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants