[BUG] image-lookup recipes match snapshots

[mbuechse]

The image-lookup-via-metadata recipes seem to have a defect whereby they also match snapshots. It seems like they could be fixed by adding a filter on visibility 'public' or 'community' (or, if possible, on type 'image').

At the same time, it became apparent that the respective standards should be made more explicit about the fact that all standard images are expected to be public and not hidden.

[mbuechse]

@toothstone can you please check whether I summarized this correctly?

[toothstone]

AFAIK this affects the base logic and thus all possible implementations, not just OpenTofu.

We implemented the lookup in Golang. Line 78 contains my quick fix to get rid of snapshots, since I wanted to avoid any reliance on non-standardized behavior such as the visibility. Unfortunately, the Glance API does not seem to have a simple "type" field, Horizon goes through two layers of "source_type" to deduce the type properly, see also gophercloud/gophercloud#3830.

[toothstone]

As a quick check you can use the openstackclient version to verify this:

openstack image list --property os_distro="ubuntu" --property os_purpose="generic" -f value -c ID -c Name --sort name:desc,created_at:desc

returns private snapshots created from the "generic" Ubuntu image on my cloud.

[mbuechse]

The lines in the Horizon code have moved since the comment you referred to was posted. Here is a permalink: https://github.com/openstack/horizon/blob/e7c9e39182dcb74579f227b1e61b30b8fa5b4ceb/openstack_dashboard/dashboards/project/images/images/tables.py#L261

[mbuechse]

The first half of the problem should be solved with #1260 (still to be debated in the SIG). The second half is to update the recipes so the visibility is taken into account.

@toothstone You don't seem to be using server-side filtering in your Go code. Is that a restriction of the library?

[toothstone]

@toothstone You don't seem to be using server-side filtering in your Go code. Is that a restriction of the library?

It's a combination of the library, the Glance API and the code being a PoC. All the metadata we care about comes as a second-level attribute, in the "Properties" field, which just makes it cumbersome to access. The library (gophercloud) offers no filter for it out of the box, but if understand this comment correctly it should still be possible to implement.

However, regarding your proposed solution to this issue via the visibility attribute, that is a first-level attribute and as such easily accessible for server-side filtering.

[toothstone]

The first half of the problem should be solved with #1260 (still to be debated in the SIG).

I was able to create a snapshot with a visibility of "community", so the change proposed in #1260 is unfortunately not a solution.

[toothstone]

@garloff and @horazont mentioned during todays SIG that properties.image_type should contain the necessary details to filter out snapshots. I was not able to find this on my cloud, closest thing I'm able to access is properties.block_device_mapping[0].source_type, but that is only present on snapshots.

Could you please provide some example? Does openstack image list --property image_type="snapshot" work for you?

I'm generally confused how I'm expected to approach this, the Glance API docs imply to me that it is up to the CSP how properties are set, I can't find any first-class attribute to differentiate the type. For reference, here is how Horizon deduces the image type.

[garloff]

There's nothing special about snapshots: You can register images with visibility community as normal user - I was wrong on this.
https://docs.openstack.org/api-ref/image/v2/#create-image

visibility (Optional):
Visibility for this image. Valid value is one of: public, private, shared, or community. At most sites, only an administrator can make an image public. Some sites may restrict what users can make an image community. Some sites may restrict what users can perform member operations on a shared image. Since the Image API v2.5, the default value is shared.

So there's two things I think we need to do:

• In the standard, require all images provided by the cloud operator to comply with the meta-data requirements and ensure that all the mandatory images are provided. We may express a preference for public visibility, but allow community as well.
• For testing, we can certainly ignore private images, as that is not how we want images to be shared by the cloud operator, but we will look at all public and community images. We'll just have to require that the tenant (project) that is provided/used for testing does not have anything unusual registered as community image. (If we want to filter out snapshots here, that may our test a bit more tolerant, but does not remove the fundamental requirement of having a relatively clean project to test against.) We can document this for users that want to run the test suite themselves.

[garloff]

How did you create those snapshots that should be filtered out, @toothstone?
I can create a snapshot from a bootable volume, create a volume from the snapshot and create an image from that volume (with visibility private, shared or community). That image however does not have any special settings (like image_type or block_device_mapping), so this does not seem to be the snapshot thing that you have created ...

[garloff]

• The other thing we could do is to filter out anything in testing that is owned by the test tenant ...