Kaniko Dependency archived - Replacement needed

[ArneTR]

Kaniko's Repository has been archived :/

• https://github.com/GoogleContainerTools/kaniko

We need a replacement now.

@ribalba Can you please put this on your todo list for the next 30 days. Not super urgent, but should also not stay open too long.

[ribalba]

Good catch :)

I would just move to a maintained fork as I am quite happy with the general functionality.

https://github.com/mzihlmann/kaniko

Or do you want to use the move for more features?

[ArneTR]

Not a bad idea.

But how do you review maintained forks? What is your scrutiny process?

By giving it a quick look the reputation of the author nor the few stars on the repo immediatly gave me a green flag ...?

[ribalba]

I think this is a bigger thing we could discuss. As we have loads of dependencies and no formal process of verification. Currently we mostly only have well known things but where is the cut off point. Also as with the ssh "hack" last year these can also happen with well known packages.

We could migrate to packaged python packages so the OS vendor checks these things. I have done this a few times with RedHat. This will limit us though in some aspects.

What we should also maybe think about is limiting network traffic somehow so that no one can push out data. Maybe an allow list of URLs in the usage scenario?

We could fork it ourselves but that would be loads of work.

I have no answer. But hardening the system is always a good idea.

[ArneTR]

HEy @ribalba:

I would like to pick this up again.

Did you make some research yet on your build error that you where experiencing with the old Kaniko?

And did you try to build the container your self so far?

[ribalba]

So the problem was something totally different. glibc and some other system libs hat updated and somehow corrupted the system so that docker was waiting for a file endpoint. After reboot everything worked again.

I did replace gcr.io/kaniko-project/executor:latest with martizih/kaniko:latest and that worked without a problem.

I looked at https://github.com/chainguard-forks/kaniko/blob/main/DEVELOPMENT.md and building a container was as easy as

git clone https://github.com/chainguard-forks/kaniko.git
cd kaniko
docker build -f deploy/Dockerfile -t lol .

[ArneTR]

Between these two variants it feels hard to choose:

• Chainguard: Two members from the original kaniko team continue the support but only commited to dependency updates
• Matizih: Way more activate, commited to also fixing know security vulnerabilities, bringing in new features, also has a "slim" image that removes some attack vectors, however so far unknown maintainer to us

How to go about it? The image building is a vulnerable part of the tool and can open many problems if we choose a too flaky implementation here. But staying on the outdated version seems also like no real option.

If we were a bigger it would be an option to also be part of the maintainers. But I feel this is not possible for us atm.

On the other hand: Kaniko is executing something inside a container. Since we launch the container ourselves it should be no less secure than all the other docker launches we are doing so far. So it should pose no additional risk no matter which container we chosse ...? Or am I missing something?

If you agree than I would change the kaniko we use to the slim image from martizih without the credential helpers, which we do not need

[ribalba]

The major problem I see is that when using the Matizih fork is that they could in theory add code that exposes the source code we map in. I don't think our server could be compromised as everything is in a container and we don't pass in anything that is critical.

I would make it configurable in the config. For the open source part we use Matizih as a default as this works in all cases. For our server we use the Chainguard rebuild to be extra secure. I can easily set this up as a GitHub action or we could even do it on the servers locally. In the future if the Matizih forks adds something we need we can easily migrate or do a code review.

[ArneTR]

Hmm, I am not in favor of that. Reason being: Since one image updates features also we might run into situations where we get a "works for me" situation due to different setups.

I tried to create more infos for better decision making by running vulnerability scans on the images:

Suprisingly both are absolute free even from low level CVEs ... that is a rare sight.

I will run on the chainguard fork for a while and see how that feels. Later we can decide if we link it as submodule and it will be built on install or if we publish a container.

[ArneTR]

The old kaniko is getting now really high CVE scores and should be replaced now.

## Overview

                   │                 Analyzed Image
───────────────────┼────────────────────────────────────────────────
 Target            │  gcr.io/kaniko-project/executor:latest
   digest          │  4e7a52dd1f14
   platform        │ linux/arm64
   provenance      │ https://github.com/GoogleContainerTools/kaniko
                   │  1d2bff595903b1887220a522a5a43c67db2da553
   vulnerabilities │    2C    22H    29M     3L
   size            │ 34 MB
   packages        │ 254

I re-evaluated current CVEs in the replacements and it seems that both have CVEs.

OSSContainertools

## Overview

                   │               Analyzed Image
───────────────────┼─────────────────────────────────────────────
 Target            │  martizih/kaniko:slim
   digest          │  be27e3e3beff
   platform        │ linux/arm64
   provenance      │ https://github.com/osscontainertools/kaniko
                   │  8d4df1dcce0133cde09d0b1077d33b56437bb7b0
   vulnerabilities │    0C     7H     3M     0L
   size            │ 14 MB
   packages        │ 195

Chainguard

## Overview

                   │                 Analyzed Image
───────────────────┼────────────────────────────────────────────────
 Target            │  lol:latest
   digest          │  74fa202c3668
   platform        │ linux/arm64
   provenance      │ https://github.com/chainguard-forks/kaniko.git
                   │  d8b20785e0f6bf2a645828f5c15e484df98959ba
   vulnerabilities │    0C     4H     1M     0L
   size            │ 15 MB
   packages        │ 225

I am still undecided which image is the best to choose but most likely the one that is downloadable via DockerHUB provides the least friction.

I will now move to martizih/kaniko:slim