Support reproducible builds

[conorsch]

See https://www.kernel.org/doc/html/latest/kbuild/reproducible-builds.html In order to support deterministic builds, set:

• KBUILD_BUILD_TIMESTAMP
• (and/or?) SOURCE_DATE_EPOCH
• KBUILD_BUILD_USER
• KBUILD_BUILD_HOST

Also of note:

If you enable CONFIG_GCC_PLUGIN_RANDSTRUCT, you will need to pre-generate the random seed in scripts/gcc-plugins/randomize_layout_seed.h so the same value is used in rebuilds.

[conorsch]

Reproducible builds are working. There are a few exemptions required, among the suite of tests that reprotest runs, but overall it works fairly well. In order to get fully deterministic builds I had to patch the kernel build logic itself, to get the debian changelog for the kernel image to support SOURCE_DATE_EPOCH, rather than calling out to date -R every time. Since the build logic in this repo already supports sideloading patches, that's not a huge deal, but calling it out for special attention.

[conorsch]

The changes in #6 got us pretty far: we now have reproducible deb builds for a very small kernel config. For SD kernels in #7, however, the debs still aren't reproducible, almost certainly due to CONFIG_GCC_PLUGIN_RANDSTRUCT seed value mentioned above.

[legoktm]

For the RANDSTRUCT issue, my current idea is we have a fixed seed stored (encrypted) in the infrastructure repo alongside the grsecurity credentials, that kernel builders export as an environment variable that the build script will dump into the correct file. Then we just need to ensure that it does not end up in the orig tarball that is distributed.

Also note that in the linked documentation, the filename has changed, but that only happened in 5.19 (torvalds/linux@be2b34f) so we still need to use the old one for our 5.15.

[legoktm]

For reference, the format of the file is:

user@0ba2b0e3ea89:/kernel/linux-5.15.60/scripts/gcc-plugins$ cat randomize_layout_seed.h 
const char randstruct_seed[] = "38b1a9d0efeb7f8ba8eec4e58d67190c8ba5deddd86c40422dc1f71d4438fc5c";

[legoktm]

I injected a fixed random seed into the build process, ran two builds, and unfortunately it was not reproducible:

user@sd-kernel-builder:~$ sha256sum build-one/*.deb
158547433df602347fdefd8105ef52d99afda63ca0269fccf66e6c92cc58915a  build-one/linux-headers-5.15.61-grsec-workstation_5.15.61-grsec-workstation-1_amd64.deb
59547d246f0608d227fd45b4f16d756af53b8b545562425c04af027e1f861e91  build-one/linux-image-5.15.61-grsec-workstation_5.15.61-grsec-workstation-1_amd64.deb
62a0075f64dd1c37c196d40a973bca55c536f6780fc2e99914597673f16fcee6  build-one/linux-image-5.15.61-grsec-workstation-dbg_5.15.61-grsec-workstation-1_amd64.deb
f203a112ac11cfd97aa386cc00a5a3d9d2b0d11daca59a22e38de5125253390a  build-one/linux-libc-dev_5.15.61-grsec-workstation-1_amd64.deb
904994e2b99665c3c5d8414e0efbe1bd26ed6484897871e0b49cc3544079fbff  build-one/securedrop-workstation-grsec_5.15.61-grsec-workstation-1_amd64.deb
user@sd-kernel-builder:~$ sha256sum build-two/*.deb
158547433df602347fdefd8105ef52d99afda63ca0269fccf66e6c92cc58915a  build-two/linux-headers-5.15.61-grsec-workstation_5.15.61-grsec-workstation-1_amd64.deb
d1bb1b3e09a1250fd0d6e2524ade449e57503cb1f2c0552a6e1b87d352b91604  build-two/linux-image-5.15.61-grsec-workstation_5.15.61-grsec-workstation-1_amd64.deb
0704a9bedee01ab6af3e5dc2363e7d49e39292b6fcb8924d4b9835ed0586498e  build-two/linux-image-5.15.61-grsec-workstation-dbg_5.15.61-grsec-workstation-1_amd64.deb
f203a112ac11cfd97aa386cc00a5a3d9d2b0d11daca59a22e38de5125253390a  build-two/linux-libc-dev_5.15.61-grsec-workstation-1_amd64.deb
904994e2b99665c3c5d8414e0efbe1bd26ed6484897871e0b49cc3544079fbff  build-two/securedrop-workstation-grsec_5.15.61-grsec-workstation-1_amd64.deb

There's something else going on, the diffoscope output was mostly useless to me (it's all assembly and stuff). I'm going to stop working on this now since it appears to require more work than expected.

[legoktm]

Discussion w/ Mickael and Conor in Montreal was that at some point these builds were reproducible. I had Claude take a look and it found 4 other things in addition to the known RANDSTRUCT problem:

• CONFIG_GRKERNSEC_RANDSTRUCT / RANDSTRUCT_FULL
• CONFIG_GCC_PLUGIN_LATENT_ENTROPY / PAX_LATENT_ENTROPY
• CONFIG_GCC_PLUGIN_TYPE_CANARY
• CONFIG_MODULE_SIG_ALL (server-only)

For the first three, the seeds are publicly available in the headers package we publish, and even then, could be reconstructed from e.g. unpacking vmlinux. So we should just switch to deterministic seeds.

For module signing, we should be able to just turn it off, the key is generated and build time and then we throw it away, so it's useless.