Hi noisycoil,
yust FYI;
* on Apr., 08. I started a new 6.1/raspi build based on the latest pull:
Got different sha256sum's than listed for your MEGA images. I assume
that's what you ment with "Talking of reproducibility".
* currently I'm not able to build 6.1/raspi fails (reproduceable) with
"E: Release signed by unknown key (key id 2453AA9CE4123A9A)" - might be
missleading / overlaying a HTTP 404 caused by my DNS hijack
* I'll stop the builds for a while and will take some time to understand
the build and your changes/commits better.
All the best!
n9iu7pk
PGP 7426 4598 B5AD 4D12 1699 C710 [ D602 E331 4D12 FFCB ]
https://keys.openpgp.org/search?q=D602E3314D12FFCB
On 03.04.24 02:24, noisycoil@??? wrote:
> Hi n9iu7pk,
>
> Thank you! Happy to hear you were able to build the other images as well.
>
>
> Talking of reproducibility, I just added two fixup commits to the various branches to make the images more reproducible (in the sense of reproducible builds). The first one adds timestamps to the contents of /EFI/debian/efi.img (efi.img is a FAT image that contains the grub binary and its config file; these are responsible for UEFI boot on ISO hybrid images). The second one fixes (or rather removes) the BCJ filter used when building the squashfs with xz compression. It turns out that the filter which I had selected ("arm") is specific to 32-bit arm, cannot be used for 64-bit arm64 (I had never actually used xz compression!), and that the "arm64" BCJ filter is not available yet neither to mksquashfs nor to the kernel decompression routines (it is in xz-utils though, AFAICS). Fun fact: adding kernel support for the arm64 filter seems to be part of a recent patchset whose merge is now frozen due to the compromise of xz-utils (see https://lkml.org/lkml/2024/3/20/1010).
>
>
> Thanks to the second commit one can now build tagged arm64 images using the following nginx configuration for DNS hijacking + URL redirection:
>
>
> server {
> server_name time-based.snapshots.deb.tails.boum.org;
>
> rewrite ^/(debian|debian-security)/[0-9]+(/?.*) http://deb.debian.org/$1$2;
> rewrite ^/torproject/[0-9]+(/?.*) http://deb.torproject.org/torproject.org$1;
> rewrite ^/[0-9.]+(/?.*) http://deb.debian.org$1;
>
> location ~ ^/(debian|debian-security|tails|torproject)/project/trace/(debian|debian-security|tails|torproject) {
> proxy_pass http://204.13.164.63:80;
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_set_header X-Forwarded-Protocol $scheme;
> proxy_set_header X-Forwarded-Host $http_host;
> }
>
> location ~ ^/tails {
> proxy_pass http://204.13.164.63:80;
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_set_header X-Forwarded-Protocol $scheme;
> proxy_set_header X-Forwarded-Host $http_host;
> }
>
> listen *:80;
> }
>
> server {
> server_name tagged.snapshots.deb.tails.boum.org;
>
> rewrite ^/[0-9a-z.-]+/(debian|debian-security)/(.*) http://deb.debian.org/$1/$2;
> rewrite ^/[0-9a-z.-]+/torproject/(.*) http://deb.torproject.org/torproject.org/$1;
>
> listen *:80;
> }
>
>
> Note the addition of tagged.snapshots.deb.tails.boum.org, which you must redirect to your webserver too (e.g. via /etc/hosts) to be able to build tagged images. Also, as I've already told you in a private email, I am now also redirecting to Tor's debian repository, which does support arm64.
>
>
> Non-tagged images have snapshot timestamps in their apt sources, which spoils reproducibility (debian-security's snapshot timestamp, in particular, gets updated every day AFAICS). Thanks also to the timestamping of efi.img's contents, tagged arm64 images should on the other hand be reproducible during the time frame in which the packages downloaded from the Debian archive don't change: if you're lucky enough to build two arm64 images with the same package versions, then those images should be identical byte-for-byte. Of course, this is not actual reproducibility (which would need actual snapshots for the arm64 component of the Debian archive so that the images are forever reproducible), but still, it's better than nothing.
>
> I created three new branches, 6.1/arm64, 6.1/asahi and 6.1/raspi, which contain the same source code as their current wip/* analogue, but with the commits squashed and a tag (6.1-arm64, 6.1-asahi and 6.1-raspi) applied at the end of them. I do not plan to modify the content of those branches, which you may thus consider as arm64 6.1 "releases" of my developer preview. The binary images built on the 6.1-* tags are in the usual MEGA folder. If you happen to build images on those tags in the near future, and the contents of the *.packages files are the same (which is essential to reproducibility), please let me know if you obtain the same sha256sums!
>
>
> The tagged arm64 and asahi images don't pass one more test scenario, "APT sources are configured correctly", because of a failure in the very last step, where it is checked that the deb.tails.boum.org distribution in its apt.sources.d list file ("6.1", in our case) matches the build's git tag ("6.1-arm64" or "6.1-asahi" or "6.1-raspi"). In theory I could make it so that the contents of the apt sources file match the git tag (e.g. by actually defining a new release version), but I believe this would make things works since deb.tails.boum.org does not provide "6.1-arm64" or "6.1-asahi" or "6.1-raspi" distributions, so apt would probably crash on update because of a non-existing repo! In practice I'm ok with this scenario failing, as long as it does for a good reason.
>
>
> Best,
>
> NC
>
>
> P.S.
>
>>> Lot of work to follow your work :D
>
> Eh eh, I hear you. I think I will take a long vacation now :-D
>
