[Tails-dev] testing squashfs-tools 4.3

Delete this message

Reply to this message
Autor: Romeo Papa
Data:  
Para: tails-dev
Assunto: [Tails-dev] testing squashfs-tools 4.3
...---...

Subject: testing squashfs-tools 4.3
To: tails-dev@???

Hello all,

* I've set up an old computer in my house to act as a Tails builder. I can't
afford to make much more than a single-core 1GB of RAM VM.
* Learning about what SquashFS was, I had a look at the documentation. Somehow
found out that squashfs-tools 4.3 has been released since 2014-05-12
(http://sourceforge.net/projects/squashfs/files/squashfs/)
* Here is the changelog:
http://sourceforge.net/p/squashfs/code/ci/master/tree/CHANGES
* We can see it mentions that some new gzip compression options were added to
this version. We can now use a lower compression level, which would supposedly
allow us to build faster testing ISOs.
* We can also see experimental support for LZ4 which is supposed the king of
all the decompression speedsters.

Debian packages are at 1:4.2 and there is no sign of a 4.3 at sid.

I went a made a 4.3 package by combining the src for squashfs-tools 4.3 and
what I found in squashfs-tools_4.2+20130409-2.debian.tar.xz, made a bogus entry
in changelog, and activated LZ4 in the makefile.

I've got my own custom deb mirror for building my custom version of Tails, so
it was fairly easy to outstage the 4.2 version that you are all currently using.

Here are some results:

BASELINE: XZ currently used in STABLE
- -comp xz -Xbcj x86 -b 1024K -Xdict-size 1024K:
real    81m43.904s (4903.904s)
user    54m10.363s (3250.363s)
sys    9m24.107s
The ISO is 970584064 bytes large.


BASELINE: Gzip vanilla currently used in TESTING
- -comp gzip:
The ISO is 1291370496 bytes large.
real    44m40.244s 2680.244
user    14m16.910s
sys     8m18.031s


TEST1: GZip with the lowest compression setting
- -comp gzip -Xcompression-level 1
The ISO is 1412929536 bytes large.
real    38m4.922s 2284.922
user    8m41.441s
sys     6m59.390s


TEST2 LZ4 vanilla
- -comp lz4
The ISO is 1801336832 bytes large.
real    38m18.538s
user    7m51.701s
sys     6m44.649s


TEST3 LZ4 High Compression
- -comp lz4 -Xhc
The ISO is 1529903104 bytes large.
real    37m59.400s
user    8m57.886s
sys     7m22.332s


Conclusions:
* The new 4.3 version appears to allow for faster test builds when using GZIP
with the lowest compression setting.
* Remarkably, it appears that LZ4 High compression is faster than LZ4 vanilla,
and even slightly faster than GZIP with the lowest compression setting. Could
LZ4 -Xhc become the new favoured 'quick-build' compression setting for the
future?

Final words:
* Would you prefer to wait for Debian to upstream the squashfs-tools 4.3
or would you be willing to distribute a custom build from the tails deb repo?
I could submit the files I used to build squashfs-tools 4.3.

PS: mksquashfs 4.3 also has a '-progress' argument you can pass it, which
is supposed to give you a nice progress bar so you can see at what point of
the compression you currently are, but sadly I haven't seen it work as
advertised.

Take care!

...---...