On Mon, Sep 10, 2012 at 03:40:31PM +0200, anonym wrote:
> * Save cache/{stages,packages}_bootstrap on disk and bind mount them
> into the build dir. This saves ~200 MiB, and speeds up the build
> quite significantly. OTOH is occupies that much extra disk space.
> * Use aufs so that the tmpfs is the rw branch, and the Tails sources
> is the ro branch. End result: we don't have to store a copy of the
> Tails source in the tmpfs, which is ~344 MiB for me. The only
> potential drawback is with performance; aufs itself has has some
> performance overhead, and also the sources has to be read from disk.
> However, linux' cache should take care of the latter if enough memory
> is available so that it in theory becomes equivalent to copying the
> sources into the tmpfs.
>
> Here's a comparison of "maximum RAM usage / build time" for building
> experimental with:
>
> no optimizations (7 GiB RAM): 6.12 GiB / 11m57.055s
> bootstrap cache: 5.87 GiB / 8m57.749s
> aufs trick: 5.84 GiB / 12m7.156s
> bootstrap cache + aufs trick: 5,64 GiB / 8m40.086s
>
> Both seem like great improvements to me both memory-wise and
> (incidentally) time-wise. I've implemented this in feature/vagrant: the
> aufs trick is always used when building in RAM, and the bootstrap cache
> can be controlled with TAILS_BUILD_OPTIONS=(no)cache, (enabled by
> default), and the TAILS_BUILD_OPTIONS=cleanall option also clears the
> bootstrap packages + stage cache. The documentation about this also
> pushed into that branch.
>
> Any one familiar with vagrant (ague?), please have a look and test this.
> Once feature/vagrant is merged into devel, we can also revert commit
> 974805d in feature/multikernel and instead merge in feature/vagrant.
This is all very lovely. It's been a while since a wanted to add a way
to cache the bootstrap stage.
Minor stuff regarding the changes:
> commit fec1126adbf22213e8359729bec6d71fd390304c
> +++ b/vagrant/provision/assets/build-tails
> +TMPFS_SIZE_BYTES=$(echo "($TMPFS_SIZE_GB * 1024^3)/1" | bc) # rounded down
I don't see the need for bc. How about dropping the floating point and
having something like:
TMPFS_SIZE_MB=6144 # 6 GB
TMPFS_SIZE_BYTES=$(($TMPFS_SIZE_GB * 1024 * 1024))
> commit c34b0621706f1ae7ad25acb034bf010cae0fd7b5
> +++ b/vagrant/provision/assets/build-tails
> + # cached yet?) so we we save the good one from the current build.
^^^^^
once is probably enough
I've done some builds with or without using RAM. It seemed to work well.
--
Ague
