Re: [Tails-dev] Debian 9: Build fails consistently, name re…

Arnaud:
> On 04/04/2017 11:49 AM, anonym wrote:
>>> So at the moment, I'm still in this situation where build fails, and I
>>> retry and retry until at some point it succeeds. I didn't have much time
>>> to work on that yet. I'll come back to you when I workaround this issue,
>>> one way or another.
>> A simple workaround for when you have cached everything needed for the build (i.e. after your have successfully built *once*) is to add "offline" to the TAILS_BUILD_OPTIONS environment variable. Of course, getting to the point when you have everything cached will be painful, but at least you will be able to rebuild as much as you like without risk of network issues.
>
> Good idea, didn't try this option yet. I guess that it will avoid any
> `apt` request, however what about the Tor download that is done with
> `wget` ? This is not cached I think, and it's quite a long download, one
> that fails

I don't know what you mean with "Tor download" -- the build system itself doesn't run or use tor.

Any way, the "offline" option is supposed to enable a true offline building mode, and if a build fails due to lack of Internet access, that is a bug. It relies on all interactions with the Internet being done with applications that respect the http_proxy environment variable (which points to apt-cacher-ng, which is configured to allow downloading and caching more than just .deb:s). Specifically, `wget` respects http_proxy, and apparently so do all the various mechanisms used to fetch firmwares and other files in the post-install scripts of all .deb:s Tails uses.

>> Here's a crazier idea to explore:
>>
>> * run apt-cacher-ng on the host (base your configuration on vagrant/provision/assets/acng.conf)
>> * add "extproxy" to TAILS_BUILD_OPTIONS
>> * set the http_proxy environment variable to point to the host's apt-cacher-ng instance
>> * the "offline" option doesn't work with external proxies, but you can set Offline mode in apt-cacher-ng's configuration instead
>> * sadly apt-cacher-ng lacks an option for retrying failed downloads, but if you can find any other HTTP proxy that can, set that up as well and then configure apt-cacher-ng to use it as a parent proxy itself (see the `Proxy` option).
>
> One the same line: did you guys consider having apt-cacher in another VM
> ? As a beginner, I'm still at the point where I modify the VM here and
> there, and at some point I'm not sure anymore what I modified, and in
> the end destroy the VM to start again from scratch. In these situations,
> having another VM for apt-cacher would be a real delight.
>
>> Happy hacking! Let us know if you find out something about the last point (parent proxy that retries). If it improves your situation without using much resources we probably want to add it to the VM's default apt-cacher-ng setup.
>
> Yep I'll tell you as soon as something interesting happens on my side ;)

If you know C, perhaps you'd want to look into implementing (optional) download retrying in apt-cacher-ng? That would, of course, be the best solution.

Cheers!