Re: [Tails-dev] Some research about mirror infrastructure

intrigeri wrote:
> I agree yet another layer of indirection, with HTTP, is the best.
>
> Your latest preferred idea (with dynamic code picking a mirror among
> the full list, running on a few "super-mirrors"), is not mentioned on
> the blueprint yet, right?
>
> I like it too, but its feasibility is conditioned by the availability
> of enough (stable, strong) mirrors that either already have a setup
> able to more or less securely run whatever PHP (or similar) we feed
> them, with whatever input data (the list of (IP, weight) pairs) we
> feed them, or are willing to set it up and keep it running.
>
> I think this requires to do a quick survey. If someone writes a draft
> email that we could send to the admins of our current fastest and most
> stable mirrors, then I'm happy to send it and report the results back.
>
> Technical details follow:
>
> * I suggest that the super-mirrors use Git over SSH, run via cron, to
> keep the code and configuration up-to-date. This requires the mirrors
> to have Git, SSH client with pubkey authentication capability, and
> cron. Add this to the survey?
>
> * Maybe we want the super-mirrors to properly check integrity,
> authenticity and up-to-date-ness of what they get. Maybe trusting
> the server that hosts this stuff *and* HTTPS or SSH crypto is good
> enough. The latter is at least as good as what we have now, so let's
> not over-engineer it for a first iteration.
>
> * Whatever we think of it, PHP is the most readily available language
> for these admins to run. Maybe I'm wrong, and it might be a good
> idea to ask in the survey if the admins can run stuff written in
> Python, Perl or Ruby.
>
> * Do we require minimal isolation of how our dynamic code runs, e.g.
> at least having it run under a dedicated UID, as opposed to mod_php
> + one single shared UID for all websites + deprecated crap such as
> open_basedir?

I'm wondering if this added complexity is really worth it for a first
fix. Could we work on this core HTTP redirection mechanism first (to
solve the pool size problem) -- with maybe lizard running a duplicate of
this script -- and work on a more distributed super-mirror mechanism (to
solve a resiliency problem) later on?

That would put more work on our side to setup lizard as a fallback, but
that would save us, for the time being, the work of coordinating,
verifying, and maintaining this set of outsources super-mirrors.

--
sajolida