Hi intrigeri,
I can't see any particular problem (except for some less than idiomatic
EcmaScript style instances, e.g. new Array()) in your code.
The only caveat for integrating it into DAVE is that every time the
extension starts/resume a download or is installed/upgraded/reloaded,
downloader.js loops through all the download jobs already "known" to the
browser's built-in download manager and, if any of them matches the URL
in the IDF, picks the active one, or if none is currently active, the
most advanced one and makes it "current", resuming it if required.
Therefore, rather than just checking for equality with the URL provided
by the IDF, we should check whether these jobs match any of the URLs
produced by combining the mirror host names with the path provided by
the IDF, and force the mirror of the "choosen" download job if any can
be made current.
Beside that, I think the JSON should be fetched every time the UI page
is reloaded (of course obeying to server-side caching directives), just
like we currently do with the IDF.
> Will you be happy to review our DAVE patch, once we have something
> that Works For Me™? Likely we will be working on it on April 6-8, and
> ideally we would need the new feature to be on AMO by the end
> of April.
Of course I will.
> Also, it would be awesome if we could ask you one or three questions
> when we hit technical difficulties along the way,
No problem, just drop me an email and if asked I'm also going to join
the #tails-dev channel ASAP.
Cheers
-- G
On 11/02/2016 14:48, intrigeri wrote:
> Hi Giorgio, sajolida and others,
>
> context: we're redesigning how we're handling the pool of HTTP mirrors
> used to download Tails ISO images. If you want details, see the "Big
> picture" section on https://tails.boum.org/blueprint/HTTP_mirror_pool/
>
> We need to add a feature to DAVE to support our new design. I'd like
> to check with Giorgio how possible/difficult it will be, in your
> opinion. And I'd like sajolida to sanity-check the plan, in particular
> checking compatibility with the IA. Anyone else in the audience is
> much welcome to help, of course :)
>
> I'll first explain what we need DAVE to do, and then I have a couple
> of questions, mainly for Giorgio.
>
>
> What we need DAVE to do
> =======================
>
> We need DAVE to retrieve a JSON configuration file from our website,
> that describes our pool of HTTP mirrors. The exact content and format
> of the config file is not clear yet, but let's assume for now that
> it'll be a list of mirrors, with information attached to each of them:
> a base URL, and a weight (0 = disabled, and the bigger the number, the
> higher the chances to pick it).
>
> And then, whenever DAVE wants to use the ISO URL it found in the ISO
> description file (IDF in the new installation process specification,
> i.e. what is called a "descriptor" in the DAVE code base), for example
> to start a download, we need DAVE to pick one mirrors from this list
> (honoring the weight but that's the trivial part I bet), and replace
> the "http://dl.amnesia.boum.org/tails/" part of the URL found in the
> IDF with the base URL corresponding to the mirror it picked.
>
> I bet it'll be simpler to modify a little bit the IDF format, to
> avoid hard-coding this part of the URL in the IDF, and write:
>
> path: /tails/stable/tails-i386-2.0/tails-i386-2.0.iso
>
> instead of the current:
>
> url: http://dl.amnesia.boum.org/tails/stable/tails-i386-2.0/tails-i386-2.0.iso
>
> ... but this is a detail at this stage, and I think we can ensure
> a smooth migration path without having to bump the IDF format to v2.
>
>
> Questions
> =========
>
> I assume that doing most of this (retrieving and loading the JSON
> config file, picking an element from a list, doing simple string/URL
> mangling) in DAVE is pretty basic stuff, that anyone moderately
> experienced in JavaScript could do, without expertise that's specific
> to Firefox add-on development. Correct?
>
> What I'm more concerned about is the integration of the basic pieces
> into the clever stuff that DAVE does. E.g. I see that it is
> maintaining a state machine (status.phase), that is used to
> retry/continue downloads, display progress, trigger verification etc.,
> so for example we need to be careful when exactly we pick a mirror and
> build the ISO URL (we have to do it before the download starts, but
> shall we re-do it every time we retry?). This is just an example of
> the class of issues I expect we will have to solve when implementing
> what we need. How hard does it sound?
>
> Any other problems you would expect us to face?
>
> Will you be happy to review our DAVE patch, once we have something
> that Works For Me™? Likely we will be working on it on April 6-8, and
> ideally we would need the new feature to be on AMO by the end
> of April.
>
> Also, it would be awesome if we could ask you one or three questions
> when we hit technical difficulties along the way, but if you can't do
> it, it's fine, I guess we can ask for help to other add-on developers
> we know :)
>
> To end with, can you please give a quick look at our existing JS code,
> and tell us if anything can't possibly work in a Firefox add-on such
> as DAVE, or really, if anything raises a red light:
>
> https://git-tails.immerda.ch/mirror-pool-dispatcher/
>
> This draft code implements something very similar to what I'm
> describing here, except it's meant to be loaded and run from a web
> page that has a download link, instead of from a Firefox extension
> (which I think we will need for downloads that are not mediated by
> DAVE).
>
> Cheers,
--
Giorgio Maone
https://maone.net
