sajolida:
> Hi,
>
> During the 2.12 release at least one of our mirrors run out of disk
> space. They followed our instructions closely and allocated 10 GiB (we
> ask for "5-10 GiB") and apparently we're now a bit above 10 GiB.
Ouch! :/
> I'd like to:
>
> 1. Make sure that we have a process set up to clean very old IUKs.
>
> For example, we could say that we only keep IUKs for the big version
> number. IUKs for 2.x until 3.0 is out as a change in the big version
> number will lead to no IUK for everybody. Right I see IUKs for 1.0~test
> on the mirrors which lead me to believe that there is only an ad-hoc
> process:
The 1.0~test IUK is a dummy IUK used for automated testing. Please ignore it! :)
> http://dl.amnesia.boum.org/tails/stable/iuk/
>
> I didn't check what the release process says about that.
At the moment the process is to always remove IUKs that are older than six months, and to remove ~alpha/~beta/~rc IUKs as soon as the final release is out (except the IUKs upgrading from those versions to the final one, of course).
> 2. Fix the current requirements which seems too low. Let's do a bit of
> calculation. Now is a quite epic time for mirrors with 12 versions in
> the 2.x cycle and a heavy preparation for 3.0 so it a good time to
> adjust our requirements.
>
> - We're now publishing two IUKs per version (n-2→n and n-1→n) and they
> are 263 MiB on average. So for 12 versions and 6 RCs, that's 30 IUKs and
> 7.7 GiB.
Remember, we provide IUKs from the previous two *planned* releases, and any unplanned (i.e. emergency) releases in between, so the *minimum* number of IUKs per version is two.
Also, this number is higher than it should be because we clean up IUKs every six months, not when Tails migrates to a new Debian version. If someone feels like it they could come up with a new number that takes that into account, and adds a few emergency releases (incl. the extra IUKs that implies for the following releases), but I'm happy with stating that 10 GiB seems enough.
> - We have 3 IUKs for 3.0~betaX and will have at least 2 more (beta4→rc1
> and rc1→3.0) and they are significantly bigger. That's 1.9 GiB.
> - We have 2 full ISO (2.12 and 3.0~beta4) and might have 3 max if we
> include an RC. That's 3.0 GiB.
Also, right around release n, both the n-1 and n ISOS will be on the mirrors, so that should be more like 4 GiB.
> So that's 12.6 GiB.
Finally, this number doesn't take into account the project/vagrant directory which currently is at 2 GiB, but OTOH we will soon be able to remove it, so let's ignore it.
> We shouldn't discard publishing even more IUKs in the future (I think
> anonym had some crazy plan about this somewhere).
FWIW, with that vague, potential plan (#11131) it's expected that the space needed for (stable) IUKs will increase to 11 GiB. But those numbers are based on data for the 2.x series. With 3.x things can go either way: in general, our ISO is growing so we can expect larger IUKs; but 3.x will be reproducible, so we can expect IUKs to be smaller. We'll see! :)
> 3. Have more explicit requirements. Apparently not every mirror has good
> monitoring of their disk space, so putting a range as requirement could
> lead to more failures than setting a strict requirement. So what about
> updating our documentation to say something like:
>
> "You will also need 15 GiB of disk space maximum."
Disk space is cheap! My expectation is that for those that can offer to run a mirror, 15 GiB is nothing, and given your current (incorrect) approximation I think we're asking for too little. I say, let's go for 30 GiB, which I bet no mirror operator will mind providing, and as a bonus I believe we'd be ready for the "crazy plan" (#11131) in case we decide to go that way.
> 4. Make sure that we don't break them again silently in the future.
> Would it be complicated to check the size of the whole thing during the
> release process? I'll let the RMs tell me if it's worth the extra work.
Sounds like a good idea! Where can I find an always up-to-date list of what directories that are mirrored (IIRC not all of the ones in our rsync directory are)? Then I can cook up a command that uses that list to calculate the current mirror disk usage.
Cheers!
