anonym:
> sajolida:
>> During the 2.12 release at least one of our mirrors run out of
>> disk space. They followed our instructions closely and allocated 10
>> GiB (we ask for "5-10 GiB") and apparently we're now a bit above 10
>> GiB.
>
> Ouch! :/
>
>> I'd like to:
>>
>> 1. Make sure that we have a process set up to clean very old IUKs.
>>
>> For example, we could say that we only keep IUKs for the big
>> version number. IUKs for 2.x until 3.0 is out as a change in the
>> big version number will lead to no IUK for everybody. Right I see
>> IUKs for 1.0~test on the mirrors which lead me to believe that
>> there is only an ad-hoc process:
>
> The 1.0~test IUK is a dummy IUK used for automated testing. Please
> ignore it! :)
Ah!
>> http://dl.amnesia.boum.org/tails/stable/iuk/
>>
>> I didn't check what the release process says about that.
>
> At the moment the process is to always remove IUKs that are older
> than six months, and to remove ~alpha/~beta/~rc IUKs as soon as the
> final release is out (except the IUKs upgrading from those versions
> to the final one, of course).
With this and the 1.0~test thingie, I understand better what I see on
the mirrors and I'm happy that we already have a good process :)
>> 2. Fix the current requirements which seems too low. Let's do a bit
>> of calculation. Now is a quite epic time for mirrors with 12
>> versions in the 2.x cycle and a heavy preparation for 3.0 so it a
>> good time to adjust our requirements.
>>
>> - We're now publishing two IUKs per version (n-2→n and n-1→n) and
>> they are 263 MiB on average. So for 12 versions and 6 RCs, that's
>> 30 IUKs and 7.7 GiB.
>
> Remember, we provide IUKs from the previous two *planned* releases,
> and any unplanned (i.e. emergency) releases in between, so the
> *minimum* number of IUKs per version is two.
>
> Also, this number is higher than it should be because we clean up
> IUKs every six months, not when Tails migrates to a new Debian
> version. If someone feels like it they could come up with a new
> number that takes that into account, and adds a few emergency
> releases (incl. the extra IUKs that implies for the following
> releases), but I'm happy with stating that 10 GiB seems enough.
>
>> - We have 3 IUKs for 3.0~betaX and will have at least 2 more
>> (beta4→rc1 and rc1→3.0) and they are significantly bigger. That's
>> 1.9 GiB. - We have 2 full ISO (2.12 and 3.0~beta4) and might have 3
>> max if we include an RC. That's 3.0 GiB.
>
> Also, right around release n, both the n-1 and n ISOS will be on the
> mirrors, so that should be more like 4 GiB.
>
>> So that's 12.6 GiB.
>
> Finally, this number doesn't take into account the project/vagrant
> directory which currently is at 2 GiB, but OTOH we will soon be able
> to remove it, so let's ignore it.
>
>> We shouldn't discard publishing even more IUKs in the future (I
>> think anonym had some crazy plan about this somewhere).
>
> FWIW, with that vague, potential plan (#11131) it's expected that the
> space needed for (stable) IUKs will increase to 11 GiB. But those
> numbers are based on data for the 2.x series. With 3.x things can go
> either way: in general, our ISO is growing so we can expect larger
> IUKs; but 3.x will be reproducible, so we can expect IUKs to be
> smaller. We'll see! :)
>
>> 3. Have more explicit requirements. Apparently not every mirror has
>> good monitoring of their disk space, so putting a range as
>> requirement could lead to more failures than setting a strict
>> requirement. So what about updating our documentation to say
>> something like:
>>
>> "You will also need 15 GiB of disk space maximum."
>
> Disk space is cheap! My expectation is that for those that can offer
> to run a mirror, 15 GiB is nothing, and given your current
> (incorrect) approximation I think we're asking for too little. I say,
> let's go for 30 GiB, which I bet no mirror operator will mind
> providing, and as a bonus I believe we'd be ready for the "crazy
> plan" (#11131) in case we decide to go that way.
Thanks for all the extra info! Summarizing your version we're talking about:
- Official ISO 2 GiB
- Official IUK 11 GiB
- Testing ISO 2 GiB
- Testing IUK 2 GiB
- TOTAL 17 GiB
I also thought that we could assume that disk space is cheap but then I
rememered how much money we actually spending on disks and that always
freaks me out :) But right, mirror operators might not store all their
stuff on SSD like we do.
I'm also not super happy to ask for much more than what we need (right
now three times more than what we use). But yeah, I guess that I'll bump
the number to 30 GiB, write all mirror operators and see if that's an
actual problem for some of them.
>> 4. Make sure that we don't break them again silently in the
>> future. Would it be complicated to check the size of the whole
>> thing during the release process? I'll let the RMs tell me if it's
>> worth the extra work.
>
> Sounds like a good idea! Where can I find an always up-to-date list
> of what directories that are mirrored (IIRC not all of the ones in
> our rsync directory are)? Then I can cook up a command that uses that
> list to calculate the current mirror disk usage.
I always forgot how our uploading mechanism work as I never interact
with it myself. But if you are not sure yourself about how to compute
this number easily then it probably means that this idea is more
complicated than it should and that we should drop it :)
