Re: [Tails-dev] Tails persistence use case

Delete this message

Reply to this message
Autore: intrigeri
Data:  
To: The Tails public development discussion list
Oggetto: Re: [Tails-dev] Tails persistence use case
Hi,

(meta: I agree on everything I don't comment on, and have therefore
snipped it.)

anonym wrote (14 Nov 2011 00:53:21 GMT) :

> Requirements
> ============


> From the roadmap and various other places in our todo item [1] our
> high-level requirements seem to be these:


> * Persistent user data store.


FTR, this was not meant to be implemented using live-snapshot at all.
A "simple" wrapper around udisk may allow to unlock and mount (possibly
read-only) a LUKS encrypted volume onto e.g. $HOME/data.

> The problems with snapshots
> ===========================


> But cpio.gz snapshots has some issues:


> 1. It is very unfriendly to flash based storage if we only do minor
>    changes to our persistent data, since *all* persistent data are
>    written back to the physical storage at *every* shutdown. I'm afraid
>    minor changes is a more typical usage where it matters. Imagine
>    having 100 MB of emails, fetching maybe 50 KB worth of new mails from
>    your inbox, and then syncing *all* the 100 MB worth of old,
>    unmodified emails back as well. That causes pretty significant write-
>    wearing in comparison to how much data that was added to the
>    snapshot.


Imagine your fetching of those 50kB new email is split into
4 different small fetches, accross a 2 hours Tails session. Each of
these small fetches is likely to update local email indexes and
whatnot other files the MUA makes sure to keep up-to-date. Say those
files, for a 100MB email store, are a few MB each. Then, snapshots
ensure those files are written only to the Flash device, while a more
synchronous method would write them four times. Depending on the
actual numbers (the 50kB / 100MB ratio, the number of fetches, the
actual size of the email indexes etc), I guess the async' vs.
sync' resulting figures wrt. write-wearing may be not *that* obvious.

> 2. On boot all snapshots' files are synced into the tmpfs, so they're
>    stored in preicious RAM. Hence snapshots cannot be very large
>    (specifically, the maximum is ~ ${RAM_SIZE}/2, and that leaves no
>    space for other file system modifications).


What kind of big files / directories do we intend to make (optionally)
persistent as part of "Persistent application-specific
configurations"? (This is no rhetorical question, I don't want to save
snapshots at all costs, but I think it will help the discussion to
know better what we are talking about.)

- /var/lib/tor : a few MB
- /var/lib/i2p : ???
- email store : generally a few dozens MBytes
- random configuration / keys listed on the wiki: a few MB

Anything else?

Note: some hardcore users may want to access many hundred MBytes, or
even above GByte email stores, or a 100MB GnuPG pubring or whatever;
I think we should not make this impossible, but I would not be shocked
if we had to tell them "then you need at least 4GB RAM and a non-Flash
external hard-drive for your persistence needs".

> The case for overlays
> =====================


[...]
> The overlay's only limitation is that it has static size and cannot
> automatically grow like a snapshot file can (snapshot partitions
> can't for the same reason).


Aren't there ways to have a loop/file-backed filesystem grow as
needed? (I might be dreaming of using qcow2 as a container.)
Probably a dead-end, but would be worth a quick search.

> Proposed solution: locally specified inclusions
> ===============================================


> We make home-{rw,sn} obsolete, only live-{rw,sn} are considered by
> live-boot (or more correctly, the scripts it adds to the initramfs).
> When a persistent media (with label/filename "live-rw") is found by
> live-boot, it looks for a file called .live-persistence.includes (but
> I'll continue calling it just ".includes") in its root. If it's not
> there, then it mounts the media (using aufs) on / just like it does for
> live-rw currently. But if .includes is present, then it doesn't mount
> anything on /, it instead bind-mounts the directories listed in
> .includes to their specified destinations.


Looks like it would perfectly suit our needs.

A problem though, is that this way to deal with things requires
a clever-enough underlying filesystem to get permissions right,
whereas good old home-sn perfectly satisfies itself, I believe, with
a FAT32 Flash stick. This is probably not a problem for Tails itself,
but making home-sn obsolete may be a problem for other people due to
that. The naming of .live-persistence.includes may be incompatible
with poor filesystems, too.

The "inclusions" naming seems misleading to me. The old
/etc/live-persistence.binds (see live-boot(7)) somehow occupies the
"bind" namespace, which makes it more complicated to find a good name.
Wouldn't live-persistence.mount do the job?

> Snapshots
> ---------


> .includes could also be used for all types of snapshots,


Right. This could be configured in another file, e.g.
.live-persistence.snapshots. So home-sn would not be made
obsolete, eventually?

> Backwards-compatibility
> -----------------------


> *If* we care for backwards-compatibility, we'd have to allow for an
> extended syntax which allows specifying source-desination pairs.
> To get the old home-{rw,sn} type persistence,
> .live-persistence.includes should then look simply like this:
>   .    /home

[...]
> Beyond backwards-compatibility, some people may find this syntax
> more convenient. Not sure this is worth the effort for implementing
> it, though.


I'm not sure either.

> There are other UI changes in live-boot's persistence handling that
> will break stuff, so people will have to be prepared for fixing
> their old setups any way.


Sure.

Cheers,
--
intrigeri <intrigeri@???>
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc
| Every now and then I get a little bit restless
| and I dream of something wild.