On 2 Jan 2017 at 15:27, intrigeri wrote:
Hi all,
first of all, i've put spender on CC as an interested party
(especially for the kexec vs. kmem hardening you're discussing
in another thread).
> We're considering dropping our current implementation, and relying on
> PAX_MEMORY_SANITIZE instead; I know that PAX_MEMORY_SANITIZE was not
> designed for this use case, but who knows: it might be good enough :)
actually this isn't the first time this case has been raised and
cleaning memory on shutdown has been on my todo list ever since,
just not at a high enough priority.
> I understand that the result won't achieve perfect protection, but our
> current implementation doesn't either. I'd like to see numbers, but
> it's hard for me to compare experimentally both approaches: our QA
> about this relies on filling memory with a known pattern from
> userspace, which works fine to measure the efficiency of our current
> implementation, but of course PAX_MEMORY_SANITIZE will erase this
> known pattern from memory… So for now I'll stick to the theoretical
> level, and experimental measurements will come later.
>
>
> If I got it right, with PAX_MEMORY_SANITIZE enabled, then:
>
> * during the lifetime of a system, any memory allocated to
> a userspace program that terminates is erased;
correct but don't forget that 'allocated to userland' means only
anonymous memory pages that are mapped into userland memory,
everything else (page cache holding file content, dentry/inode/etc
metadata) is subject to different rules, see next.
> * during the lifetime of a system, any kernel memory that's
> explicitly freed, e.g. with kfree(), is erased;
well, 'free' is a bit abstract concept to reason about at this
level ;).
the kernel has more than one way to allocate (and thus free) memory
and SANITIZE doesn't (directly) apply to all of them. in practice
most memory is managed by the buddy allocator and a few layers that
build on top of it (slab, vmalloc, etc). the first (and original)
layer where SANITIZE kicks in is the buddy allocator where memory
is managed in groups of pages. this becomes important for the layers
built on top of it since they can very well allocate/free memory at
their level without every freeing back to the buddy allocator. for
this reason a while ago Mathias Krause added support for sanitizing
at the slab layer as well (slab manages C level objects, you can
consider it as the kernel's equivalent of malloc) but for performance
reasons not all slabs are sanitized by default (for tails you'll want
to enable all of it i guess, see the config help on the boot time
command line option or just patch the code to change the default
which is probably safer for your case).
> * on system shutdown, all processes are killed, and thus their memory
> is erased.
yes if (presumably :) you meant a clean shutdown (and subject to
the above details). i'm just pointing it out because in the panic
case you mentioned a clean shutdown may not be possible.
> So, what remains after system shutdown boils down to:
>
> * kernel memory allocated and then freed, during the lifetime of the
> system, through means that are not covered by PAX_MEMORY_SANITIZE:
> is there any such thing? I'm interested in a (possibly incomplete)
> list of these memory areas or allocation methods.
while the buddy allocator acts as the lowest layer of memory manager
for the kernel (so it sees everything as far as C code is concerned),
there's also boot time memory allocated that is kept around and i'm
pretty sure that some of it is never freed. there can also be
arch-specific memory reservations that may need special handling
(hugetlbfs, CMA, etc).
> * kernel memory, that was still in use during shutdown, and that the
> kernel does not explicitly free: again, is there any such thing?
as hinted at above, there's lots of memory that is not process memory
and that will be kept around as long as something uses that memory (or
even when not). shutdown in linux isn't about undoing everything that
happened before, it's simply just another function to execute that
happens to never return. this leaves some memory in-use. just so you
can get an idea, look at /proc/slabinfo from your shutdown script and
count all the still allocated objects... now not all of them have
sensitive information that you care about, but some definitely do.
with all that said, let me comment on your current approach:
> Current Tails implementation
> ============================
>
> Our current protection against this scenario is: when I unplug my
> Tails USB stick, an emergency shutdown procedure is triggered, that
> kexec's another kernel with some special command line parameter, and
> then the initramfs erases all the memory that's available from
> userspace.
i'd like to interject here and point out a less than obvious sideeffect
of SANITIZE (not really documented because the effect isn't relevant
for the original purpose of SANITIZE): when the kernel transitions
from the bootmem allocator to the buddy allocator during early boot,
it does so by simulating a free of all that memory back to the buddy
allocator. this in turn activates the erase-on-free logic of SANITIZE
pretty much for free (pun intended, sorry).
in other words, if you were to kexec into a SANITIZE enabled kernel,
you'd get your memory clearing for free automatically, earlier than
any initramfs would execute even and it'd cover most kernel memory
that the kernel ever cares about (or cared in its previous incarnation
at least).
now this brings us to the other topic you raised about grsecurity's
KMEM hardening. technically it's not incompatible with kexec, so you
can re-enable kexec, however note that until some signed kexec mechanism
enters the kernel, it carries a risk of executing potentially malicious
kernels (but maybe that's not a problem in your use cases). perhaps
embedding or loading the kexec kernel from initramfs would get around
those concerns for good.
that's it for now, let us know if you have questions.
cheers,
PaX Team
