Re: [Tails-dev] Please review'n'merge test/5959-antitest-mem…

28/12/13 23:50, intrigeri wrote:
> Hi,
>
> test/5959-antitest-memory-erasure (#5959) tests that, without erasing
> memory, we can retrieve the known pattern. I had to do a bit of
> refactoring and improvements of the erase_memory feature along
> the way.
>
> Please review'n'merge into devel once test/rjb-migration is merged.

Code looks great, but during testing I had problems:

> +  def reset
> +    # ruby-libvirt 0.4 does not support the reset method.
> +    # XXX: Once we use Jessie, use @domain.reset instead.
> +    system("virsh reset " + @domain_name) if is_running?
> +  end

For whatever reason, the TailsToaster (i.e. `@domain_name`) domain isn't
present for me (also checked with `virsh list`), so I get the following
error in both anti-tests:

>     When I fill the guest's memory with a known pattern without verifying # features/step_definitions/erase_memory.rb:78                                                                      
> error: failed to get domain 'TailsToaster'
> error: An error occurred, but the cause is unknown

>

>     And I reboot without wiping the memory                                # features/step_definitions/erase_memory.rb:159                                                                     
>       FindFailed: can not find TailsBootSplashPostReset.png on the screen.
>       Line ?, in File ? (RuntimeError)

Indeed, when observing the session I can verify that no reset took
place. If I switch back to sysrq-resetting (revert commit 443d2d9) it
works fine.

Then I have two probably unrelated problems:

1. I consistently get:

>     And at least 8 GiB of RAM was detected                                # features/step_definitions/erase_memory.rb:46
>       Detected 8393662464 bytes of RAM
>       Didn't detect enough RAM (RuntimeError)
>       ./features/step_definitions/erase_memory.rb:53:in `/^at least (\d+) ([[:alpha:]]+) of RAM was detected$/'                                                                               
>       features/erase_memory.feature:16:in `And at least 8 GiB of RAM was detected'

We currently allow a 128 MiB "gap", but now the gap is 187 MiB. I
suppose the switch from a i686-pae to a amd64 kernel is the cause. Can
you reproduce this?

2. I consistently get:

>   Scenario: Memory erasure on an old computer                     # features/erase_memory.feature:55                                                                                          

[...]

>     Then I find very few patterns in the guest's memory           # features/step_definitions/erase_memory.rb:141
>       Pattern coverage: 0.164% (6097552 bytes)
>       0.164% of the memory is filled with the pattern, but less than 0.100% was expected (RuntimeError)
>       ./features/step_definitions/erase_memory.rb:145:in `/^I find very few patterns in the guest's memory$/'
>       features/erase_memory.feature:69:in `Then I find very few patterns in the guest's memory'

The exact pattern coverage of course varies but it's always been
slightly above 0.1%. We currently only allow 0.001 = 0.1% coverage after
memory erasure, so we may want to bump it to 0.5%, which still I guess
is pretty ok (I guess? It feels like we're completely arbitrary here :)).

So, just to be clear, with the changes below I'm able to finish the
erase_memory feature:

diff --git a/features/step_definitions/erase_memory.rb
b/features/step_definitions/erase_memory.rb
index d23e8ac..33ee555 100644
--- a/features/step_definitions/erase_memory.rb
+++ b/features/step_definitions/erase_memory.rb
@@ -49,7 +49,7 @@ Given /^at least (\d+) ([[:alpha:]]+) of RAM was
detected$/ do |min_ram, unit|
puts "Detected #{@detected_ram_b} bytes of RAM"
min_ram_b = convert_to_bytes(min_ram.to_i, unit)
# All RAM will not be reported by `free`, so we allow a 128 MB gap
- gap = convert_to_bytes(128, "MiB")
+ gap = convert_to_bytes(196, "MiB")
assert(@detected_ram_b + gap >= min_ram_b, "Didn't detect enough RAM")
end

@@ -141,7 +141,7 @@ end
 Then /^I find very few patterns in the guest's memory$/ do
   next if @skip_steps_while_restoring_background
   coverage = pattern_coverage_in_guest_ram()
-  max_coverage = 0.001
+  max_coverage = 0.005
   assert(coverage < max_coverage,
          "#{"%.3f" % (coverage*100)}% of the memory is filled with the " +
          "pattern, but less than #{"%.3f" % (max_coverage*100)}% was
expected")
@@ -158,7 +158,9 @@ end

When /^I reboot without wiping the memory$/ do
next if @skip_steps_while_restoring_background
- @vm.reset
+ @vm.execute('echo 1 > /proc/sys/kernel/sysrq')
+ @vm.spawn('echo b > /proc/sysrq-trigger')
+
@screen.wait('TailsBootSplashPostReset.png', 30)
end


> Then, only #6483 will remain a blocker to drop this part of our manual
> test suite, eventually.

Excellent! Since we now use an amd64 kernel we can close that ticket
once 0.23 is out.

Cheers!