Re: [Tails-dev] [review'n'merge:1.1] #6559, #7062, #6275: te…

Delete this message

Reply to this message
Author: intrigeri
Date:  
To: The Tails public development discussion list
Subject: Re: [Tails-dev] [review'n'merge:1.1] #6559, #7062, #6275: test/6559-adapt-test-suite-for-Wheezy
anonym wrote (13 May 2014 15:53:17 GMT) :
> My suggestion would be that we don't use the emergency shitdown, and
> simply sends a `halt`, which we did before, and which was much more
> reliable. I know you want us to the same ways we except our users to
> use, and I agree, but well... can't we just make a dedicated test for
> the emergency shutdown instead?


Yes, it would be good to have the USB feature a bit more robust, while
not dropping a useful test => fine with me, please go ahead.

>> 2. Scenario: Booting Tails from a USB drive upgraded from DVD with persistence enabled # features/usb_install.feature:182
>>      [...]
>>      And the boot device has safe access rights                                        # features/step_definitions/usb.rb:326
>>      And the expected persistent files are present in the filesystem                   # features/step_definitions/usb.rb:423
>>        Could not find expected file in persistent directory /etc/NetworkManager/system-connections (RuntimeError)

>>
>>    Same in "Booting Tails from a USB drive upgraded from USB with
>>    persistence enabled" and "Booting a USB drive upgraded from ISO
>>    with persistence enabled".

>>
>>    And on next run, I cannot reproduce this. Weird.


> It's notable that that particular persistence preset's directory was
> changed in feature/wheezy. What was your --old-iso when you ran the
> first test vs the second?


A build from the devel branch, from a few days earlier.

> Could it have been a Wheezy-based image from
> before the persistence preset was changed?


IIRC, this change was made a while ago, and I don't think I have any
such ISO anymore.

>> 3. Scenario: Iceweasel should not have any plugins enabled                # features/torified_browsing.feature:26
> [...]
> This is fixed now.


Congrats! \o/

>> 4. Scenario: Memory erasure on an old computer                     # features/erase_memory.fe
>>    [...]
>>      And I shutdown and wait for Tails to finish wiping the memory # features/step_definitions/erase_memory.rb:164
>>      Then I find very few patterns in the guest's memory           # features/step_definitions/erase_memory.rb:140
>>        Pattern coverage: 0.314% (11 MiB)
>>        0.314% of the memory is filled with the pattern, but less than 0.250% was expected (RuntimeError)

>>
>>    I got this once out of two tries. Is it an acceptable drawback of
>>    how the test suite works, or a real problem?


> To me it just shows that with the particular kernel (or whatever) that
> we happen to use now require us to bump the highly arbitrary 0.25% to
> 0.5%, perhaps. Without some more rigorous guideline to what we think is
> acceptable, arbitrary is what we've got. What do you think?


Fair enough. Maybe we want a low-priority Research ticket to look at
this later, at least to document that we have an issue here?

Cheers,
--
intrigeri
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc