Re: [Tails-dev] [review'n'merge:1.1] #6559, #7062, #6275: te…

Delete this message

Reply to this message
Author: anonym
Date:  
To: The Tails public development discussion list
Subject: Re: [Tails-dev] [review'n'merge:1.1] #6559, #7062, #6275: test/6559-adapt-test-suite-for-Wheezy
13/05/14 22:12, intrigeri wrote:
> anonym wrote (13 May 2014 15:53:17 GMT) :
>> My suggestion would be that we don't use the emergency shitdown, and
>> simply sends a `halt`, which we did before, and which was much more
>> reliable. I know you want us to the same ways we except our users to
>> use, and I agree, but well... can't we just make a dedicated test for
>> the emergency shutdown instead?
>
> Yes, it would be good to have the USB feature a bit more robust, while
> not dropping a useful test => fine with me, please go ahead.


See commit fc95510.

>>> 2. Scenario: Booting Tails from a USB drive upgraded from DVD with persistence enabled # features/usb_install.feature:182
>>>      [...]
>>>      And the boot device has safe access rights                                        # features/step_definitions/usb.rb:326
>>>      And the expected persistent files are present in the filesystem                   # features/step_definitions/usb.rb:423
>>>        Could not find expected file in persistent directory /etc/NetworkManager/system-connections (RuntimeError)

>>>
>>>    Same in "Booting Tails from a USB drive upgraded from USB with
>>>    persistence enabled" and "Booting a USB drive upgraded from ISO
>>>    with persistence enabled".

>>>
>>>    And on next run, I cannot reproduce this. Weird.

>
>> It's notable that that particular persistence preset's directory was
>> changed in feature/wheezy. What was your --old-iso when you ran the
>> first test vs the second?


Perhaps it's not so notable since that's the first preset that will be
tested, so probably all of them were not there.

> A build from the devel branch, from a few days earlier.
>
>> Could it have been a Wheezy-based image from
>> before the persistence preset was changed?
>
> IIRC, this change was made a while ago, and I don't think I have any
> such ISO anymore.


Sorry, then I have no idea.

>>> 4. Scenario: Memory erasure on an old computer                     # features/erase_memory.fe
>>>    [...]
>>>      And I shutdown and wait for Tails to finish wiping the memory # features/step_definitions/erase_memory.rb:164
>>>      Then I find very few patterns in the guest's memory           # features/step_definitions/erase_memory.rb:140
>>>        Pattern coverage: 0.314% (11 MiB)
>>>        0.314% of the memory is filled with the pattern, but less than 0.250% was expected (RuntimeError)

>>>
>>>    I got this once out of two tries. Is it an acceptable drawback of
>>>    how the test suite works, or a real problem?

>
>> To me it just shows that with the particular kernel (or whatever) that
>> we happen to use now require us to bump the highly arbitrary 0.25% to
>> 0.5%, perhaps. Without some more rigorous guideline to what we think is
>> acceptable, arbitrary is what we've got. What do you think?
>
> Fair enough.


Done in commit bf1795b.

> Maybe we want a low-priority Research ticket to look at
> this later, at least to document that we have an issue here?


Filed as #7313.

Cheers!