Re: [Tails-dev] Progress report on the automated test suite

Delete this message

Reply to this message
Author: anonym
Date:  
To: The Tails public development discussion list
Subject: Re: [Tails-dev] Progress report on the automated test suite
03/11/12 17:18, Ague Mill wrote:
> anonym:
>> Next I'd like to announce that the automated test suite, in its current
>> unfinished state, actually has found its very first Tails bug. Here's
>> the cucumber output of when it was found:
>>
>>     [...]
>>     And all Internet traffic has only flowed through Tor
>>       # cucumber/iceweasel/step_definitions/torified_browsing.rb:66

>>
>>       The following IPv6 hosts were contacted:
>>       ff02::1
>>       Full network capture available at: [...censored...]
>>     There were network leaks! (RuntimeError) [...]

>>
>> In other words, our firewall leaks link-local IPv6 broadcasts even
>> though it should block everything IPv6 (right?). This is promising (not
>> that Tails has this particular bug, but that the test suite found it)!
>
> I did not run the code itself, but are you sure that these packets came
> from Tails and not from the host system?


Unless there's a bug in tcpdump, yes, I'm sure. tcpdump is configured
with the filter "src host ${IP} or src host ${IP6}", where IP* = the
guest running Tails.

>> Saving/restoring VM snapshots
>> =============================
>>
>> This is how I intend to use it for a given feature:
>>
>>   Background:
>>     Given I restore the background snapshot if it exists
>>     [ ... "real" background steps ... ]
>>     And I save the background snapshot if it does not exist

>>
>> [ ... Scenarios ... ]
>
> Those lines feel like noise: they are an implementation detail and
> should not appear in the scenarios.
>
> Cucumber offer tags and hooks that should be usable to achieve something
> similar while keeping the scenarios as lean as possible. See:
> <https://github.com/cucumber/cucumber/wiki/Hooks> and
> <http://stackoverflow.com/questions/9994797/cucumber-when-to-use-tags-hooks-vs-backgrounds>


I looked at them but couldn't find anything that I thought suited. I
need one hook that runs exactly before the background, and one that runs
exactly after.

>> An issue with restoring past state like this is that our Tor's circuit
>> state may get out-of-sync with the circuit state of the relays they use.
>> For instance, I ran 10 tests that restored to the same post-background
>> state and all but the first two failed to fetch a web page. Then I ran
>> 10 tests where I do the following after each snapshot restore:
>>
>> 1. Stop Tor.
>> 2. Sync time from host to guest.
>> 3. Start Tor.
>>
>> And then all 10 tests succeeded, so it seems resetting Tor like this is
>> highly necessary.
>
> Indeed, as restoring from a snapshot is likely to break all existing TCP
> connections. Have you tried to see if a SIGHUP sent to Tor is sufficient?


>From previous experiences (e.g. "tordate") Tor doesn't behave well if it

experience a time jump, even if given a HUP, so in order to avoid future
mysterious failed tests due to Tor being confused, I thought restarting
it was for the best.

> Side note: your `try_for` function is very unidiomatic Ruby.
> I suggest you have a look at the part about blocks on
> <http://www.ruby-doc.org/docs/ProgrammingRuby/html/tut_containers.html>,
> and the `yield` and `block_given?` methods.


Thanks, I'll look into it (again)! I actually used `yield` at first, but
didn't get it to work, whereas anonymous functions worked fine. I
suspect those issues might have been due to an unrelated problem (the
stock 'timeout' module in Ruby, which I used at first, can lockup if
syscall get involved, apparently).

Cheers!