Re: [Tails-dev] Progress report on the automated test suite

03/11/12 09:55, intrigeri wrote:
>> Next I'd like to announce that the automated test suite, in its
>> current unfinished state, actually has found its very first Tails
>> bug. [...] In other words, our firewall leaks link-local IPv6
>> broadcasts even though it should block everything IPv6 (right?).
>
> Ouch.
>
> WAN hat on: please report it (ticket + email) separately so that it
> does not get lost in the middle of this "report on the automated test
> suite" thread.

For the ticket, see todo/really_block_ipv6.

>> I'd like to present the last two with a bit more depth and hear your
>> opinions, especially w.r.t. the fact that they alter Tails or "cheat" in
>> the testing process, so I wonder how "ethical" they are in the context
>> of test-driven development.
>
>> Running arbitrary commands inside the guest VM
>> ==============================================
>
>> This is very valuable as it makes many tests that would be truly
>> awkward to do with sikuli into something trivial. libvirt doesn't
>> seem to have something like VirtualBox' `vboxmanage guestcontrol
>> execute` (provided by the VirtualBox guest additions), so
>> I implemented a simple remote shell (read: a backdoor (listening on
>> port 1337 + firewall exception) so expect havoc on the Tails forum!)
>> that starts on the guest when the boot parameter
>> "autotest_never_use_me" is present on the kernel cmdline.
>
> "autotest_never_use_me" looks to me like "(speaking to) autotest:
> never use me". What about "backdoor_for_autotest"?

I'm not sure I want to mention the word "backdoor". Sure, I do it in the
remote shell server script, but then it's mentioned in a context where
sane people should have no reason to be worried. Makes sense? Other
naming suggestions?

>> Saving/restoring VM snapshots
>> =============================
>> [...]
>
> For both features, to reply on the 'how "ethical" they are in the
> context of test-driven development' topic, I'd need a concrete example
> of how this would be used in practice.

I'm sure there are specific/concrete situations where this is not a good
idea. I was more interested if you saw any fundamental flaws with this
approach since it's a step away from black box testing (same applies to
the remote shell).

Cheers!