Re: [Tails-dev] Progress report on the automated test suite

Üzenet törlése

Válasz az üzenetre
Szerző: intrigeri
Dátum:  
Címzett: The Tails public development discussion list
Tárgy: Re: [Tails-dev] Progress report on the automated test suite
hi,

anonym wrote (13 Nov 2012 13:01:41 GMT) :
> 03/11/12 09:55, intrigeri wrote:
>>> that starts on the guest when the boot parameter
>>> "autotest_never_use_me" is present on the kernel cmdline.
>>
>> "autotest_never_use_me" looks to me like "(speaking to) autotest:
>> never use me". What about "backdoor_for_autotest"?


> I'm not sure I want to mention the word "backdoor". Sure, I do it in the
> remote shell server script, but then it's mentioned in a context where
> sane people should have no reason to be worried.


Why should the boot parameter name be mentioned in contexts where this
does not apply? (Not a rhetorical question.)

>>> Saving/restoring VM snapshots
>>> =============================
>>> [...]
>>
>> For both features, to reply on the 'how "ethical" they are in the
>> context of test-driven development' topic, I'd need a concrete example
>> of how this would be used in practice.


> I'm sure there are specific/concrete situations where this is not a good
> idea. I was more interested if you saw any fundamental flaws with this
> approach since it's a step away from black box testing (same applies to
> the remote shell).


Sorry, I'm not in a mood to think about fundamental flaws without
examples. Anyhow, I'll try to contribute a bit, hoping that helps.

So, I think that:

1. There are serious shortcomings that come with these features.
Every time one cheats and uses them, one should know what they are
actually *not* testing, and think if/how that could be tested.
2. Trying at all costs to totally avoid to use these features is
probably not a good use of our time.
3. Generally, I like to exercise systems under test at different
levels, and the remote shell feature is probably the most efficient
way to zoom-in and run something like "unit tests". This may not
totally replace more zoomed-out, behavioral testing, though.

That's all I feel I can answer, on a general level.

I'd rather be pointed to cases when it looks much easier or even
needed to resort to such hacks, and then, discuss whether that is
acceptable on a case by case basis. But that may happen later, once
actual reasons to use these features arise.

Cheers,
--
intrigeri
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc