Re: [Tails-dev] How to deal with tests expected to fail? [Wa…

Nachricht löschen

Nachricht beantworten
Autor: intrigeri
Datum:  
To: The Tails public development discussion list
Betreff: Re: [Tails-dev] How to deal with tests expected to fail? [Was: #6559, #7062, #6275: test/6559-adapt-test-suite-for-Wheezy]
anonym wrote (13 May 2014 15:52:53 GMT) :
> Ok, I get the problem you're describing and agree that the current
> behaviour of @skip is bad.


I'm glad we agree on this.

> For the record, I didn't implement the @skip mechanism for speed. To me
> it just doesn't seem right to have tests that are expected to fail, and
> let them fail all the time. In particular it would make pointless any
> sort of automated run of these tests on a future server of ours that
> email us whenever we commit something that breaks a test.


Full ACK.

> It's true that this probably is some distance into the future.


IIRC, one of us has a substantial subset on this on their plate, to be
done between July and September. Granted, the tests won't run
automatically yet, but once we've got the platform ready to run it by
hand, having it run automatically is not that far.

>> I realize that the current implementation (--tags
>> ~@skip) is trivial, and that implementing the logic I'm suggesting
>> above would likely be more involved (right?).


> Indeed. I did a quick investigation.


[... snipping very useful investigation, congrats again! ...]

> I think we can just drop the whole @skip/@todo thing completely until we
> have a mechanism that works like you describe above. This probably means
> getting a new feature into upstream cucumber.


Wow, I'm surprised this is not provided out-of-the-box, and I'm
curious how others do.

>> Maybe CUCUMBER_OPTS should not loseIn
>> a potential existing value of itself, when initialized?


> Sure. However, since I guess the code that introduced the variable will
> be dropped, is that something you'd wished to be added any way?


Not really.

Cheers!
--
intrigeri
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc