Re: [Tails-project] Live Demo proposal: Blackbox automated t…

Delete this message

Reply to this message
Autor: intrigeri
Data:  
Dla: Public mailing list about the Tails project
Temat: Re: [Tails-project] Live Demo proposal: Blackbox automated testing of operating systems: how Tails does it
Hi,

thanks for all the kind words :)

bertagaz wrote (08 Aug 2015 19:08:01 GMT) :
> the dpkg trigger bug could have been found far more sooner with such
> a tool,


In passing, piuparts is way better suited for this task, and
https://piuparts.debian.org/ is amazing.

Still in passing: in my experience, such bugs, that happen only
occasionally / in specific contexts, are hard to identify with an
automated test suite, for a number of reasons including the fact that
it's always tempting to call it a false positive, transient error, you
name it; and the fact that test suites run in controlled environment
(which is usually seen as a good thing) tend to not expose crazy race
conditions that only happen under load, or with a specific combination
of packages, etc. The fact that we currently run our own test suite on
various systems, some of them being "production" environments as well,
helps us on the latter. The fact that it's been historically brittle
and prone to transient errors hasn't been helping us on the former...
but thankfully, this has changed a lot recently :)

> as well as having functional graphical upgrade in Debian...


ACK!

Cheers,
--
intrigeri