Re: [Tails-dev] Automated tests specification

Hi,

On Tue, Sep 01, 2015 at 06:57:05PM +0200, anonym wrote:
> On 09/01/2015 12:04 PM, intrigeri wrote:
> > bertagaz wrote (28 Aug 2015 14:24:51 GMT) :
> >> I've also added a new section about the result to keep:
> >
> >> ## What kind of result shall be kept
> >
> >> The test suite produces different kind of artifacts: logfiles, screen
> >> captures for failing steps, snapshots of the test VM, and also videos of
> >> the running test session.
> >
> >> Videos may be a bit too much to keep, given they slow down the test
> >> suite
> >
> > Do we have data about how much they slow down the test suite on our
> > current Jenkins slaves? See #10001 for partial data about how we could
> > make it less resources-hungry...
>
> Speculation: Didn't we plan for an extra core for this? I suppose we'd
> need fours cores, for: video capture, sikuli, libvirt (USB emulation in
> particular), cucumber-and-other-stuff. Hmm, perhaps
> cucumber-and-other-stuff are happy with the left over from the other
> cores? So three cores?

We planned for 3 cores per isotester and to be honest I don't think we
have room to add more. We're currently using 43 of the 48 cores, so
reserving 4 more of them won't leave much for the host.

> > And by the way, perhaps having videos optionally split per-scenario
> > would help a lot making good use of such videos: faster download,
> > easier to jump to the failing part, less storage needs on our infra.
> > anonym and Kill Your TV, if you agree it would be useful and not
> > stupid an idea, I can file a non-blocking ticket about it.
>
> Yes, per-scenario videos would be great (my plan was to do this when we
> have #8947, but whatever, nothing prevents us from having it now). They
> would be more useful than per-feature videos, and actually easier to
> implement AFAICT. Please file a ticket!

+1.

bert.