Re: [Tails-dev] Automated tests specification

On 09/01/2015 06:57 PM, anonym wrote:
> On 09/01/2015 12:04 PM, intrigeri wrote:
>> bertagaz wrote (28 Aug 2015 14:24:51 GMT) :
>>> and might take quite a bit of disk space to store.
>>
>> ... and smaller (#10001). I'm curious how much space a full set of
>> test suite videos take.
>
> I'll try to remember to collect this info in my next full run.

I got a video that was 309 MiB large with test/6094-improved-snapshots
checked out on an isotester. I don't think the errors I got affected the
runtime (or amount of changes on the screen, which matters more w.r.t.
video compression). Any way, I believe the per-(failed-)scenario
approach is what we really want; videos of the passed scenarios is just
distracting and wasteful.

Failing Scenarios:
cucumber features/time_syncing_bridges.feature:29 # Scenario: Clock way
in the past in bridge mode
cucumber features/time_syncing_bridges.feature:45 # Scenario: Clock way
in the future in bridge mode
cucumber features/tor_stream_isolation.feature:48 # Scenario: Explicitly
torify-wrapped applications are using the default SocksPort
cucumber features/torified_git.feature:22 # Scenario: Cloning git
repository over SSH

186 scenarios (4 failed, 182 passed)
1345 steps (4 failed, 5 skipped, 1336 passed)
348m33.765s

It's interesting to note that this runtime is not different from what I
normally get when running this branch on the isotesters (I said "~350
minutes" earlier in this thread), which makes me wonder if the main
concern of #10001, that --capture increases the runtime (even when
enough CPU is available), is actually valid. I'm gonna investigate that
now...

Cheers!