Hi,
Some months have passed since we added support for Additional Software
and VeraCrypt in Tails in 3.9 on September 5.
I computed some metrics to see the impact of our work on our users.
User base
=========
See boots.png in attachment.
Since July 2017 (Tails 3.0), the number of boots per day had stagnated.
I attributed this partly to the removal of hardware support for 32-bit
computers in 3.0 which I estimated at ~5% of users (#8183).
Since September 2019 the graph is going up again and we have 20% more
boots overall. Yay!
I'm guessing that this can be explained by a combination of:
- Slow recovery from dropping support for 32-bit computers in 3.0. Old
computers eventually die and people have newer ones. If we wanted to
be optimistic we could say that the graph was already slowly
recovering starting in 2018.
- No releases in July and August 2019. There's a tendency to get an
increase in boots when we do releases and a decrease in boots when we
don't. People upgrade and that makes more boot.
Unfortunately, we don't have metrics on the number of unique Tails
installations. But might have plans for doing this in a privacy-
preserving way (#16535).
- The great work we did on Additional Software and VeraCrypt :)
Additional Software
===================
I did some stats on the usage of Additional Software on the WhisperBack
reports that we received before and after the release.
- The numbers are pretty small so doing stats on them is very fragile.
The Additional Software usage changes a lot from one month to another.
- I don't think that the people reporting WhisperBack bugs are
representative of our overall user base. You need to speak English
(mostly), be confident, have time & energy for reporting bugs, etc.
- The same people send many reports. Like this 1 person sending 7
reports in September (and many more since then!).
| | With AS | Total | % |
|----------------+---------+-------+----|
| July | 3 | 51 | 6 |
| August | 3 | 55 | 5 |
| September | 29 | 104 | 28 |
| October | 13 | 105 | 12 |
| November | 16 | 57 | 28 |
| December | 15 | 67 | 22 |
| January | 14 | 70 | 20 |
| February | 10 | 62 | 16 |
| March | 5 | 56 | 9 |
|----------------+---------+-------+----|
| Average before | | | 6 |
| Average after | | | 20 |
NB: Before 3.9, you could configure Additional Software but you had to
do it on the command line.
In terms of the packages that WhisperBack reporters added the most
popular ones were:
- vlc
- florence (seems like the same person reporting a lot)
- vim
- youtube-dl
- emacs
A lot of packages seemed to come from very geeky users (text editors,
network tools, programming tools, sysadmin utilities, etc.) but not only.
I also saw:
- games (pingus)
- document editors (dia)
- ebook readers (calibre)
- video tools (recordmydesktop)
- messaging app (telegram-desktop)
- security tools (diceware)
- and most importantly, someone else than me using
gnome-shell-pomodoro in Tails! :)
But also scary stuff like transmission-gtk or openvpn.
The usage of Additional Software more than tripled with the introduction
of the graphical interface. Yay!
It didn't sound much to me but we should keep in mind that one of the
big objective of Additional Software is be to allow us to remove things
from Tails more easily (#15291) or offer the user to add packages to
better support their language (#15807#note-27). And this flexibility
will benefit everybody using Tails since the initial download will get
smaller.
Also, as part of this work we couldn't replace Synaptic with GNOME
Software. Synaptic worked all-right during our usability tests with
people who've never seen it before but GNOME Software is definitely
better at presenting packages and help people find out what they need.
VeraCrypt integration
=====================
In November 2017, before doing the work on VeraCrypt, we conducted a
survey to gather a baseline of the usage of VeraCrypt amongst Tails
users. See
https://tails.boum.org/blueprint/veracrypt/#survey
This month I conducted another survey to compare our current user base
with this baseline. I only copied 3 questions from the initial survey:
Q1: Which operating system other than Tails do you use the most?
Q2: How familiar are you with GNOME Disks?
Q3: Do you use VeraCrypt?
See survey-export.ods.
Overall tech-savviness
----------------------
I used Q1 and Q2 as a very rough indicator of the tech-savviness of our
user base and adoption (relevance × ease of use) by less tech-savvy people.
The big picture hasn't changed much since 2017:
- We have 1.5% more of macOS users. Yay!
Note that the change to USB images also happened during the same
period so it might explain this more than Additional Software (why
would macOS users like Additional Software more than others?) or
VeraCrypt (macOS users are not really into VeraCrypt, see below).
- The percentage of Windows users hasn't changed much.
This disappointed me a bit :(
- The portion of Debian and Ubuntu users hasn't changed much.
The percentage of "Other" decreased as much as the portion of macOS
increased. "Other" include other Linux (mostly), some people answering
nothing or "Tails", and a couple of BSD and mobile OS.
- The percentage of users who don't know what is GNOME Disks has
increased a tiny bit. That's a good sign :)
VeraCrypt usage
---------------
The people who use VeraCrypt in Tails almost doubled (16% to 27%).
It's good but I expected more...
- Half of this number comes from the percentage of people who answered
"only outside of Tails" in 2017 which went down from 24% to 19%. These
are people who we expected to be blocked from using VeraCrypt in Tails
because it was not integrated. This is where I would have expected
better results.
- The other half comes from the percentage of non-VeraCrypt users in
2017. The decrease in this number could related to people for whom
Tails became more relevant because they could use VeraCrypt in it.
Having a closer look of VeraCrypt usage by operating system:
- The biggest increase in VeraCrypt adoption in Tails comes from Debian
and Ubuntu users (6% to 11% of total users). This increase is
compensated by a decrease of non-VeraCrypt users on Debian and Ubuntu.
These could be the people for whom Tails became more relevant because
they could use VeraCrypt in it.
- The second biggest increases in VeraCrypt adoption in Tails come from
Windows users (from 7% to 12% of total users). Windows users are still
the biggest VeraCrypt users (21% of total users). This increase is
compensated by a decrease of people who were only using VeraCrypt
outside of Tails in 2017. These could be the people who were blocked
from using VeraCrypt in Tails because it was not integrated.
- macOS users still barely don't use VeraCrypt (3% of total users).
Summary and reflection
======================
The results are good, Tails is seeing a steady increase of its user base
again, Additional Software is being used and will be very useful for
further developments, and VeraCrypt probably helped getting more users.
But I expected a bigger increase in VeraCrypt usage and a bigger change
in the overall tech-savviness of our user base.
This lead me to reflect on how we promote changes and new features to
our users. Currently, as a Tails user, I only have 3 ways of learning
about Additional Software, VeraCrypt, or any new feature in Tails:
- I read every release notes (or Twitter on the day of the release).
I suspect that an important part of our user base either don't take
the time to read them or uses Tails only occasionally (or frequently
during certain periods of time) and won't see them at the right time.
- I search explicitly for VeraCrypt in Tails or something about
Additional Software, either online or trying things out in Tails.
- I read the full index of our documentation or somehow stumbled upon
our doc pages while reading our website.
- Word-of-mouth.
I came to believe that an important part of our old user base has a hard
time learning about the new things in Tails. When interviewing Tails
users or talking to security trainers, I often realize that they are not
up-to-date with the changes in Tails even many months afterwards.
I think we could get more impact from all the great work that we do in
Tails by communicating more about it and letting infrequent or older
users know what's new.
The first ideas that came to my mind to improve on this are:
- A "Learn what's new" when doing an upgrade. We would know which
version the user is upgrading from and could customize the message.
- A "Did you know" message on /home about the changes we did in the past
year or so.
--
sajolida
