Hi,
More success metrics! This time to evaluate the impact of distributing
USB images :)
For the record, starting with Tails 3.12 (January 29), instead of
downloading an ISO image (a format originally designed for CDs), people
now download Tails as a USB image: an image of the data as it needs to
be written to the USB stick. This makes the installation process is
simpler and faster for all operating systems.
See
https://tails.boum.org/news/version_3.12/
Method
======
I used logs from our website, specifically:
- Hits on the download buttons (both direct and BitTorrent) that we have
on our download page.
See "hitCounter" in
https://tails.boum.org/install/inc/js/download.js.
- Hits on the "back" URLs that people use to open our installation
instructions before starting on Tails.
For example:
https://tails.boum.org/install/win/usb?back=1.
I compared values:
- Before 3.12: Dec 26 → Jan 23, 28 days
- After 3.12: Feb 19 → Mar 19, 28 days
Both time spans didn't include a release but the first set was further
away from its previous release (15 days vs 6 days). I think this
influenced the total figures too much (eg. total number of downloads)
and that's why I'm not comparing them. I'm only comparing the percentage
of users per OS and I guess that these were not really influenced by the
distance of the logs to their previous release.
As always I'm assuming that Windows and macOS users are less tech-savvy
than Linux users and treat more Windows and macOS users are a good sign.
Not that I like Windows or macOS more than Linux :)))
I'm attaching a spreadsheet with all the results and here is a summary.
Results
=======
The percentage of downloads from Windows and macOS increased by 1.2% and
3.4% respectively. We know have:
- 73% of downloads from Windows (+1.2%)
- 7% of downloads from macOS (+3.4%)
- 20% of downloads from Linux (-4.5%)
The percentage of "back" URLs from macOS increased by 7.5%:
- 77% of "back" from Windows (-2.0%)
- 13% of "back" from macOS (+7.5%)
- 10% of "back" from Linux (-5.5%)
I'm not worried to see a decrease in "back" from Windows. The new
installation process is shorter for Windows users but not significantly
easier. On the other hand, the new installation process for macOS became
a lot easier (from command line to Etcher). The important result for
Windows is that the percentage of "back" from Linux decreased more than
the percentage of "back" from Windows.
Looking at conversion rates ("back" / downloads) seems to provide a
better summary of the UX improvements per OS:
- Windows -0.1%, installation is as easy until "back"
- macOS +4.6%, installation is much easier until "back"
- Linux -1.2%, installation is bit harder until "back"
I'm surprised to see a decrease in the conversion rate for Linux users
because it actually became easier for most of them (no need to configure
backports in Debian anymore or go through the command line in Ubuntu).
Similarly as for the analysis of the VeraCrypt success metrics [1], it
might be a bit early to see really how the introduction of USB image
will impact our user base on the long run. Unlike for VeraCrypt, we are
talking here about downloads, which might include a good amount of
people who want to install Tails for the first time (especially the hits
on "back") but the general results might continue to evolve in the right
direction as more time passes and a bigger part of our target audience
change their perception of Tails. For example, the fact that Tails was
extremely hard to install on macOS might stick around for some more time
until more Mac users interested in Tails learn that it is now much
easier to install.
I'll update these stats again in the future to see how things evolve.
[1]:
http://lists.autistici.org/message/20190326.173900.138d9b45.en.html
Comparing the percentage of downloads with the percentage of users (as
per the self-reported data from the VeraCrypt survey) also could also
reflect how hard it is for Windows and macOS users to actually start
Tails ("back" is right before trying to restart on Tails) and adopt
Tails for real:
- Windows are 73% of download but only 45% of Tails users
- macOS are 7% of downloads but 8% of Tails users
This might be related to the fact that it is still much harder to start
Tails on PC than on Mac.
To finish with, we're now serving 76% of USB images and our download
buttons are hit 3970 times a day on average. This doesn't include people
following the 'expert' scenario and downloading from the command line
(otherwise 4.8% of hits on "back" URLs).
--
sajolida
