Hello,
here is some feedback we gathered at the first Tails UX session with
NUMA experiments.
First of all, it was really a nice experience. The method seems very
interesting. Some of the feedback which follows has been translated into
tickets on Redmine. Other points seem to collide a little bit with
Tails' policy, for example on *not* recommending any third party e-mail
service.
I invite you all to discuss the tickets rather then the whole thread.
Also, if there is a point not addressed by the tickets, please start a
new thread on the mailing list to talk about it otherwise we might get
lost, as this whole thing is quite long.
The next steps:
- a second usability/user observation workshop
- a workshop to brainstorm on solutions
- starting to implement solutions
Right now, we (tchou and NUMA mostly) are trying to fix some new dates
for a new usability testing session.
There will probably be a second observation workshop at Pas Sage En
Seine.
At the HackFest we shall redo such a conception workshop, or even start
to implement some of the solutions.
Cheers.
u.
==
If you're only interested in the tickets, here they are:
Tails:
-
https://labs.riseup.net/code/issues/5417
Improve user experience when wiping memory at shutdown
-
https://labs.riseup.net/code/issues/7440
Implement a progress indicator while establishing a connection to Tor
-
https://labs.riseup.net/code/issues/7436
Contact Riseup on how we could help towards publishing a list of
recommended email providers
-
https://labs.riseup.net/code/issues/7433
Have Tails OpenPGP Applet in the menu
-
https://labs.riseup.net/code/issues/7449
UX: Add Seahorse to application menu
-
https://labs.riseup.net/code/issues/7448
UX: Check whether we might be on a captive portal if Tor fails to connect
MAT:
-
https://labs.riseup.net/code/issues/7444
Be able to add multiple files more easily
-
https://labs.riseup.net/code/issues/7442
Make it easier to see the current metadata in MAT
-
https://labs.riseup.net/code/issues/7445
Icon usage in MAT
-
https://labs.riseup.net/code/issues/7446
UX: make preferences clearer
==
There were 15 people subscribed to the workshop.
Only 4 of them came plus one person who was not subscribed.
We do not really know what caused this but would like to find out.
===
How the workshop was organized.
We prepared 4 missions.
There should have been 3 groups of people who would carry out one
mission. (As there were only 5 people, we carried out only 3 missions.)
Every group member would carry out his or her mission individually.
People receive the mission in an envelope, together with 3 red cards.
After doing the mission, everybody writes down what they do exactly to
accomplish the mission.
During the mission, if the person is blocked by some problem, they can
lift a red card and ask for help. However, they are encouraged to look
for the solution themselves by every means possible. But if that takes
too long, they could use the card.
The 3 people of NUMA sat next to a person in particular to observe them.
During the second phase, everybody would reconstitute their mission
through putting post-its for each step on a white board. They would
explain publicly where they blocked, and what worked and what not. If
they blocked somewhere, they would put a cross on the post it. If they
used a red card, they would paste the red card to the white board too.
===
This is the feedback from the NUMA UX experts (i am merely copy-pasting
this):
Usability Test: Debriefing and Recommendations
All missions included booting on Tails, selecting keyboard and language
and activating persistence and/or root password if the mission seemed to
require it.
Mission A
Create a new encrypted document of your choice (photo, pdf, etc..) and
save it to Tails, using persistence.
User profile:
Male, Developer, Windows user, Uses Tails on a virtual machine.
Had persistence on his key, mission took him 3 minutes without problems.
2nd User profile:
Male, crypto guy.
Did not have persistence. Needed to create persistent volume, but his
key was a first copy. So he did it with a key we provided. Set up
persistence, was no problem, but he did not read the message saying that
he needs to reboot to make the whole thing work.
Mission B
Find out the number of Tails downloads this month.
Pass on this information using GPG via email to xxx@??? either
using the available public key or the shared secret: "numacestcool"
User profile:
Male, Developer, Windows user, Uses Tails on a virtual machine.
User flow:
Launching Tails : He uses Tails on a virtual machine.
He launches Tails. He thinks it’s positive that the Admin mode is
available only at the launching screen and it can’t be activated inside
the system.
He gets a message “Warning: virtual machine, your screen can be
accessed from windows”. He guesses (but is not sure) that the security
is maintained even if he is using a virtual machine.
No problem in connecting to NUMA network (when he understands that
it is a captive portal he asks why he could connect to it, maybe linked
to the virtual machine ? )
Observation: The user is not sure about its level of security.
Recommendation: Tails means "secure". Be precise about what's secure and
what is not. Vagueness is not acceptable.
Finding the information: He visit TAILS website and looks for the
information. He visits a search engine and sees it's "enhanced by
Google”. He asks why Tails use Google which is non anonymous by
definition. He goes on wikipedia looking for GPG repositories. He tries
gnupgp.com. The website asks for a login so he rather goes for the MIT
website (pgp.mit.edu).
He finds the address and the key.
Looks for GPG in the menu and can’t find any tool (another user says
he found it on the top right corner but a right click made it disappear)
Why isn't it in the menu ?
He wants to create a temporary identity to use the PGP key
He is surprised that the system asks him to sign because -he says-
he only wants to send a message
He decides to do that using the Terminal because “I already know
what I want to do”.
Observation: The user can't find the PGP applet. The user was an expert
with the Terminal and could encrypt the message, but this would an
almost impossible task for a first-time user.
Recommandation: Put the PGP applet with other Apps. On the website,
present some simple scenarios for tasks a Tails user will probably have
to do.
Writing and sending an email: He now need a mail client. He doesn’t
like the default Tails client (too complex to set up) so he look for a
web service. He would like a webmail without login.
Tries Mailnesia: doesn’t work because the IP is banned.
Tries Hushmail, and finds out he needs to pay.
He doesn’t want to use Gmail because he thinks the message will be
delivered unencrypted
Observation: The user has a goal but can't find the right tool online.
When the service is not locally installed it becomes complex to choose
the right web tool for a task.
Recommendation: Tails should recommend this kind of services (as useful
Bookmarks or in a Tails toolbox on the website?)
Main problems:
Not sure about security level inside the virtual machine.
Couldn’t find a PGP applet.
Looked for a secure mail client that he could use without login but
couldn’t find it.
Mission C
Find one or more images or PDF files and inspect their metadata. (You
can use your own files or find some on the internet). Clean
up these files to erase any metadata.
User Profile:
Female, Journalist, Debian user, Uses Tails to teach her students about
Internet anonymity.
User Flow:
Network connection: she uses the non-secure NUMA network at first,
as the network config doesn't allow a secure connection she needs to
open up a non-secure browser session first, this apparently initiates
the connection.
Observations:
From a first-time user perspective the messages displayed during
this phase might not be very clear, if I remember this correctly the
system tells the user the connection is not secure, but does not
indicate a solution: open an insecure browser session first.
Recommendations:
It seems that the necessary steps to initiate a secure connection
are quite simple and could probably be automated: the system could for
example ask the user using a pop-up or a tooltip if he wants to open a
temporary insecure connection for the initial "handshake" , then perform
the handshake in the background and notify the user once the secure
connection has been established (all the while displaying a progress
bar/wheel)
Tor connection: after the first insecure browsing session the user
closes the browser and waits, she says she is "waiting for the green
onion to appear"
Observations:
This user knows that she needs to wait for something to happen
before she can browse securely, but a first-time user wouldn't have this
information, nor would he know how long the wait will last (which can be
a problem even for an experienced user)
Recommendations:
In general when something needs to happen before a user can do
something else it is advisable to let the user know: a) what needs to
happen and b) how long this is going to take.
A quick fix for this could be to display some progress bar/wheel in
the status bar that indicates that Tor is connecting, initially you can
also couple this with a tooltip message along the lines of "Establishing
Tor secure connection" with a smaller "do not show this anymore"
button/link to kill the tooltip. For examples of ongoing connection
status icons you can look at how Dropbox or Skype implement connection
and synching visual cues. Frewall programs like Little Snitch also
display more complex network information on hover.
Closing Tor: when the user closes Tor a cryptic message appears
stating that the program is closing and that in the meanwhile some
confidential data might still be exposed.
Observations:
this message was rather mysterious in its wording because it
highlighted a potential threat, but didn't clearly state what was
happening in the meanwhile, how long would this take and whether the
user could do something about it.
Recommendations:
for an OS that promises security it doesn't seem appropriate to give
the user such vague information about potential threats with unclear
outcomes. Again, it is advisable to state clearly what is happening, if
the issue is being auto-resolved by the system (in the case of a memory
clean up) give an approximate idea of how long it is going to take. When
instead there are other actions a user can perform to solve the issue
use clear interface elements to give the user the quickest path to fix
the issue.
Downloading a picture: the user finds a picture using a secure
browser connection. She tries to right-click to see if she can access
the metadata directly in the browser before downloading it, but the
browser doesn't provide this functionality, she downloads the image to
the desktop.
Using Metadata Anonymisation Toolkit: the user reports knowing about
MAT, she says she usually clears metadata through a Terminal command. At
first she has some trouble finding the program, but finally manages to
locate it inside the accessories menu. Once inside MAT, she has trouble
displaying the metadata as the program doesn't seem to provide a "view
metadata" option within the menu nor in the toolbar. She finally manages
to bring up the metadata dialog with a double-click on the filename in
the list. She prefers to use the menu options rather than the icons in
the toolbar to verify and clean up metadata. She tries to drag multiple
files at once to the file list, but only one file is added. Once she has
cleared metadata from the files in the list she clears the list and then
adds the files again to double check that they contain no metadata.
Looking at the "inspect metadata" function description that states
"verify sensitive metadata inside the file" the user asks how does the
program decide which data is sensitive and which is not. She clicks on
this icon, but the system apparently provides no noticeable feedback.
Observations:
a) In MAT the metadata information of files doesn't seem to be
easily accessible as it requires the user to guess that double-clicking
will bring up this info.
b) The fact that the user prefers using the menu items rather than
the icons might indicate that the icons alone are not expressive enough.
c) There seems to be some confusion regarding the way that the
program speaks about "sensitive metadata"
Recommendations:
a) Since MAT is a metadata removal tool it may be wise to give the
metadata better visibility by displaying it at a primary level, this
could be in the form of a contextual panel (have a look at how Adobe
Bridge and other media management programs implement this).
b) Iconography might need to be couple with text to make its
function clearer
c) If the program promises to identify what metadata is sensitive it
has to clearly show what bit of information is sensitive and what not
(using color coding for example), if on the other hand it only allows to
detect whether metadata is present or not the goal for the interface is
to clearly show whether metadata is present or not. The fact that the
user clicks multiple times on the "inspect" function is symptomatic of a
lack of appropriate feedback, which can come either in the form of a
window or simply associated to the file in the list using color coding,
iconography or other types of indicators (similarly to how virus
scanners might display an infected file in a scan report for example).
This uncertainty about the completion of the cleanup task is further
confirmed by the fact that the user clears the list and drags the files
back in for a second check, as if the system hasn't given her enough
reassurance.
Tails Shutdown: the user shuts down Tails by removing the USB key. A
black screen appears with some messages regarding sensitive data
exposure and many lines of hard-to-read status messages. When asked what
this screen means the user says that it's basically telling you that
it's cleaning up the sensitive data and that she doesn't really read or
understand what the rest of the status message means.
Observations: the function of this screen probably wouldn't be clear
to a first-time user and even to an expert user it doesn't mean much in
its current state. At first glance it looks like a system crash screen,
more alarming than reassuring.
Recommendations: it seems that the objective of this screen would be
to a) inform the user that her data is being wiped from memory and b) to
conceal the user's activity in case of an abrupt shutdown prompted by a
physical confidentiality threat (someone entering the room for example).
You could for example mask this shutdown screen as a common Windows or
OSX shutdown process, with a progress bar / wheel that informs the user
that the "cache is being cleared", "the system is shutting down" or
whatever other message is more appropriate.
Product Development and Communication Suggestions
Tails as a secure activities launchpad (guarantor)
Communicating your value proposition
Engaging a new audience
Design Challenges
Tails is a synonym of “anonymous” and “secure”. Users look for more
than the software itself, they look for a secure environment. This
involves contexts of use and using other web services.
- On the web, Tails should recommend some tools and link them to
usual scenarios (like a journalist exchanging files with a source).
- A double entry toolbox (tools / scenarios) would be a value added
service for the community.
Tool X used in scenario Y as in example W.
“Security” doesn’t accept uncertainty and is quite binary: the user
must know if he is in a secure area or not (cf. when launching tails on
a virtual machine).
Deal with “security” as a zone the user can enter and exit.
- Explain when and why to wait for a secure service to be active
(cf. when establishing a TOR connection) and inform the user of the
approximate time it will take / of the progress of the operation.
- Consider using a red/green light to notify the user he is/ he is
not in a secure zone. If you believe that security is your main value
proposition, it would make sense to explore ways of making this emerge
to the main layer of the interface by permanently visualizing the states
of all security areas, more or less the same way that status indicators
inside an airplane cockpit or space rocket give a constant overview of
all the critical parameters.
- When the system tells the user he is not in a secure zone (e.g.
when using a virtual machine) it should detail what are the risks he
undergoes, if a solution can be automated provide a direct shortcut
through a functional button / multiple choice, if a solution cannot be
automated inform the user what to do next.
- In the browser, notify the user when he tries to use a service
that will take him out of a secure zone (ex: gmail).
It’s hard to explain the difference between actions the user
shouldn’t do and actions the user can’t do. To clarify:
I'm not sure this is something you should do by default, I'd rather
suggest this be done on a case-by-case basis, the same way that
firewalls learn which programs are or aren't allowed to establish
connections the first time they try to do so. I'd say: when a threat
presents itself inform the user and give him a choice, for example: a
program runs over an insecure connection, do you want to allow this?
-YES for this session; -YES forever; - NO forever; -etc...
- Give examples of things the user shouldn't do. You can seize the
opportunity to educate your users when a threat presents itself, for
example when a user downloads an executable file you can ask
confirmation and provide a "more info" button that leads to a full
description of the possible threats.
The lack of personalisation makes TAILS interface more similar to a
ticket booth than an OS: every time the user is a new one, no
information is kept in the system memory.
- Keep that in mind while designing and organise the interface so that
the user can go through personalisation steps easily (setting language,
mail etc.)
- Allow users to store certain preferences in the persistent drive (this
should remain an option since preferences are personal data in a way).
