[Tails-dev] Results from usability tests of Tor Connection i…

Delete this message

Reply to this message
Author: sajolida
Date:  
To: The Tails public development discussion list
New-Topics: [Tails-dev] Results from usability tests of first-time use in São Paulo
Subject: [Tails-dev] Results from usability tests of Tor Connection in São Paulo
Hi,

In August, I conducted moderated in-person usability tests of Tor 
Connection in São Paulo as part of our work on Sponsor 03.

https://gitlab.tails.boum.org/tails/tails/-/issues/18762

I recruited 4 representative participants who were interested in privacy 
tools but had never used Tails before. I asked them to connect to Tor 
from 3 different networks and observed them while doing so:

- 1 network allowed connecting to Tor directly
- 1 network was behind a captive portal
- 1 network was blocking Tor

I already tested Tor Connection in August 2021. Back then, I deemed the 
results as catastrophic. None of the test participants managed to finish 
the 3 tasks within 2 hours. None of them managed to use a custom bridge.

In São Paulo, things were quite better, but general usability is still 
very bad.

The good stuff:

- 1 participant managed to perform all 3 tasks without major issues.

- All 4 participants connected to Tor directly without major issues.
   All of them used the fixes from #18587 and #18584!

- 3 participants managed to connect to Tor pretty much on their own
   using a bridge using the upcoming QR code feature. (#18219)

- The last 2 participants managed to sign in to the network using a
   captive portal pretty much on their own after I improved a bit some
   phrasing. (#19168)

The bad stuff:

- P1 ended up connecting to SecureDrop using the Unsafe Browser!

- Connecting to a captive portal took 50, 64, and 66 minutes to P1, P2,
   and P4 respectively, sometimes even after serious hand-holding by me.

- 3 participants thought that the issue was a missing Tor bridge when
   behind a captive portal.

- Several participants faced a dead slow progress bar that lasted
   several minutes when connecting to Tor, despite the ISP connection
   being very fast and the obfs4 bridge working decently. (#19173)

- All 4 participants thought that "Hiding" was safer than "Autconnect"
   in the consent question based on the sensitivity of the task at hand
   (connecting to SecureDrop) and not the need to hide their connection
   to Tor.

In general, the main issue was poor diagnostic when Tor fails to start. 
People don't know what the issue is, try everything in vain, and get 
very lost.

I created 16 new GitLab issues for possible solutions:

- #19095: Disable HTTPS-only mode in Unsafe Browser
- #19163: Improve Mac animation based on feedback from Brazil
- #19164: Don't display offline warning of Tor Browser when opening
           local doc
- #19166: Remove (easier) and (safer) label from consent question
- #19167: Always select "Ask for a bridge by email" by default when
           "Hiding"
- #19168: Explain better the Unsafe Browser from Tor Connection
- #19169: Add label to bridge line
- #19170: Add diagram with phone to troubleshooting doc
- #19171: Hide “Fix clock”, “Proxy”, and “Captive Portal” sections when
           time sync was successful already
- #19172: Fix links from Tor Connection to the offline documentation
- #19173: Investigate progress bar getting unstuck after restarting Tor
           Connection
- #19174: Mention captive portals in the troubleshooting documentation
           about connecting to Tor
- #19175: Explain how to get more bridges in the troubleshooting
           documentation about Tor
- #19176: Improve list of example website in Unsafe Browser home page
- #19177: Display "Tails" instead of "EFI Boot" or "Windows" in macOS
           Startup Manager
- #19208: Have a separate progress bar for the 2nd step of connecting to
           Tor

For detailed results, see the rainbow table in attachment.

-- 
sajolida
Tails — https://tails.boum.org/
UX · Fundraising · Technical Writing