On 7/21/14, intrigeri <intrigeri@???> wrote:
> Hi,
>
> Jacob Appelbaum wrote (24 Jun 2014 10:56:54 GMT) :
>> I think agreeing on a specific user agent and having a central place
>> to find it makes the job much easier to tackle. In any case, I think
>> setting a few shell aliases would not hurt and if they source a common
>> file for a user agent, it should be straight forward to keep things in
>> sync with perhaps no upstream modifications?
>
>> For example:
>
>> wget --user-agent="$useragent"
>> curl --user-agent "$useragent"
>> GET -H "User-Agent:$useragent"
>
> This would definitely work. We ship a getTorBrowserUserAgent program,
> that's used by the curl processes started by htpdate. Its results
> could be cached at ISO build time, and then used by these aliases.
Seems fine, yes.
>
> One should look for other instances of using wget, curl, LWP and
> friends without going through the shell, too. Any taker?
Not it. :)
>
>> For the discussion at hand, I sniffed my own sessions and saw the
>> following data transmissions.
>
> Woohoo \o/ .. and sorry for the delay.
>
>> wget:
>> [...]
>> Accept: */*
>> Connection: keep-alive
>
>> curl:
>> [...]
>> Accept: */*
>> Proxy-Connection: Keep-Alive
>
>> GET:
>> [...]
>> TE: deflate,gzip;q=0.3
>> Connection: TE, close
>
>> This is Tor Browser on Tails for the same file but on a different web
>> server:
>> [...]
>> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>> Accept-Language: en-us,en;q=0.5
>> Accept-Encoding: gzip, deflate
>> Connection: keep-alive
>
> So, this shows that we have an identifying set of headers for each of
> these four clients, even if we ignore the user-agent information.
...
>
> ... but, in the following tests (with a forged user-agent), most of
> these discrepancies disappear, so I'm confused:
>
>> Here are the same clients with a forged User Agent:
>
>> wget --user-agent="User-Agent:Mozilla/5.0 (Windows NT 6.1; rv:24.0)
>> Gecko/20100101 Firefox/24.0"
>> [...]
>> Accept: */*
>> Connection: keep-alive
>
>> curl --user-agent "User-Agent:Mozilla/5.0 (Windows NT 6.1; rv:24.0)
>> Gecko/20100101 Firefox/24.0"
>> http://people.torproject.org/~ioerror/misc/tor-ips.txt; shows:
>> [...]
>> Accept: */*
>> Connection: keep-alive
>
> I'm surprised: without faking the user-agent, we had
> "Proxy-Connection" instead of "Connection". Is one of these results
> wrong, or is curl behaving erratically, or is there another
> rational explanation?
>
I don't remember? :-)
>> GET -H "User-Agent:Mozilla/5.0 (Windows NT 6.1; rv:24.0)
>> Gecko/20100101 Firefox/24.0"
>> [...]
>> Connection: keep-alive
>
> Same here, we had "Connection: TE, close" previously => same question.
>
>> My conclusion is that setting the user agent for curl and wget to
>> match Tor Browser isn't a horrible idea. It even seems like on a
>> single GET request, it would be helpful for privacy and anonymity set
>> reasons. It certainly reduces the version information leakage that is
>> absolutely useful for fingerprinting and exploitation. For `GET` - we
>> might also add -H="Accept: */*" and then all three would be aligned.
>
> I'll wait for the surprising things highlighted above to be clarified,
> before commenting on this one.
>
I think the first set was wrong or weird and the second set was mostly
correct. It would be good if someone could re-run the tests on Tails
1.1 anyway. Any takers?
All the best,
Jacob
