Android Electrum breaks on dual-stack (IPv6+IPv4) servers, desktop versions misbehave as well

[gits7r]

Hello,

When the mobile device has IPv6 internet connectivity (public address assigned via IPv6 prefix delegation, by the router), it can connect just fine to any IPv6 internet destination using the web browser for example, but Electrum wallet 3.3.4 will not connect to the server if it's a dual stacked one (if the hostname resolves to both AAAA IPv6 and A IPv4). It will just hang as "connecting" forever.

We can confirm the issue is with Electrum because the same dual stack server that runs Electrum server service also has a webserver, and it can be browsed from the same mobile device, so the destination is actually reachable, it's only Electrum that cannot connect. To reproduce this, simply connect a mobile device to a router that has IPv6 enabled for clients and try to connect to any dual stacked server hostname. If you connect to one that resolves only to IPv4, it works fine.

The desktop version also misbehaves, if the host running Electrum wallet has both IPv6 internet and IPv4 internet, when you try to connect to an Electrum server that resolves to both IPv6 and IPv4, it connects to IPv4 (doesn't hang forever, like Android version), but if you try to ping the hostname of the Electrum server you properly ping its IPv6 address. This can be simply reproduced as well.

There is a bug in Python, I am not sure if it's related but quoting it here, in case this is the fault we should know what to do:
https://bugs.python.org/issue35302

There is also a similar issue with ElectrumX server, one we think it's related to the above upstream Python bug, which is logged as:
kyuupichan/electrumx#682

[SomberNight]

Do you run/know of such Electrum servers that can be used to test with?

[gits7r]

Yes.
Try with electrum2.eff.ro (50002).

I can give you testing environment if you don't have native IPv6 anywhere (like an OpenVPN tunnel in TAP mode that connects over IPv4 endpoints but assigns you a public /64 IPv6 prefix you can use to reach the internet). That can be run also on an Android mobile device. Or if you prefer a virtual machine, just ping me by email and i'll create it for you. But I think you already have native IPv6 though.

[gits7r]

I have done more digging to confirm / infirm this bug.

The bug is confirmed. Dual stacked servers:
electrum.eff.ro
electrum2.eff.ro

Electrum 3.0.5 connects to any of them using IPv6, as it should in any environment.
Electrum 3.3.4 connects to any of them using IPv4, in the same environment with IPv6 enabled, while the servers are totally listening and being reachable on IPv6. If you ping from the same environment the hostnames of the server, it properly sends ICMP requests to the IPv6 address and receives replies with 0 loss.

Android is broken entirely, it reports server is offline if the hostname resolves to IPv6 also.

To reproduce simply run tcpdump / netstat and try to connect using electrum 3.3.4 from a box with IPv6 internet access. Then try to ping the hostname of the server from the same box. Then try with 3.0.5 and see the difference (maybe other versions work too, this is the one I had at hand to test).

[gits7r]

While connected, netstat -an at ElectrumX server side reports 1 (one) ESTABLISHED connection with the IPv4 address of the Electrum 3.3.4 client. When session is terminated by closing the Electrum 3.3.4 application, netstat -an at ElectrumX server side reports 2 (two) TIME_WAIT connections with the IPv4 address of the Electrum 3.3.4 client (but with different source ports of course). Is this normal?

[SomberNight]

Note: I've just tested now and I can reproduce. I can connect to your server on my phone if I have the usual IPv4 VPN enabled, but I cannot without the VPN on my IPv6 enabled WiFi.

[SomberNight]

Testing on desktop, on a dual stack host, it seems to connect via IPv6.

In the console tab:

>> network.interface.session.peer_address()
('2001:678:7dc:3::2', 50002, 0, 0)

$ netstat -a | grep 50002
tcp6       0      0 user-VirtualBox:45020   electrum2.eff.ro:50002  ESTABLISHED

And with only IPv4 available locally:

>> network.interface.session.peer_address()
('195.135.194.3', 50002)

$ netstat -a | grep 50002
tcp        0      0 user-VirtualBox:43924   electrum2.eff.ro:50002  ESTABLISHED

Tested with Python 3.6.7 and 3.7.2

The desktop version also misbehaves, if the host running Electrum wallet has both IPv6 internet and IPv4 internet, when you try to connect to an Electrum server that resolves to both IPv6 and IPv4, it connects to IPv4

So I cannot reproduce that ^. Can you check the output of network.interface.session.peer_address()?

[SomberNight]

I suppose the issue on Android can be explained by https://github.com/kivy/python-for-android/blob/f16a66590c92bb57c69b214b2b69dfaa6dc1e0ed/pythonforandroid/recipes/python3/__init__.py#L33-L43
It seems the bundled Python interpreter is built with IPv6 disabled.

I've opened an issue upstream: kivy/python-for-android#1771

[gits7r]

ok, looks like we nailed for Android.

I am not sure what is different in your environment that it actually connects over IPv6. In mine, still no:

>> network.interface.session.peer_address() ('195.135.194.6', 50002)

It does not connect over IPv6. I have tried using Electrum 3.3.4. Of course native IPv6 is working on the host, if I ping the ElectrumX hostname I get ICMP replies over IPv6 (as it should).

There's absolutely nothing changed on my host besides regular native IPv6 with normal preference of IPv6 over IPv4 when both are available. I tried this from 2 different computers and one virtual machine, same result.

[gits7r]

I can see you mention the Python version which means you built Electrum from source. Try for example with the standalone executable for Windows or the AppImage for Linux. This is what I am using, and this appears to be the only difference.

[SomberNight]

Can you try specifying the IPv6 address directly instead of a domain in Electrum? as the server. Does it connect?

[gits7r]

I tried to specify the IPv6 address directly on both SSL as well as TCP (to prevent any kind of self-signed certificate pinning mismatch) and neither connects, it just stays as not connected. Tried also putting the IPv6 address between [ ] and still it did not connect.

[gits7r]

I've opened an issue upstream: kivy/python-for-android#1771

This is a very good catch, I see no reason for disabling IPv6 on packaged python, but shouldn't we have something at the application level to prevent such misconfigurations, something that will automatically fall back to IPv4? I agree this is besides the scope of the application, the problem is upstream of course. I just don't know why for the desktop version it connects using IPv4 instead of reporting offline forever.

Use the standalone bundled executable and try to connect from an IPv6 enabled host and run network.interface.session.peer_address() -- you should get the IPv4 address instead.

[SomberNight]

So I'm now testing with the official 3.3.4 AppImage on Ubuntu 18.04 running in VirtualBox, with IPv6 connectivity, and it looks to be working for me.

>> network.interface.session.peer_address()
('2001:678:7dc:3::2', 50002, 0, 0)

Weird that it does not work for you.

[gits7r]

I'll give it another try with Linux as well, spinning again a Debian VM to check out the AppImage. I have tried with the executable on Windows just now and it connects over IPv4. Could it be only on Windows? Windows executables have anything different?

[SomberNight]

Ok, I'll try with Windows later.

Windows executables have anything different?

There are subtle differences between all the binaries.
E.g. regarding Python itself:

• for the AppImage, we build it ourselves directly
• for Windows, we just download official binaries from python.org
• for Mac, we use pyenv (which I believe builds Python locally)
• for Android, we use python-for-android, which builds Python locally

[SomberNight]

The desktop version also misbehaves, if the host running Electrum wallet has both IPv6 internet and IPv4 internet, when you try to connect to an Electrum server that resolves to both IPv6 and IPv4, it connects to IPv4

Ok, so I can reproduce this on Windows. It's specific to a workaround Electrum does on Windows only: see #4421 and #4486
We don't let Python to do its own DNS resolution, rather we do our own, but only A records are requested.

[gits7r]

ok, this makes sense. I have tested the AppImage under Debian and it worked over IPv6. Is there any particular reason why only A records are requested? Why not both A and AAAA and use IPv6 in case the host can do it? We could do the same for Android in case upstream kivy has a particular reason to have IPv6 disabled in packaged python.

[SomberNight]

Is there any particular reason why only A records are requested?

I simply did not think about IPv6 at the time. dnspython is used to resolve hostnames, and it just defaults to asking for A records.