DNS issues with VPN usage

[ldionmarcil]

httpx version:

v1.10.0

Current Behavior:

When using a VPN on Linux, httpx becomes unreliable due to intermittent DNS failures. Running the same command a few times will yield different results since the DNS resolution will fail, seemingly at random.

Expected Behavior:

A somewhat deterministic output every time I run the command on a given data set.

Steps To Reproduce:

1. Connect to VPN (tested with Private Internet Access)
2. Run httpx with a handful of hosts
3. Observe failures

Anything else:

Here are a few executions with public servers, showing the DNS failures:

$ echo "https://perdu.com\nhttps://lapresse.ca/\nhttps://ledevoir.com" | httpx -v 2>&1 | grep -e http -e DBG
[INF] Current httpx version v1.10.0 (latest)
https://ledevoir.com
[DBG] Failed 'https://lapresse.ca/': Get "https://lapresse.ca/": cause="no address found for host"
https://perdu.com

$ echo "https://perdu.com\nhttps://lapresse.ca/\nhttps://ledevoir.com" | httpx -v 2>&1 | grep -e http -e DBG
[INF] Current httpx version v1.10.0 (latest)
https://perdu.com
[DBG] Failed 'https://lapresse.ca/': Get "https://lapresse.ca/": cause="no address found for host"
https://ledevoir.com

$ echo "https://perdu.com\nhttps://lapresse.ca/\nhttps://ledevoir.com" | httpx -v 2>&1 | grep -e http -e DBG
[INF] Current httpx version v1.10.0 (latest)
https://lapresse.ca/
[DBG] Failed 'https://ledevoir.com': Get "https://ledevoir.com": cause="no address found for host"
https://perdu.com

None of these have weird DNS, in fact they all resolve properly:

$ dig +short perdu.com
172.64.80.1
$ dig +short lapresse.ca
52.70.7.144
32.195.53.148
3.82.167.114
$ dig +short ledevoir.com
151.101.66.132
151.101.2.132
151.101.130.132
151.101.194.132

Something strange I found is that if I specify a public resolver in httpx, the DNS resolution works fine every time, even if its the same DNS server that I have configured in /etc/resolv.conf:

$ cat /etc/resolv.conf
nameserver 1.1.1.1

$ echo "https://perdu.com\nhttps://lapresse.ca/\nhttps://ledevoir.com" | httpx -v 2>&1 | grep -e http -e DBG
[INF] Current httpx version v1.10.0 (latest)
[DBG] Failed 'https://ledevoir.com': Get "https://ledevoir.com": cause="no address found for host"
https://lapresse.ca/
https://perdu.com

$ echo "https://perdu.com\nhttps://lapresse.ca/\nhttps://ledevoir.com" | httpx -v -r 1.1.1.1 2>&1 | grep -e http -e DBG
[INF] Current httpx version v1.10.0 (latest)
[DBG] Using resolvers: 1.1.1.1
https://ledevoir.com
https://lapresse.ca/
https://perdu.com

This leads me to believe that DNS resolution is broken for whatever reason when using a VPN.

I checked with dnsx and it also fails to do DNS lookups when on VPN:

$ dnsx -v

      _             __  __
   __| | _ __   ___ \ \/ /
  / _' || '_ \ / __| \  /
 | (_| || | | |\__ \ /  \
  \__,_||_| |_||___//_/\_\

                projectdiscovery.io

[INF] Current dnsx version 1.2.3 (outdated)
$ echo "perdu.com\nlapresse.ca\nledevoir.com" | dnsx -v 2>&1  | grep -e DBG -e write
[DBG] query error for lapresse.ca: could not resolve, max retries exceeded
write udp 10.7.18.114:58914->149.112.112.112:53: write: operation not permitted
[DBG] query error for perdu.com: could not resolve, max retries exceeded
write udp 10.7.18.114:43895->9.9.9.9:53: write: operation not permitted
[DBG] query error for ledevoir.com: could not resolve, max retries exceeded
write udp 10.7.18.114:39107->208.67.222.222:53: write: operation not permitted
[DBG] incomplete response for perdu.com (all retries exhausted)
[DBG] incomplete response for ledevoir.com (all retries exhausted)
[DBG] incomplete response for lapresse.ca (all retries exhausted)

But it works with a hardcoded DNS server flag.

[INF] Current dnsx version 1.2.3 (outdated)
ledevoir.com
lapresse.ca
perdu.com