Enable other containers to join network namespace of the none network #3443

Shubhranshu153 · 2024-09-20T17:50:55Z

Fixes: Unable to attach to container network created with none

Description

There is a use case with pause containers, where other containers attaches to the pause container network. The pause container is launched with network none. This requires the pause container have a copy of hosts, hostname and resolv conf.
It also seems to share a net namespace, the containers must also share a user namespace.

The solution is to have a copy of the hosts/hostname and resolv.conf. In case of container network, add userns and netns both.

Want to confirm is this an acceptable solution and i can send out an PR for it.
Steps to reproduce the issue

Create a pause container with network none.
Create another container with --net container:

Describe the results you received and expected

It would display errors with resolv.conf not found and once those configs are added would see an error with sys fs.
Expected result is to be able to connect to the network of pause container.
What version of nerdctl are you using?

1.7.5
Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None
Host information

lima vm (fedora image), but can be reproduced in any architecture.

AkihiroSuda · 2024-09-20T18:14:43Z

Want to confirm is this an acceptable solution

Yes if it works with Docker.

Needs an integration test.

pkg/containerutil/container_network_manager.go

apostasie · 2024-09-20T23:06:45Z

pkg/containerutil/container_network_manager.go

+	}
+
+	resolvConfPath := filepath.Join(stateDir, "resolv.conf")
+	copyFileContent("/etc/resolv.conf", resolvConfPath)


That would make containers using none to now use the host resolv.conf, right?
This does not seem right.

checked with docker and they have it for none, personally i dont think it should be part of none, but if it is not attaching to a none network namespace (custom use cases) seems to throw error.

It does not look like they have it here...

docker run --net none debian cat /etc/resolv.conf

# Generated by Docker Engine. # This file can be edited; Docker Engine will not make further changes once it # has been modified. nameserver 192.168.5.2 search . # Based on host file: '/run/systemd/resolve/resolv.conf' (legacy) # Overrides: []

vs.

cat /etc/resolv.conf

nameserver 127.0.0.53 options edns0 trust-ad search .

this is my output (docker 25)

docker run --network none alpine:latest cat /etc/resolv.conf <redacted> nameserver 10.4.4.10 options timeout:1 attempts:2

sudo docker run --network none alpine:latest cat /etc/hostname 9d25f3a65441

to be fair i tested in another system i got same as yours too, so not sure which one is correct, but for some system is taking from /run/systemd/resolv/resolv.conf?

It likely does not matter much, since there will be no network anyhow. Concern here is more about information leakage.
I would suggest we either leave it empty or mimic the behavior of cniNetworkManager

not sure if information leak is a concern in network none, if one can ssh from host they have access to /etc/resolv.conf of the host anyway.

I like the idea of mimicing the behavior of cniNetworkManager, that seems to take into account adding host, dns etc.

while we are on the topic, if we use --network container:containerd id with

nameServers = m.netOpts.DNSServers searchDomains = m.netOpts.DNSSearchDomains dnsOptions = m.netOpts.DNSResolvConfOptions

dont we want to add them to the resolv.conf, similarly if --network host with these options, not sure they are getting added

[shubhum@lima-finch nerdctl]$ sudo nerdctl run --network host --hostname default --dns test --dns-search testname alpine:latest cat /etc/resolv.conf # This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8). # Do not edit. # # This file might be symlinked as /etc/resolv.conf. If you're looking at # /etc/resolv.conf and seeing this text, you have followed the symlink. # # This is a dynamic resolv.conf file for connecting local clients to the # internal DNS stub resolver of systemd-resolved. This file lists all # configured search domains. # # Run "resolvectl status" to see details about the uplink DNS servers # currently in use. # # Third party programs should typically not access this file directly, but only # through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a # different way, replace this symlink by a static file or a different symlink. # # See man:systemd-resolved.service(8) for details about the supported modes of # operation for /etc/resolv.conf. nameserver 127.0.0.53 options edns0 trust-ad search ant.amazon.com amazon.com

changed this configs to some generic default values

pkg/containerutil/container_network_manager.go

apostasie · 2024-10-27T14:02:22Z

@Shubhranshu153 will be slow to review - travelling right now. Will do asap.

apostasie

Left a note about WriteDefaultResolveConf.
Otherwise, these changes seem good and will bring us more in line with what docker does, so +1.

This needs tests - especially around your use case of another container using the none container net.

apostasie · 2024-10-28T20:06:59Z

pkg/containerutil/container_network_manager.go

@@ -150,6 +150,11 @@ type noneNetworkManager struct {
 	client        *containerd.Client
 }

+func WriteDefaultResolvConf(filePath string) error {


I think we should use the existing resolvconf.Build.
It is not currently safe to use it (not atomic, nor locking), but it is better IMO than duplicating the problem.

apostasie · 2024-10-28T20:22:23Z

pkg/containerutil/container_network_manager.go

+	}
+
+	resolvConfPath := filepath.Join(stateDir, "resolv.conf")
+	if err := WriteDefaultResolvConf(resolvConfPath); err != nil {


See note above.

Shubhranshu153 · 2024-10-29T22:24:56Z

will add tests.

Shubhranshu153 · 2024-10-30T05:45:00Z

Left a note about WriteDefaultResolveConf. Otherwise, these changes seem good and will bring us more in line with what docker does, so +1.

This needs tests - especially around your use case of another container using the none container net.

so overall behaviour wise it is like docker but the configs dont still match, which is probably fine as we dont expose host details if we add specifically the things we want to expose.

Shubhranshu153 · 2024-10-30T06:29:42Z

@apostasie added the tests, for the configs i have made it docker non compat because of the explanation above.
PTAL.
Thanks.

cmd/nerdctl/container/container_run_network_linux_test.go

go.mod

apostasie · 2024-10-30T12:16:06Z

@apostasie added the tests, for the configs i have made it docker non compat because of the explanation above. PTAL. Thanks.

Thanks @Shubhranshu153

I left a few comments on the tests. I will give this a spin locally later today too, but otherwise looks good.

Shubhranshu153 · 2024-10-30T21:15:37Z

@apostasie PTAL
Not sure why FreeBSD is failing

Signed-off-by: Shubharanshu Mahapatra <[email protected]>

Shubhranshu153 · 2024-10-31T00:05:03Z

@AkihiroSuda @apostasie
need to add some test but need some guidance here
In Docker you can add something like:

docker run --rm   --dns 8.8.8.8   --dns 8.8.4.4 --add-host somewhat.example.com:127.0.0.1   --dns-search example.com  --network none  -it ubuntu:latest

and you can get them added

dev-dsk-shubhum-2b-5b633f8c % docker run --rm   --dns 8.8.8.8   --dns 8.8.4.4 --add-host somewhat.example.com:127.0.0.1   --dns-search example.com  --network none  -it ubuntu:latest cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
127.0.0.1       somewhat.example.com

(24-10-31 0:01:58) <0> [~]  
dev-dsk-shubhum-2b-5b633f8c % docker run --rm   --dns 8.8.8.8   --dns 8.8.4.4 --add-host somewhat.example.com:127.0.0.1   --dns-search example.com  --network none  -it ubuntu:latest cat /etc/resolv.conf 
search example.com
nameserver 8.8.8.8
nameserver 8.8.4.4
options timeout:1 attempts:2

So added these functionalities to none network, want some guidance if am missing functionality wise, i reused the code from host network as it seems it just used values from netopts. But i plan to refactor it to a single function (cleanup wise)

apostasie · 2024-10-31T18:02:04Z

Will have a look as time allows.
Thanks @Shubhranshu153

Shubhranshu153 marked this pull request as ready for review September 20, 2024 18:03

apostasie reviewed Sep 20, 2024

View reviewed changes

pkg/containerutil/container_network_manager.go Outdated Show resolved Hide resolved

apostasie reviewed Sep 20, 2024

View reviewed changes

pkg/containerutil/container_network_manager.go Outdated Show resolved Hide resolved

Shubhranshu153 force-pushed the fix-network-none branch 2 times, most recently from e99fd1d to 29760dc Compare October 26, 2024 14:54

Shubhranshu153 requested a review from apostasie October 26, 2024 14:59

apostasie reviewed Oct 28, 2024

View reviewed changes

Shubhranshu153 force-pushed the fix-network-none branch 6 times, most recently from 1a627e3 to e80e165 Compare October 30, 2024 05:40

Shubhranshu153 force-pushed the fix-network-none branch from e80e165 to 516b8c0 Compare October 30, 2024 05:53

Shubhranshu153 force-pushed the fix-network-none branch 2 times, most recently from a2a9820 to ebc6d87 Compare October 30, 2024 09:42

apostasie reviewed Oct 30, 2024

View reviewed changes

cmd/nerdctl/container/container_run_network_linux_test.go Outdated Show resolved Hide resolved

apostasie reviewed Oct 30, 2024

View reviewed changes

cmd/nerdctl/container/container_run_network_linux_test.go Outdated Show resolved Hide resolved

apostasie reviewed Oct 30, 2024

View reviewed changes

go.mod Outdated Show resolved Hide resolved

Shubhranshu153 force-pushed the fix-network-none branch 4 times, most recently from c64c4f7 to cdb4929 Compare October 30, 2024 19:21

Shubhranshu153 force-pushed the fix-network-none branch 2 times, most recently from e2dc511 to eba7437 Compare October 30, 2024 20:36

Shubhranshu153 force-pushed the fix-network-none branch from eba7437 to 3bc581d Compare October 30, 2024 21:22

enable host name and default resolv.conf and hosts file in none network

b477824

Signed-off-by: Shubharanshu Mahapatra <[email protected]>

Shubhranshu153 force-pushed the fix-network-none branch from 3bc581d to b477824 Compare October 30, 2024 23:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable other containers to join network namespace of the none network #3443

Enable other containers to join network namespace of the none network #3443

Shubhranshu153 commented Sep 20, 2024

AkihiroSuda commented Sep 20, 2024

apostasie Sep 20, 2024

Shubhranshu153 Sep 20, 2024

apostasie Sep 21, 2024

Shubhranshu153 Sep 21, 2024 •

edited

Loading

Shubhranshu153 Sep 21, 2024

apostasie Sep 21, 2024

Shubhranshu153 Sep 21, 2024

Shubhranshu153 Oct 26, 2024

apostasie commented Oct 27, 2024

apostasie left a comment

apostasie Oct 28, 2024

apostasie Oct 28, 2024

Shubhranshu153 commented Oct 29, 2024

Shubhranshu153 commented Oct 30, 2024

Shubhranshu153 commented Oct 30, 2024 •

edited

Loading

apostasie commented Oct 30, 2024

Shubhranshu153 commented Oct 30, 2024

Shubhranshu153 commented Oct 31, 2024

apostasie commented Oct 31, 2024

Enable other containers to join network namespace of the none network #3443

Are you sure you want to change the base?

Enable other containers to join network namespace of the none network #3443

Conversation

Shubhranshu153 commented Sep 20, 2024

Fixes: Unable to attach to container network created with none

Description

Describe the results you received and expected

AkihiroSuda commented Sep 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shubhranshu153 Sep 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apostasie commented Oct 27, 2024

apostasie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shubhranshu153 commented Oct 29, 2024

Shubhranshu153 commented Oct 30, 2024

Shubhranshu153 commented Oct 30, 2024 • edited Loading

apostasie commented Oct 30, 2024

Shubhranshu153 commented Oct 30, 2024

Shubhranshu153 commented Oct 31, 2024

apostasie commented Oct 31, 2024

Shubhranshu153 Sep 21, 2024 •

edited

Loading

Shubhranshu153 commented Oct 30, 2024 •

edited

Loading