<nwarn>: Failed to write container stdin - podman exec pipes

[mstinsky]

I want to backup containerized mariadb through streams and pipe stdout to another container which gets the backup through stdin.
Doing the following command:
/bin/podman exec galera mariadb-backup --backup --stream=xbstream | /bin/podman exec -i restic restic backup --stdin --stdin-filename mariadb.xb --tag MariaDB

The backup seems to work fine and I can restore it BUT conmon is logging the following a couple of times while doing the backup: conmon ce75eb3303d4cc55f995 <nwarn>: Failed to write container stdin

As I am doing the backup through stdout/stdin I am a bit worried that something is not working correctly.
But I am not sure how to debug where conmon fails and how to fix it.

Does anyone has a debug idea? :)

conmon --version
conmon version 2.1.10
commit: 3ea3d7f99779af0fcd69ec16c211a7dc3b4efb60

podman --version
podman version 4.9.4-rhel

[haircommander]

try running podman with --log-level info or --log-level error and those warnings should be suppressed

[mstinsky]

I dont want to suppress the warning, I would like to understand why conmon has problems to write the container stdin in the first place.
Running podman with --log-level debug did not provide more information for the warning.

[jnovy]

The error seems to be caused by writing to a local socket in conn_sock.c:563:

static void sock_try_write_to_local_sock(struct remote_sock_s *sock)
{
	struct local_sock_s *local_sock = sock->dest;
	ssize_t w = 0;

	if (!sock->remaining || *(local_sock->fd) < 0)
		return;

	if (local_sock->is_stream) {
		w = write(*(local_sock->fd), sock->buf + sock->off, sock->remaining);
	} else {
		w = sendto(*(local_sock->fd), sock->buf + sock->off, sock->remaining, MSG_DONTWAIT | MSG_NOSIGNAL,
			   (struct sockaddr *)local_sock->addr, sizeof(*(local_sock->addr)));
	}
	if (w < 0) {
		nwarnf("Failed to write %s", local_sock->label);
	} else {
		sock->off += w;
		sock->remaining -= w;
	}
}

Maybe we should use print strerror(errno) in nwarnf() warnings too similarly to pwarn() to not to hide the actual reason why socket write failed? What do you think @haircommander ?

[jnovy]

The error seems to be caused by writing to a local socket in conn_sock.c:563:

static void sock_try_write_to_local_sock(struct remote_sock_s *sock)
{
	struct local_sock_s *local_sock = sock->dest;
	ssize_t w = 0;

	if (!sock->remaining || *(local_sock->fd) < 0)
		return;

	if (local_sock->is_stream) {
		w = write(*(local_sock->fd), sock->buf + sock->off, sock->remaining);
	} else {
		w = sendto(*(local_sock->fd), sock->buf + sock->off, sock->remaining, MSG_DONTWAIT | MSG_NOSIGNAL,
			   (struct sockaddr *)local_sock->addr, sizeof(*(local_sock->addr)));
	}
	if (w < 0) {
		nwarnf("Failed to write %s", local_sock->label);
	} else {
		sock->off += w;
		sock->remaining -= w;
	}
}

Maybe we should use print strerror(errno) in nwarnf() warnings too similarly to pwarn() to not to hide the actual reason why socket write failed? What do you think @haircommander ?

@giuseppe what do you think about the above?

[giuseppe]

printing the errno seems like a good idea to me so we can understand better what is going on

[francoism90]

If any debug information would be useful, please let me know.

[jnovy]

Thank you @francoism90 for offering help! Would you mind re-running your scenario with #554 applied?

[francoism90]

@jnovy That's going to be difficult, but I'll try. I've never done a package (re)build from MicroOS before, so it may take a few tries.

In these cases it would be great if you could checkout a binary or build package somewhere. :)

I'll keep you updated.