Permission denied on starting dnsdist on Rocky 9.2

[b1tw0rker]

• [ x] This is not a support question, I have read about opensource and will send support questions to the IRC channel, Github Discussions or the mailing list.
• [ x] I have read and understood the 'out in the open' support policy

• Program: dnsdist
• Issue type: Bug report

Short description

Permission denied on starting dnsdist on Rocky 9.2 using letsencrypt
when starting dnsdist with: systemctl start dnsdist i get these error:

Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:8000000D:system library:file_ctrl:Permission denied:crypto/bio/bss_file.c:297:calling fopen(/etc/letsencrypt/live/somedomain.com/cert.pem, r)
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:10080002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:300:
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:0A080002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:448:
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: Fatal error: An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/somedomain.com/cert.pem

Environment

• Operating system: Rocky Linux 9.2
• Software version: dnsdist 1.8.2
• Software source: PowerDNS repository

Steps to reproduce

Permission denied on starting dnsdist on Rocky 9.2 using letsencrypt
when starting dnsdist with: systemctl start dnsdist i get these error:

Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:8000000D:system library:file_ctrl:Permission denied:crypto/bio/bss_file.c:297:calling fopen(/etc/letsencrypt/live/somedomain.com/cert.pem, r)
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:10080002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:300:
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:0A080002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:448:
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: Fatal error: An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/somedomain.com/cert.pem

1. Implement some certificate on your server

2. configure /etc/dnsdist/dnsdist.conf
addDOHLocal("127.0.0.1:443", {"/etc/letsencrypt/live/somedomain.com/cert.pem"}, {"/etc/letsencrypt/live/somedomain.com/privkey.pem"})

3. systemctl start dnsdist

Expected behaviour

Start the server

Actual behaviour

Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:8000000D:system library:file_ctrl:Permission denied:crypto/bio/bss_file.c:297:calling fopen(/etc/letsencrypt/live/somedomain.com/cert.pem, r)
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:10080002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:300:
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: C05043065E7F0000:error:0A080002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:448:
Oct 17 07:23:42 localhost.localdomain dnsdist[439201]: Fatal error: An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/somedomain.com/cert.pem

Other information

Movinf the cert to /tmp dir does not help:

Oct 17 07:35:54 localhost.localdomain dnsdist[440966]: Configuration '/etc/dnsdist/dnsdist.conf' OK!
Oct 17 07:35:54 localhost.localdomain dnsdist[440966]: Configuration '/etc/dnsdist/dnsdist.conf' OK!
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: dnsdist 1.8.2 comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: Added downstream server 127.0.0.1:5353
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: Listening on 192.168.0.3:53
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: Listening on 192.168.0.3:9443 for TLS
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: C0A082CBAD7F0000:error:80000002:system library:file_ctrl:No such file or directory:crypto/bio/bss_file.c:297:calling fopen(/tmp/cert1.pem, r)
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: C0A082CBAD7F0000:error:10080002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:300:
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: C0A082CBAD7F0000:error:0A080002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:448:
Oct 17 07:35:54 localhost.localdomain dnsdist[440967]: Fatal error: An error occurred while trying to load the TLS server certificate file: /tmp/cert1.pem
Oct 17 07:35:54 localhost.localdomain systemd[1]: dnsdist.service: Main process exited, code=exited, status=1/FAILURE

Curious error: No such file or directory:crypto/bio/bss_file.c:297:calling fopen(/tmp/cert1.pem, r)

cat /tmp/cert1.pem shows the cert like charme.

Running: dnsdist -C /etc/dnsdist/dnsdist.conf on the console brings the server up like charme. only "systemctl start dnsdist" does not work well

[rgacogne]

I'm moving this issue to a discussion since in my humble opinion this is in fact a support question.

The first issue arises because our systemd unit starts dnsdist under an unprivileged user named dnsdist, so this user needs to have access to the certificate and key in /etc/letsencrypt/live/somedomain.com/ and clearly it doesn't. This is actually documented in https://dnsdist.org/guides/dns-over-https.html:

A particular attention should be taken to the permissions of the certificate and key files. Many ACME clients used to get and renew certificates, like CertBot, set permissions assuming that services are started as root, which is no longer true for dnsdist as of 1.5.0. For that particular case, making a copy of the necessary files in the /etc/dnsdist directory is advised, using for example CertBot’s --deploy-hook feature to copy the files with the right permissions after a renewal.

Moving the certificate and key to /tmp does not help because our systemd unit sandboxes dnsdist, in particular is enables PrivateTmp 1 which means that the dnsdist process does not have access to the system-wide /tmp folder.

[b1tw0rker]

Got it. And you are right. Moving cert to: /etc/dnstdist does the trick. (looks like chown privkey to dnsdist user is mandatory, if you want not change permissons from rw----)

Thanks for the help.