os-caddy: caddy lost dns.providers.route53 on last update

[permezel]

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

• [x ] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
• [ x] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
• [ x] The title contains the plugin to which this issue belongs

Describe the bug
Caddy stopped running on recent update. If I try to apply it, it complains about a missing dns.providers.route53

To Reproduce
Steps to reproduce the behavior:

1. Go to 'services/caddy', a previously functioning reverse proxy utilising route53
2. Try to start it
3. Observe that it fails
4. See error

Expected behavior

I expect the caddy service to start and run.
I expect all my reverse proxies to be available and functioning.

Relevant log files
Error: adapting config using caddyfile: parsing caddyfile tokens for 'tls': getting module named 'dns.providers.route53': module not registered: dns.providers.route53, at /usr/local/etc/caddy/Caddyfile:41

Additional context
I read the change log and there is buried in there some comment about marking route53 optional and something about caddy-add-package being run from the command line. caddy add-package is available from the command line, but does not function to add route53.

Environment
OPNsense 24.7.12_4-amd64

[permezel]

# caddy add-package dns.providers.route53
2025/02/07 06:35:38.485 INFO    this executable will be replaced        {"path": "/usr/local/bin/caddy"}
2025/02/07 06:35:38.485 INFO    requesting build        {"os": "freebsd", "arch": "amd64", "packages": ["github.com/caddyserver/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/mholt/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/mholt/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/mholt/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]", "dns.providers.route53", "github.com/caddy-dns/[email protected]", "github.com/caddy-dns/[email protected]"]}
Error: download failed: download failed: HTTP 400: dns.providers.route53 is not a registered Caddy module package path (id=54f72a99-f4ad-4ec0-92fa-8806ae685a52)

[permezel]

I just upgraded to 25.1, and caddy was still (as expected) non-functioning, but I decided to try:

caddy add-package github.com/caddy-dns/route53

and that appears to have replaced my /usr/local/bin/caddy binary with one which did start, and appears to be working.

The new binary seems to have all the modules:

# caddy list-modules
admin.api.load
admin.api.metrics
admin.api.pki
admin.api.reverse_proxy
caddy.adapters.caddyfile
caddy.config_loaders.http
caddy.filesystems
caddy.listeners.http_redirect
caddy.listeners.proxy_protocol
caddy.listeners.tls
caddy.logging.cores.mock
caddy.logging.encoders.append
caddy.logging.encoders.console
caddy.logging.encoders.filter
caddy.logging.encoders.filter.cookie
caddy.logging.encoders.filter.delete
caddy.logging.encoders.filter.hash
caddy.logging.encoders.filter.ip_mask
caddy.logging.encoders.filter.query
caddy.logging.encoders.filter.regexp
caddy.logging.encoders.filter.rename
caddy.logging.encoders.filter.replace
caddy.logging.encoders.json
caddy.logging.writers.discard
caddy.logging.writers.file
caddy.logging.writers.net
caddy.logging.writers.stderr
caddy.logging.writers.stdout
caddy.storage.file_system
events
http
http.authentication.hashes.bcrypt
http.authentication.providers.http_basic
http.encoders.gzip
http.encoders.zstd
http.handlers.acme_server
http.handlers.authentication
http.handlers.copy_response
http.handlers.copy_response_headers
http.handlers.encode
http.handlers.error
http.handlers.file_server
http.handlers.headers
http.handlers.intercept
http.handlers.invoke
http.handlers.log_append
http.handlers.map
http.handlers.metrics
http.handlers.push
http.handlers.request_body
http.handlers.reverse_proxy
http.handlers.rewrite
http.handlers.static_response
http.handlers.subroute
http.handlers.templates
http.handlers.tracing
http.handlers.vars
http.ip_sources.static
http.matchers.client_ip
http.matchers.expression
http.matchers.file
http.matchers.header
http.matchers.header_regexp
http.matchers.host
http.matchers.method
http.matchers.not
http.matchers.path
http.matchers.path_regexp
http.matchers.protocol
http.matchers.query
http.matchers.remote_ip
http.matchers.tls
http.matchers.vars
http.matchers.vars_regexp
http.precompressed.br
http.precompressed.gzip
http.precompressed.zstd
http.reverse_proxy.selection_policies.client_ip_hash
http.reverse_proxy.selection_policies.cookie
http.reverse_proxy.selection_policies.first
http.reverse_proxy.selection_policies.header
http.reverse_proxy.selection_policies.ip_hash
http.reverse_proxy.selection_policies.least_conn
http.reverse_proxy.selection_policies.query
http.reverse_proxy.selection_policies.random
http.reverse_proxy.selection_policies.random_choose
http.reverse_proxy.selection_policies.round_robin
http.reverse_proxy.selection_policies.uri_hash
http.reverse_proxy.selection_policies.weighted_round_robin
http.reverse_proxy.transport.fastcgi
http.reverse_proxy.transport.http
http.reverse_proxy.upstreams.a
http.reverse_proxy.upstreams.multi
http.reverse_proxy.upstreams.srv
pki
tls
tls.ca_pool.source.file
tls.ca_pool.source.http
tls.ca_pool.source.inline
tls.ca_pool.source.pki_intermediate
tls.ca_pool.source.pki_root
tls.ca_pool.source.storage
tls.certificates.automate
tls.certificates.load_files
tls.certificates.load_folders
tls.certificates.load_pem
tls.certificates.load_storage
tls.client_auth.verifier.leaf
tls.get_certificate.http
tls.get_certificate.tailscale
tls.handshake_match.local_ip
tls.handshake_match.remote_ip
tls.handshake_match.sni
tls.handshake_match.sni_regexp
tls.issuance.acme
tls.issuance.internal
tls.issuance.zerossl
tls.leaf_cert_loader.file
tls.leaf_cert_loader.folder
tls.leaf_cert_loader.pem
tls.leaf_cert_loader.storage
tls.permission.http
tls.stek.distributed
tls.stek.standard

  Standard modules: 124

caddy.listeners.layer4
dns.providers.acmedns
dns.providers.acmeproxy
dns.providers.azure
dns.providers.bunny
dns.providers.cloudflare
dns.providers.desec
dns.providers.directadmin
dns.providers.dnsmadeeasy
dns.providers.duckdns
dns.providers.gandi
dns.providers.hetzner
dns.providers.hexonet
dns.providers.infomaniak
dns.providers.inwx
dns.providers.ionos
dns.providers.linode
dns.providers.mailinabox
dns.providers.namecheap
dns.providers.namedotcom
dns.providers.netcup
dns.providers.ovh
dns.providers.porkbun
dns.providers.powerdns
dns.providers.rfc2136
dns.providers.route53
dns.providers.scaleway
dns.providers.vultr
dynamic_dns
dynamic_dns.ip_sources.interface
dynamic_dns.ip_sources.simple_http
dynamic_dns.ip_sources.static
dynamic_dns.ip_sources.upnp
http.handlers.rate_limit
http.reverse_proxy.transport.http_ntlm
layer4
layer4.handlers.echo
layer4.handlers.proxy
layer4.handlers.proxy_protocol
layer4.handlers.socks5
layer4.handlers.subroute
layer4.handlers.tee
layer4.handlers.throttle
layer4.handlers.tls
layer4.matchers.clock
layer4.matchers.dns
layer4.matchers.http
layer4.matchers.local_ip
layer4.matchers.not
layer4.matchers.openvpn
layer4.matchers.postgres
layer4.matchers.proxy_protocol
layer4.matchers.quic
layer4.matchers.rdp
layer4.matchers.regexp
layer4.matchers.remote_ip
layer4.matchers.socks4
layer4.matchers.socks5
layer4.matchers.ssh
layer4.matchers.tls
layer4.matchers.winbox
layer4.matchers.wireguard
layer4.matchers.xmpp
layer4.proxy.selection_policies.first
layer4.proxy.selection_policies.ip_hash
layer4.proxy.selection_policies.least_conn
layer4.proxy.selection_policies.random
layer4.proxy.selection_policies.random_choose
layer4.proxy.selection_policies.round_robin
tls.handshake_match.alpn

  Non-standard modules: 70

  Unknown modules: 0

For now, I suppose, it might be OK to use this, but it seems a bit wild. I'd prefer just the 'approved' caddy plus the few optional modules I need.

Or, perhaps it is time to abandon caddy and go back to the other reverse proxy approach I was using which had problems causing me to try caddy again.

[Monviech]

Choices had to be made to decrease large dependencies getting pulled into the binary.

What you did was correct.

#4437

[permezel]

Will I have to do it again next update? Sent with Proton Mail secure email.

…

On Friday, 7 February 2025 at 17:39, Monviech ***@***.***> wrote: Choices had to be made to decrease large dependencies getting pulled into the binary. What you did was correct. #4437 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[Monviech]

When os-caddy (the plugin glue in this plugin repository) is updated, no.

When caddy itself (the binary) is updated, yes. e.g. version bump.
Example: opnsense/ports@643196a

There sadly is no good solution here. The requirement to compile everything in, and not having a plugin framework that be external from caddy itself, limits flexibility. It's an issue I cannot solve.

Its an inherent golang issue:

https://pkg.go.dev/plugin