"File Not Found: /data/nginx/certs/ofm_le.cert"

[mcmikemn]

I'm trying to self-host OFS (just http-host), but am getting a "file not found" error regarding the LE cert:

...
Building wheels for collected packages: http_host_lib
Building editable for http_host_lib (pyproject.toml): started
Building editable for http_host_lib (pyproject.toml): finished with status 'done'
Created wheel for http_host_lib: filename=http_host_lib-0.0.0-0.editable-py3-none-any.whl size=2747 sha256=a52f04658f4704e91fbc9344d77c479bd13bb8d2bbe01b43d90c147ecc2a303a
Stored in directory: /tmp/pip-ephem-wheel-cache-q6fubp80/wheels/61/7f/24/856d900ad990328dece0446ab79f0d5e60688ec8d380fbbe37
Successfully built http_host_lib
Installing collected packages: urllib3, pycurl, idna, click, charset-normalizer, certifi, requests, http_host_lib
Successfully installed certifi-2024.8.30 charset-normalizer-3.4.0 click-8.1.7 http_host_lib-0.0.0 idna-3.10 pycurl-7.45.3 requests-2.32.3 urllib3-2.2.3
Running http_host.py sync --force
---
2024-10-09 18:34:47.694153+00:00
Starting sync
Syncing local version files
deployed version planet: 20241002_001001_pt
deployed version monaco: 20241008_231001_pt
Downloading asset fonts
skipping asset: fonts
Downloading asset styles
skipping asset: styles
Downloading asset natural_earth
skipping asset: natural_earth
Downloading sprites
skipping sprite version: ofm_f384
Downloading btrfs: monaco 20241008_231001_pt
file exists, skipping download
Downloading btrfs: monaco 20241008_231001_pt
file exists, skipping download
Downloading btrfs: planet 20241002_001001_pt
file exists, skipping download
Downloading btrfs: planet 20241002_001001_pt
file exists, skipping download
Running auto clean btrfs
keeping runs for planet: ['20241002_001001_pt']
keeping runs for monaco: ['20241008_231001_pt']
Running auto mount
creating fstab
created fstab entry for /mnt/ofm/planet-20241002_001001_pt
created fstab entry for /mnt/ofm/monaco-20241008_231001_pt
running mount -a
Writing nginx config
Traceback (most recent call last):
File "/data/ofm/http_host/bin/http_host.py", line 115, in <module>
cli()
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/data/ofm/http_host/bin/http_host.py", line 105, in sync
full_sync(force)
File "/data/ofm/http_host/bin/http_host_lib/sync.py", line 43, in full_sync
write_nginx_config()
File "/data/ofm/http_host/bin/http_host_lib/nginx.py", line 49, in write_nginx_config
shutil.copyfile(Path('/etc/nginx/ssl/dummy.crt'), le_cert)
File "/usr/lib/python3.10/shutil.py", line 256, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/data/nginx/certs/ofm_le.cert'
Command failed: sudo -S -p '[sudo] password: ' bash -c "/data/ofm/venv/bin/python -u /data/ofm/http_host/bin/http_host.py sync --force"
Error: Traceback (most recent call last):
File "/data/ofm/http_host/bin/http_host.py", line 115, in <module>
cli()
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/data/ofm/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/data/ofm/http_host/bin/http_host.py", line 105, in sync
full_sync(force)
File "/data/ofm/http_host/bin/http_host_lib/sync.py", line 43, in full_sync
write_nginx_config()
File "/data/ofm/http_host/bin/http_host_lib/nginx.py", line 49, in write_nginx_config
shutil.copyfile(Path('/etc/nginx/ssl/dummy.crt'), le_cert)
File "/usr/lib/python3.10/shutil.py", line 256, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/data/nginx/certs/ofm_le.cert'

Here is my .env:

# Leave it empty if you use SSH keys
SSH_PASSWD=

# Direct subdomain, using Let's Encrypt certificates
DOMAIN_LE=ofs.mydomain.com

# Let's Encrypt account email
[email protected]

# Skip the full planet download, useful for testing (true/false)
SKIP_PLANET=false

# --- Let's Encrypt DNS related variables, not needed for self-hosting
DOMAIN_LEDNS=

# --- host list
HTTP_HOST_LIST=

# --- Load Balancer script
TELEGRAM_TOKEN=
TELEGRAM_CHAT_ID=

Do you have any suggestions as to what I'm doing wrong?

(By the way, could DOMAIN_LE include a wildcard, like *.mydomain.com?)

[hyperknot]

Can you tell me which OS version are you using? I'll try to reproduce your error on an identical setup on a new, clean host.

[mcmikemn]

I tried installing from Debian Bookworm to a Debian Bookwork, from Debian Bookworm to Ubuntu 22.04, and from Ubuntu 22.04 to Ubuntu 22.04.

[hyperknot]

The installation definitely should be a single line command. If it is not then it's a bug what needs to be fixed, which I'll work on.

[hyperknot]

@mcmikemn I was able to reproduce your error on Ubuntu 22, I'll be fixing it now.

[hyperknot]

@mcmikemn I've found multiple bugs which I've fixed now. I've also updated the self-hosting docs. Can you have a look at it and see if this works for you?

[mcmikemn]

Thanks for looking into this, @hyperknot. I made fresh ubuntu 22.04 VMs (the workstation I'm installing from and the VM I'm installing to) and started installing OFM from scratch.

This time I think the process got further, but failed at the ACME challenge:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: omf.thewooskeys.com
Type:   unauthorized
Detail: 167.99.23.13: Invalid response from https://omf.thewooskeys.com/.well-known/acme-challenge/CF5SE7t8KeowcIdH-a-dlTndMjNgq9Vhrr4Xj4-J3Ic: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

It looks to me like this is using the HTTP-01 challenge, which fails for me because I'm testing this behind a VPN and I don't want to forward a domain to this VM yet. If I'm correct that this is why the installation is failing, I could try again by installing to a DO droplet or something outside of my VPN, but I'm probably not going to be able to get to that for a while.

[hyperknot]

Great to know! Yes, it's normal that if HTTP-01 is not supported then it won't work. I think the only solution here would be to make a switch SKIP_LETSENCRYPT and which would let you configure your own certificate handling (DNS, etc.)

[hyperknot]

I created this issue to skip Let's Encrypt:
#30

[hyperknot]

Added SKIP_LETSENCRYPT variable just for this use case.