Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Fresh install on FreeBSD 14.1: jails cannot access internet via the host #703

Open
waseigo opened this issue Jun 4, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@waseigo
Copy link

waseigo commented Jun 4, 2024

[MANDATORY] Describe the bug [MANDATORY]
Bastille was running just fine on a FreeBSD 14.0 host. I upgraded the host to 14.1 and all jails lost access to the internet.
To verify that this was not something specific to that one host, I tried again with a fresh install of bastille on a fresh 14.1 install on a VM. Same behavior.

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)

0.10.20231125
14.1-RELEASE
14.1-RELEASE
14.1-RELEASE

[MANDATORY] How did you install bastille? (port/pkg/git)

pkg

[optional] Steps to reproduce?

  1. Install FreeBSD 14.1 or upgrade existing host to 14.1.
  2. Install bastille from pkg/latest.
  3. Follow bastille's Getting Started guide, including pf.conf.
  4. Create new jail.
  5. Attempt to pkg install ....
  6. See below. Host resolution fails.
root@gandalf:~ # bastille list
   JID  IP Address      Hostname                      Path
     2  172.16.33.90    minio                         /usr/local/bastille/jails/minio/root
root@gandalf:~ # bastille pkg minio install -y htop minio
[minio]:
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly, please wait...
pkg: Error fetching https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly/Latest/pkg.txz: Host does not resolve
Address resolution failed for https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly.
Consider changing PACKAGESITE.

The problem persists after a reboot (just to make sure that eveyrthing that's supposed to run also runs at boot.

user@gandalf:~ $ doas bastille pkg 
Usage: bastille pkg [-H|--host] TARGET command [args]
user@gandalf:~ $ doas bastille pkg minio install minio
[minio]:
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly, please wait...
pkg: Error fetching https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly/Latest/pkg.txz: Host does not resolve
Address resolution failed for https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly.
Consider changing PACKAGESITE.

Failed to apply on some jails, please check logs

[optional] Expected behavior

Host resolution succeeds.

[optional] Other information

uname -a

root@gandalf:~ # uname -a 
FreeBSD gandalf 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64

/etc/rc.conf

root@gandalf:~ # cat /etc/rc.conf
hostname="gandalf"
ifconfig_vtnet0="DHCP"
local_unbound_enable="YES"
sshd_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
powerd_enable="YES"
moused_nondefault_enable="NO"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
bastille_enable="YES"
cloned_interfaces="lo1"
ifconfig_lo1_name="bastille0"
pf_enable="YES"

ifconfig

root@gandalf:~ # ifconfig
vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
	ether 52:54:00:07:a5:32
	inet 192.168.88.6 netmask 0xffffff00 broadcast 192.168.88.255
	media: Ethernet autoselect (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet 127.0.0.1 netmask 0xff000000
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bastille0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet 172.16.33.90 netmask 0xffffff00
	inet6 fe80::1%bastille0 prefixlen 64 scopeid 0x3
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

/etc/pf.conf

root@gandalf:~ # cat /etc/pf.conf 
ext_if="vtnet0"

set block-policy return
scrub in on $ext_if all fragment reassemble
set skip on lo

table <jails> persist
nat on $ext_if from <jails> to any -> ($ext_if:0)
rdr-anchor "rdr/*"

block in all
pass out quick keep state
antispoof for $ext_if inet
pass in inet proto tcp from any to any port ssh flags S/SA keep state
@waseigo waseigo added the bug Something isn't working label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant