Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement systemd-networkd configuration functions #12

Open
stgraber opened this issue Feb 20, 2025 · 1 comment · May be fixed by #17
Open

Implement systemd-networkd configuration functions #12

stgraber opened this issue Feb 20, 2025 · 1 comment · May be fixed by #17
Assignees

Comments

@stgraber
Copy link
Member

stgraber commented Feb 20, 2025

We need to extend the internal/systemd package with functions related to networkd:

  • GenerateNetworkConfiguration => Clears any existing configuration in /run, turns our network definition struct into networkd configuration files
  • ApplyNetworkConfiguration => Implement a RestartUnit in systemctl.go and then call that for systemd-networkd

My current thinking for the network config is something like this (as YAML):

interfaces:
  - name: san1
    addresses:
     - 10.0.101.10/24
     - fd40:1234:1234:101::10/64
    hwaddr: AA:BB:CC:DD:EE:01
    roles:
     - storage

  - name: san2
    addresses:
     - 10.0.102.10/24
     - fd40:1234:1234:102::10/64
    hwaddr: AA:BB:CC:DD:EE:01
    roles:
     - storage

bonds:
  - name: management
    mode: 802.3ad
    mtu: 9000
    vlan: 100
    addresses:
     - 10.0.100.10/24
     - fd40:1234:1234:100::10/64
    routes:
     - to: 0.0.0.0/0
       via: 10.0.100.1
     - to: ::/0
       via: fd40:1234:1234:100::1
    members:
     - hwaddr: AA:BB:CC:DD:EE:03
     - hwaddr: AA:BB:CC:DD:EE:04
    roles:
     - management
     - instances

vlans:
  - name: uplink
    parent: management
    id: 1234
    mtu: 1500
    roles:
     - ovn-uplink

For a much simpler example:

interfaces:
  - name: management
    addresses:
     - dhcp4
     - slaac
    routes:
     - to: 0.0.0.0/0
       via: dhcp4
     - to: ::/0
       via: slaac
    hwaddr: AA:BB:CC:DD:EE:01
    roles:
     - management
     - instances

Behind the scenes we should:

  • Rename all physical ethernet interfaces to something consistent, say enMAC so enAABBCCDDEE01 for example. We track everything based on MAC so that will be easier than dealing with the usual variety of naming schemes.
  • Every interface, bond or vlan defined above should be internally implemented as a bridge in which we then place either the physical interface, a bond that we created internally or a vlan that we created internally.
  • The bridge itself is what the user names, the intermediate bond and vlan interfaces are named by us and their naming doesn't really matter (other that we should use a consistent prefix so we can easily ignore those).
  • Anything that's not defined in the configuration should be set to link DOWN and generally be ignored.
  • All bridges should have VLAN filtering enabled and be configured to allow all VLAN tags into the bridge
  • If a VLAN id is directly set on an interface or bond, it means that the resulting bridge should have that VLAN id set as its physical/access VLAN rather than the default (0)
  • VLAN interfaces are used when multiple bridges are needed and so one can't just use the vlan field on the parent. The example above show one such case where the management bridge (bond) uses the VLAN 100 as its access VLAN, but there's also VLAN 1234 that we want to split out of that bridge and into its own bridge (in this case to give it to OVN).
  • DHCPv4, DHCPv6 and SLAAC are implemented through special address and route destinations, they can be used for addressing only (no routes) but cannot be used for route-only.
  • Default is for all dynamic addressing (DHCPv4, DHCPv6, SLAAC) to be completely disabled on any managed interface
  • The roles don't directly come into play with networkd generated configuration but they will be used for bootstrapping as a way to select specific interfaces/addresses across multiple machines by asking for whatever interface has a given role to be used rather than having to provide the interface name or address
@stgraber
Copy link
Member Author

For anyone wondering, yes, there are a lot of similarities with netplan :)

We can't quite use netplan directly as we're trying to keep our dependencies very minimal and so having to pull python3 and a bunch of python packages would be a bit problematic (though we still need to untangle that with the OVN packages too).

This also only implements a small subset of what netplan can do and then does a few things that are very specific to our environment, like the fact that we generate bridges for everything and always enable VLAN filtering (which netplan still doesn't support).

We also will be shipping OpenVswitch as part of Incus OS, but we intend to keep things purposefully separate with networkd being used for all the host level networking and then OpenVswitch being used exclusively for OVN. This avoids a potential OVN related issue being able to take down the entire system by corrupting the OVS database.

@gibmat gibmat linked a pull request Mar 12, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging a pull request may close this issue.

2 participants