diff --git a/docs/apps/lnw/es2k/es2k-linux-networking.md b/docs/apps/lnw/es2k/es2k-linux-networking.md index 1c3db9c..5afa305 100644 --- a/docs/apps/lnw/es2k/es2k-linux-networking.md +++ b/docs/apps/lnw/es2k/es2k-linux-networking.md @@ -290,9 +290,20 @@ Current Linux Networking support for the networking recipe has the following lim - Only OvS bridges are supported. - Configure p4rt-ctl runtime rules before OvS configuration. - Double vlan tag is NOT supported. +- Source port value used in P4RT rules should be VSI ID + 16 +- Delete the route if added manually using `ip route` command +- Bring the port down or remove the IP address before removing the port from OVS bridge or deleting the bridge. - Add all ACC PR's to VSI group 1. - On ACC, firewall needs to be disabled. Otherwise, this service will block encapsulated packets. - systemctl stop firewalld - LAG and ECMP are mutually exclusive. Both can't co-exist in the system configuration at the same time. - LAG configuration done via bonding driver is supported. - The supported modes are active-backup and active-active with 802.3ad (LACP). + +## Additional limitations for Linux Networking V3 + +- After tearing the setup, it's possible that entries are present in `vm_src_ip4_mac_map_table` and + `vm_dst_ip4_mac_map_table` since entries in these tables are learnt from OVS flows in MAC learning phase + and OVS might not issue a callback to delete these unless aging happens. Delete these enties using + `p4rt-ctl del-entry` command +- IPv6 is not supported diff --git a/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md b/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md index 8d3c771..36dc921 100644 --- a/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md +++ b/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md @@ -31,21 +31,34 @@ for more details on this feature. Prerequisites: -- Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` p4 artifacts. Go through `Limitations` specified in `README` and bringup the setup accordingly. -- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +- For Linux Networking V2 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. Modify `load_custom_pkg.sh` with following parameters for linux_networking package: -```text - sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG - sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG - sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` -``` +- For Linux Networking V3 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v3` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +Modify `load_custom_pkg.sh` with following parameters for linux_networking package: + + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/mod_num_pages = 1;/mod_num_pages = 2;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` - Download `IPU_Documentation` TAR file specific to the build and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below ```bash - echo 16 > /sys/class/net/ens802f0/device/sriov_numvfs + echo 8 > /sys/class/net/ens802f0/device/sriov_numvfs ``` Notes about topology: @@ -67,7 +80,11 @@ System under test will have above topology running the networking recipe. Link P ## Creating the topology -Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` p4 program. +- For Linux Networking V2 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` P4 program. + +- For Linux Networking V3 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v3` P4 program. ### Port Mapping @@ -106,13 +123,22 @@ These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC. Once the application is started, set the forwarding pipeline config using P4Runtime Client `p4rt-ctl` set-pipe command -```bash -$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ - $OUTPUT_DIR/p4info.txt -``` +- For Linux Networking V2 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` + +- For Linux Networking V3 -Note: Assuming `fxp-net_linux-networking-v2.pb.bin` and `p4info.txt` -along with other P4 artifacts are created as per the steps mentioned in the previous section. + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v3.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` + +Note: Assuming `pb.bin` and `p4info.txt` along with other P4 artifacts are created as per +the steps mentioned in the previous section. ### Configure VSI Group and add a netdev @@ -205,12 +231,25 @@ Example: - Corresponding port representor VSI value 9 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-27). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" +- For Linux Networking V2 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" + ``` +- For Linux Networking V3 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27/2047,action=linux_networking_control.set_source_port(43)" + ``` + +- Common P4RT rules + +```bash # Create a mapping between overlay VF (VSI-27/source port-43) and ACC port representor (VSI-9) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=43,zero_padding=0,action=linux_networking_control.fwd_to_vsi(25)" @@ -240,7 +279,7 @@ Example: - If a VSI is used as an action, add an offset of 16 to the VSI value ```bash -# Create a source port for a physical port (Phy port-0). Source port action can be any value. +# Create a source port for a physical port (Phy port-0). Source port value should be VSI ID + 16. # For simplicity consider the same value as phy port id. p4rt-ctl add-entry br0 linux_networking_control.rx_source_port \ "vmeta.common.port_id=0,zero_padding=0,action=linux_networking_control.set_source_port(0)" @@ -268,14 +307,26 @@ Example: - Corresponding port representor VSI value 18 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-24). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" +- For Linux Networking V2 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24/2047,action=linux_networking_control.set_source_port(40)" + ``` +- Common P4RT rules -# Create a mapping between overlay VF (VSI-24/source port-40) and ACC port representor (VSI-18) +```bash +# Create a mapping between APF netdev (VSI-24/source port-40) and ACC port representor (VSI-18) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=40,zero_padding=0,action=linux_networking_control.fwd_to_vsi(34)" @@ -301,31 +352,61 @@ For TCAM entry configure LPM LUT table Create a dummy LAG bypass table for all 8 hash indexes -```bash - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" +- For Linux Networking V2 - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" -``` + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" + ``` + +- For Linux Networking V3 + + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=0/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=1/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=2/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=3/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=4/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=5/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=6/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=7/7,action=linux_networking_control.bypass" + ``` ### Create integration bridge and add ports to the bridge