From 8301854d1550fc09717dabd250ccf1d3c4bbfbce Mon Sep 17 00:00:00 2001 From: nupuruttarwar Date: Thu, 4 Apr 2024 10:56:22 -0700 Subject: [PATCH] Update documentation for Linux networking V3 Signed-off-by: nupuruttarwar --- docs/apps/lnw/es2k/es2k-linux-networking.md | 11 ++ docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md | 170 ++++++++++++++----- docs/apps/lnw/es2k/es2k-lnw-underlay-ecmp.md | 166 +++++++++++++----- docs/apps/lnw/es2k/es2k-lnw-underlay-frr.md | 165 +++++++++++++----- docs/apps/lnw/es2k/es2k-lnw-underlay-lag.md | 98 ++++++++--- 5 files changed, 460 insertions(+), 150 deletions(-) diff --git a/docs/apps/lnw/es2k/es2k-linux-networking.md b/docs/apps/lnw/es2k/es2k-linux-networking.md index 1c3db9c..5afa305 100644 --- a/docs/apps/lnw/es2k/es2k-linux-networking.md +++ b/docs/apps/lnw/es2k/es2k-linux-networking.md @@ -290,9 +290,20 @@ Current Linux Networking support for the networking recipe has the following lim - Only OvS bridges are supported. - Configure p4rt-ctl runtime rules before OvS configuration. - Double vlan tag is NOT supported. +- Source port value used in P4RT rules should be VSI ID + 16 +- Delete the route if added manually using `ip route` command +- Bring the port down or remove the IP address before removing the port from OVS bridge or deleting the bridge. - Add all ACC PR's to VSI group 1. - On ACC, firewall needs to be disabled. Otherwise, this service will block encapsulated packets. - systemctl stop firewalld - LAG and ECMP are mutually exclusive. Both can't co-exist in the system configuration at the same time. - LAG configuration done via bonding driver is supported. - The supported modes are active-backup and active-active with 802.3ad (LACP). + +## Additional limitations for Linux Networking V3 + +- After tearing the setup, it's possible that entries are present in `vm_src_ip4_mac_map_table` and + `vm_dst_ip4_mac_map_table` since entries in these tables are learnt from OVS flows in MAC learning phase + and OVS might not issue a callback to delete these unless aging happens. Delete these enties using + `p4rt-ctl del-entry` command +- IPv6 is not supported diff --git a/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md b/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md index 8d3c771..8949856 100644 --- a/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md +++ b/docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md @@ -31,21 +31,33 @@ for more details on this feature. Prerequisites: -- Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` p4 artifacts. Go through `Limitations` specified in `README` and bringup the setup accordingly. -- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +- For Linux Networking V2 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. Modify `load_custom_pkg.sh` with following parameters for linux_networking package: -```text - sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG - sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG - sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` -``` +- For Linux Networking V3 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v3` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +Modify `load_custom_pkg.sh` with following parameters for linux_networking package: + + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 28;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/mod_num_pages = 1;/mod_num_pages = 2;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` - Download `IPU_Documentation` TAR file specific to the build and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below ```bash - echo 16 > /sys/class/net/ens802f0/device/sriov_numvfs + echo 8 > /sys/class/net/ens802f0/device/sriov_numvfs ``` Notes about topology: @@ -67,7 +79,11 @@ System under test will have above topology running the networking recipe. Link P ## Creating the topology -Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` p4 program. +- For Linux Networking V2 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` P4 program. + +- For Linux Networking V3 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v3` P4 program. ### Port Mapping @@ -106,13 +122,22 @@ These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC. Once the application is started, set the forwarding pipeline config using P4Runtime Client `p4rt-ctl` set-pipe command -```bash -$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ - $OUTPUT_DIR/p4info.txt -``` +- For Linux Networking V2 -Note: Assuming `fxp-net_linux-networking-v2.pb.bin` and `p4info.txt` -along with other P4 artifacts are created as per the steps mentioned in the previous section. + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` + +- For Linux Networking V3 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v3.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` + +Note: Assuming `pb.bin` and `p4info.txt` along with other P4 artifacts are created as per +the steps mentioned in the previous section. ### Configure VSI Group and add a netdev @@ -205,12 +230,25 @@ Example: - Corresponding port representor VSI value 9 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-27). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" +- For Linux Networking V2 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" + ``` +- For Linux Networking V3 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27/2047,action=linux_networking_control.set_source_port(43)" + ``` + +- Common P4RT rules + +```bash # Create a mapping between overlay VF (VSI-27/source port-43) and ACC port representor (VSI-9) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=43,zero_padding=0,action=linux_networking_control.fwd_to_vsi(25)" @@ -240,7 +278,7 @@ Example: - If a VSI is used as an action, add an offset of 16 to the VSI value ```bash -# Create a source port for a physical port (Phy port-0). Source port action can be any value. +# Create a source port for a physical port (Phy port-0). Source port value should be VSI ID + 16. # For simplicity consider the same value as phy port id. p4rt-ctl add-entry br0 linux_networking_control.rx_source_port \ "vmeta.common.port_id=0,zero_padding=0,action=linux_networking_control.set_source_port(0)" @@ -268,14 +306,26 @@ Example: - Corresponding port representor VSI value 18 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-24). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" +- For Linux Networking V2 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24/2047,action=linux_networking_control.set_source_port(40)" + ``` +- Common P4RT rules -# Create a mapping between overlay VF (VSI-24/source port-40) and ACC port representor (VSI-18) +```bash +# Create a mapping between APF netdev (VSI-24/source port-40) and ACC port representor (VSI-18) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=40,zero_padding=0,action=linux_networking_control.fwd_to_vsi(34)" @@ -301,31 +351,61 @@ For TCAM entry configure LPM LUT table Create a dummy LAG bypass table for all 8 hash indexes -```bash - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" +- For Linux Networking V2 - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" -``` + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" + ``` + +- For Linux Networking V3 + + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=0/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=1/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=2/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=3/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=4/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=5/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=6/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=7/7,action=linux_networking_control.bypass" + ``` ### Create integration bridge and add ports to the bridge diff --git a/docs/apps/lnw/es2k/es2k-lnw-underlay-ecmp.md b/docs/apps/lnw/es2k/es2k-lnw-underlay-ecmp.md index 10ce9d6..6e52058 100644 --- a/docs/apps/lnw/es2k/es2k-lnw-underlay-ecmp.md +++ b/docs/apps/lnw/es2k/es2k-lnw-underlay-ecmp.md @@ -31,15 +31,29 @@ for more details on this feature. Prerequisites: -- Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` p4 artifacts. Go through `Limitations` specified in `README` and bring up the setup accordingly. -- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +- For Linux Networking V2 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. Modify `load_custom_pkg.sh` with following parameters for linux_networking package: -```text - sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG - sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG - sed -i 's/acc_apf = 4;/acc_apf = 8;/g' $CP_INIT_CFG -``` + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` + +- For Linux Networking V3 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v3` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +Modify `load_custom_pkg.sh` with following parameters for linux_networking package: + + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 28;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/mod_num_pages = 1;/mod_num_pages = 2;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` - Download `IPU_Documentation` TAR file specific to the build and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below @@ -68,7 +82,11 @@ System under test will have above topology running the networking recipe. Link P ## Creating the topology -Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` p4 program. +- For Linux Networking V2 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` P4 program. + +- For Linux Networking V3 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v3` P4 program. ### Port mapping @@ -101,12 +119,21 @@ These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC. Once the application is started, set the forwarding pipeline config using P4Runtime Client `p4rt-ctl` set-pipe command -```bash -$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ - $OUTPUT_DIR/p4info.txt -``` +- For Linux Networking V2 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` -Note: Assumes that `fxp-net_linux-networking-v2.pb.bin`, `p4info.txt` and other P4 artifacts, are created by following the steps in the previous section. +- For Linux Networking V3 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v3.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` + +Note: Assumes that `pb.bin`, `p4info.txt` and other P4 artifacts, are created by following the steps in the previous section. ### Configure VSI Group and add a netdev @@ -198,11 +225,25 @@ Example: - Overlay VF1 has a VSI value 27, its corresponding port representor has VSI value 9 - If a VSI is used as an action, add an offset of 16 to the VSI value +- For Linux Networking V2 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27/2047,action=linux_networking_control.set_source_port(43)" + ``` + +- Common P4RT rules + ```bash -# Create a source port for an overlay VF (VSI-27). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" # Create a mapping between overlay VF (VSI-27/source port-43) and ACC port representor (VSI-9) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ @@ -259,14 +300,27 @@ Example: - APF netdev 1 on HOST has a VSI value 24, its corresponding port representor has VSI value 18 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-24). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" +- For Linux Networking V2 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24/2047,action=linux_networking_control.set_source_port(40)" + ``` +- Common P4RT rules -# Create a mapping between overlay VF (VSI-24/source port-40) and ACC port representor (VSI-18) +```bash + +# Create a mapping between APF netdev (VSI-24/source port-40) and ACC port representor (VSI-18) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=40,zero_padding=0,action=linux_networking_control.fwd_to_vsi(34)" @@ -292,31 +346,61 @@ For TCAM entry configure LPM LUT table Create a dummy LAG bypass table for all 8 hash indexes -```bash - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" +- For Linux Networking V2 - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" -``` + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" + ``` + +- For Linux Networking V3 + + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=0/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=1/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=2/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=3/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=4/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=5/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=6/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=7/7,action=linux_networking_control.bypass" + ``` ### Create integration bridge and add ports to the bridge diff --git a/docs/apps/lnw/es2k/es2k-lnw-underlay-frr.md b/docs/apps/lnw/es2k/es2k-lnw-underlay-frr.md index 7c82c77..33e5f83 100644 --- a/docs/apps/lnw/es2k/es2k-lnw-underlay-frr.md +++ b/docs/apps/lnw/es2k/es2k-lnw-underlay-frr.md @@ -31,15 +31,29 @@ for more details on this feature. Prerequisites: -- Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` p4 artifacts. Go through `Limitations` specified in `README` and bring up the setup accordingly. -- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +- For Linux Networking V2 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. Modify `load_custom_pkg.sh` with following parameters for linux_networking package: -```text - sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG - sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG - sed -i 's/acc_apf = 4;/acc_apf = 8;/g' $CP_INIT_CFG -``` + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` + +- For Linux Networking V3 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v3` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +Modify `load_custom_pkg.sh` with following parameters for linux_networking package: + + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 28;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/mod_num_pages = 1;/mod_num_pages = 2;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` - Download `IPU_Documentation` TAR file specific to the build and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below @@ -68,7 +82,11 @@ System under test will have above topology running the networking recipe. Link P ## Creating the topology -Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` p4 program. +- For Linux Networking V2 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` P4 program. + +- For Linux Networking V3 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v3` P4 program. ### Port mapping @@ -100,12 +118,20 @@ These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC. Once the application is started, set the forwarding pipeline config using P4Runtime Client `p4rt-ctl` set-pipe command -```bash -$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ - $OUTPUT_DIR/p4info.txt -``` +- For Linux Networking V2 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` -Note: Assumes that `fxp-net_linux-networking-v2.pb.bin`, `p4info.txt` and other P4 artifacts, are created by following the steps in the previous section. +- For Linux Networking V3 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v3.pb.bin \ + $OUTPUT_DIR/p4info.txt + +Note: Assumes that `pb.bin`, `p4info.txt` and other P4 artifacts, are created by following the steps in the previous section. ### Configure VSI Group and add a netdev @@ -197,11 +223,25 @@ Example: - Overlay VF1 has a VSI value 27, its corresponding port representor has VSI value 9 - If a VSI is used as an action, add an offset of 16 to the VSI value +- For Linux Networking V2 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27/2047,action=linux_networking_control.set_source_port(43)" + ``` + +- Common P4RT rules + ```bash -# Create a source port for an overlay VF (VSI-27). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" # Create a mapping between overlay VF (VSI-27/source port-43) and ACC port representor (VSI-9) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ @@ -258,14 +298,27 @@ Example: - APF netdev 1 on HOST has a VSI value 24, its corresponding port representor has VSI value 18 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-24). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" +- For Linux Networking V2 + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24/2047,action=linux_networking_control.set_source_port(40)" + ``` + +- Common P4RT rules -# Create a mapping between overlay VF (VSI-24/source port-40) and ACC port representor (VSI-18) +```bash + +# Create a mapping between APF netdev (VSI-24/source port-40) and ACC port representor (VSI-18) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=40,zero_padding=0,action=linux_networking_control.fwd_to_vsi(34)" @@ -291,31 +344,61 @@ For TCAM entry configure LPM LUT table Create a dummy LAG bypass table for all 8 hash indexes -```bash - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" +- For Linux Networking V2 - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass" - p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ - "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" -``` + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass" + ``` + +- For Linux Networking V3 + + ```bash + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=0/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=1/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=2/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=3/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=4/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=5/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=6/7,action=linux_networking_control.bypass" + + p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \ + "user_meta.cmeta.lag_group_id=0/255,hash=7/7,action=linux_networking_control.bypass" + ``` ### Create integration bridge and add ports to the bridge diff --git a/docs/apps/lnw/es2k/es2k-lnw-underlay-lag.md b/docs/apps/lnw/es2k/es2k-lnw-underlay-lag.md index e6b1994..504985d 100644 --- a/docs/apps/lnw/es2k/es2k-lnw-underlay-lag.md +++ b/docs/apps/lnw/es2k/es2k-lnw-underlay-lag.md @@ -31,16 +31,28 @@ for more details on this feature. Prerequisites: -- Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` p4 artifacts. Go through `Limitations` specified in `README` and bring up the setup accordingly. -- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +- For Linux Networking V2 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. Modify `load_custom_pkg.sh` with following parameters for linux_networking package: -```text - sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG - sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG - sed -i 's/acc_apf = 4;/acc_apf = 8;/g' $CP_INIT_CFG - sed -i 's/allow_change_mac_address = false;/allow_change_mac_address = true;/g' $CP_INIT_CFG -``` + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG + ``` + +- For Linux Networking V3 + - Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v3` P4 artifacts. Go through `Limitations` specified in `README` and bring up the config accordingly. + + - Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package. +Modify `load_custom_pkg.sh` with following parameters for linux_networking package: + + ```text + sed -i 's/sem_num_pages = 1;/sem_num_pages = 28;/g' $CP_INIT_CFG + sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG + sed -i 's/allow_change_mac_address = false;/allow_change_mac_address = true;/g' $CP_INIT_CFG + sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG - Download `IPU_Documentation` TAR file specific to the build and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below @@ -72,7 +84,11 @@ System under test will have above topology running the networking recipe. Link P ## Creating the topology -Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` p4 program. +- For Linux Networking V2 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` P4 program. + +- For Linux Networking V3 + Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v3` P4 program. ### Port mapping @@ -104,10 +120,19 @@ These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC. Once the application is started, set the forwarding pipeline config using P4Runtime Client `p4rt-ctl` set-pipe command -```bash -$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ - $OUTPUT_DIR/p4info.txt -``` +- For Linux Networking V2 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` + +- For Linux Networking V3 + + ```bash + $P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v3.pb.bin \ + $OUTPUT_DIR/p4info.txt + ``` Note: Assumes that `fxp-net_linux-networking-v2.pb.bin`, `p4info.txt` and other P4 artifacts, are created by following the steps in the previous section. @@ -201,11 +226,25 @@ Example: - Overlay VF1 has a VSI value 27, its corresponding port representor has VSI value 9 - If a VSI is used as an action, add an offset of 16 to the VSI value +- For Linux Networking V2 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=27/2047,action=linux_networking_control.set_source_port(43)" + ``` + +- Common P4RT rules + ```bash -# Create a source port for an overlay VF (VSI-27). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)" # Create a mapping between overlay VF (VSI-27/source port-43) and ACC port representor (VSI-9) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ @@ -262,14 +301,27 @@ Example: - APF netdev 1 on HOST has a VSI value 24, its corresponding port representor has VSI value 18 - If a VSI is used as an action, add an offset of 16 to the VSI value -```bash -# Create a source port for an overlay VF (VSI-24). Source port action can be any value. -# For simplicity add 16 to VSI ID. - p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ - "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" +- For Linux Networking V2 + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)" + ``` + +- For Linux Networking V3 + + ```bash + # Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16. + p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \ + "vmeta.common.vsi=24/2047,action=linux_networking_control.set_source_port(40)" + ``` + +- Common P4RT rules + +```bash -# Create a mapping between overlay VF (VSI-24/source port-40) and ACC port representor (VSI-18) +# Create a mapping between APF netdev (VSI-24/source port-40) and ACC port representor (VSI-18) p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \ "user_meta.cmeta.source_port=40,zero_padding=0,action=linux_networking_control.fwd_to_vsi(34)"