ipdk-io · ffoulkes · Sep 26, 2023 · Sep 26, 2023
@@ -38,7 +38,7 @@ System under test will have above topology running the networking recipe. Link P
 ## Create P4 artifacts and start Infrap4d process
 
 - Use Linux networking p4 program present in the directory `/opt/p4/p4sde/share/mev_reference_p4_files/linux_networking` for this scenario.
-- Refer to [Running Infrap4d on Intel IPU E2100](/guides/es2k/running-infrap4d.md) for compiling `P4 artifacts`, `bringing up ACC` and running `infrap4d` on ACC.
+- See [Running Infrap4d on Intel IPU E2100](/guides/es2k/running-infrap4d.md) for compiling `P4 artifacts`, `bringing up ACC` and running `infrap4d` on ACC.
 
 ## Creating the topology
 
@@ -50,7 +50,8 @@ Once the application is started, set the forwarding pipeline config using
 P4Runtime Client `p4rt-ctl` set-pipe command
 
 ```bash
-$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/linux_networking.pb.bin $OUTPUT_DIR/linux_networking.p4info.txt
+$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/linux_networking.pb.bin \
+    $OUTPUT_DIR/linux_networking.p4info.txt
 ```
 
 Note: Assuming `linux_networking.pb.bin` and `linux_networking.p4info.txt` along with other P4 artifacts are created as per the steps mentioned in previous section.
@@ -63,7 +64,8 @@ Use one of the IPDF netdevs on ACC to receive all control packets from overlay V
 # SEM_DIRECT_MAP_PGEN_CTRL: LSB 11-bit is for vsi which need to map into vsig
 devmem 0x20292002a0 64 0x8000050000000008
 
-# SEM_DIRECT_MAP_PGEN_DATA_VSI_GROUP : This will set vsi (set in SEM_DIRECT_MAP_PGEN_CTRL register LSB) into VSIG-3
+# SEM_DIRECT_MAP_PGEN_DATA_VSI_GROUP : This will set vsi
+# (set in SEM_DIRECT_MAP_PGEN_CTRL register LSB) into VSIG-3
 devmem 0x2029200388 64 0x3
 
 # SEM_DIRECT_MAP_PGEN_CTRL: LSB 11-bit is for vsi which need to map into vsig
@@ -89,8 +91,8 @@ ip addr add 99.0.0.2/24 dev <Netdev connected to VF2>
 ifconfig <Netdev connected to VF> up
 ```
 
-Option 2: If we are unable to spawn VM's on top of the VF's, for this use case we can also leverage kernel network namespaces.
-Move each VF to a network namespace and assign IP addresses
+Option 2: If we are unable to spawn VM's on top of the VF's, we can leverage kernel network namespaces.
+Move each VF to a network namespace and assign IP addresses:
 
 ```bash
 ip netns add VM0
@@ -111,25 +113,26 @@ Legacy OvS is used as a control plane for source MAC learning of overlay VM's. O
 ```bash
 export RUN_OVS=/tmp
 rm -rf $RUN_OVS/etc/openvswitch
-rm -rf $RUN_OVS/var/run/openvswitch 
+rm -rf $RUN_OVS/var/run/openvswitch
 mkdir -p $RUN_OVS/etc/openvswitch/
 mkdir -p $RUN_OVS/var/run/openvswitch
-
-ovsdb-tool create $RUN_OVS/etc/openvswitch/conf.db /opt/p4/p4-cp-nws/share/openvswitch/vswitch.ovsschema 
+
+ovsdb-tool create $RUN_OVS/etc/openvswitch/conf.db \
+    /opt/p4/p4-cp-nws/share/openvswitch/vswitch.ovsschema
 
 ovsdb-server $RUN_OVS/etc/openvswitch/conf.db \
     --remote=punix:$RUN_OVS/var/run/openvswitch/db.sock \
     --remote=db:Open_vSwitch,Open_vSwitch,manager_options \
     --pidfile=$RUN_OVS/var/run/openvswitch/ovsdb-server.pid \
     --unixctl=$RUN_OVS/var/run/openvswitch/ovsdb-server.ctl \
-    --detach 
+    --detach
 
 ovs-vswitchd --detach \
     --pidfile=$RUN_OVS/var/run/openvswitch/ovs-vswitchd.pid \
     --no-chdir unix:$RUN_OVS/var/run/openvswitch/db.sock \
     --unixctl=$RUN_OVS/var/run/openvswitch/ovs-vswitchd.ctl \
     --mlockall \
-    --log-file=/tmp/ovs-vswitchd.log 
+    --log-file=/tmp/ovs-vswitchd.log
 
 alias ovs-vsctl="ovs-vsctl --db unix:$RUN_OVS/var/run/openvswitch/db.sock"
 ovs-vsctl set Open_vSwitch . other_config:n-revalidator-threads=1
@@ -149,10 +152,11 @@ ifconfig vlan1 up
 ifconfig vlan2 up
 ```
 
-Note: Here the assumption is, we have created 2 overlay VM's and creating 2 port representers for those VM's.
-Port representer should always be in the format: `lowercase string 'vlan'+'vlanID'`
+Note: Here the assumption is, we have created 2 overlay VM's and creating
+2 port representers for those VM's.  Port representer should always be in
+the format: `lowercase string 'vlan'+'vlanID'`
 
-### Create intergation bridge and add ports to the bridge
+### Create integration bridge and add ports to the bridge
 
 Create OvS bridge, VxLAN tunnel and assign ports to the bridge.
 
@@ -165,7 +169,8 @@ ovs-vsctl add-port br-int vlan2
 ifconfig vlan1 up
 ifconfig vlan2 up
 
-ovs-vsctl add-port br-int vxlan1 -- set interface vxlan1 type=vxlan options:local_ip=30.1.1.1 options:remote_ip=40.1.1.1 options:dst_port=4789
+ovs-vsctl add-port br-int vxlan1 -- set interface vxlan1 type=vxlan \
+    options:local_ip=30.1.1.1 options:remote_ip=40.1.1.1 options:dst_port=4789
 ```
 
 Note: Here we are creating VxLAN tunnel with VNI 0, you can create any VNI for tunneling.
@@ -179,38 +184,54 @@ Below configuration assumes
 - Overlay VF1 has a VSI value 14
 - Overlay VF2 has a VSI value 15
 
-These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC. This command provides VSI ID, Vport ID, and corresponding MAC addresses for all
+These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC.
+This command provides VSI ID, Vport ID, and corresponding MAC addresses for all:
 
 - IDPF netdevs on ACC
 - VF's on HOST
 - IDPF netdevs on HOST (if IDPF driver loaded by you on HOST)
 - Netdevs on IMC
 
 ```bash
-
-# Rules for control packets coming from overlay VF(VSI-14), IPU will add a VLAN tag 1 and send to HOST1(VSI-8)
-
- p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_host_to_ovs_and_ovs_to_wire_table "vmeta.common.vsi=14,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.add_vlan_and_send_to_port(1,24)" 
- p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_host_to_ovs_table "vmeta.common.vsi=14,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(24)"
- p4rt-ctl add-entry br0 linux_networking_control.vlan_push_mod_table "vmeta.common.mod_blob_ptr=1,action=linux_networking_control.vlan_push(1,0,1)"
-
-# Rules for control packets coming from overlay VF(VSI-15), IPU will add a VLAN tag 2 and send to HOST1(VSI-8)
-
- p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_host_to_ovs_and_ovs_to_wire_table "vmeta.common.vsi=15,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.add_vlan_and_send_to_port(2,24)"
-  p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_host_to_ovs_table "vmeta.common.vsi=15,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(24)"
-  p4rt-ctl add-entry br0 linux_networking_control.vlan_push_mod_table "vmeta.common.mod_blob_ptr=2,action=linux_networking_control.vlan_push(1,0,2)"
-
-# Rules for control packets coming from HOST1(VSI-8), IPU will remove the VLAN tag 1 and send to overlay VF(VSI-14)
-
- p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_ovs_to_host_table "vmeta.common.vsi=8,hdrs.dot1q_tag[vmeta.common.depth].hdr.vid=1,action=linux_networking_control.remove_vlan_and_send_to_port(1,30)"
- p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_ovs_to_host_table "vmeta.misc_internal.vm_to_vm_or_port_to_port[27:17]=14,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(30)"
- p4rt-ctl add-entry br0 linux_networking_control.vlan_pop_mod_table "vmeta.common.mod_blob_ptr=1,action=linux_networking_control.vlan_pop"
-
-# Rules for control packets coming from HOST1(VSI-8), IPU will remove the VLAN tag 2 and send to overlay VF(VSI-15)
-
- p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_ovs_to_host_table "vmeta.common.vsi=8,hdrs.dot1q_tag[vmeta.common.depth].hdr.vid=2,action=linux_networking_control.remove_vlan_and_send_to_port(2,31)"
- p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_ovs_to_host_table "vmeta.misc_internal.vm_to_vm_or_port_to_port[27:17]=15,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(31)"
- p4rt-ctl add-entry br0 linux_networking_control.vlan_pop_mod_table "vmeta.common.mod_blob_ptr=2,action=linux_networking_control.vlan_pop"
+# Rules for control packets coming from overlay VF (VSI-14).
+# IPU will add a VLAN tag 1 and send to HOST1 (VSI-8).
+
+p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_host_to_ovs_and_ovs_to_wire_table \
+    "vmeta.common.vsi=14,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.add_vlan_and_send_to_port(1,24)"
+p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_host_to_ovs_table \
+    "vmeta.common.vsi=14,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(24)"
+p4rt-ctl add-entry br0 linux_networking_control.vlan_push_mod_table \
+    "vmeta.common.mod_blob_ptr=1,action=linux_networking_control.vlan_push(1,0,1)"
+
+# Rules for control packets coming from overlay VF (VSI-15).
+# IPU will add a VLAN tag 2 and send to HOST1 (VSI-8).
+
+p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_host_to_ovs_and_ovs_to_wire_table \
+    "vmeta.common.vsi=15,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.add_vlan_and_send_to_port(2,24)"
+p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_host_to_ovs_table \
+    "vmeta.common.vsi=15,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(24)"
+p4rt-ctl add-entry br0 linux_networking_control.vlan_push_mod_table \
+    "vmeta.common.mod_blob_ptr=2,action=linux_networking_control.vlan_push(1,0,2)"
+
+# Rules for control packets coming from HOST1 (VSI-8).
+# IPU will remove the VLAN tag 1 and send to overlay VF (VSI-14).
+
+p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_ovs_to_host_table \
+    "vmeta.common.vsi=8,hdrs.dot1q_tag[vmeta.common.depth].hdr.vid=1,action=linux_networking_control.remove_vlan_and_send_to_port(1,30)"
+p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_ovs_to_host_table \
+    "vmeta.misc_internal.vm_to_vm_or_port_to_port[27:17]=14,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(30)"
+p4rt-ctl add-entry br0 linux_networking_control.vlan_pop_mod_table \
+    "vmeta.common.mod_blob_ptr=1,action=linux_networking_control.vlan_pop"
+
+# Rules for control packets coming from HOST1 (VSI-8).
+# IPU will remove the VLAN tag 2 and send to overlay VF (VSI-15).
+
+p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_ovs_to_host_table \
+    "vmeta.common.vsi=8,hdrs.dot1q_tag[vmeta.common.depth].hdr.vid=2,action=linux_networking_control.remove_vlan_and_send_to_port(2,31)"
+p4rt-ctl add-entry br0 linux_networking_control.handle_rx_loopback_from_ovs_to_host_table \
+    "vmeta.misc_internal.vm_to_vm_or_port_to_port[27:17]=15,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(31)"
+p4rt-ctl add-entry br0 linux_networking_control.vlan_pop_mod_table \
+    "vmeta.common.mod_blob_ptr=2,action=linux_networking_control.vlan_pop"
 ```
 
 ### Configure rules for underlay control packets
@@ -224,22 +245,26 @@ Below configuration assumes
 
 ```bash
 # Configuration for control packets between physical port 0 to underlay IDPF netdev VSI-10
- p4rt-ctl add-entry br0 linux_networking_control.handle_rx_from_wire_to_ovs_table "vmeta.common.port_id=0,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(26)"
+p4rt-ctl add-entry br0 linux_networking_control.handle_rx_from_wire_to_ovs_table \
+    "vmeta.common.port_id=0,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(26)"
 
 # Configuration for control packets between underlay IDPF netdev VSI-10 to physical port 0
- p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_host_to_ovs_and_ovs_to_wire_table "vmeta.common.vsi=10,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(0)"
+p4rt-ctl add-entry br0 linux_networking_control.handle_tx_from_host_to_ovs_and_ovs_to_wire_table \
+    "vmeta.common.vsi=10,user_meta.cmeta.bit32_zeros=0,action=linux_networking_control.set_dest(0)"
 ```
 
 ### Underlay configuration
 
-Create a dummy interface which is used for TEP termination and IDPF netdev for underlay connectivity. Use BGP protocol with FRR, for route redistribution.
+Create a dummy interface which is used for TEP termination and IDPF netdev for
+underlay connectivity. Use BGP protocol with FRR, for route redistribution.
 
 Below configuration assumes
 
 - Underlay IDPF netdev has a VSI value 10
 
 ```bash
-p4rt-ctl add-entry br0 linux_networking_control.ecmp_lpm_root_lut "user_meta.cmeta.bit32_zeros=4/255.255.255.255,priority=2,action=linux_networking_control.ecmp_lpm_root_lut_action(0)"
+p4rt-ctl add-entry br0 linux_networking_control.ecmp_lpm_root_lut \
+    "user_meta.cmeta.bit32_zeros=4/255.255.255.255,priority=2,action=linux_networking_control.ecmp_lpm_root_lut_action(0)"
 
 nmcli device set <IDPF netdev for VSI 10> managed no
 ip link add dev TEP0 type dummy
@@ -248,7 +273,7 @@ ip link add dev TEP0 type dummy
 FRR running configuration
 
 ```bash
-# <Install FRR on ACC and enable BGP protocol>  
+# <Install FRR on ACC and enable BGP protocol>
 # <FRR VTYSH running configuration>
 
 interface TEP0