Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update documentation for Linux networking V3 #16

Merged
merged 3 commits into from
Apr 9, 2024
Merged

Update documentation for Linux networking V3 #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8301854
Update documentation for Linux networking V3
nupuruttarwar Apr 4, 2024
51490dd
Addressing comments
nupuruttarwar Apr 4, 2024
fe5d246
Merge branch 'main' into update_lnw_v3
nupuruttarwar Apr 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/apps/lnw/es2k/es2k-linux-networking.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,9 +290,20 @@ Current Linux Networking support for the networking recipe has the following lim
- Only OvS bridges are supported.
- Configure p4rt-ctl runtime rules before OvS configuration.
- Double vlan tag is NOT supported.
- Source port value used in P4RT rules should be VSI ID + 16
- Delete the route if added manually using `ip route` command
- Bring the port down or remove the IP address before removing the port from OVS bridge or deleting the bridge.
- Add all ACC PR's to VSI group 1.
- On ACC, firewall needs to be disabled. Otherwise, this service will block encapsulated packets.
- systemctl stop firewalld
- LAG and ECMP are mutually exclusive. Both can't co-exist in the system configuration at the same time.
- LAG configuration done via bonding driver is supported.
- The supported modes are active-backup and active-active with 802.3ad (LACP).

## Additional limitations for Linux Networking v3

- After teardown, it's possible that entries continue to exist in `vm_src_ip4_mac_map_table` and
`vm_dst_ip4_mac_map_table` since entries in these tables are learnt from OVS flows in MAC learning phase
and OVS might not issue a callback to delete these unless aging happens. Delete these enties using
nupuruttarwar marked this conversation as resolved.
Show resolved Hide resolved
`p4rt-ctl del-entry` command
- IPv6 is not supported
172 changes: 126 additions & 46 deletions docs/apps/lnw/es2k/es2k-lnw-overlay-vms.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,33 @@ for more details on this feature.

Prerequisites:

- Download `hw-p4-programs` TAR file specific to the build and extract it to get `fxp-net_linux-networking-v2` p4 artifacts. Go through `Limitations` specified in `README` and bringup the setup accordingly.
- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package.
- For Linux Networking v2
- Download `hw-p4-programs` TAR file compliant with the CI build image and extract it to get `fxp-net_linux-networking-v2` P4 artifacts. Check `README` for bringup guide, and be sure to check `Limitations` for known issues.
- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package.
Modify `load_custom_pkg.sh` with following parameters for linux_networking package:

```text
sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG
sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG
sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG
```text
sed -i 's/sem_num_pages = 1;/sem_num_pages = 25;/g' $CP_INIT_CFG
sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG
sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG
```

```
- For Linux Networking v3
- Download `hw-p4-programs` TAR file compliant with the CI build image and extract it to get `fxp-net_linux-networking-v3` P4 artifacts. Check `README` for bringup guide, and be sure to check `Limitations` for known issues.
- Follow steps mentioned in [Deploying P4 Programs for E2100](/guides/es2k/deploying-p4-programs) for bringing up IPU with a custom P4 package.
Modify `load_custom_pkg.sh` with following parameters for linux_networking package:

```text
sed -i 's/sem_num_pages = 1;/sem_num_pages = 28;/g' $CP_INIT_CFG
sed -i 's/lem_num_pages = 1;/lem_num_pages = 10;/g' $CP_INIT_CFG
sed -i 's/mod_num_pages = 1;/mod_num_pages = 2;/g' $CP_INIT_CFG
sed -i 's/acc_apf = 4;/acc_apf = 16;/g' $CP_INIT_CFG
```

- Download `IPU_Documentation` TAR file specific to the build and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below
- Download `IPU_Documentation` TAR file compliant with the CI build image and refer to `Getting Started Guide` on how to install compatible `IDPF driver` on host. Once an IDPF driver is installed, bring up SRIOV VF by modifying the `sriov_numvfs` file present under one of the IDPF network devices. Example as below

```bash
echo 16 > /sys/class/net/ens802f0/device/sriov_numvfs
echo 8 > /sys/class/net/ens802f0/device/sriov_numvfs
```

Notes about topology:
Expand All @@ -67,7 +79,11 @@ System under test will have above topology running the networking recipe. Link P

## Creating the topology

Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` p4 program.
- For Linux Networking v2
Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v2` P4 program.

- For Linux Networking v3
Follow steps mentioned in [Running Infrap4d on Intel E2100](/guides/es2k/running-infrap4d.md) for starting `infrap4d` process and creating protobuf binary for `fxp-net_linux-networking-v3` P4 program.

### Port Mapping

Expand Down Expand Up @@ -106,13 +122,22 @@ These VSI values can be checked with `/usr/bin/cli_client -q -c` command on IMC.
Once the application is started, set the forwarding pipeline config using
P4Runtime Client `p4rt-ctl` set-pipe command

```bash
$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \
$OUTPUT_DIR/p4info.txt
```
- For Linux Networking v2

Note: Assuming `fxp-net_linux-networking-v2.pb.bin` and `p4info.txt`
along with other P4 artifacts are created as per the steps mentioned in the previous section.
```bash
$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v2.pb.bin \
$OUTPUT_DIR/p4info.txt
```

- For Linux Networking v3

```bash
$P4CP_INSTALL/bin/p4rt-ctl set-pipe br0 $OUTPUT_DIR/fxp-net_linux-networking-v3.pb.bin \
nupuruttarwar marked this conversation as resolved.
Show resolved Hide resolved
$OUTPUT_DIR/p4info.txt
```

Note: Assuming `pb.bin` and `p4info.txt` along with other P4 artifacts are created as per
the steps mentioned in the previous section.

### Configure VSI Group and add a netdev

Expand Down Expand Up @@ -205,12 +230,25 @@ Example:
- Corresponding port representor VSI value 9
- If a VSI is used as an action, add an offset of 16 to the VSI value

```bash
# Create a source port for an overlay VF (VSI-27). Source port action can be any value.
# For simplicity add 16 to VSI ID.
p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \
"vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)"
- For Linux Networking v2

```bash
# Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16.
p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \
"vmeta.common.vsi=27,zero_padding=0,action=linux_networking_control.set_source_port(43)"
```

- For Linux Networking v3

```bash
# Create a source port for an overlay VF (VSI-27). Source port value should be VSI ID + 16.
p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \
"vmeta.common.vsi=27/2047,action=linux_networking_control.set_source_port(43)"
nupuruttarwar marked this conversation as resolved.
Show resolved Hide resolved
```

- Common P4RT rules

```bash
# Create a mapping between overlay VF (VSI-27/source port-43) and ACC port representor (VSI-9)
p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \
"user_meta.cmeta.source_port=43,zero_padding=0,action=linux_networking_control.fwd_to_vsi(25)"
Expand Down Expand Up @@ -240,7 +278,7 @@ Example:
- If a VSI is used as an action, add an offset of 16 to the VSI value

```bash
# Create a source port for a physical port (Phy port-0). Source port action can be any value.
# Create a source port for a physical port (Phy port-0). Source port value should be VSI ID + 16.
# For simplicity consider the same value as phy port id.
p4rt-ctl add-entry br0 linux_networking_control.rx_source_port \
"vmeta.common.port_id=0,zero_padding=0,action=linux_networking_control.set_source_port(0)"
Expand Down Expand Up @@ -268,14 +306,26 @@ Example:
- Corresponding port representor VSI value 18
- If a VSI is used as an action, add an offset of 16 to the VSI value

```bash
# Create a source port for an overlay VF (VSI-24). Source port action can be any value.
# For simplicity add 16 to VSI ID.
p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \
"vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)"
- For Linux Networking v2

```bash
# Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16.
p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \
"vmeta.common.vsi=24,zero_padding=0,action=linux_networking_control.set_source_port(40)"
```

- For Linux Networking v3

```bash
# Create a source port for an APF netdev (VSI-24). Source port value should be VSI ID + 16.
p4rt-ctl add-entry br0 linux_networking_control.tx_source_port_v4 \
"vmeta.common.vsi=24/2047,action=linux_networking_control.set_source_port(40)"
```

- Common P4RT rules

# Create a mapping between overlay VF (VSI-24/source port-40) and ACC port representor (VSI-18)
```bash
# Create a mapping between APF netdev (VSI-24/source port-40) and ACC port representor (VSI-18)
p4rt-ctl add-entry br0 linux_networking_control.source_port_to_pr_map \
"user_meta.cmeta.source_port=40,zero_padding=0,action=linux_networking_control.fwd_to_vsi(34)"

Expand All @@ -301,31 +351,61 @@ For TCAM entry configure LPM LUT table

Create a dummy LAG bypass table for all 8 hash indexes

```bash
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass"
- For Linux Networking v2

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass"
```bash
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=0,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass"
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=1,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass"
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=2,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass"
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=3,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass"
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=4,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass"
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=5,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass"
```
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=6,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0,hash=7,action=linux_networking_control.bypass"
```

- For Linux Networking v3

```bash
p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=0/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=1/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=2/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=3/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=4/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=5/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=6/7,action=linux_networking_control.bypass"

p4rt-ctl add-entry br0 linux_networking_control.tx_lag_table \
"user_meta.cmeta.lag_group_id=0/255,hash=7/7,action=linux_networking_control.bypass"
```

### Create integration bridge and add ports to the bridge

Expand Down
Loading