[Sonic-DASH] Dash Tunnel and FNIC changes #1911

prsunny · 2025-02-06T01:42:11Z

Updated to rev 2.4 for:

Dash Tunnel behavior
PA validation updates
Switch attributes

mssonicbld · 2025-02-06T01:42:14Z

/azp run

azure-pipelines · 2025-02-06T01:42:20Z

No pipelines are associated with this pull request.

prsunny · 2025-02-06T01:44:50Z

@theasianpianist , @mukeshmv , @prabhataravind , @kperumalbfn , @r12f, @oleksandrivantsiv, @marian-pritsak for viz

prsunny · 2025-02-06T01:59:33Z

@KrisNey-MSFT

mssonicbld · 2025-02-06T02:47:19Z

/azp run

azure-pipelines · 2025-02-06T02:47:25Z

No pipelines are associated with this pull request.

mukeshmv · 2025-02-06T22:16:25Z

doc/dash/dash-sonic-hld.md

@@ -696,10 +711,17 @@ key                      = DASH_TUNNEL_TABLE:tunnel_name; tunnel name used for r
 ; field                  = value
 endpoints                = list of addresses for ecmp tunnel
 encap_type               = vxlan or nvgre
-vni                      = vni value for encap
+vni                      = vni value for encap, create only attribute


should we mention create only attribute for encap_type as well ?

mukeshmv · 2025-02-06T22:20:28Z

doc/dash/dash-sonic-hld.md

+
+For single endpoint, implmentation shall simply create a sai_dash_tunnel object with ```SAI_DASH_TUNNEL_ATTR_DIP=endpoint IP``` and ```SAI_DASH_TUNNEL_ATTR_MAX_MEMBER_SIZE=1```
+
+For ECMP, implementation shall create ```sai_dash_tunnel_member``` and ```sai_dash_tunnel_next_hop``` with appropriate ```SAI_DASH_TUNNEL_ATTR_MAX_MEMBER_SIZE```


Please mention that though members can be updated, at any point the number of members cannot exceed the count provided at Dash tunnel create

mukeshmv · 2025-02-06T22:25:48Z

doc/dash/dash-sonic-hld.md

@@ -89,6 +92,7 @@ At a high level the following should be supported:
    - Telemetry and Monitoring
    - Private Link
    - Private Link NSG
+    - Express Route GW Bypass


can we have a config example for express route gw bypass ?

Will add as a next iteration (another PR)

added an example

mssonicbld · 2025-02-07T00:31:36Z

/azp run

azure-pipelines · 2025-02-07T00:31:42Z

No pipelines are associated with this pull request.

mukeshmv · 2025-02-07T06:33:05Z

doc/dash/dash-sonic-hld.md

@@ -482,6 +491,7 @@ DASH_ROUTING_APPLIANCE_TABLE:{{appliance_id}}:
        "addresses": {{list of addresses}} 
        "encap_type": {{encap type}}
        "vni": {{vni}}
+        "region_id": {{local region id}}


Region ID is already added to the APPLIANCE table. This is the wrong deprecated Routing Appliance table

Yes. Prince, the routing appliance table is not the appliance table.

Thanks for catching. addressed

mukeshmv · 2025-02-07T06:35:47Z

doc/dash/dash-sonic-hld.md

@@ -499,6 +510,8 @@ DASH_APPLIANCE_TABLE:{{appliance_id}}
    "sip": {{ip_address}}
    "vm_vni": {{vni}}
    "local_region_id": {{region_id}}
+    "outbound_direction_lookup": {{dst_mac/src_mac}}


do we still need this attribute if we have Floating NIC mode ?

Yes, if there is fnic mode disabled and need to change the lookup attribute. Basically, aligning with SAI model

mukeshmv

couple more comments

r12f · 2025-02-07T17:17:21Z

doc/dash/dash-sonic-hld.md

-        "subtype": "Appliance",
-        "type": "SonicHost",
+        "subtype": "SmartSwitch",
+        "type": "SonicDpu",


Should we swap the type and subtype?

This is just aligning with sonic-buildimage and copied from existing definitions.

r12f · 2025-02-07T17:20:05Z

doc/dash/dash-sonic-hld.md

@@ -482,6 +491,7 @@ DASH_ROUTING_APPLIANCE_TABLE:{{appliance_id}}:
        "addresses": {{list of addresses}} 
        "encap_type": {{encap type}}
        "vni": {{vni}}
+        "region_id": {{local region id}}


Yes. Prince, the routing appliance table is not the appliance table.

r12f · 2025-02-07T17:22:09Z

doc/dash/dash-sonic-hld.md

@@ -499,6 +510,8 @@ DASH_APPLIANCE_TABLE:{{appliance_id}}
    "sip": {{ip_address}}
    "vm_vni": {{vni}}
    "local_region_id": {{region_id}}
+    "outbound_direction_lookup": {{dst_mac/src_mac}}


Do we have a vni table? The will be multiple VNIs needing this.

We don't have a VNI table currently.

r12f · 2025-02-07T17:24:33Z

doc/dash/dash-sonic-hld.md


-For more scale numbers, please refer to the [doc](https://github.com/sonic-net/DASH/blob/main/documentation/express-route-service/express-route-gateway-bypass.md)
+Expected max number of 4K PA_VALIDATION entries. For more scale numbers, please refer to the [doc](https://github.com/sonic-net/DASH/blob/main/documentation/express-route-service/express-route-gateway-bypass.md)


Maybe better to mention this in scaling requirements too, if missed.

Or simply move to there, in case inconsistent in future.

added to scale section and removed from here

mssonicbld · 2025-02-10T19:06:13Z

/azp run

azure-pipelines · 2025-02-10T19:06:20Z

No pipelines are associated with this pull request.

oleksandrivantsiv · 2025-02-11T21:14:36Z

doc/dash/dash-sonic-hld.md

@@ -695,11 +707,18 @@ DASH_TUNNEL_TABLE:{{tunnel_name}}
 key                      = DASH_TUNNEL_TABLE:tunnel_name; tunnel name used for referencing in mapping table
 ; field                  = value
 endpoints                = list of addresses for ecmp tunnel
-encap_type               = vxlan or nvgre
-vni                      = vni value for encap
+encap_type               = vxlan or nvgre, create only attribute


For the encap_type can we add an explicit definishion of what is expected return packet?

added a statement

mssonicbld · 2025-02-12T01:20:12Z

/azp run

azure-pipelines · 2025-02-12T01:20:18Z

No pipelines are associated with this pull request.

prsunny · 2025-02-12T04:11:22Z

@mzms, please review the expected dash config section for er gw

mssonicbld · 2025-02-12T18:07:45Z

/azp run

azure-pipelines · 2025-02-12T18:07:53Z

No pipelines are associated with this pull request.

prabhataravind · 2025-02-12T19:40:47Z

doc/dash/dash-sonic-hld.md

@@ -507,6 +519,8 @@ key                      = DASH_APPLIANCE_TABLE:id ; attributes specific for the
 sip                      = source ip address, to be used in encap
 vm_vni                   = VM VNI that is used for setting direction. Also used for inbound encap to VM
 local_region_id          = Region where this appliance is located
+outbound_direction_lookup= dst_mac or src_mac; Default is src_mac. This attribute overrides to dst_mac


Is this optional or mandatory?

prabhataravind · 2025-02-12T20:31:01Z

doc/dash/dash-sonic-hld.md

+    1. This packet shall be transformed IPv6 packet from PL endpoint
+    2. Outer SRC_PA:50.1.2.3, Outer DST_PA:10.250.20.19
+    3. Reverse transpositions applied (v6->v4)
+    4. Transformed packet tunneled to one of ER GW endpoint IP as configured in DASH_TUNNEL_TABLE


what is the behavior when there are multiple endpoints configured in DASH_TUNNEL_TABLE? Which endpoint is taken?

mukeshmv · 2025-02-12T23:40:59Z

doc/dash/dash-sonic-hld.md

@@ -127,6 +131,7 @@ Following are the minimal scaling requirements
 | Total active connections      | 32M (Bidirectional)           |
 | Metering Buckets per ENI      | 4000                          |
 | CPS                           | 3M                            |
+| Max PA validation entries     | 4k                            |


Can we add DASH_TUNNEL / NEXTHOP / max MEMBERs per TUNNEL scale ?

prsunny added 2 commits February 5, 2025 16:35

Dash tunnel and FNIC changes

4900026

Dash tunnel and FNIC changes (#1910)

7acceee

Override mac

de5b63e

r12f self-requested a review February 6, 2025 04:34

mukeshmv reviewed Feb 6, 2025

View reviewed changes

Updated for create only

d6c72be

mukeshmv approved these changes Feb 7, 2025

View reviewed changes

mukeshmv reviewed Feb 7, 2025

View reviewed changes

r12f reviewed Feb 7, 2025

View reviewed changes

Address review comments

16da30a

oleksandrivantsiv reviewed Feb 11, 2025

View reviewed changes

Added and example for ER GW

51c5ba5

prsunny requested a review from mukeshmv February 12, 2025 03:17

Address review comment

2d1e38d

prabhataravind reviewed Feb 12, 2025

View reviewed changes

mukeshmv reviewed Feb 12, 2025

View reviewed changes

mukeshmv approved these changes Feb 13, 2025

View reviewed changes


		For single endpoint, implmentation shall simply create a sai_dash_tunnel object with ```SAI_DASH_TUNNEL_ATTR_DIP=endpoint IP``` and ```SAI_DASH_TUNNEL_ATTR_MAX_MEMBER_SIZE=1```

		For ECMP, implementation shall create ```sai_dash_tunnel_member``` and ```sai_dash_tunnel_next_hop``` with appropriate ```SAI_DASH_TUNNEL_ATTR_MAX_MEMBER_SIZE```


		For more scale numbers, please refer to the [doc](https://github.com/sonic-net/DASH/blob/main/documentation/express-route-service/express-route-gateway-bypass.md)
		Expected max number of 4K PA_VALIDATION entries. For more scale numbers, please refer to the [doc](https://github.com/sonic-net/DASH/blob/main/documentation/express-route-service/express-route-gateway-bypass.md)

[Sonic-DASH] Dash Tunnel and FNIC changes #1911

Are you sure you want to change the base?

[Sonic-DASH] Dash Tunnel and FNIC changes #1911

Conversation

prsunny commented Feb 6, 2025 • edited Loading

mssonicbld commented Feb 6, 2025

azure-pipelines bot commented Feb 6, 2025

prsunny commented Feb 6, 2025 • edited Loading

prsunny commented Feb 6, 2025

mssonicbld commented Feb 6, 2025

azure-pipelines bot commented Feb 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mssonicbld commented Feb 7, 2025

azure-pipelines bot commented Feb 7, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mukeshmv Feb 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mukeshmv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mssonicbld commented Feb 10, 2025

azure-pipelines bot commented Feb 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mssonicbld commented Feb 12, 2025

azure-pipelines bot commented Feb 12, 2025

prsunny commented Feb 12, 2025

mssonicbld commented Feb 12, 2025

azure-pipelines bot commented Feb 12, 2025

Choose a reason for hiding this comment

prabhataravind Feb 12, 2025 • edited Loading

Choose a reason for hiding this comment

mukeshmv Feb 12, 2025 • edited Loading

Choose a reason for hiding this comment

prsunny commented Feb 6, 2025 •

edited

Loading

prsunny commented Feb 6, 2025 •

edited

Loading

mukeshmv Feb 7, 2025 •

edited

Loading

prabhataravind Feb 12, 2025 •

edited

Loading

mukeshmv Feb 12, 2025 •

edited

Loading