TELCODOCS-1506 Telco CORE Reference Design Specification

ref config updates

Adding generated YAML + modules from the RDS

Reorganizing TOC

Adding latest RDS updates

RDS doc updates

RDS YAML updates

Shanes review

latest updates + new components overview diagram

David J's review comments

CCS attributes update

Ian's review comments

adding Hari's deviations update

updating 4.14 link URLs

Generalizes scope and deviation topics for RAN and Core

updates for RDS prod version

update deviations wording

Updating deviation and scope topics

consolidate dev and scope topics + intro

Adding link to ztp-site-generate procedure

typo

final changes for RDS

typos

Ian's comments

update for RDS terminology

remove core CRs note

add GH core CRs link

Author: kquinn1204

.vale/styles/Vocab/OpenShiftDocs/accept.txt

@@ -7,21 +7,24 @@
 [Mm]idhaul
 [Pp]assthrough
 [Pp]ostinstall
+[Pp]recaching
 [Pp]reinstall
 [Rr]ealtime
 [Tt]elco
 Assisted Installer
 Control Plane Machine Set Operator
 custom resources?
+GHz
 gpsd
 gpspipe
+hyperthreads?
+KPIs?
 linuxptp
 Mbps
+MBps
 Mellanox
 MetalLB
 NICs?
-Operator
-Operators
 Operators?
 pmc
 ubxtool

_attributes/common-attributes.adoc

@@ -128,6 +128,16 @@ endif::[]
 :TempoShortName: distributed tracing platform (Tempo)
 :TempoOperator: Tempo Operator
 :TempoVersion: 2.3.0
+//telco
+ifdef::telco-ran[]
+:rds: telco RAN DU
+:rds-caps: Telco RAN DU
+:rds-first: Telco RAN distributed unit (DU)
+endif::[]
+ifdef::telco-core[]
+:rds: telco core
+:rds-caps: Telco core
+endif::[]
 //logging
 :logging-title: logging subsystem for Red Hat OpenShift
 :logging-title-uc: Logging subsystem for Red Hat OpenShift

_distro_map.yml

@@ -231,7 +231,7 @@ openshift-dpu:
       name: '4.10'
       dir: container-platform-dpu/4.10
 openshift-telco:
-  name: OpenShift Container Platform for Telco
+  name: OpenShift Container Platform
   author: OpenShift Documentation Project <[email protected]>
   site: commercial
   site_name: Documentation

_topic_maps/_topic_map.yml

@@ -2875,10 +2875,32 @@ Name: Reference design specifications
 Dir: telco_ref_design_specs
 Distros: openshift-telco
 Topics:
-- Name: Telco RAN reference design specification
-  File: ztp-ran-reference-design
+- Name: Telco reference design specifications
+  File: telco-ref-design-specs-overview
+- Name: Telco RAN DU reference design specification
+  Dir: ran
+  Topics:
+  - Name: Telco RAN DU reference design overview
+    File: telco-ran-ref-design-spec
+  - Name: Telco RAN DU use model overview
+    File: telco-ran-du-overview
+  - Name: RAN DU reference design components
+    File: telco-ran-ref-du-components
+  - Name: RAN DU reference design configuration CRs
+    File: telco-ran-ref-du-crs
+  - Name: Telco RAN DU software specifications
+    File: telco-ran-ref-software-artifacts
 - Name: Telco core reference design specification
-  File: cnf-core-reference-design
+  Dir: core
+  Topics:
+  - Name: Telco core reference design overview
+    File: telco-core-rds-overview
+  - Name: Telco core use model overview
+    File: telco-core-rds-use-cases
+  - Name: Core reference design components
+    File: telco-core-ref-design-components
+  - Name: Core reference design configuration CRs
+    File: telco-core-ref-crs
 ---
 Name: Specialized hardware and driver enablement
 Dir: hardware_enablement

images/473_OpenShift_Telco_Core_Reference_arch_1123.png

@@ -78,7 +78,7 @@ metadata:
   name: numaresourcesscheduler
 spec:
   imageSpec: "registry.redhat.io/openshift4/noderesourcetopology-scheduler-container-rhel8:v{product-version}"
-  cacheResyncPeriod: "5s" <1> 
+  cacheResyncPeriod: "5s" <1>
 ----
 <1> Enter an interval value in seconds for synchronization of the scheduler cache. A value of `5s` is typical for most implementations.
 +

modules/cnf-deploying-the-numa-aware-scheduler-with-manual-performance-settings.adoc

@@ -39,7 +39,7 @@ registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
 /usr/bin/test-run.sh -ginkgo.v -ginkgo.focus="cyclictest"
 ----
 +
-The command runs the `cyclictest` tool for 10 minutes (600 seconds). The test runs successfully when the maximum observed latency is lower than `MAXIMUM_LATENCY` (in this example, 20 μs). Latency spikes of 20 μs and above are generally not acceptable for telco RAN workloads.
+The command runs the `cyclictest` tool for 10 minutes (600 seconds). The test runs successfully when the maximum observed latency is lower than `MAXIMUM_LATENCY` (in this example, 20 μs). Latency spikes of 20 μs and above are generally not acceptable for {rds} workloads.
 +
 If the results exceed the latency threshold, the test fails.
 +

modules/cnf-performing-end-to-end-tests-running-cyclictest.adoc

@@ -0,0 +1,56 @@
+// Module included in the following assemblies:
+//
+// * telco_ref_design_specs/ran/telco-ran-ref-design-spec.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="telco-core-whats-new-ref-design_{context}""]
+= {product-title} {product-version} features for {rds}
+
+The following features that are included in {product-title} {product-version} and are leveraged by the {rds} reference design specification (RDS) have been added or updated.
+
+.New features for {rds} in {product-title} {product-version}
+[cols="1,3", options="header"]
+|====
+|Feature
+|Description
+
+//CNF-7349 Rootless DPDK pods
+|Support for running rootless Data Plane Development Kit (DPDK) workloads with kernel access by using the TAP CNI plugin
+a|DPDK applications that inject traffic into the kernel can run in non-privileged pods with the help of the TAP CNI plugin.
+
+* link:https://docs.openshift.com/container-platform/4.14/networking/hardware_networks/using-dpdk-and-rdma.html#nw-running-dpdk-rootless-tap_using-dpdk-and-rdma[Using the TAP CNI to run a rootless DPDK workload with kernel access]
+
+//CNF-5977 Better pinning of the networking stack
+|Dynamic use of non-reserved CPUs for OVS
+a|With this release, the Open vSwitch (OVS) networking stack can dynamically use non-reserved CPUs.
+The dynamic use of non-reserved CPUs occurs by default in performance-tuned clusters with a CPU manager policy set to `static`.
+The dynamic use of available, non-reserved CPUs maximizes compute resources for OVS and minimizes network latency for workloads during periods of high demand.
+OVS cannot use isolated CPUs assigned to containers in `Guaranteed` QoS pods. This separation avoids disruption to critical application workloads.
+
+//CNF-7760
+|Enabling more control over the C-states for each pod
+a|The `PerformanceProfile` supports `perPodPowerManagement` which provides more control over the C-states for pods. Now, instead of disabling C-states completely, you can specify a maximum latency in microseconds for C-states. You configure this option in the `cpu-c-states.crio.io` annotation, which helps to optimize power savings for high-priority applications by enabling some of the shallower C-states instead of disabling them completely.
+
+* link:https://docs.openshift.com/container-platform/4.14/scalability_and_performance/cnf-low-latency-tuning.html#node-tuning-operator-pod-power-saving-config_cnf-master[Optional: Power saving configurations]
+
+//CNF-7741 Permit to disable NUMA Aware scheduling hints based on SR-IOV VFs
+|Exclude SR-IOV network topology for NUMA-aware scheduling
+a|You can exclude advertising Non-Uniform Memory Access (NUMA) nodes for the SR-IOV network to the Topology Manager. By not advertising NUMA nodes for the SR-IOV network, you can permit more flexible SR-IOV network deployments during NUMA-aware pod scheduling.
+
+For example, in some scenarios, you want flexibility for how a pod is deployed. By not providing a NUMA node hint to the Topology Manager for the pod's SR-IOV network resource, the Topology Manager can deploy the SR-IOV network resource and the pod CPU and memory resources to different NUMA nodes. In previous {product-title} releases, the Topology Manager attempted to place all resources on the same NUMA node.
+
+* link:https://docs.openshift.com/container-platform/4.14/networking/hardware_networks/configuring-sriov-device.html#nw-sriov-exclude-topology-manager_configuring-sriov-device[Exclude the SR-IOV network topology for NUMA-aware scheduling]
+
+//CNF-8035 MetalLB VRF Egress interface selection with VRFs (Tech Preview)
+|Egress service resource to manage egress traffic for pods behind a load balancer (Technology Preview)
+a|With this update, you can use an `EgressService` custom resource (CR) to manage egress traffic for pods behind a load balancer service.
+
+You can use the `EgressService` CR to manage egress traffic in the following ways:
+
+* Assign the load balancer service's IP address as the source IP address of egress traffic for pods behind the load balancer service.
+
+* Configure the egress traffic for pods behind a load balancer to a different network than the default node network.
+
+* link:https://docs.openshift.com/container-platform/4.14/networking/ovn_kubernetes_network_provider/configuring-egress-traffic-for-vrf-loadbalancer-services.html#configuring-egress-traffic-loadbalancer-services[Configuring an egress service]
+
+|====

modules/telco-core-414-whats-new-ref-design.adoc

@@ -0,0 +1,28 @@
+// Module included in the following assemblies:
+//
+// * telco_ref_design_specs/ran/telco-core-ref-components.adoc
+
+:_mod-docs-content-type: REFERENCE
+[id="telco-core-cluster-network-operator_{context}"]
+= Cluster Network Operator (CNO)
+
+New in this release::
+
+Not applicable.
+
+Description::
+
+The CNO deploys and manages the cluster network components including the default OVN-Kubernetes network plugin during {product-title} cluster installation. It allows configuring primary interface MTU settings, OVN gateway modes to use node routing tables for pod egress, and additional secondary networks such as MACVLAN.
++
+In support of network traffic segregation, multiple network interfaces are configured through the CNO. Traffic steering to these interfaces is configured through static routes applied by using the NMState Operator. To ensure that pod traffic is properly routed, OVN-K is configured with the `routingViaHost` option enabled. This setting uses the kernel routing table and the applied static routes rather than OVN for pod egress traffic.
++
+The Whereabouts CNI plugin is used to provide dynamic IPv4 and IPv6 addressing for additional pod network interfaces without the use of a DHCP server.
+
+Limits and requirements::
+
+* OVN-Kubernetes is required for IPv6 support.
+* Large MTU cluster support requires connected network equipment to be set to the same or larger value.
+
+Engineering considerations::
+* Pod egress traffic is handled by kernel routing table with the `routingViaHost` option. Appropriate static routes must be configured in the host.
+

modules/telco-core-cluster-network-operator.adoc

@@ -0,0 +1,49 @@
+// Module included in the following assemblies:
+//
+// * telco_ref_design_specs/ran/telco-core-ref-components.adoc
+
+:_mod-docs-content-type: REFERENCE
+[id="telco-core-cpu-partitioning-performance-tune_{context}"]
+= CPU partitioning and performance tuning
+
+New in this release::
+
+Open vSwitch (OVS) is removed from CPU partitioning. OVS manages its cpuset dynamically to automatically adapt to network traffic needs. Users no longer need to reserve additional CPUs for handling high network throughput on the primary container network interface (CNI). There is no impact on the configuration needed to benefit from this change.
+
+Description::
+
+CPU partitioning allows for the separation of sensitive workloads from generic purposes, auxiliary processes, interrupts, and driver work queues to achieve improved performance and latency. The CPUs allocated to those auxiliary processes are referred to as `reserved` in the following sections. In hyperthreaded systems, a CPU is one hyperthread.
++
+For more information, see https://docs.openshift.com/container-platform/latest/scalability_and_performance/cnf-low-latency-tuning.html#cnf-cpu-infra-container_cnf-master[Restricting CPUs for infra and application containers].
++
+Configure system level performance.
+For recommended settings, see link:https://docs.openshift.com/container-platform/latest/scalability_and_performance/ztp_far_edge/ztp-reference-cluster-configuration-for-vdu.html#ztp-du-configuring-host-firmware-requirements_sno-configure-for-vdu[Configuring host firmware for low latency and high performance].
+
+Limits and requirements::
+* The operating system needs a certain amount of CPU to perform all the support tasks including kernel networking.
+** A system with just user plane networking applications (DPDK) needs at least one Core (2 hyperthreads when enabled) reserved for the operating system and the infrastructure components.
+* A system with Hyper-Threading enabled must always put all core sibling threads to the same pool of CPUs.
+* The set of reserved and isolated cores must include all CPU cores.
+* Core 0 of each NUMA node must be included in the reserved CPU set.
+* Isolated cores might be impacted by interrupts. The following annotations must be attached to the pod if guaranteed QoS pods require full use of the CPU:
++
+----
+cpu-load-balancing.crio.io: "disable"
+cpu-quota.crio.io: "disable"
+irq-load-balancing.crio.io: "disable"
+----
+* When per-pod power management is enabled with `PerformanceProfile.workloadHints.perPodPowerManagement` the following annotations must also be attached to the pod if guaranteed QoS pods require full use of the CPU:
++
+----
+cpu-c-states.crio.io: "disable"
+cpu-freq-governor.crio.io: "performance"
+----
+
+Engineering considerations::
+* The minimum reserved capacity (`systemReserved`) required can be found by following the guidance in  link:https://access.redhat.com/solutions/5843241["Which amount of CPU and memory are recommended to reserve for the system in OCP 4 nodes?"]
+* The actual required reserved CPU capacity depends on the cluster configuration and workload attributes.
+* This reserved CPU value must be rounded up to a full core (2 hyper-thread) alignment.
+* Changes to the CPU partitioning will drain and reboot the nodes in the MCP.
+* The reserved CPUs reduce the pod density, as the reserved CPUs are removed from the allocatable capacity of the OpenShift node.
+* The real-time workload hint should be enabled if the workload is real-time capable.
+* Hardware without Interrupt Request (IRQ) affinity support will impact isolated CPUs. To ensure that pods with guaranteed CPU QoS have full use of allocated CPU, all hardware in the server must support IRQ affinity.