Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

draft: improve CPU cores affinity to NUMA nodes v6 #12248

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

lukashino
Copy link
Contributor

@lukashino lukashino commented Dec 8, 2024

Followup of #11706

Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/7036

This work allows more precise thread assignment and it is done either

  • automatically - by picking assigned cores of the same NUMA locality as the interface from the worker-cpu-list
  • manually - you can specify per-iface settings in the threading section

This works with AF-PACKET and DPDK (and should with other capture modes as well). Primary target is workers runmode.

Dependency - hwloc - for the automatic core assignment.

At this point I am not sure why pkg-config doesn't find hwloc in github ci - it works on my machines - in case it fails on yours add LIBS="$LIBS -lhwloc" to your configure.ac

To try this PR:

  1. compile Suri from this branch
  2. Modify the newly generated suricata.yaml according to your setup.
  3. Run as usual.

Describe changes:

v6

  • support for autofp mode supported - receive-cpu-set can now contain per interface CPU affinity settings - here comes a question - should worker-cpu-set in autofp be somehow configurable to the receive-cpu-set threads?
  • YAML node "per-iface" changed to "interface-specific-cpu-set"
  • list form of (management/worker/...)-cpu-set was changed to simple tree nodes
  • cpu/prio/mode properties of (management/worker/...)-cpu-set nodes moved one YAML layer up
  • DPDK (net_bonding / PCIe address / ... ) / AFP interfaces are supported

On YAML changes:

I tried other YAML designs such as:

Specifying the NUMA affinity in the individual capture modes

Example

dpdk:
  - interface: "3b:00.0"
    threads: [ 2, 4, 6, 8]

But management / other threads would still be required to configure elsewhere. I liked the proposed approach more because all affinity-related settings are centralized.

Not having a list nodes in the YAML

threading:
  worker-cpu-set:
    net_bonding0:
      cpu: [ 1, 3, 5 ]

YAML library changes net_bonding to net-bonding when YAML is loaded - so this is a nogo.

Have interface-related properties as children

threading:
  worker-cpu-set:
    - interface: net_bonding0
        cpu: [ 1, 3, 5 ]

This is a disallowed construct in YAML, so another nogo. A similar pattern is currently with *-cpu-set nodes but the different here is that the parent list item has a value (in this case net_bonding0). That is what is causing problems. Interface related therefore need to right under "interface".

PRO TIP: A good way to visualize different YAML structures is by using YAML to JSON converters. Currently it creates object like:

[
  {
    "interface": "3b:00.0",
    "mode": "exclusive",
  }
]

Lukas Sismis added 7 commits December 8, 2024 15:30
This work let's you set threading settings either:
- automatically - to let Suricata decide what cores from what NUMA
  is better for the given interface
- manually - you can per-interface configure threading settings

This requires hwloc-devel / hwloc-dev to be installed
Part of Ticket 2321 work to remove unnecessary lists from
the config file.

Ticket: 2321
@suricata-qa
Copy link

ERROR:

ERROR: QA failed on build_asan.

Pipeline 23725

@ct0br0 ct0br0 added the skip qa label Dec 9, 2024
@ct0br0
Copy link

ct0br0 commented Dec 9, 2024

added 'skip qa' since this isn't going to build without updates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

3 participants