From 3a2f55b85d2635c21760425b9bb9a6eb904a063b Mon Sep 17 00:00:00 2001 From: "Fedotov, Aleksei" Date: Tue, 12 Nov 2024 13:06:57 +0100 Subject: [PATCH 1/2] Add RFC for creation and use of NUMA arenas --- .../numa-arenas-creation-and-use.org | 147 ++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 rfcs/proposed/numa_support/numa-arenas-creation-and-use.org diff --git a/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org b/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org new file mode 100644 index 0000000000..78fa560a14 --- /dev/null +++ b/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org @@ -0,0 +1,147 @@ +#+title: API to Facilitate Instantiation and Use of oneTBB's Task Arenas Constrained to NUMA Nodes + +*Note:* This is a sub-RFC of the https://github.com/oneapi-src/oneTBB/pull/1535. + +* Introduction +Let's consider the example from "Setting the preferred NUMA node" section of the +[[https://oneapi-src.github.io/oneTBB/main/tbb_userguide/Guiding_Task_Scheduler_Execution.html][Guiding Task Scheduler Execution]] page of oneTBB Developer Guide. + +** Motivating example +#+begin_src C++ +std::vector numa_indexes = tbb::info::numa_nodes(); // [0] +std::vector arenas(numa_indexes.size()); // [1] +std::vector task_groups(numa_indexes.size()); // [2] + +for(unsigned j = 0; j < numa_indexes.size(); j++) { + arenas[j].initialize(tbb::task_arena::constraints(numa_indexes[j])); // [3] + arenas[j].execute([&task_groups, &j](){ // [4] + task_groups[j].run([](){/*some parallel stuff*/}); + }); +} + +for(unsigned j = 0; j < numa_indexes.size(); j++) { + arenas[j].execute([&task_groups, &j](){ task_groups[j].wait(); }); // [5] +} +#+end_src + +Usually the users of oneTBB employ this technique to tie oneTBB worker threads +up within NUMA nodes and yet have all the parallelism of a platform utilized. +The pattern allows to find out how many NUMA nodes are on the system. With that +number user creates that many ~tbb::task_arena~ objects, constraining each to a +dedicated NUMA node. Along with ~tbb::task_arena~ objects user instantiates the +same number of ~tbb::task_group~ objects, with which the oneTBB tasks are going +to be associated. The ~tbb::task_group~ objects are needed because they allow +waiting for the work completion as the ~tbb::task_arena~ class does not provide +synchronization semantics on its own. Then the work gets submitted in each of +arena objects, and waited upon their finish at the end. + +** Interface issues and inconveniences: +- [0] - Getting the number of NUMA nodes is not the task by itself, but rather a + necessity to know how many objects to initialize further. +- [1] - Explicit step for creating the number of ~tbb::task_arena~ objects per + each NUMA node. Note that by default the arena objects are constructed with a + slot reserved for master thread, which in this particular example usually + results in undersubscription issue as the master thread can join only one + arena at a time to help with work processing. +- [2] - Separate step for instantiation the same number of ~tbb::task_group~ + objects, in which the actual work is going to be submitted. Note that user + also needs to make sure the size of ~arenas~ matches the size of + ~task_groups~. +- [3] - Actual tying of ~tbb::task_arena~ instances with corresponding NUMA + nodes. Note that user needs to make sure the indices of ~tbb::task_arena~ + objects match corresponding indices of NUMA nodes. +- [4] - Actual work submission point. It is relatively easy to make a mistake + here by using the ~tbb::task_arena::enqueue~ method instead. In this case not + only the work submission might be done after the synchronization point [5], + but also the loop counter ~j~ can be mistakenly captured by reference, which + at least results in submission of the work into incorrect ~tbb::task_group~, + and at most a segmentation fault, since the loop counter might not exist by + the time the functor starts its execution. +- [5] - Synchronization point, where user needs to again make sure corresponding + indices are used. Otherwise, the waiting might be done in unrelated + ~tbb::task_arena~. It is also possible to mistakenly use + ~tbb::task_arena::enqueue~ method with the same consequences as were outlined + in the previous bullet, but since it is a synchronization point, usually the + blocking call is used. + +The proposal below addresses these issues. + +* Proposal +Introduce simplified interface to: +- Contstrain a task arena to specific NUMA node, +- Submit work into constrained task arenas, and +- To wait for completion of the submitted work. + +Since the new interface represents a constrained ~tbb::task_arena~ , the +proposed name is ~tbb::constrained_task_arena~. Not including the word "numa" +into the name would allow it for extension in the future for other types of +constraints. + +** Usage Example +#+begin_src C++ +std::vector numa_arenas = + tbb::initialize_numa_constrained_arenas(); + +for(unsigned j = 0; j < numa_arenas.size(); j++) { + numa_arenas[j].enqueue( (){/*some parallel stuff*/} ); +} + +for(unsigned j = 0; j < numa_arenas.size(); j++) { + numa_arenas[j].wait(); +} +#+end_src + +** New arena interface +The example above requires new class named ~tbb::constrained_task_arena~. On one +hand, it is a ~tbb::task_arena~ class that isolates the work execution from +other parallel stuff executed by oneTBB. On the other hand, it is a constrained +arena that represents an arena associated to a certain NUMA node and allows +efficient and error-prone work submission in this particular usage scenario. + +#+begin_src C++ +namespace tbb { + +class constrained_task_arena : protected task_arena { +public: + using task_arena::is_active(); + using task_arena::terminate(); + + using task_arena::max_concurrency(); + + using task_arena::enqueue; + + void wait(); +private: + constrained_task_arena(tbb::task_arena::constraints); + friend std::vector initialize_numa_constrained_arenas(); +}; + +} +#+end_src + +The interface exposes only necessary methods to allow submission and waiting of +a parallel work. Most of the exposed function members are taken from the base +~tbb::task_arena~ class. Implementation-wise, the new task arena would include +associated ~tbb::task_group~ instance, with which enqueued work will be +implicitly associated. + +The ~tbb::constrained_task_arena::wait~ method waits for the work in associated +~tbb::task_group~ to finish, if any was submitted using the +~tbb::constrained_task_arena::enqueue~ method. + +The instance of the ~tbb::constrained_task_arena~ class can be created only by +~tbb::initialize_numa_constrained_arenas~ function, whose sole purpose is to +instantiate a ~std::vector~ of initialized ~tbb::constrained_task_arena~ +instances, each constrained to its own NUMA node of the platform and does not +include reserved slots, and return this vector back to caller. + +* Open Questions +1. Should the interface for creation of constrained task arenas support other + construction parameters (e.g., max_concurrency, number of reserved slots, + priority, other constraints) from the very beginning or it is enough as the + first iteration and these parameters can be added in the future when the need + arise? +2. Should the new task arena allow initializing it with, probably, different + parameters after its creation? +3. Should the new task arena interface allow copying of its settings by exposing + its copy-constructor similarly to what ~tbb::task_arena~ does. From a96d1b414dedc66340840624d0d1c9a9016fbebc Mon Sep 17 00:00:00 2001 From: "Fedotov, Aleksei" Date: Thu, 14 Nov 2024 12:01:39 +0100 Subject: [PATCH 2/2] Address Mike's remarks --- rfcs/proposed/numa_support/numa-arenas-creation-and-use.org | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org b/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org index 78fa560a14..038a9c5718 100644 --- a/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org +++ b/rfcs/proposed/numa_support/numa-arenas-creation-and-use.org @@ -26,8 +26,8 @@ for(unsigned j = 0; j < numa_indexes.size(); j++) { Usually the users of oneTBB employ this technique to tie oneTBB worker threads up within NUMA nodes and yet have all the parallelism of a platform utilized. -The pattern allows to find out how many NUMA nodes are on the system. With that -number user creates that many ~tbb::task_arena~ objects, constraining each to a +The pattern starts by finding the number of NUMA nodes on the system. With that +number, user creates that many ~tbb::task_arena~ objects, constraining each to a dedicated NUMA node. Along with ~tbb::task_arena~ objects user instantiates the same number of ~tbb::task_group~ objects, with which the oneTBB tasks are going to be associated. The ~tbb::task_group~ objects are needed because they allow