OnnxRuntime with OpenVino EP on AWS Lambda - hwloc/linux: failed

[hjander]

Hi,

i am trying to create a serverless inference service on AWS Lambda with ORT and OpenVino, when running the inference code, i get the following error:

hwloc/linux: failed to find sysfs cpu topology directory, aborting linux discovery.

The first lines of Lambda startup log this:

14:12:21.741000 INIT_START Runtime Version: provided:al2023.v18 Runtime Version ARN: arn:aws:lambda:eu-central-1::runtime:a9b531e267d3347428b53cd65bd0722a2ab5fec8cab891a2a739679ed048ced2 14:12:21.831000 INFO Loaded ONNX Runtime dylib with version "1.17.3" 14:12:21.918000 DEBUG Environment not yet initialized, creating a new one 14:12:21.953000 DEBUG Environment created env_ptr="0x55c929816850" 14:12:21.955000 DEBUG Session Options { execution_mode:0 execution_order:DEFAULT enable_profiling:0 optimized_model_filepath: enable_mem_pattern:1 enable_mem_reuse:1 enable_cpu_mem_arena:1 profile_file_prefix:onnxruntime_profile_ session_logid: session_log_severity_level:-1 session_log_verbosity_level:0 max_num_graph_transformation_steps:10 graph_optimization_level:3 intra_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str: set_denormal_as_zero: 0 } inter_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str: set_denormal_as_zero: 0 } use_per_session_threads:1 thread_pool_allow_spinning:1 use_deterministic_compute:0 config_options: { } } 14:12:21.955000 DEBUG Flush-to-zero and denormal-as-zero are off 14:12:21.955000 DEBUG Creating and using per session threadpools since use_per_session_threads_ is true 14:12:21.955000 DEBUG Dynamic block base set to 0 14:12:23.292000 DEBUG [OpenVINO-EP]No runtime device selection option provided. 14:12:23.292000 DEBUG [OpenVINO-EP]Choosing Device: CPU , Precision: FP32 14:12:23.370000 hwloc: Registered discovery component "no_os" phases 0x2 with priority 40 (statically build) 14:12:23.370000 hwloc: Registered discovery component "xml" phases 0x1 with priority 30 (statically build) 14:12:23.370000 hwloc: Registered discovery component "synthetic" phases 0x1 with priority 30 (statically build) 14:12:23.370000 hwloc: Registered discovery component "linux" phases 0x7a with priority 50 (statically build) 14:12:23.370000 hwloc: Registered discovery component "x86" phases 0x2 with priority 45 (statically build) 14:12:23.370000 hwloc: Enabling discovery component "linux" with phases 0x7a (among 0x7a) 14:12:23.370000 hwloc: Enabling discovery component "x86" with phases 0x2 (among 0x2) 14:12:23.370000 hwloc: Enabling discovery component "no_os" with phases 0x2 (among 0x2) 14:12:23.370000 hwloc: Excluding discovery component "xml" phases 0x1, conflicts with excludes 0x1 2024/06/29/[$LATEST]7b1432047d12449cb338d953643e37a7 14:12:23.370000 hwloc: Excluding discovery component "synthetic" phases 0x1, conflicts with excludes 0x1 14:12:23.370000 hwloc: Final list of enabled discovery components: linux(0x7a),x86(0x2),no_os(0x2) 14:12:23.370000 hwloc/linux: failed to find sysfs cpu topology directory, aborting linux discovery. 14:12:23.398000 DEBUG Initializing session. 14:12:23.398000 DEBUG Adding default CPU execution provider. 14:12:23.398000 DEBUG Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0 2024

Setup:

• AWS Lambda with rust 1.79 and al.2023 (Amazon Linux) runtime, intel x86
• Onnx Runtime for rust v2.0.0-rc.2 -
• OpenVino 2023.3 - i copied all lib from openvino/onnxruntime_ep_ubuntu20 container

All libs are loaded it seems and with CPUExecutionProvider everything works fine.
I tried other hwloc implementations (synthetic, no_os) but to no avail and i don't know how to use the xml one

Any hints or pointers how i could circumvent this problem ?

[itPodo]

Did you Solve it? Have the same issue