Skip to content

cost_distance: numpy dask map_overlap chunk path skips the _check_memory guard #3343

@brendancol

Description

@brendancol

Description

The numpy dask path of cost_distance skips the working-memory guard that every other backend applies.

_make_chunk_func (xrspatial/cost_distance.py:1122) builds a closure that calls _cost_distance_kernel directly inside da.map_overlap. That kernel allocates dist, h_keys, h_rows, h_cols, visited, and out, each sized height * width, for the chunk it receives. The map_overlap chunk also includes the overlap depth, so the per-chunk footprint can exceed the nominal chunk size.

The other paths already guard this allocation:

  • numpy: _cost_distance_numpy calls _check_memory(height, width) before the kernel.
  • cupy and dask+cupy: _cost_distance_cupy calls _check_gpu_memory(height, width).
  • iterative tile Dijkstra: _cost_distance_dask_iterative estimates total RAM and raises MemoryError.

The numpy map_overlap path is the only one without a guard. A single oversized chunk allocates several height * width arrays and can exhaust a worker's memory. The other paths raise a MemoryError that points the user at max_cost= or smaller chunks; here the numpy allocator just fails with an opaque error instead.

Expected behavior

The numpy map_overlap chunk function should call _check_memory(h, w) before running the Dijkstra kernel, matching _cost_distance_numpy and the other backends, so an oversized chunk raises a MemoryError that names the remedy.

Severity

Medium. The chunk size is user-controlled, so this is a defense-in-depth consistency gap rather than a remote vulnerability. It is the only allocation path in the module without a guard.

Found during a security sweep of the cost_distance module.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdaskDask backend / chunked arraysinput-validationInput validation and error messagesoomOut-of-memory risk with large datasets

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions