Usage of enclosedCells() is not valid for hi boundaries

[mbkuhn]

Bug description

A few places in the code (affiliated with filling boundary cells), the "enclosedCells" call is used to convert face-centered boxes to cell-centered boxes. However, this is not a legitimate way to do it because it is only valid for lo boundaries. The orientation (hi/lo) needs to be considered to be valid for both.

https://github.com/Exawind/amr-wind/blob/main/amr-wind/wind_energy/ABLBoundaryPlane.cpp#L915
https://github.com/Exawind/amr-wind/blob/main/amr-wind/wind_energy/ABLModulatedPowerLaw.cpp#L178
this one doesn't matter because we don't have any temperature fields located at faces, but it would be good to correct it as well.
https://github.com/Exawind/amr-wind/blob/main/amr-wind/wind_energy/ABLModulatedPowerLaw.cpp#L255

Steps to reproduce

I added a print statement to output the i index when the populate_data call is executed for filling umac.
when running the "abl_bndry_input_amr_native_xhi" reg test, the print statement shows that the index 48 is being written to, even though this is a valid face in the domain, not a ghost face, as the routine intends.

Steps to reproduce the behavior:

1. Compiler used
   • GCC
   • LLVM
   • oneapi (Intel)
   • nvcc (NVIDIA)
   • rocm (AMD)
   • with MPI
   • other:
2. Operating system
   • Linux
   • OSX
   • Windows
   • other (do tell ;)):
3. Hardware:
   • CPU
   • GPU

Additional context

This is likely why there are diffs for this reg test for the PR #1274

[marchdf]

Nice find!

[marchdf]

Can you use some of these other ones:

 /**
    * \brief Convert the BoxND from the current type into the
    * argument type.  This may change the BoxND coordinates:
    * type CELL -> NODE : increase coordinate by one on high end
    * type NODE -> CELL : reduce coordinate by one on high end
    * other type mappings make no change.
    */
    AMREX_GPU_HOST_DEVICE
    BoxND& convert (const IntVectND<dim>& typ) noexcept; 3 refs|0 ref

    //! Convert to NODE type in all directions.
    AMREX_GPU_HOST_DEVICE
    BoxND& surroundingNodes () noexcept; 2 refs

    //! Convert to NODE type in given direction.
    AMREX_GPU_HOST_DEVICE
    BoxND& surroundingNodes (int dir) noexcept; 13 refs|0 ref

    AMREX_GPU_HOST_DEVICE
    BoxND& surroundingNodes (Direction d) noexcept { return surroundingNodes(static_cast<int>(d)); } 1 ref|1 ref

    //! Convert to CELL type in all directions.
    AMREX_GPU_HOST_DEVICE
    BoxND& enclosedCells () noexcept; 8 refs

    //! Convert to CELL type in given direction.
    AMREX_GPU_HOST_DEVICE
    BoxND& enclosedCells (int dir) noexcept; 2 refs|0 ref

    AMREX_GPU_HOST_DEVICE
    BoxND& enclosedCells (Direction d) noexcept { return enclosedCells(static_cast<int>(d)); } 1 ref|1 ref

[mbkuhn]

I thought that the problem was enclosedCells(), but that is only part of the picture. After the enclosedCells calls, the box is intersected with a boundary box that is cell-centered. If I correctly convert the enclosedCells call, I will also need to convert the boundary box before doing the intersection. I think it's easier to keep enclosedCells(), do the intersection, and then do a conditional shift afterward.