Run Parcels with GPUs?

[iuryt]

Hello everyone,

I’ve been working with Parcels for particle tracking and was wondering if there’s any support for running it on GPUs. Given the computational cost of large-scale simulations, it would be great to take advantage of GPU acceleration for better performance.

• Are there any existing implementations or experimental features that allow Parcels to utilize GPUs (e.g., via CUDA, or other parallel computing frameworks)?
• If not, are there any ongoing discussions or plans to incorporate GPU support in future versions?
• Would anyone be able to share experiences or alternative approaches for optimizing performance?

Looking forward to any insights or suggestions from the community!

Thanks!

[JamiePringle]

I can't comment on GPU's. But in my experience, you can get very substantial speed ups by careful choice of chunking of input, killing particles as needed, paying attention to and specifying chunking on output, etc.

However, for me to comment usefully, it would be useful to have a brief description of your problem. What is the input? When are the particles launched? Are you running with MPI? If you have an output file, what is the chunking in the output file?

To get the chunking of a netcdf input file , use ncdump -h -s filename.nc. It is often good to specify a chunk size when defining the fieldSet -- this can greatly speed things up. Some experimentation is useful, but a good place to start is to imitate the chunking of the input files. But play with this.

To get the chunking of an output zarr file, try the following commands

import zarr
jnk=zarr.open('output.zarr')
jnk.lat.info

which will return something like

Name               : /lat
Type               : zarr.core.Array
Data type          : float64
Shape              : (6197300, 4)
Chunk shape        : (1549325, 1)
Order              : C
Read-only          : False
Compressor         : Blosc(cname='lz4', clevel=5, shuffle=SHUFFLE, blocksize=0)
Store type         : zarr.storage.DirectoryStore
No. bytes          : 198313600 (189.1M)
No. bytes stored   : 86857672 (82.8M)
Storage ratio      : 2.3
Chunks initialized : 16/16

[iuryt]

Hi Jamie, thanks for sharing your experience. I wasn't thinking about a specific project, but rather whether there are any ideas for implementing GPU capabilities for Parcels and if this is interesting enough for us to invest effort in as a community. I believe that, at the very least, the predefined kernels, particularly the advection ones, could benefit from this implementation.

[iuryt]

Now, regarding your advice on chunking fields, I'm curious to learn more about how you prepare the fields before running the Lagrangian simulations. Accessing and downloading CMEMS reanalysis usually results in storing fields in a single NetCDF file, which might not be optimal. Would you chunk the data and save it in Zarr before running the particles on the fields? How would you approach this?

[JamiePringle]

See the thread I created at #1951

[erikvansebille]

Hi @iuryt, thanks for starting this discussion. We haven't thoroughly investigated/explored GPU-accelerated implementation of Parcels yet (are currently too busy with v4-development).

However, in discussions it was always brought up that most of our typical Lagrangian simulations are IO-bound (especially reading hydrodynamic input fields), and as far as I understand GPUs are not specifically much better in IO than normal CPUs. Or am I mistaken?

This heavy burden on input-reading is one of the reasons why we are now working on leveraging much more xarray-functionality directly in the new Parcels v4 development. It could be that this gives such a boost that GPU-acceleration becomes relevant to explore next. Would you be interested?

[VeckoTheGecko]

It is possible to run Xarray on GPUs, so this could be something to explore with v4 of Parcels. See the CuPy-Xarray: Xarray on GPUs! blog post for more info

[iuryt]

I have not yet used CuPy-Xarray, but I have access to GPUs and have experience with GPU processing and analysis. That could be a very interesting project.

[JamiePringle]

The issue of chunking the input file that @iuryt raised above is important, but not really directly relevant to GPU acceleration. So I have started a separate discussion thread, #1951 , describing my experience with it and how easily to chunk input data files. Jamie