RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

[kerstin921]

I've tried a minimal example:

wavelength = 1e-6
speed_of_light = 277972458

# Initialize FDTD grid
grid = fdtd.Grid(shape=(20*wavelength, 50*wavelength, 1), grid_spacing=wavelength/10)

# Initialize perfectly matched layer (PML) boundaries
grid[0:10, :, :] = fdtd.PML(name="pml_xlow")
grid[-10:, :, :] = fdtd.PML(name="pml_xhigh")
grid[:, 0:10, :] = fdtd.PML(name="pml_ylow")
grid[:, -10:, :] = fdtd.PML(name="pml_yhigh")

# Initialize source
grid[:, 10, 0] = fdtd.LineSource(period=wavelength/speed_of_light, name="source")

# Initialize objects
#grid[10:190, 50:100, :] = fdtd.AbsorbingObject(permittivity=2.5, conductivity=1e-6, name="absorbing_object")

# Run simulation
start_time = timeit.default_timer()

for i in range(200):
    grid.step()  # Run simulation one timestep at a time and animate
    if i % 10 == 0:
        grid.visualize(z=0, cmap='plasma', animate=True, index=i, save=False)
        plt.title(f"Simulation step {i}")
        #clear_output(wait=True)  # Only necessary in Jupyter notebooks

print(timeit.default_timer() - start_time)

But I'm getting this runtime error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [51], in <cell line: 17>()
     13 grid[:, 10, 0] = fdtd.LineSource(period=wavelength/299792458, name="source")
     16 # Initialize objects
---> 17 grid[10:190, 50:100, :] = fdtd.AbsorbingObject(permittivity=2.5, conductivity=1e-6, name="absorbing_object")
     20 # Run simulation
     21 start_time = timeit.default_timer()

File C:\Program Files\Python38\lib\site-packages\fdtd\grid.py:365, in Grid.__setitem__(self, key, attr)
    362 else:
    363     raise KeyError("maximum number of indices for the grid is 3")
--> 365 attr._register_grid(
    366     grid=self,
    367     x=self._handle_single_key(x),
    368     y=self._handle_single_key(y),
    369     z=self._handle_single_key(z),
    370 )

File C:\Program Files\Python38\lib\site-packages\fdtd\objects.py:193, in AbsorbingObject._register_grid(self, grid, x, y, z)
    187     conductivity = conductivity[..., None]
    188 self.conductivity = bd.broadcast_to(
    189     conductivity, self.inverse_permittivity.shape
    190 )
    192 self.absorption_factor = (
--> 193     0.5
    194     * self.grid.courant_number
    195     * self.inverse_permittivity
    196     * self.conductivity
    197     * self.grid.grid_spacing
    198     * const.eta0
    199 )

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

[kerstin921]

By the way, is it allowed to have objects protruding into the PML region or is this something that should be avoided?

[flaport]

I am sorry @kerstin921 , I ran your code snippet with different versions of pytorch and for all the different backends but I can't seem to reproduce this error.

To give you some context. It seems in your case that one of the following four tensors is placed on the cpu while the rest is on the gpu (or vice versa):

• self.grid.courant_number
• self.inverse_permittivity
• self.conductivity
• self.grid.grid_spacing
• const.eta0

Maybe you set one of those values manually?

Can you do a conda env export or a pip list to show which packages are in your environment?

Thanks

[kerstin921]

Thanks, this is my pip list output:

Package              Version
-------------------- -----------
anyio                3.6.1
argon2-cffi          21.3.0
argon2-cffi-bindings 21.2.0
asttokens            2.0.5
async-generator      1.10
atomicwrites         1.4.1
attrs                21.4.0
backcall             0.2.0
beautifulsoup4       4.11.1
bleach               5.0.1
certifi              2022.6.15
cffi                 1.15.1
chardet              5.0.0
charset-normalizer   2.1.0
colorama             0.4.5
cycler               0.11.0
Cython               0.29.30
DateTime             4.5
debugpy              1.6.2
decorator            5.1.1
defusedxml           0.7.1
entrypoints          0.4
executing            0.8.3
fastjsonschema       2.16.1
fdtd                 0.2.5
fonttools            4.34.4
future               0.18.2
gprMax               3.1.4
graphviz             0.20
h5py                 3.7.0
icc-rt               2019.0
idna                 3.3
importlib-metadata   4.12.0
importlib-resources  5.9.0
iniconfig            1.1.1
intel-openmp         2022.1.0
ipykernel            6.15.1
ipython              8.4.0
ipython-genutils     0.2.0
ipywidgets           7.7.1
jedi                 0.18.1
Jinja2               3.1.2
jsonschema           4.7.2
jupyter              1.0.0
jupyter-client       7.3.4
jupyter-console      6.4.4
jupyter-core         4.11.1
jupyter-server       1.18.1
jupyter-telemetry    0.1.0
jupyterlab-pygments  0.2.2
jupyterlab-widgets   1.1.1
kiwisolver           1.4.4
line-profiler        3.5.1
littleutils          0.2.2
llvmlite             0.39.0
MarkupSafe           2.1.1
matplotlib           3.5.2
matplotlib-inline    0.1.3
mistune              0.8.4
nbclassic            0.4.3
nbclient             0.6.6
nbconvert            6.5.0
nbformat             5.4.0
nest-asyncio         1.5.5
notebook             6.4.12
notebook-shim        0.1.0
numba                0.56.0
numpy                1.22.4
nvidia-ml-py3        7.352.0
nvidia-smi           0.1.3
oauthlib             3.2.0
packaging            21.3
pandas               1.4.3
pandocfilters        1.5.0
parso                0.8.3
pickleshare          0.7.5
Pillow               9.2.0
pip                  22.2
pipdeptree           2.2.1
pluggy               1.0.0
prometheus-client    0.14.1
prompt-toolkit       3.0.30
psutil               5.9.1
pure-eval            0.2.2
py                   1.11.0
py-thorlabs-tsp      0.0.8
pycparser            2.21
Pygments             2.12.0
pyparsing            3.0.9
pyrsistent           0.18.1
pytest               7.1.2
python-dateutil      2.8.2
python-json-logger   2.0.4
pytz                 2022.1
pywin32              304
pywinpty             2.0.6
pyzmq                23.2.0
qtconsole            5.3.1
QtPy                 2.1.0
requests             2.28.1
ruamel.yaml          0.17.21
ruamel.yaml.clib     0.2.6
scipy                1.8.1
seaborn              0.11.2
Send2Trash           1.8.0
setuptools           63.2.0
six                  1.16.0
sniffio              1.2.0
sorcery              0.2.2
soupsieve            2.3.2.post1
stack-data           0.3.0
stdlib-list          0.8.0
terminado            0.15.0
terminaltables       3.1.10
tinycss2             1.1.1
tomli                2.0.1
torch                1.12.0
tornado              6.2
tqdm                 4.64.0
traitlets            5.3.0
typing_extensions    4.3.0
urllib3              1.26.10
wcwidth              0.2.5
webencodings         0.5.1
websocket-client     1.3.3
wheel                0.37.1
widgetsnbextension   3.6.1
wrapt                1.14.1
zipp                 3.8.1
zmq                  0.0.0
zope.interface       5.4.0

[kerstin921]

The only permittivity value I modified is inside a

grid[a:b, c:d, e:f] = fdtd.Object(permittivity=100, name='something')

[IceCream613]

Very cool project! I look forward to seeing where it goes.

I have the same error, when I use 'fdtd.AbsorbingObject'. Maybe there is something wrong with this function?

[zaneky]

In fact, it is a bit of bug. In the root file FDTD -> objects.py, line. 170, as the object of fdtd.AbsorbingObject initialized, the auther defined 'self.conductivity = conductivity', which is a type of numpy, or cpu. It will be well to modified 'self.conductivity = bd.array(conductivity)'.

[flaport]

thanks @zaneky , I made the proposed change. Should be available in fdtd 0.2.8.