You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are many parts of the C backend which use the single precision float without there being a clear reason (and in some cases converting between the two several times). While it makes sense to store the largest grids in single precision, most of the internal calculations (notably the interpolation tables) should be able to be in double without taking a performance hit.
We should run some performance tests replacing some of the float calculations with double throughout the code. Having a more uniform data size throughout will make development a little easier and the double precision should be a little more forgiving in terms of floating point errors that could accumulate.
The text was updated successfully, but these errors were encountered:
There are many parts of the C backend which use the single precision
float
without there being a clear reason (and in some cases converting between the two several times). While it makes sense to store the largest grids in single precision, most of the internal calculations (notably the interpolation tables) should be able to be indouble
without taking a performance hit.We should run some performance tests replacing some of the
float
calculations withdouble
throughout the code. Having a more uniform data size throughout will make development a little easier and the double precision should be a little more forgiving in terms of floating point errors that could accumulate.The text was updated successfully, but these errors were encountered: