Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration toward double precision floating point numbers #361

Open
daviesje opened this issue Mar 27, 2024 · 0 comments
Open

Migration toward double precision floating point numbers #361

daviesje opened this issue Mar 27, 2024 · 0 comments
Assignees

Comments

@daviesje
Copy link
Contributor

There are many parts of the C backend which use the single precision float without there being a clear reason (and in some cases converting between the two several times). While it makes sense to store the largest grids in single precision, most of the internal calculations (notably the interpolation tables) should be able to be in double without taking a performance hit.

We should run some performance tests replacing some of the float calculations with double throughout the code. Having a more uniform data size throughout will make development a little easier and the double precision should be a little more forgiving in terms of floating point errors that could accumulate.

@daviesje daviesje mentioned this issue Aug 8, 2024
@daviesje daviesje linked a pull request Aug 8, 2024 that will close this issue
@daviesje daviesje self-assigned this Aug 8, 2024
@daviesje daviesje removed a link to a pull request Aug 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

No branches or pull requests

1 participant