Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saving 3D array into .npy file for large dimension gets corrupted output #100

Open
sahilhassanzee opened this issue Dec 23, 2024 · 1 comment

Comments

@sahilhassanzee
Copy link

I'm trying to put a 3D array of dimension (10000, 160, 346) in an .npy file using the append mode as shown in the code below:

#include "cnpy.h"
#include <vector>
#include <cstdlib>
constexpr int N = 346*160;

int main(int argc, char* argv[]) {
    int ROWS = 10000;

    // Create a vector
    std::vector<double> vec (N,0);

    // Fill the vector with data
    for(int i=0; i < N; i++) {
        vec[i] = i;
    }

    // Append the vector ROWS number of times into an npz file
    for (int t=0; t<ROWS; t++){
        cnpy::npy_save("sample.npy", &vec[0], {1,160,346}, "a");
    }

    return 0;
}

If I now try to open this file using the following Python code:

import numpy as np
x = np.load('sample.npy')

I get the following error:

ValueError: cannot reshape array of size 553599998 into shape (10000,160,346)

So, the size of array in the saved sample.npy is 2 Bytes smaller than what's expected. I have observed arrays upto 16 Bytes off from the expected size for larger size. Could you help resolve this issue?

@sahilhassanzee
Copy link
Author

By the way, my numpy version is 2.1.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant