We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I have a json named temp.json.
{"ref1": "ABC", "ref2": "DEF", "cuts":[{"cut1": 3, "cut2": 5}]}
I want to load it.
ds = datasets.load_dataset('json', data_files="./temp.json", features=datasets.Features({ 'ref1': datasets.Value('string'), 'ref2': datasets.Value('string'), 'cuts': datasets.Sequence({ "cut1": datasets.Value("uint16"), "cut2": datasets.Value("uint16") }) }))
The above code does not work. However, I can load it without giving features.
ds = datasets.load_dataset('json', data_files="./temp.json")
Is it possible to load integers as uint16 to save some memory?
As in the bug description.
The data are loaded and integers are uint16.
Copy-and-paste the text below in your GitHub issue.
datasets
huggingface_hub
fsspec
The text was updated successfully, but these errors were encountered:
Hi ! Sequence has a weird behavior for dictionaries (from tensorflow-datasets), use a regular list instead:
Sequence
ds = datasets.load_dataset('json', data_files="./temp.json", features=datasets.Features({ 'ref1': datasets.Value('string'), 'ref2': datasets.Value('string'), 'cuts': [{ "cut1": datasets.Value("uint16"), "cut2": datasets.Value("uint16") }] }))
Sorry, something went wrong.
Hi ! Sequence has a weird behavior for dictionaries (from tensorflow-datasets), use a regular list instead: ds = datasets.load_dataset('json', data_files="./temp.json", features=datasets.Features({ 'ref1': datasets.Value('string'), 'ref2': datasets.Value('string'), 'cuts': [{ "cut1": datasets.Value("uint16"), "cut2": datasets.Value("uint16") }] }))
Thank you!
It works.
No branches or pull requests
Describe the bug
I have a json named temp.json.
I want to load it.
The above code does not work. However, I can load it without giving features.
Is it possible to load integers as uint16 to save some memory?
Steps to reproduce the bug
As in the bug description.
Expected behavior
The data are loaded and integers are uint16.
Environment info
Copy-and-paste the text below in your GitHub issue.
datasets
version: 2.21.0huggingface_hub
version: 0.24.5fsspec
version: 2024.5.0The text was updated successfully, but these errors were encountered: