Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize ELascii objects #19

Open
davebraze opened this issue Jun 6, 2016 · 3 comments
Open

Serialize ELascii objects #19

davebraze opened this issue Jun 6, 2016 · 3 comments

Comments

@davebraze
Copy link
Owner

Look into schemes for serializing ELascii objects (readELascii.R). Consider packages jsonlite or XML. Just now, I'm leaning toward the former.

@davebraze
Copy link
Owner Author

davebraze commented Jun 15, 2016

I've been rethinking this. Rather than go for a text serialization I think it would be better to just build thin wrappers for reading and writing ELascii objects to/from rdata files. After all, the point is to provide efficient read/write to file, not inter-operability as such.

@davebraze
Copy link
Owner Author

davebraze commented Nov 14, 2017

Also consider the feather format, implemented in the eponymous R package: https://cran.r-project.org/web/packages/feather/index.html. I think, that feather would be preferable to the HDF5 format, although should probably think about this a bit more.

There is some discussion of relative merits here: https://news.ycombinator.com/item?id=11384577.

One concern about feather is whether it will have staying power, or end up abandonware. HDF5, at least, has been around for a long time and is widely used. Note there are two R packages for interacting with HDF5 files: H5, and rhdf5, the latter on bioconductor. An older package, HDF5, has been removed from CRAN and seems deprecated.

Still shouldn't rule out json or XML.

@davebraze
Copy link
Owner Author

davebraze commented Nov 17, 2017

Here, wesm/feather#318, it says of feather that "you shouldn't rely on it for long-term storage, at least for now." That pretty much rules out use of feather for my purposes.

In that same place "Parquet files" are recommended for safe long term storage., cf https://rud.is/b/2017/01/22/create-parquet-files-from-r-data-frames-with-sergeant-apache-drill-a-k-a-make-parquet-files-great-again-in-r/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant