From 844235eb7f5bb0bea6e15f421d7d238853a26512 Mon Sep 17 00:00:00 2001 From: Alex Gardner Date: Fri, 28 Jul 2023 19:46:37 -0700 Subject: [PATCH 1/6] Add documentation for JLD2 I/O --- docs/src/man/importing_and_exporting.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/src/man/importing_and_exporting.md b/docs/src/man/importing_and_exporting.md index 35cf19b9b..e7347aa10 100644 --- a/docs/src/man/importing_and_exporting.md +++ b/docs/src/man/importing_and_exporting.md @@ -1,5 +1,24 @@ # Importing and Exporting Data (I/O) +## JLD2 Files + +A convenient format for reading and writing DataFrames in JLD2 which saves and loads +Julia data structures in a format comprising a subset of HDF5. Unlike several other +formats, JLD2 preserves custom Types. The save and load functions, provided by FileIO, +provide a mechanism to read/write `DataFrame` from/to a JLD2 file. + +A `DataFrame` can be saved as a JLD2 file at path `output.jld2` using +```julia +using FileIO +df = DataFrame(x=1, y=2) +save("output.jld2", Dict("df" => df)) +``` + +and can be read using +```julia +load("output.jld2") # -> ict{String, Any} with 1 entry: "df" => 1×2 DataFrame +``` + ## CSV Files For reading and writing tabular data from CSV and other delimited text files, From 8f32a2fc0f190e7c704bfb15f73147d34a5757d4 Mon Sep 17 00:00:00 2001 From: Alex Gardner Date: Mon, 31 Jul 2023 13:34:10 -0700 Subject: [PATCH 2/6] add JLD2.jl --- docs/src/man/importing_and_exporting.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/src/man/importing_and_exporting.md b/docs/src/man/importing_and_exporting.md index e7347aa10..f2e57135f 100644 --- a/docs/src/man/importing_and_exporting.md +++ b/docs/src/man/importing_and_exporting.md @@ -10,6 +10,8 @@ provide a mechanism to read/write `DataFrame` from/to a JLD2 file. A `DataFrame` can be saved as a JLD2 file at path `output.jld2` using ```julia using FileIO +using Pkg; Pkg.add("JLD2") + df = DataFrame(x=1, y=2) save("output.jld2", Dict("df" => df)) ``` From f5ef5d3b3d7ab64e1229fdc3d4466c006ae20f11 Mon Sep 17 00:00:00 2001 From: Alex Gardner Date: Mon, 31 Jul 2023 13:42:19 -0700 Subject: [PATCH 3/6] missing "D" in "Dict" --- docs/src/man/importing_and_exporting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/man/importing_and_exporting.md b/docs/src/man/importing_and_exporting.md index f2e57135f..1f8e0687b 100644 --- a/docs/src/man/importing_and_exporting.md +++ b/docs/src/man/importing_and_exporting.md @@ -18,7 +18,7 @@ save("output.jld2", Dict("df" => df)) and can be read using ```julia -load("output.jld2") # -> ict{String, Any} with 1 entry: "df" => 1×2 DataFrame +load("output.jld2") # -> Dict{String, Any} with 1 entry: "df" => 1×2 DataFrame ``` ## CSV Files From aff1770f7d00a66dec80469657629f86eac07d73 Mon Sep 17 00:00:00 2001 From: Alex Gardner Date: Tue, 1 Aug 2023 15:02:32 -0700 Subject: [PATCH 4/6] small text changes --- docs/src/man/importing_and_exporting.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/src/man/importing_and_exporting.md b/docs/src/man/importing_and_exporting.md index 1f8e0687b..97405fb0a 100644 --- a/docs/src/man/importing_and_exporting.md +++ b/docs/src/man/importing_and_exporting.md @@ -2,12 +2,13 @@ ## JLD2 Files -A convenient format for reading and writing DataFrames in JLD2 which saves and loads +A convenient format for reading and writing data frames in JLD2, which saves and loads Julia data structures in a format comprising a subset of HDF5. Unlike several other -formats, JLD2 preserves custom Types. The save and load functions, provided by FileIO, -provide a mechanism to read/write `DataFrame` from/to a JLD2 file. +formats, JLD2 preserves custom Types. The `save` and `load` functions, provided by FileIO.jl, +allow to read/write a data frame from/to a JLD2 file. + +A data frame can be saved as a JLD2 file output.jld2 using -A `DataFrame` can be saved as a JLD2 file at path `output.jld2` using ```julia using FileIO using Pkg; Pkg.add("JLD2") From 67ec270c867baeaf496e39004c748540683d7a71 Mon Sep 17 00:00:00 2001 From: Alex Gardner Date: Tue, 26 Sep 2023 14:32:45 -0700 Subject: [PATCH 5/6] move CSV above JLD2 --- docs/src/man/importing_and_exporting.md | 44 ++++++++++++------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/src/man/importing_and_exporting.md b/docs/src/man/importing_and_exporting.md index 97405fb0a..baa3d8b97 100644 --- a/docs/src/man/importing_and_exporting.md +++ b/docs/src/man/importing_and_exporting.md @@ -1,27 +1,5 @@ # Importing and Exporting Data (I/O) -## JLD2 Files - -A convenient format for reading and writing data frames in JLD2, which saves and loads -Julia data structures in a format comprising a subset of HDF5. Unlike several other -formats, JLD2 preserves custom Types. The `save` and `load` functions, provided by FileIO.jl, -allow to read/write a data frame from/to a JLD2 file. - -A data frame can be saved as a JLD2 file output.jld2 using - -```julia -using FileIO -using Pkg; Pkg.add("JLD2") - -df = DataFrame(x=1, y=2) -save("output.jld2", Dict("df" => df)) -``` - -and can be read using -```julia -load("output.jld2") # -> Dict{String, Any} with 1 entry: "df" => 1×2 DataFrame -``` - ## CSV Files For reading and writing tabular data from CSV and other delimited text files, @@ -135,6 +113,28 @@ As you can see the code required to transform `iris` into a proper input to the format is not easy. Therefore CSV.jl is the preferred package to write CSV files for data stored in data frames. +## JLD2 Files + +A convenient format for reading and writing data frames in JLD2, which saves and loads +Julia data structures in a format comprising a subset of HDF5. Unlike several other +formats, JLD2 preserves custom Types. The `save` and `load` functions, provided by FileIO.jl, +allow to read/write a data frame from/to a JLD2 file. + +A data frame can be saved as a JLD2 file output.jld2 using + +```julia +using FileIO +using Pkg; Pkg.add("JLD2") + +df = DataFrame(x=1, y=2) +save("output.jld2", Dict("df" => df)) +``` + +and can be read using +```julia +load("output.jld2") # -> Dict{String, Any} with 1 entry: "df" => 1×2 DataFrame +``` + ## Other formats Other data formats are supported for reading and writing in the following packages From 8b1bcbae99b4325fb44887cd375d58a4fbeb40ce Mon Sep 17 00:00:00 2001 From: Alex Gardner Date: Wed, 27 Sep 2023 10:51:10 -0700 Subject: [PATCH 6/6] improved description of jld2 read/write --- docs/src/man/importing_and_exporting.md | 42 ++++++++++++++++++++----- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/docs/src/man/importing_and_exporting.md b/docs/src/man/importing_and_exporting.md index baa3d8b97..bac495545 100644 --- a/docs/src/man/importing_and_exporting.md +++ b/docs/src/man/importing_and_exporting.md @@ -115,24 +115,52 @@ for data stored in data frames. ## JLD2 Files -A convenient format for reading and writing data frames in JLD2, which saves and loads -Julia data structures in a format comprising a subset of HDF5. Unlike several other -formats, JLD2 preserves custom Types. The `save` and `load` functions, provided by FileIO.jl, -allow to read/write a data frame from/to a JLD2 file. +JLD2 is a HDF5-compatible file format that allows reading and writing data frames. +A valuable feature of JLD2 format is that it preserves custom column types of the stored data frame. + +The `save` and `load` functions, provided by FileIO.jl, allow to read/write a data frame from/to a JLD2 file. A data frame can be saved as a JLD2 file output.jld2 using +If you have not used the FileIO and JLD2 packages before then you may need to install it first: +```julia +using Pkg; +Pkg.add("FileIO") +Pkg.add("JLD2") +``` + +The FileIO and JLD2 functions are not loaded automatically and must be imported into the session. ```julia using FileIO -using Pkg; Pkg.add("JLD2") +using JLD2 +``` +We can now create a simple data frame and save it as a jld2 file using `save`. `save` +accepts an AbstractDict yielding the key/value pairs, where the key is a string representing +the name of the dataset and the value represents its contents: + +```julia df = DataFrame(x=1, y=2) save("output.jld2", Dict("df" => df)) ``` -and can be read using +A jld2 file can be read in using `load`. If `load` is called with a single dataset name, +load returns the contents of that dataset from the file: +```julia +df = load("output.jld2", "df") +1×2 DataFrame + Row │ x y + │ Int64 Int64 +─────┼────────────── + 1 │ 1 2 +``` + +If `load` is called with a filename argument only, the load function loads all datasets +from the given file into a Dict: ```julia -load("output.jld2") # -> Dict{String, Any} with 1 entry: "df" => 1×2 DataFrame +df = load("output.jld2") +Dict{String, Any} with 1 entry: + "df" => 1×2 DataFrame… ``` ## Other formats