Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON decoders chapter #68

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
1b37b65
Decode from JsonData ADT to Student ADT.
monmcguigan Jul 7, 2024
f56f150
Merge pull request #1 from monmcguigan/wip-json-decoding
monmcguigan Jul 7, 2024
181e4c0
Improve getListStudents function
monmcguigan Jul 7, 2024
4617335
Improve checkStudentsObj function
monmcguigan Jul 7, 2024
ce3a786
Use JsonDecoder alias and list decoder.
monmcguigan Jul 9, 2024
0729c05
Remove unused functions.
monmcguigan Jul 9, 2024
8a639c1
Consolidate errors for decoding, add Boolean to JsonData
monmcguigan Jul 14, 2024
21d14e1
Merge pull request #2 from monmcguigan/improvements
monmcguigan Jul 14, 2024
600c866
Comment out boolean.
monmcguigan Jul 14, 2024
263e2d4
Merge pull request #3 from monmcguigan/decoding
monmcguigan Jul 14, 2024
25c62fa
Checkpoint: condensed StudentHandler that decodes as expected.
monmcguigan Jul 14, 2024
fc4d12b
Checkpoint: Fixed error types so I can have type annotations.
monmcguigan Jul 14, 2024
a464212
Checkpoint: Add Boolean to JsonData and Bool decoder.
monmcguigan Jul 14, 2024
0581670
Checkpoint: Decoding testing and Student handling refinements.
monmcguigan Jul 14, 2024
8c87672
Improve consistency in StudentHandler.
monmcguigan Jul 15, 2024
1e0b64f
Rename error tag.
monmcguigan Jul 15, 2024
3850049
Bring in most recent changes to deocder lib and encoder WIP
monmcguigan Jul 31, 2024
6e0d7a5
OneOf decoder implementation.
monmcguigan Aug 2, 2024
1b10ca7
Bring in changes from PR
monmcguigan Aug 4, 2024
0ad4bba
Tag union decoder, using type discriminators.
monmcguigan Aug 8, 2024
3abafd0
Updates for talk
monmcguigan Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions json/DataTypes/DecodeStudent.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
module [readStudents]

import Student exposing [Student, Module]
import Decoding exposing [JsonDecoder, list, field, string, number, bool, map2, map3, or]

Students : List Student

readStudents : JsonDecoder Students
readStudents = \json ->
studentsField = \s -> (list readStudent) s
(field "students" studentsField) json

readStudent : JsonDecoder Student
readStudent = \json ->
nameField = field "name" string
modulesField = field "modules" (\m -> (list readModule) m)
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved
currentGrade = field "currentGrade" number
finalGrade = field "finalGrade" number
currentStudent = map3 nameField modulesField currentGrade (\(name, mods, cg) -> CurrentStudent { name: name, modules: mods, currentGrade: cg })
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved
graduatedStudent = map3 nameField modulesField finalGrade (\(name, mods, fg) -> GraduatedStudent { name: name, modules: mods, finalGrade: fg })
(or currentStudent graduatedStudent) json

readModule : JsonDecoder Module
readModule = \json ->
nameField = field "name" string
creditsField = field "credits" number
enrolledField = field "enrolled" bool
(map3 nameField creditsField enrolledField \(name, credits, enrolled) -> { name: name, credits: credits, enrolled: enrolled }) json
Copy link
Contributor

@rtfeldman rtfeldman Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if you want to get into record builder syntax in this chapter (or if it seems too advanced - also, the new version of this syntax just landed in the compiler, so it may not have existed when this part of the chapter was first written!) but it can make this function a lot more concise:

readModule : JsonDecoder Module
readModule = \json ->
    { map2 <-
        name: field "name" string,
        credits: field "credits" number,
        enrolled: field "enrolled" bool,
    } json

By the way, I know we have a compiler bug that means right now you have to do \json -> (...) json (which in general should be able to be written as just (...)), but hopefully we can fix that in time to simplify this to just readModule = { map2 <- ... } without the need for the enclosing lambda!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, although I think in order for this to work, map2 would need the function it accepts to take 2 arguments instead of a tuple:

-map2 : JsonDecoder a, JsonDecoder b, ((a, b) -> c) -> JsonDecoder c
+map2 : JsonDecoder a, JsonDecoder b, (a, b -> c) -> JsonDecoder c

That's the map2 signature the { foo <- syntax sugar is designed to work with 😄

81 changes: 81 additions & 0 deletions json/DataTypes/DecodeStudentLong.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
module [getListStudents]

import JsonData exposing [Json]
import Student exposing [Student, Module]

JsonErrors : [
FieldNotFound Str,
ExpectedJsonObject Str,
ExpectedJsonArray Str,
]

getListStudents : Json -> Result (List Student) JsonErrors
getListStudents = \json ->
when json is
Object obj ->
when Dict.get obj "students" is
Ok students -> checkStudentList students
_ -> Err (FieldNotFound "Expected field with name students in object")
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

_ -> Err (ExpectedJsonObject "Expected an Object")
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

checkStudentList : Json -> Result (List Student) JsonErrors
checkStudentList = \json ->
when json is
Arr students ->
List.mapTry (List.map students checkStudentObj) \x -> x
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

_ -> Err (ExpectedJsonArray "Expected an Array")

checkStudentObj : Json -> Result Student JsonErrors
checkStudentObj = \json ->
when json is
Object obj ->
when Dict.get obj "name" is
Ok (String name) ->
when Dict.get obj "modules" is
Ok modules ->
when checkModulesList modules is
Ok goodMods ->
when Dict.get obj "finalGrade" is
Ok (Number fg) -> Ok (GraduatedStudent { name: name, modules: goodMods, finalGrade: fg })
_ ->
when Dict.get obj "currentGrade" is
Ok (Number cg) -> Ok (CurrentStudent { name: name, modules: goodMods, currentGrade: cg })
_ -> Err (FieldNotFound "Expected field ")
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

Err e -> Err e

_ -> Err (FieldNotFound "Expected field with name modules in object")

_ -> Err (FieldNotFound "Expected field with name name in object")

_ -> Err (ExpectedJsonObject "Expected an Object")

checkModulesList : Json -> Result (List Module) JsonErrors
checkModulesList = \json ->
when json is
Arr modules ->
List.mapTry (List.map modules readModuleObject) \x -> x
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

_ -> Err (ExpectedJsonArray "Expected an Array")

readModuleObject : Json -> Result Module JsonErrors
readModuleObject = \json ->
when json is
Object fields ->
when Dict.get fields "name" is
Ok (String name) ->
when Dict.get fields "credits" is
Ok (Number credits) ->
when Dict.get fields "enrolled" is
Ok (Boolean enrolled) ->
Ok ({ name: name, credits: credits, enrolled: enrolled })
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

_ -> Err (FieldNotFound "Expected field with name \"credits\" in object")

_ -> Err (FieldNotFound "Expected field with name \"credits\" in object")

_ -> Err (FieldNotFound "Expected field with name \"name\" in object")

_ -> Err (ExpectedJsonObject "Expected an Object")
167 changes: 167 additions & 0 deletions json/DataTypes/Decoding.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
module [string, number, bool, list, field, map, map2, map3, andThen, or, JsonDecoder, DecodingErrors]
import JsonData exposing [Json]

DecodingErrors : [
FieldNotFound Str,
ExpectedJsonObject Str,
ExpectedJsonArray Str,
WrongJsonType Str,
KeyNotFound,
DecodingFailed
]

JsonDecoder t : Json -> Result t DecodingErrors
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

string : JsonDecoder Str
string = \json ->
when json is
String str -> Ok str
_ -> Err (WrongJsonType "Expected a String when decoding, found $(printJsonType json)")
monmcguigan marked this conversation as resolved.
Show resolved Hide resolved

number : JsonDecoder U64
number = \json ->
when json is
Number num -> Ok num
_ -> Err (WrongJsonType "Expected a Number when decoding")

bool : JsonDecoder Bool
bool = \json ->
when json is
Boolean b -> Ok b
_ -> Err (WrongJsonType "Expected a Bool when decoding")

list : JsonDecoder a -> JsonDecoder (List a)
list = \decoderA ->
\data ->
when data is
Arr jsonValues -> List.mapTry jsonValues decoderA
_ -> Err (ExpectedJsonArray "Expected an Arr when decoding")

field : Str, JsonDecoder a -> JsonDecoder a
field = \name, decoder ->
\data ->
when data is
Object dict ->
when Dict.get dict name is
Ok v -> decoder v
Err s -> Err s

_ -> Err (WrongJsonType "Expected an Object when decoding")

map : JsonDecoder a, (a -> b) -> JsonDecoder b
map = \decoderA, f ->
\data -> Result.map (decoderA data) f
# \data -> decoderA data |> Result.map f
# decoderA |> Result.map f

map2 : JsonDecoder a, JsonDecoder b, ((a, b) -> c) -> JsonDecoder c
map2 = \decoderA, decoderB, f ->
\data ->
when (decoderA data, decoderB data) is
(Ok a, Ok b) -> Ok (f (a, b))
(Err a, _) -> Err a
(_, Err b) -> Err b

map3 : JsonDecoder a, JsonDecoder b, JsonDecoder c, ((a, b, c) -> d) -> JsonDecoder d
map3 = \decoderA, decoderB, decoderC, f ->
\data ->
when (decoderA data, decoderB data, decoderC data) is
(Ok a, Ok b, Ok c) -> Ok (f (a, b, c))
(Err a, _, _) -> Err a
(_, Err b, _) -> Err b
(_, _, Err c) -> Err c

andThen : (a -> JsonDecoder b), JsonDecoder a -> JsonDecoder b
andThen = \toB, aDecoder ->
\data ->
when aDecoder data is
Ok a -> (toB a) data
Err a -> Err a



or : JsonDecoder a, JsonDecoder a -> JsonDecoder a
or = \decoderA, decoderB ->
\data ->
decoderA data
|> Result.onErr \_ -> (decoderB data)


# TODO - Sum and Product decoders
# product=\a, b, c, decoderA, decoderB, decoderC, f ->
# could you take in a dictionary of Dict Str JsonDecoder a?
product : Str, Str, Str, JsonDecoder a, JsonDecoder b, JsonDecoder c, ((a, b, c) -> d) -> JsonDecoder d
altPr : List (Str, JsonDecoder a)

# sum : JsonDecoder a, JsonDecoder b -> JsonDecoder c
# sum : [JsonDecoder a] -> JsonDecoder a
# sum =\ds ->
# \data ->
# List.mapTry ds (\decoder -> decoder data)
# when decoderA data is
# Ok a -> a
# _ -> when decoderB data is
# Ok b -> b
# Err b -> Err b
# sum: List (JsonDecoder a) -> JsonDecoder a ⁠

# sum : [JsonDecoder a] -> JsonDecoder a
# sum =\ decoders ->
# \data ->
# List.walkTry decoders \d -> d



# TESTS
mathsMod = Object
(
Dict.empty {}
|> Dict.insert "name" (String "Maths 101")
|> Dict.insert "credits" (Number 200)
|> Dict.insert "enrolled" (Boolean Bool.true)
)
phyMod = Object
(
Dict.empty {}
|> Dict.insert "name" (String "Physics 101")
|> Dict.insert "credits" (Number 200)
)
nameDecoder = field "name" string
creditsDecoder = field "credits" number
enrolledDecoder = field "enrolled" bool

# map tests
expect (map nameDecoder \name -> { name: name }) phyMod == Ok ({ name: "Physics 101" })
expect (map enrolledDecoder \name -> { name: name }) phyMod == Err(KeyNotFound)

# map2 tests
expect (map2 nameDecoder creditsDecoder \(name, credits) -> { name: name, credits: credits }) (phyMod) == Ok ({ name: "Physics 101", credits: 200 })

# map3 tests
expect (map3 nameDecoder creditsDecoder enrolledDecoder \(name, credits, enrolled) -> { name: name, credits: credits, enrolled: enrolled }) (mathsMod) == Ok ({ name: "Maths 101", credits: 200, enrolled: Bool.true })

# list tests
myList = Arr [String "hello", String "world"]
expect (list string) myList == Ok (["hello", "world"])
expect (list string) mathsMod == Err (ExpectedJsonArray "Expected an Arr when decoding")

# field tests
expect nameDecoder mathsMod == Ok ("Maths 101")
expect nameDecoder myList == Err (WrongJsonType "Expected an Object when decoding")
expect (field "blah" string) mathsMod == Err KeyNotFound

# primitive types
expect string (String "hello") == Ok ("hello")
expect string (Number 123) == Err (WrongJsonType "Expected a String when decoding, found Number 123")
expect bool (Boolean Bool.true) == Ok Bool.true
expect number (Number 400) == Ok (400)
# expect null (Null) == Ok ("null")

printJsonType : Json -> Str
printJsonType = \json ->
when json is
String str -> "String $(str)"
Number num -> "Number $(Num.toStr num)"
Boolean _ -> "Boolean"
Object _ -> "Object"
Arr _ -> "Arr"
41 changes: 41 additions & 0 deletions json/DataTypes/Encoding.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
module []
import JsonData exposing [Json]

JsonEncoder t : t -> Json

string : JsonEncoder Str
string = \str ->
String str

number : JsonEncoder U64
number = \num ->
Number num

bool : JsonEncoder Bool
bool = \b ->
Boolean b

list : JsonEncoder a -> JsonEncoder (List a)
list = \encoderA ->
\data ->
json = List.map data encoderA
Arr json

# field : Str, JsonEncoder a -> JsonEncoder (Str, Json)
field = \ fieldName, encoderA ->
\data ->
j = encoderA data
(fieldName, j)
# TESTS
# primitives
expect string "hello" == (String "hello")
expect number 50 == (Number 50)
expect bool Bool.true == (Boolean Bool.true)
# there is no null type in Roc

# list tests
strs = ["hello", "hello again"]
expect (list string) strs == Arr[(String "hello"), (String "hello again")]

# field encoder
expect (field "fieldName" string) "fieldValue" == ("fieldName", String "fieldValue")
9 changes: 9 additions & 0 deletions json/DataTypes/JsonData.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
module [Json]

Json : [
String Str,
Number U64,
Boolean Bool,
Object (Dict Str Json),
Arr (List Json),
]
20 changes: 20 additions & 0 deletions json/DataTypes/Student.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
module [Student, Module]

Student : [
CurrentStudent {
name : Str,
modules : List (Module),
currentGrade : U64
},
GraduatedStudent {
name : Str,
modules : List (Module),
finalGrade : U64
}
]

Module : {
name : Str,
credits : U64,
enrolled : Bool
}
31 changes: 31 additions & 0 deletions json/DataTypes/StudentData.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"students": [
{
"name": "Alice",
"modules": [
{
"name": "Physics 101",
"credits": 10,
"enrolled": true
},
{
"name": "Maths 101",
"credits": 20,
"enrolled": true
}
],
"currentGrade": 75
},
{
"name": "Bob",
"modules": [
{
"name": "Maths 101",
"credits": 20,
"enrolled": false
}
],
"finalGrade": 70
}
]
}
5 changes: 5 additions & 0 deletions json/DataTypes/main.roc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package [
JsonData,
Student,
DecodeStudent
]{}
Loading