FHIR JSON Serializer Problems

[Sneedd]

Help! Been stuck a while now. Tried different approaches to make it work, but so far nothing worked.

Basically I am trying to serialize and deserialize an custom resource. Tried the System.Text.Json and Newtonsoft approach.

Here is my code, if you like to try it out: Create a new MSTest .NET6 project, add the Hl7.Fhir.R4 (v5.3.0) library and paste the following code. I added the results over the tests, where 5 of 8 are failing.

// Hl7.Fhir.R4 5.3.0
using Hl7.Fhir.Introspection;
using Hl7.Fhir.Model;
using Hl7.Fhir.Serialization;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
using System.Text.Json;


namespace Example
{
    public class Workspace
    {
        public string Id { get; set; }
        public string? Name { get; set; }
        public bool? Active { get; set; }
    }

    [Serializable]
    [DataContract]
    [FhirType("FhirWorkspace", "http://hl7.org/fhir/StructureDefinition/FhirWorkspace", IsResource = true)]
    public class FhirWorkspace : DomainResource
    {
        private readonly Workspace _original;

        [IgnoreDataMember]
        public Workspace Original => _original;

        public override string TypeName => nameof(FhirWorkspace);

        public override IEnumerable<ElementValue> NamedChildren
        {
            get
            {
                foreach (var elementPair in base.NamedChildren)
                {
                    yield return elementPair;
                }
                if (Name != null)
                {
                    yield return new ElementValue("name", Name);
                }
                if (Active != null)
                {
                    yield return new ElementValue("active", Active);
                }
            }
        }

        [DataMember]
        [FhirElement("name", Order = 90)]
        public FhirString Name
        {
            get => new FhirString(_original.Name);
            set => _original.Name = value?.Value;
        }

        [DataMember]
        [FhirElement("active", Order = 100)]
        public FhirBoolean Active
        {
            get => new FhirBoolean(_original.Active);
            set => _original.Active = value?.Value;
        }

        public FhirWorkspace()
            : this(new Workspace())
        {
        }

        public FhirWorkspace(Workspace original)
        {
            _original = original;
        }

        public override IDeepCopyable DeepCopy()
        {
            return new FhirWorkspace(new Workspace { Name = _original.Name });
        }

        protected override IEnumerable<KeyValuePair<string, object>> GetElementPairs()
        {
            foreach (var elementPair in base.GetElementPairs())
            {
                yield return elementPair;
            }
            if (Name != null)
            {
                yield return new KeyValuePair<string, object>("name", Name);
            }
            if (Active != null)
            {
                yield return new KeyValuePair<string, object>("active", Active);
            }
        }
    }

    [TestClass]
    public class FhirSerialization
    {
        private void TestJson1SerializerWith(Workspace workspace)
        {
            var serializer = new FhirJsonSerializer();
            var jsonContent = serializer.SerializeToString(new FhirWorkspace(workspace));

            var parser = new FhirJsonParser();
            var fhirWorkspace = parser.Parse<FhirWorkspace>(jsonContent);
            Assert.IsNotNull(fhirWorkspace);
            var workspace2 = fhirWorkspace.Original;

            Assert.AreEqual(workspace.Name, workspace2.Name);
            Assert.AreEqual(workspace.Active, workspace2.Active);
        }

        // Fails: System.FormatException: While building a POCO: Element 'name' must not repeat (at FhirWorkspace.name[0])
        [TestMethod]
        public void SerializerJson1Test()
        {
            TestJson1SerializerWith(new Workspace { Name = "ABC", Active = true });
        }

        // Fails: System.FormatException: While building a POCO: Element 'active' must not repeat (at FhirWorkspace.active[0])
        [TestMethod]
        public void SerializerJson1_EmptyString_Test()
        {
            TestJson1SerializerWith(new Workspace { Name = "", Active = true });
        }

        // Fails: System.FormatException: While building a POCO: Element 'active' must not repeat (at FhirWorkspace.active[0])
        [TestMethod]
        public void SerializerJson1_NullString_Test()
        {
            TestJson1SerializerWith(new Workspace { Name = null, Active = true });
        }

        // Ok
        [TestMethod]
        public void SerializerJson1_AllNull_Test()
        {
            TestJson1SerializerWith(new Workspace { Name = null, Active = null });
        }

        private void TestJson2SerializerWith(Workspace workspace)
        {
            var modelInspector = new ModelInspector(Hl7.Fhir.Specification.FhirRelease.R4);
            modelInspector.ImportType(typeof(FhirWorkspace));

            var options = new JsonSerializerOptions()
                .ForFhir(modelInspector);

            var jsonContent = JsonSerializer.Serialize(new FhirWorkspace(workspace), typeof(FhirWorkspace), options);

            var fhirWorkspace = JsonSerializer.Deserialize(jsonContent, typeof(FhirWorkspace), options) as FhirWorkspace;
            Assert.IsNotNull(fhirWorkspace);
            var workspace2 = fhirWorkspace.Original;

            Assert.AreEqual(workspace.Name, workspace2.Name);
            Assert.AreEqual(workspace.Active, workspace2.Active);
        }

        // OK
        [TestMethod]
        public void SerializerJson2Test()
        {
            TestJson2SerializerWith(new Workspace { Name = "ABC", Active = true });
        }

        // Fails: Properties cannot be empty strings
        [TestMethod]
        public void SerializerJson2_EmptyString_Test()
        {
            TestJson2SerializerWith(new Workspace { Name = "", Active = true });
        }

        // OK
        [TestMethod]
        public void SerializerJson2_NullString_Test()
        {
            TestJson2SerializerWith(new Workspace { Name = null, Active = true });
        }

        // Fails: An object needs to have at least one property
        [TestMethod]
        public void SerializerJson2_AllNull_Test()
        {
            TestJson2SerializerWith(new Workspace { Name = null, Active = null });
        }
    }
}

So my problems are:

• Should not the two serializers behave the same?
• I do not understand the "Element 'xx' must not repeat" messages.
• Is it really not allowed to have an entity without any property value?
• Why empty strings are a problem?
• What I am doing wrong?

[Sneedd]

About the problem with the Newtonsoft implementation, I figured out that I can provide an ModelInspector but that does not help. So for now I give up for this one, but if someone is intressted.

private void TestJson1SerializerWith(Workspace workspace)
{
    var modelInspector = new ModelInspector(Hl7.Fhir.Specification.FhirRelease.R4);
    modelInspector.ImportType(typeof(FhirWorkspace));

    var serializer = new CommonFhirJsonSerializer(modelInspector);           
    var jsonContent = serializer.SerializeToString(new FhirWorkspace(workspace));

    var parser = new BaseFhirParser(modelInspector);            
    string rootName = modelInspector.GetFhirTypeNameForType(typeof(FhirWorkspace));
    ISourceNode node = FhirJsonNode.Parse(jsonContent, rootName);
    var fhirWorkspace = parser.Parse<FhirWorkspace>(node);
    Assert.IsNotNull(fhirWorkspace);
    var workspace2 = fhirWorkspace.Original;

    Assert.AreEqual(workspace.Name, workspace2.Name);
    Assert.AreEqual(workspace.Active, workspace2.Active);
}

About my problems with System.Text.Json serializer, I also debugged deeper into it. And found this issue #2235, which means I need to validate my resource first myself. Never thought of that ... for performance reasons, hm ok ... but separating the code does not make it faster, only when you look at it in a specific angle. And what about bundles, I mean when the data structure resembles a hierarchy.

Here a validation which only works for the tests, more worse than anything else. Honestly I do not like this solution and will only just workaround these problems for me now.

// Base.Validate() will not work, because it is mostly not implemented e.g. Bundle
//     this means I need do something like this?
private void Validate(Resource resource)
{
    Stack<Base> stack = new Stack<Base>(new[] { resource });
    while (stack.Count > 0)
    {
        var current = stack.Pop();

        bool hasValue = false;
        foreach (var child in current.NamedChildren)
        {
            if (child.Value == null)
            {
                throw new InvalidOperationException("No nulls");
            }
            if (child.Value is FhirString value && string.IsNullOrEmpty(value.Value))
            {
                throw new InvalidOperationException("No null or empty strings");
            }
            if (child.Value is BackboneElement res)
            {
                stack.Push(res);
            }
            if (child.Value is Resource res2)
            {
                stack.Push(res2);
            }
            hasValue = true;
        }
        if (!hasValue)
        {
            throw new InvalidOperationException("No empty objects");
        }
    }
}

[ewoutkramer]

At first glance, it looks like you're doing the right thing. I'll create an issue to check out what's going on, since we haven't thoroughly tested doing custom resources.