Dealing with numbers that are too large to be parsed into a decimal

[chgl]

I don't think this is an actual bug, so I am mostly looking for guidance/ideas.

The following code fails:

using Hl7.Fhir.Model;
using Hl7.Fhir.Serialization;

var fhirJson = @"{
  ""valueQuantity"": {
    ""value"": 6.5742407583884805e+84,
  }
}";

var parser = new FhirJsonParser();
var obs = parser.Parse<Observation>(fhirJson);

with System.FormatException: Invalid Json encountered. Details: Input string '6.5742407583884805e+84' is not a valid decimal. which seems fair. However I am unsure what the best way to move forward is. The value comes from a source system and the resource is generated using custom Python code (using the fhir.resources library). I am now trying to parse and process it further using a dotnet service and this library. In my particular case I am actually not interested in the value. Is it possible to skip some paths when deserializing?

[ewoutkramer]

It's an interesting question that pops up now and then (with codes, date times, decimals....). Whenever we have a design session for issues like this, we realize that the follow-up question is: if we allowed this data to pass validation and parsing, what should the value of the Quantity be? We're afraid that if we make this a setting, and, say, we set the data to null, this will cause missed or wrongly interpreted data later (will your colleague who uses your code realize you have disabled giving parsing errors for this field?)).

You probably don't want to always allow incorrect data in datetimes or decimals, but maybe only in certain instances.

What would you ideally like the SDK to do in this case?

[ewoutkramer]

This is exactly what we currently do for incorrect enum values - but it still means it's easy for the user of the data to thing there was no value there (while there is a value, but it was incorrect). That feels unsafe, so, although we have done this in the past for enums, I am wondering whether this is the right way to proceed.

@marcovisserFurore @brianpos, @brendankowitz - do you have an opinion on this?

[brianpos]

Another thought would be to put it into an Annotation on the object for that value, and report the outcome into the processing instance.
I could try that out with my new XML parser to see if that is any better?
This would then not "dirty" up the object model, but permit you to have the real value in there too.
Not sure that I'd want to change the way we're doing enums or dates though. As they already have that string representation in the model.
But I guess could use the same annotation onto the model also.

[brianpos]

Re-looking at the notes above, are you really looking to hold onto the values themselves as a part of the model, and not just get past the parser exceptions?

[brendankowitz]

I'm not sure there are any great options here. Like @brianpos points out, unless there is support via a "BigDecimal" structure in dotnet or otherwise, everything else is working around the problem.

Is this just a problem when using the POCOs, if the processing is read-only is FhirJsonNode.Parse an option here?

[chgl]

Re-looking at the notes above, are you really looking to hold onto the values themselves as a part of the model, and not just get past the parser exceptions?

Unfortunately at least in my case the processing isn't just read-only: I am modifying parts of the resource and then serialize and pass forward the (entire) changed resource - effectively what FHIR-Tools-for-Anonymization does. So while I am not using the numeric value, it should somehow be kept until the resource is serialized again.

[brianpos]

I don't know if we should consider using something like this (and can't speak for the quality of this code either)
https://gist.github.com/nberardi/2667136
I believe there are really only a few places where we actually need to use the full scale of the type, such as in Observation.
And a few other options listed here
https://stackoverflow.com/questions/2863388/what-is-the-equivalent-of-the-java-bigdecimal-class-in-c