null support for NutritionalValues

[Dieterbe]

the following fields should be nullable. they can be null if the data is missing in OFF. currently we set them to 0, which is a bug

• fiber
• sugar
• maybe also fatSaturated, sodium // need to confirm exactly which fields can be null.

the wger backend is supposed to already handle this properly, but we don't properly handle this in flutter.

[Denpeer]

For the ingredient model in the backend we have:

language: Not Null
created: Not Null
last_update: Can be Null
uuid: Not Null
name: Not Null
energy: Not Null
protein: Not Null
carbohydrates: Not Null
carbohydrates_sugar: Can be Null
fat: Not Null
fat_saturated: Can be Null
fiber: Can be Null
sodium: Can be Null
code: Can be Null
remote_id: Can be Null
source_name: Can be Null
source_url: Can be Null
last_imported: Can be Null
common_name: Can be Null
category: Can be Null
brand: Can be Null

This would translate to

  double energy;
  double protein;
  double carbohydrates;
  double? carbohydratesSugar;
  double fat;
  double? fatSaturated;
  double? fiber;
  double? sodium;

How do we want to handle cases where a list of NutritionalValues objects has some objects with a null for fatSaturated (for example) and some without. Adding all the values and returning will give a lower total fatSaturated than in is really in the products.

We can:

• Add all the NutritionalValues with a non null value for the nullable objects and accept that aggregate Nutritional information (like in a meal) might have lower, possibly incorrect values like we do now.
• Do some UI rework to show that some Ingredients in a meal do not have their fatSaturated (for example) values defined so you can not glance the amount of saturated fat from this summary.

The second option feels more correct, but it would increase the complexity. I know a commercial app like myfitnesspal chooses the first option presumably for this reason. @Dieterbe what are your thoughts?

[Dieterbe]

So, we are effectively talking about aggregates of nullable values.
Your first proposal is treat null as zero (which, as you say, by itself is incorrect), your second is to make the aggregate itself null (more correct but also not that useful)
Perhaps there is a 3rd way, sort of a combination of both: ignore nulls (treat them as zero) but add an additional marker that the information may be incomplete. Therefore the returned value is a "lower limit". To achieve this, we could introduce a new custom datatype. (Basically a double and bool to describe whether it is "complete" or should be interpreted as a low limit)

I have thought in the past there was a use case for upper limits as well, but I can't remember what it was.

[Denpeer]

@Dieterbe What do you think of something like this for the visuals?


With Tooltip:

[Dieterbe]

ok i just remembered some use cases for having an upper limit:

• sugars cannot be more than carbs. (although still need to figure out how OFF deals with american vs europen labeling. in EU we don't classify sugar under carbs)
• sugars is limited by kcals
• saturated fats is limited by total fats
• fibers is limited to total ingredient weight
• sodium is limited to total ingredient weight (although in practice it should typically be much less than the ingredient itself, except for when you're eating pure salt, an electrolyte mix, etc)
• in the future we may support some kind of "free meals" where you can log a meal or ingredient but not so precisely. e.g. "i ate something that was between 400 and 600 kcal, it had between 20 and 40 g protein" (from this we can also calculate ranges of carbs/sugars/fats)

therefore i suggest we introduce a new custom datatype, called DoubleRange or something, which has a low and a high double value. they could equal (for precisely known values), and they could also be 0 or double.infinite. obviously math operations such as addition etc needs to be built so we can still compute aggregates based on them, as accurately as possible

we then need to be able to render them. my suggestion is something like this:

case	vis proposal 1	vis proposal 2
precise value	123 g	123 g
low limit	123< g	123 - ∞ g
hi limit	<123 g	0 - 123 g
range	112+-12 g	100-123 g

i wouldn't use those warning triangles you show, however if we have any non-precise values, i would show a banner similar to what you show, which on tap woud show a dialog with more information about what these values mean

[rolandgeider]

Those are good rules! At the moment we only apply some very simple ones when importing the ingredients. I can run them on the DB and see if there are any that cause problems.

Also, the upper limit will always be 100% of the product weight, no need for infinity:)

[rolandgeider]

FYI, from the 2.443.821 ingretients in the db:

	No filter	USDA	OFF
saturated fats	192.995	115.809	76.272
sugars	168.815	115.809	52.089
fiber	1.277.948.	19.539	1.257.455
sodium	291.769	398	290429

are NULL