Compare DataModel Reflection Decode Perfomance

[Atralupus]

#1940 I was trying to solve this issue
However, more conditional codes were required for null type
So I changed the decode code using Reflection
Reflection codes are heavy, so I need to check the performance difference

https://github.com/planetarium/libplanet/releases/tag/0.34.0 Code Bench result

BenchDataModel.Encode: DefaultJob
Runtime = .NET Core 6.0.4 (CoreCLR 6.0.422.16404, CoreFX 6.0.422.16404), Arm64 RyuJIT; GC = Concurrent Workstation
Mean = 265.537 us, StdErr = 0.233 us (0.09%), N = 15, StdDev = 0.903 us
Min = 264.305 us, Q1 = 264.885 us, Median = 265.462 us, Q3 = 266.026 us, Max = 267.576 us
IQR = 1.141 us, LowerFence = 263.174 us, UpperFence = 267.737 us
ConfidenceInterval = [264.572 us; 266.502 us] (CI 99.9%), Margin = 0.965 us (0.36% of Mean)
Skewness = 0.53, Kurtosis = 2.48, MValue = 2
-------------------- Histogram --------------------
[263.825 us ; 268.056 us) | @@@@@@@@@@@@@@@
---------------------------------------------------

BenchDataModel.Decode: DefaultJob
Runtime = .NET Core 6.0.4 (CoreCLR 6.0.422.16404, CoreFX 6.0.422.16404), Arm64 RyuJIT; GC = Concurrent Workstation
Mean = 140.451 us, StdErr = 0.028 us (0.02%), N = 13, StdDev = 0.100 us
Min = 140.280 us, Q1 = 140.388 us, Median = 140.442 us, Q3 = 140.516 us, Max = 140.666 us
IQR = 0.129 us, LowerFence = 140.195 us, UpperFence = 140.709 us
ConfidenceInterval = [140.331 us; 140.571 us] (CI 99.9%), Margin = 0.120 us (0.09% of Mean)
Skewness = 0.35, Kurtosis = 2.55, MValue = 2
-------------------- Histogram --------------------
[140.224 us ; 140.722 us) | @@@@@@@@@@@@@
---------------------------------------------------

| Method |     Mean |   Error |  StdDev |
|------- |---------:|--------:|--------:|
| Encode | 272.0 us | 4.19 us | 3.50 us |
| Decode | 143.1 us | 0.90 us | 0.80 us |

Fix Reflection Decode Code Bench result

BenchDataModel.Encode: DefaultJob
Runtime = .NET Core 6.0.4 (CoreCLR 6.0.422.16404, CoreFX 6.0.422.16404), Arm64 RyuJIT; GC = Concurrent Workstation
Mean = 265.836 us, StdErr = 0.152 us (0.06%), N = 14, StdDev = 0.569 us
Min = 264.637 us, Q1 = 265.854 us, Median = 266.017 us, Q3 = 266.197 us, Max = 266.446 us
IQR = 0.342 us, LowerFence = 265.341 us, UpperFence = 266.710 us
ConfidenceInterval = [265.194 us; 266.477 us] (CI 99.9%), Margin = 0.641 us (0.24% of Mean)
Skewness = -1.07, Kurtosis = 2.63, MValue = 2
-------------------- Histogram --------------------
[264.327 us ; 266.756 us) | @@@@@@@@@@@@@@
---------------------------------------------------

BenchDataModel.Decode: DefaultJob
Runtime = .NET Core 6.0.4 (CoreCLR 6.0.422.16404, CoreFX 6.0.422.16404), Arm64 RyuJIT; GC = Concurrent Workstation
Mean = 611.511 us, StdErr = 0.184 us (0.03%), N = 14, StdDev = 0.687 us
Min = 610.398 us, Q1 = 611.133 us, Median = 611.232 us, Q3 = 612.191 us, Max = 612.472 us
IQR = 1.058 us, LowerFence = 609.547 us, UpperFence = 613.778 us
ConfidenceInterval = [610.736 us; 612.285 us] (CI 99.9%), Margin = 0.775 us (0.13% of Mean)
Skewness = 0.05, Kurtosis = 1.41, MValue = 2
-------------------- Histogram --------------------
[610.024 us ; 612.846 us) | @@@@@@@@@@@@@@
---------------------------------------------------

| Method |     Mean |   Error |  StdDev |
|------- |---------:|--------:|--------:|
| Encode | 265.8 us | 0.64 us | 0.57 us |
| Decode | 611.5 us | 0.77 us | 0.69 us |

We need to discuss whether we can apply it.🤔
if we doesn't want to use a reflection. check above link or more conditional codes
https://discord.com/channels/928926944937013338/929910863123591199/974213134367739914

---

bench file

using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using BenchmarkDotNet.Attributes;
using Libplanet.Store;
using Libplanet.Tests.Store;
using BTypes = Bencodex.Types;

namespace Libplanet.Benchmarks
{
    public class BenchDataModel
    {
        private DataModelTree3 dataTree;
        private BTypes.Dictionary encodedDataTree;

        public BenchDataModel()
        {
            dataTree = new DataModelTree3();
            encodedDataTree = dataTree.Encode();
        }

        [Benchmark]
        public void Encode()
        {
            dataTree.Encode();
        }

        [Benchmark]
        public void Decode()
        {
            DataModelTree3 temp = new DataModelTree3(encodedDataTree);
        }

        private class DataModelForBench : DataModel
        {
            public DataModelForBench()
                : base()
            {
                BigList = Enumerable.Repeat(10, 1000).AsEnumerable().ToImmutableList<int>();
                BigDict = Enumerable
                    .Range(0, 1000)
                    .Select(_ => new KeyValuePair<string, string>(
                        "test",
                        "test"))
                    .ToImmutableDictionary();
            }

            public DataModelForBench(Bencodex.Types.Dictionary encoded)
                : base(encoded)
            {
            }

            public ImmutableList<int> BigList { get; private set; }

            public ImmutableDictionary<string, string> BigDict { get; private set; }
        }

        private class DataModelTree1 : DataModel
        {
            public DataModelTree1()
                : base()
            {
                Data1 = new DataModelForBench();
                Data2 = new DataModelTest.RootModel();
            }

            public DataModelTree1(Bencodex.Types.Dictionary encoded)
                : base(encoded)
            {
            }

            public DataModelForBench Data1 { get; private set; }

            public DataModelTest.RootModel Data2 { get; private set; }
        }

        private class DataModelTree2 : DataModel
        {
            public DataModelTree2()
                : base()
            {
                Data1 = new DataModelTree1();
                Data2 = new DataModelForBench();
            }

            public DataModelTree2(Bencodex.Types.Dictionary encoded)
                : base(encoded)
            {
            }

            public DataModelTree1 Data1 { get; private set; }

            public DataModelForBench Data2 { get; private set; }
        }

        private class DataModelTree3 : DataModel
        {
            public DataModelTree3()
                : base()
            {
                Data1 = new DataModelTree2();
                Data2 = new DataModelForBench();
            }

            public DataModelTree3(Bencodex.Types.Dictionary encoded)
                : base(encoded)
            {
            }

            public DataModelTree2 Data1 { get; private set; }

            public DataModelForBench Data2 { get; private set; }
        }
    }
}

Reflection Decode

private static object DecodeFromListIValue(BTypes.List list, Type type)
{
    if (type.IsGenericType && type.GetGenericTypeDefinition() == typeof(ImmutableList<>))
    {
        Type[] genericTypes = type.GetGenericArguments();
        Type genericType = genericTypes[0];

        var tempList = Activator.CreateInstance(
            typeof(List<>).MakeGenericType(genericType), true)
                ?? throw new NullReferenceException();
        var method = tempList.GetType().GetMethod("Add")
            ?? throw new NullReferenceException();

        foreach (var elem in list.Select(x => DecodeFromIValue(x, genericType)))
        {
            method.Invoke(tempList, new object[] { elem });
        }

        switch (tempList)
        {
            case List<bool> listBool:
                return listBool.ToImmutableList();
            case List<int> listInt:
                return listInt.ToImmutableList();
            case List<long> listLong:
                return listLong.ToImmutableList();
            case List<BigInteger> listBigInteger:
                return listBigInteger.ToImmutableList();
            case List<ImmutableArray<byte>> listBytes:
                return listBytes.ToImmutableList();
            case List<Address> listAddress:
                return listAddress.ToImmutableList();
            case List<string> listString:
                return listString.ToImmutableList();
            default:
                throw new ArgumentException();
        }
    }
    else
    {
        throw new ArgumentException(
            $"Invalid target property type {type} encountered.");
    }
}

[greymistcube]

I don't think 4x performance discrepancy can be easily ignored. Also it would be nice to see the difference for each level.

Something like

[Benchmark]
public void DecodeFlatOriginal()
{
	DataModelFlatOriginal temp = new DataModelFlatOriginal(encodedFlatData);
}

[Benchmark]
public void DecodeFlatNew()
{
	DataModelFlatNew temp = new DataModelFlatNew(encodedFlatData);
}

[Benchmark]
public void DecodeNestedOriginal()
{
	DataModelNestedOriginal temp = new DataModelNestedOriginal(encodedNestedData);
}

[Benchmark]
public void DecodeNestedNew()
{
	DataModelNestedNew temp = new DataModelNestedNew(encodedNestedData);
}

where

public class DataModelFlatOriginal : DataModel { ... }
public class DataModelNestedOriginal : DataModel { ... }
public class DataModelFlatNew : DataModelNew { ... }
public class DataModelNestedNew : DataModelNew { ... }

with DataModel being the original as in #1935, and DataModelNew being the one with additional reflection above, might give a better picture with what we are dealing with.

[greymistcube]

Here's some preliminary result using #1935 and generally how it is implemented in lib9c. You can see the code I used at https://gist.github.com/greymistcube/a1e63e5e1f0a160ed9ecf0697115be33.

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19044
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.1.413
  [Host]     : .NET Core 3.1.19 (CoreCLR 4.700.21.41101, CoreFX 4.700.21.41603), X64 RyuJIT
  DefaultJob : .NET Core 3.1.19 (CoreCLR 4.700.21.41101, CoreFX 4.700.21.41603), X64 RyuJIT

Method	Mean	Error	StdDev
EncodeRoot	4,777.8 μs	91.61 μs	105.50 μs
EncodeLeaf	1,546.5 μs	23.65 μs	20.96 μs
EncodePlainValueLeaf	1,192.8 μs	13.04 μs	12.20 μs
DecodeRoot	3,022.3 μs	49.53 μs	46.33 μs
DecodeLeaf	971.4 μs	18.99 μs	19.50 μs
DecodePlainValueLeaf	801.8 μs	15.42 μs	21.61 μs

[Atralupus]

#1956
Another way has been discovered