Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Farkle benchmarks. #129

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

teo-tsirpanis
Copy link

Hello, I have created a parsing library called Farkle. Its API bears a strong resemblance to parser combinators, but in contrast with ordinary parser combinator libraries like Pidgin, Sprache and FParsec, it uses bottom-up LALR(1) parsing under the hood. It would be fun to add it to the benchmarks, and would help tuning the performance of both Farkle and Pidgin.

I ran it on my machine, and the enclosed results are interesting. In the JSON benchmarks, Farkle is a solid contender, finishing always in the top three in both time and memory, and finishing first in the deep JSON files benchmark. In the expression benchmarks, Farkle always finishes second in time, but uses significantly more memory than Pidgin and FParsec.

Let me know if you have any questions.

JSON benchmarks.
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19044
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=7.0.100-preview.2.22153.17
  [Host]     : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
  DefaultJob : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
Method Mean Error StdDev Median Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated
BigJson_Pidgin 456.5 us 5.86 us 5.20 us 456.0 us 1.00 0.00 33.2031 - - 101.7 KB
BigJson_Sprache 3,579.2 us 71.09 us 157.54 us 3,575.0 us 7.81 0.31 1746.0938 - - 5349.63 KB
BigJson_Superpower 2,277.5 us 37.81 us 33.52 us 2,284.5 us 4.99 0.10 296.8750 - - 913.43 KB
BigJson_FParsec 361.2 us 7.22 us 15.70 us 359.1 us 0.81 0.04 76.6602 - - 235.93 KB
BigJson_Farkle 545.6 us 7.40 us 6.92 us 546.7 us 1.20 0.02 68.3594 - - 209.54 KB
LongJson_Pidgin 414.5 us 8.13 us 11.40 us 412.9 us 1.00 0.00 33.6914 - - 104.25 KB
LongJson_Sprache 2,783.8 us 55.17 us 95.17 us 2,761.2 us 6.76 0.30 1406.2500 - - 4311.36 KB
LongJson_Superpower 1,809.0 us 13.58 us 12.04 us 1,806.1 us 4.34 0.10 230.4688 - - 706.79 KB
LongJson_FParsec 314.0 us 5.43 us 8.13 us 311.2 us 0.76 0.03 74.7070 - - 230.3 KB
LongJson_Farkle 449.6 us 4.60 us 4.08 us 448.6 us 1.08 0.03 57.1289 - - 175.31 KB
DeepJson_Pidgin 533.0 us 9.77 us 26.92 us 524.8 us 1.00 0.00 66.4063 - - 205.29 KB
DeepJson_Sprache 3,108.7 us 63.99 us 183.59 us 3,074.9 us 5.86 0.47 730.4688 222.6563 - 2946.56 KB
DeepJson_FParsec 328.3 us 6.54 us 17.23 us 325.7 us 0.62 0.04 62.0117 - - 190.43 KB
DeepJson_Farkle 318.7 us 5.73 us 8.92 us 315.9 us 0.57 0.03 48.3398 - - 148.17 KB
WideJson_Pidgin 262.9 us 4.64 us 4.34 us 261.6 us 1.00 0.00 15.6250 - - 48.42 KB
WideJson_Sprache 1,580.5 us 25.60 us 34.17 us 1,572.4 us 6.05 0.17 912.1094 - - 2797.28 KB
WideJson_Superpower 1,145.3 us 22.73 us 31.86 us 1,139.4 us 4.38 0.14 148.4375 - - 459.74 KB
WideJson_FParsec 213.2 us 4.01 us 4.29 us 211.8 us 0.81 0.02 33.9355 - - 104.63 KB
WideJson_Farkle 350.5 us 6.99 us 18.29 us 345.2 us 1.32 0.06 34.1797 - - 105.56 KB
Expression benchmarks.
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19044
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=7.0.100-preview.2.22153.17
  [Host]     : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
  DefaultJob : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
Method Mean Error StdDev Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated
LongInfixL_Pidgin 426,214.4 ns 8,470.92 ns 12,678.88 ns 11.78 0.34 - - - 136 B
LongInfixR_Pidgin 422,279.7 ns 1,854.75 ns 1,644.19 ns 11.50 0.05 - - - 128 B
LongInfixL_FParsec 36,710.6 ns 107.03 ns 89.37 ns 1.00 0.00 0.0610 - - 200 B
LongInfixR_FParsec 42,116.8 ns 227.04 ns 212.38 ns 1.15 0.01 0.0610 - - 200 B
LongInfixL_Farkle 309,182.5 ns 908.00 ns 804.92 ns 8.42 0.04 51.2695 - - 160976 B
LongInfixR_Farkle 320,691.2 ns 1,242.24 ns 1,161.99 ns 8.73 0.03 66.4063 - - 208832 B
ShortInfixL_Pidgin 1,420.0 ns 5.79 ns 5.13 ns 12.88 0.08 0.0420 - - 136 B
ShortInfixR_Pidgin 1,427.4 ns 5.90 ns 5.52 ns 12.94 0.09 0.0401 - - 128 B
ShortInfixL_FParsec 110.3 ns 0.54 ns 0.48 ns 1.00 0.00 0.0637 - - 200 B
ShortInfixR_FParsec 109.9 ns 0.17 ns 0.15 ns 1.00 0.00 0.0637 - - 200 B
ShortInfixL_Farkle 679.9 ns 1.70 ns 1.59 ns 6.17 0.03 0.4129 - - 1296 B
ShortInfixR_Farkle 679.8 ns 2.38 ns 2.11 ns 6.17 0.04 0.4129 - - 1296 B

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant