Add threshold for statistics #465
Conversation
| let private printThresholdStats (scnStats: ScenarioStats) = | ||
| let headers = ["threshold"; "status"] | ||
| let rows = | ||
| scnStats.ThresholdStats | ||
| |> Option.map (ReportHelper.ThresholdStats.createTableRows Console.okEscColor Console.errorEscColor) | ||
| |> Option.defaultValue List.empty | ||
|
|
||
| [ Console.addLine $"thresholds for scenario: {Console.okColor scnStats.ScenarioName}" | ||
| Console.addTable headers rows ] |
There was a problem hiding this comment.
That's how it looks like:
I'm not sure if this would be sufficient, and if not, I'm not sure at the moment how we could print a string representation for the thresholds predicates.
There was a problem hiding this comment.
For F# we could use https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/code-quotations
But on the other hand, it will be a deal-breaker for C#.
I am not sure but maybe this project can help here https://mbraceproject.github.io/FsPickler/
I don't exactly know if it is possible to somehow fetch the AST representation of the Threshold function/delegate but it would be awesome.
There was a problem hiding this comment.
Also, we could have an additional API that will accept the optional string argument that the user will fill, and it will be printed if it's not empty. But maybe, for now, we can leave all this "Fetching Meta Info" machinery as for v2?
There was a problem hiding this comment.
Hi @pavlogrushetsky
You can take a look how this library is working for F#
https://github.com/SwensenSoftware/unquote
There was a problem hiding this comment.
Thank you! I will take a look.
There was a problem hiding this comment.
@AntyaDev It looks like I could utilize Unquote library 😀. I've played with it a little bit, and was able to specify a threshold using quotation expression. The work is still in progress, but I think, I would need to separate Threshold model to have one in Contracts with function union cases (like int -> bool), and one in DomainTypes with Quotations.Expr union cases (like Quotations.Expr<int -> bool>). This would allow us to keep the consumer code as simple as possible, to not to have something like the following:
|> Scenario.withThresholds [
RequestAllCount(<@ fun x -> x = 3 @>)
]and instead be able to specify thresholds using plain functions:
|> Scenario.withThresholds [
RequestAllCount(fun x -> x = 3)
]I would just need to translate Contracts.Threshold model to DomainTypes.Threshold model, like you do for Scenario.
@AntyaDev Please, let me know if this makes sense to you.
There was a problem hiding this comment.
Yes, it sounds reasonable!
There was a problem hiding this comment.
Hey @AntyaDev 👋 I've played a bit with quotation expressions, and it turned out that the text representation for the decompiled predicates isn't really human-readable. I think, it's not worth implementing something very complicated, at least at this point.
So, I've added an ability to override a default threshold description. Here is how it looks in C#:
In F#, it looks a little bit ugly:
To improve this syntax for F# users, I've implemented a simple computation expression, so that it would be possible to specify a description as a real optional parameter, like the following:
I've also improved the console output to make it more human readable. Here is an example:
@AntyaDev Please, let me know what you think about this 😅
There was a problem hiding this comment.
Looks very good, I like how F# and C# look like.
Btw what was the issue with F# quotations? I thought that you would not be able to use it for C# and to use it for F#, you should wrap the predicate into <@ predicate @>
There was a problem hiding this comment.
Btw what was the issue with F# quotations? I thought that you would not be able to use it for C# and to use it for F#, you should wrap the predicate into <@ predicate @>
Yep, I was able to wrap the predicates in <@ @> in the domain logic and keep using just plain lambdas like fun x -> x > 10 in both C# and F#, and the thresholds evaluation logic worked good. And main reason to do this was to get a string representation for those quotation expressions, but it turned out that it is not really human readable when I tried to de-compile them.
|
@AntyaDev One more question 😅 Would the following functionality make sense to you?:
|
|
"Specifying thresholds through the configuration files" - seems can be skipped for now :) |
Great! Makes sense to me, thank you! |
…ride-description Implement thresholds CE
AntyaDev
force-pushed
the
dev
branch
4 times, most recently
from
4678a91 to
f2fbc64
Compare
|
Hi @pavlogrushetsky |
Oh, I've been busy recently, sorry. I'll try to push something this week. |
AntyaDev
force-pushed
the
dev
branch
3 times, most recently
from
9960ae0 to
824134b
Compare
AntyaDev
force-pushed
the
dev
branch
2 times, most recently
from
b6fd14d to
14b6b47
Compare
AntyaDev
force-pushed
the
dev
branch
5 times, most recently
from
60b2d02 to
16d6057
Compare
AntyaDev
force-pushed
the
dev
branch
6 times, most recently
from
cee3bec to
7295146
Compare
AntyaDev
force-pushed
the
dev
branch
2 times, most recently
from
4ca4c86 to
fc702e8
Compare
This is a draft PR for #355 (Add threshold for statistics). The work is still in progress, and the PR is rather for review and conversation about the overall approach.
Some other things TODO:
DataTransferStatsSpecifying thresholds through the configuration filesI would appreciate any feedback on this draft implementation 🙇♂️.