Specifying problem.yaml, small changes suggested
#372
Comments
I'll get to the rest of your comments later, but I think it is a bad idea to make assumptions about what is or isn't a name. |
|
(I clarified under Q1 above why this is a question at all.) |
|
yes that key is meaningless for non multi-pass problems |
|
Q1: Yes, the correct way would be #Person = string | { name: string, email: string }I’ve tried to make that point some time ago. (It’s how we handle #Person = string | { name!: string, email?: string, orcid?: string }C4: Another issue I’ve flagged before. The number of passes should not be given in two different places in A more readable, useful, parseable and maintainable schema would like something like this: scoring?: *false | true
interactive?: *false | true
limits?:
validation_passes?: *1 | int &>1
*{ submit_answer?: false } | {
submit_answer: true,
interactive: false,
limit.validation_passes : 1
} |
|
C3: Interesting. So I think you’re saying time_multipliers?: {
ac_to_time_limit?: float & >=1 | *2.0
time_limit_to_tle?: float & >=1 | *1.5
}
time_resolution?: float & >0 | *1.0
time_limit: float & >0
["compilation_time" | "validation_time" ] : int & >0
[#SizeLimit]: int & >0This sounds good to me. |
|
I agree and prefer this (we also don't use something like output limit to decide interactive). But I don't understand the issue in the first place ^^' What is the problem with (btw. I would say that validation passes must be at least 2, its multi-pass not single pass...) |
We can do that, and I quite like the idea. That’s a different schema again. (This is exactly why we’re here.) Let me see if I can write that: if (type & [...]) != _|_ { // is type a list?
if list.Contains(type, "multi-pass") { // does the list contain "multi-pass"?
validation: passes: *2 | int & >1 // allow this field (else it's forbidden) with default 2
}
}Do we like this? (I do.) |
|
Yes, looking good 😄 Can two nested Also note that, in BAPCtools, we currently allow |
|
(No, CUE very deliberately does not allow that kind of combination. Evaluation order must be independent. There’s a deep reason for this, going back to compositionality, which I will not elucidate here.) Yes, the fact that if (type & [...]) != _|_ {
if list.Contains(type, "multi-pass") { validation_passes: *2 | int & >=2 }
}
if (type & string) != _|_ {
if type == "multi-pass" { validation_passes: *2 | int & >=2 }
}I observe again that scoring: *false | true
validation_passes: *1 | int & >= 1
# and so on.Readable, easy-to parse, accessible, easy to communicate, etc. No rules need ever be explained or checked or parsed or validated. A schema-aware editor would give useful feedback, etc. etc. |
|
With the above schema, we validate all of: ---
# Specify validation passes for multi-pass problem
limits:
validation_passes: 2 # this is the default
type: "multi-pass"
name: Hello
uuid: 550e8400-e29b-41d4-a716-446655440000
problem_format_version: 2023-02
---
# limits.validation_passes would default to 2
type: "multi-pass"
name: Hello
uuid: 550e8400-e29b-41d4-a716-446655440000
problem_format_version: 2023-02
---
# Type can be list
type: ["multi-pass"]
name: Hello
uuid: 550e8400-e29b-41d4-a716-446655440000
problem_format_version: 2023-02
---
type: ["multi-pass"]
limits:
validation_passes: 2 # this is the default
name: Hello
uuid: 550e8400-e29b-41d4-a716-446655440000
problem_format_version: 2023-02and reject both of the following: # Don't specify validation passes for problems that aren't multi-pass
limits:
validation_passes: 1
name: Hello
uuid: 550e8400-e29b-41d4-a716-446655440000
problem_format_version: 2023-02
---
# Multi-pass problems need at least two passes
type: "multi-pass"
limits:
validation_passes: 1
name: Hello
uuid: 550e8400-e29b-41d4-a716-446655440000
problem_format_version: 2023-02 |
|
(By the way, we're now continuing the discussion from #131 and #167, linking them for good measure) Your schema with two nested
Whether we split off |
|
I’ve summarised my thoughts on |
Here’s a CUE schema for
problem.yaml:You can play around with it live at https://cuelang.org/play/?id=w85MrRg1lkY#w=function&i=cue&f=eval&o=cue . In particular, play around with the YAML snippets at the bottom, which contain many examples of validated
problem.yaml. Please try to break the schema by inventing YAML that you think should validate (but doesn’t) and vice versa.Changes from Current Draft
The schema solidifies several unclear details in the prose specification, but I also introduced some changes. Here goes.
C1. Keywords
The spec says
I say
C2. Single Translators
The spec says
which disallows the spec's own example of
I say
where
C3. Limit time types
The spec is not clear about what numerical values are allowed for most of the limits. I have
C4. Default validation_passes
The spec has
validation_passesdefaulting to 2. I think it should beC5. Empty type sequence
The spec allows
which I find meaningless. I suggest
where
#Type: "pass-fail" | "scoring" | "multi-pass" | "interactive" | "submit-answer"C6. Mandatory credits.authors
I have
so that
authorsmust be present in acreditsstruct. This is more stringent than the draft spec.Q1. Question: #Person format
The draft spec has
I just have
This allows somebody called "@Lice" or "Shifty << McShiftyson". Should we forbid such names? How?
For instance, in the schema for
submissions.yamlI now maintain at https://github.com/thorehusfeldt/problem-package-schemas/blob/master/cue/submissions.cue I havewhich is slightly more restrictive with respect to
<and>; it would disallow "Shifty << McShiftyson".Q2. Question: Lenient Natural Language Codes
I have
rather than a large regex of the ISO-specified language codes that actually exist. I think this is fine; I don’t want us to have an opinion on whether
filexists compared toxyxorpo-BR.The text was updated successfully, but these errors were encountered: