-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Revisit samples #291
Revisit samples #291
Conversation
48f2e3c
to
846d49c
Compare
available for download, with any `.out` files renamed to `.ans` | ||
(replacing existing `.ans` files). When that also does not exist, the test cases in | ||
`data/sample` are available for download. `data/sample/download` is required for | ||
interactive problems. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a harsh requirement. What about cases where you don't provide a testing tool and thus don't have anything useful for contestants to download? Leaving data/sample/download/ empty is unfortunately not a viable option with git.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would there be no relevant files at all to download in these cases?
With the current spec, I believe you would get the .in
, and .ans
(I think?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would say there are no relevant files to download, indeed. In theory you could provide the .interaction file, but I think it'd be rather useless and probably confusing.
With current Kattis at least you're not allowed to download samples at all for interactive problems, a change which iirc was made ~3 years ago, and definitely one for the better, because before that we regularly had contestants confused about what the .in
and .ans
files meant (they weren't show in the statement online or their format explained in any way). I don't know if the old spec said anything about downloading samples in the first place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leaving data/sample/download/ empty is unfortunately not a viable option with git.
You can make adata/sample/download/.gitkeep
, and then we could specify that this particular.gitkeep
file is ignored in the download? (Kinda ugly, I know.)
The reason that I made data/sample/download
required is indeed that data/sample/statement
is never what you want to offer for download. But I hadn't considered that Kattis already doesn't make anything available for download anyway. Domjudge definitely does allow downloading the 'wrong' .in
and .ans
currently.
Instead we could say that for interactive problems, data/sample/download
doesn't fall back to data/sample/statement
but instead is empty by default?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That works, or you could say that it falls back to /statement but that .interaction files are excluded from download.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm that could maybe work. I'm wondering how often the 'internal' data/sample/*.{in,ans}
files for interactive problems have the same syntax as the public version for the testing tool.
Ah actually: providing .in
and .ans
really only makes sense if you have a testing tool, which should anyway go into data/sample/download
.
So either you do have a testing tool, and you choose the .in
and .ans
you want to provide with the corresponding 'public' syntax (which may or may not be the same as the 'private' syntax of data/sample/*.{in,ans}
), or you don't have a testing tool, and in that case providing .in
and .ans
files is kinda useless anyway.
So yes, making data/sample/download
empty by default for interactive problems instead of requiring it sounds good.
For each test case, `data/sample/statement` must contain one of the following for default or custom | ||
validation (non-interactive) problems: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that an .interaction file can only exist if there is an .in file in data/sample? That seems very restrictive -- I find that it somewhat frequently happens that you want to add an illustrative (perhaps incomplete because a full interaction might involve e.g. 10^5 queries?) sample interaction that doesn't have a corresponding .in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As written I would interpret it to mean that, yes.
That was not the intention. You should be allowed to "create" new "test cases" for the statement only by adding them here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, the for each test case
here is meant to be parsed as something like 'for each basename for which a data/sample/statement/basename.ext
is found with some recognised extension, ...'.
Maybe we should additionally clarify that no other files may exist at all anyway.
- A `.in` and `.out` file, which are both shown, neither of which is validated. | ||
- A `.interaction` file that contains lines starting with `<` and `>`, | ||
containing an interaction log with output from the interactor (`<`) and | ||
submission (`>`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(completely unrelated to the spec changes, but: it might be nice to allow "..." ellipsis lines too)
Fix #263, hopefully