Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit samples #291

Merged
merged 9 commits into from
Aug 18, 2024
Merged

Revisit samples #291

merged 9 commits into from
Aug 18, 2024

Conversation

RagnarGrootKoerkamp
Copy link
Collaborator

@RagnarGrootKoerkamp RagnarGrootKoerkamp commented Aug 18, 2024

Fix #263, hopefully

@RagnarGrootKoerkamp RagnarGrootKoerkamp changed the title Problem statement samples and download samples Revisit samples Aug 18, 2024
available for download, with any `.out` files renamed to `.ans`
(replacing existing `.ans` files). When that also does not exist, the test cases in
`data/sample` are available for download. `data/sample/download` is required for
interactive problems.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a harsh requirement. What about cases where you don't provide a testing tool and thus don't have anything useful for contestants to download? Leaving data/sample/download/ empty is unfortunately not a viable option with git.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be no relevant files at all to download in these cases?

With the current spec, I believe you would get the .in, and .ans (I think?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say there are no relevant files to download, indeed. In theory you could provide the .interaction file, but I think it'd be rather useless and probably confusing.

With current Kattis at least you're not allowed to download samples at all for interactive problems, a change which iirc was made ~3 years ago, and definitely one for the better, because before that we regularly had contestants confused about what the .in and .ans files meant (they weren't show in the statement online or their format explained in any way). I don't know if the old spec said anything about downloading samples in the first place.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving data/sample/download/ empty is unfortunately not a viable option with git.
You can make a data/sample/download/.gitkeep, and then we could specify that this particular .gitkeep file is ignored in the download? (Kinda ugly, I know.)

The reason that I made data/sample/download required is indeed that data/sample/statement is never what you want to offer for download. But I hadn't considered that Kattis already doesn't make anything available for download anyway. Domjudge definitely does allow downloading the 'wrong' .in and .ans currently.

Instead we could say that for interactive problems, data/sample/download doesn't fall back to data/sample/statement but instead is empty by default?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works, or you could say that it falls back to /statement but that .interaction files are excluded from download.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm that could maybe work. I'm wondering how often the 'internal' data/sample/*.{in,ans} files for interactive problems have the same syntax as the public version for the testing tool.

Ah actually: providing .in and .ans really only makes sense if you have a testing tool, which should anyway go into data/sample/download.
So either you do have a testing tool, and you choose the .in and .ans you want to provide with the corresponding 'public' syntax (which may or may not be the same as the 'private' syntax of data/sample/*.{in,ans}), or you don't have a testing tool, and in that case providing .in and .ans files is kinda useless anyway.

So yes, making data/sample/download empty by default for interactive problems instead of requiring it sounds good.

spec/2023-07-draft.md Show resolved Hide resolved
spec/2023-07-draft.md Show resolved Hide resolved
Comment on lines +600 to +601
For each test case, `data/sample/statement` must contain one of the following for default or custom
validation (non-interactive) problems:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that an .interaction file can only exist if there is an .in file in data/sample? That seems very restrictive -- I find that it somewhat frequently happens that you want to add an illustrative (perhaps incomplete because a full interaction might involve e.g. 10^5 queries?) sample interaction that doesn't have a corresponding .in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As written I would interpret it to mean that, yes.

That was not the intention. You should be allowed to "create" new "test cases" for the statement only by adding them here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, the for each test case here is meant to be parsed as something like 'for each basename for which a data/sample/statement/basename.ext is found with some recognised extension, ...'.
Maybe we should additionally clarify that no other files may exist at all anyway.

- A `.in` and `.out` file, which are both shown, neither of which is validated.
- A `.interaction` file that contains lines starting with `<` and `>`,
containing an interaction log with output from the interactor (`<`) and
submission (`>`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(completely unrelated to the spec changes, but: it might be nice to allow "..." ellipsis lines too)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mixing of .in and .interaction samples
4 participants