UI/Design Change: Renaming Version (.vX) to Something less Confusing.

[LennartPurucker]

Currently, the website/UI has a button for dataset versions (e.g., https://openml.org/search?type=data&sort=version&status=any&order=asc&exact_name=nyc-taxi-green-dec-2016), but as far as I can tell, these are not continuous versions of the same dataset but rather a tag for all datasets with the same name.

This has caused much confusion among researchers and practitioners trying to find source datasets or continuously versioning their datasets.
Furthermore, computer scientists likely associate versions with something akin to a git history.

I propose to rename this feature in the website to something less ambiguous (like name-alikes)
This would be a relatively small change only in the website/UI to avoid many problems in practice. We could still rename it back, once a "real" version history exists.

What do you think? @PGijsbers @joaquinvanschoren

Rel.: openml/openml.org#95

[PGijsbers]

I would agree that the current use of version is rather unintuitive. Especially since there's no enforcement that they're actually based from the same data (it is as you said: just based on the "name" of the dataset). It does make some sense that e.g., a binarized version of multiclass classification is "some version of" the same dataset, but I can imagine that many people will consider the more strict association with versioning initially (as would I); primarily to remove errors from the data or maybe in some cases adding additional samples.

But then the question becomes: how should we do it?
Should user B be able to create an iteration/version of dataset A uploaded by user A? Or should only user A be able to do that? How do we check that it's really "a version" akin to git history as opposed to some other relationship? Can there be two distinct datasets with the same name? E.g., a penguin dataset with tabular data vs. a penguin dataset with images of penguins?

This change will also impact the client libraries (e.g., python allows (get_dataset(name="penguin", version=42)). I would want more clarity on what we are working towards before changing things. We could add a simple explanation to the website in the meantime.

[LennartPurucker]

Thanks for transforming this into a discussion (I did not think of it)!

The perfect solution would be a git history per dataset with forks and clones. This would be a huge change and a lot of work. I see this as a future future goal. Moreover, I think that any dataset could have any name and that we should instead opt for titles of the dataset (akin to paper tiles for dataset papers) rather than short nicknames as we have them right now. The nicknames could then become tags, which is essentially what versions are now.

Rel. client: We could rename it on the webpage but keep it the same in the API (with an updated docstring). This kind of confusion is better, as the user would need to read the docstring even to find what the version parameter does/is.

I think only a few people use versions to retrieve datasets using get_dataset(name="penguin", version=42). Furthermore, I firmly believe users should not do this anyway; instead, they should use the ID. I would even go as far as not allowing the use of names or versions for data retrieval. I suggest avoiding bad behavior by design, even if it limits edge use cases.

An explanation would be a good start, but it is likely insufficient.
How likely are users to read the explanations? The name is too strong of a signal to most users.

[PGijsbers]

This kind of confusion is better, as the user would need to read the docstring even to find what the version parameter does/is.

Eh, I agree that we should have clear communication of what a version vs commit is (or whatever the terms may become), but I don't believe that using distinct terms based on the interface is a good thing. OpenML already has a lot of different concepts that people sometimes have problems understanding, introducing multiple terms for each only compounds that problem.

I would even go as far as not allowing the use of names or versions for data retrieval.

I would agree if OpenML were only used for research, but it's also a used as an educational resource, whether that's in e.g., scikit-learn documentation, or university courses. For those applications, I think being able to retrieve datasets by name makes OpenML more accessible, while requiring users to use an identifier has little added value.