CVOC : Multiple services for 1 term field

[luddaniel]

Following the last Community Call August 22, 2023 Notes it appears that it would be nice to have one issue describing a new use case of external vocabulary using the supported generic external vocabulary support mechanism of Dataverse.

Right now, the setting :CVocConf (that uses a configuration file cvoc-conf.json as value) allows to implement 1 service url for 1 field (or a group of 1 parent field and many childs fields).

For our Dataverse, we are thinking about having multiple services for the same bloc of termName/vocabularyUri fields.
For example :

1. https://thesaurus.inrae.fr/thesaurus-inrae/fr/ (SKOMOS)
2. https://skosmos.loterre.fr/en/ (SKOMOS)
3. https://agroportal.lirmm.fr/ontologies (OntoPortal)

This feature mainly concerns the java code which will have to manage the cache and indexing needs for the multilingual search results (search result page/api and dataset metadata tab view) of each service while the javascript modifications only concern the dataset edit view.

Feedback: Each instance is working well independently using the generic external vocabulary mechanism with the good configuration of the json file.

@qqmyers @4tikhonov @qqmyers @DS-INRA

[pdurbin]

@luddaniel thanks for creating this issue and discussing it on the community call.

When you mention the need to modify the Java code, I assume this involves changes to the old/current frontend, which is written in JavaServer Faces (JSF). In case you or others aren't aware, the JSF frontend is deprecated and a new equivalent frontend is being written from scratch in React.

Because of this ongoing rewrite, I wonder how you feel about deferring this work until the new React frontend is ready to receive it.

You can get a sense of the state of the new React frontend (early days) by watching the recent video at https://github.com/IQSS/dataverse-frontend#demo-videos . The roadmap for the new React front end can be seen at https://groups.google.com/g/dataverse-community/c/2hfhWIHQddE/m/hATuWGGSAAAJ

Obviously, if you or others want to work on backend (API and database) stuff, all of that work will carry forward into the future indefinitely.

[4tikhonov]

Some time ago I have developed Semantic Gateway https://github.com/Dans-labs/semantic-gateway as a middleware which will allow to connect any number of Skosmos or other CV services to Dataverse term fields. You need to create semantic mappings and put them in the configuration file, and Semantic Gateway will "imitate" Skosmos instance by providing the same interface to CV plugin which we've incorporated earlier.

[qqmyers]

FWIW: There's nothing stopping a JavaScript from using multiple services now. The limitation from the Dataverse side is that the registration mechanism only supports providing one service URL so today a JavaScript would need to hardcode other endpoints and/or do its own lookup of which services to use, or use a gateway as above, etc.

[cmbz]

2025-03-24

• Reviewed during prioritization meeting. @qqmyers could you chime in here with a bit more information about what might need to be done in the DV codebase to support this feature? Thanks.

[qqmyers]

As I mentioned before, an individual script can do this now - basically custom JavaScript. (Similarly, as Slava said, since the Skosmos script already handles multiple vocabularies, if all the vocabs are in the same Skosmos service, you can do it that way.) To make it something that can be done by assigning multiple scripts to one field would require redesigning the config mechanism to support it, and then making changes in the code wherever CVoC is handled to deal with the fact that identifiers might be for one script or another. There's back-end code involved, as well as UI code - where we'd need some standard mechanism to let people select which script to invoke before the individual scripts can run to do their normal display/edit functionality. My guess is that it would make sense to do this at the same time as changing the mechanism to do config separately rather than through one :CVocConf setting, and changing to allow configuring one script for multiple fields at a time, etc. I'd suggest starting with a design doc that covers these and any related changes we want in the same timeframe so we can assure that everything is covered, we have a path forward for any current scripts, etc.

[cmbz]

Thanks very much @qqmyers! @scolapasta please see Jim's comment about recommendations: #9828 (comment)