Enhancement: EDAM ontology tools view improvements

[neoformit]

With @hechth @bgruening
The EDAM ontology tools view is a great concept but would be much more useful if the following were addressed:

1. Text-searchable ontologies. Currently I have to scroll through a long list until I see an ontology that appeals to my analysis
2. Tree-view for ontologies. Why are we displaying hierarchically ordered data in flat lists?
3. Better ontology annotation coverage across tool wrappers (outside the scope of this issue)
4. More comprehensive ontologies, particularly in emerging fields such as proteomics and metabolomics (also outside the scope of this issue).

I think 2) should be solved by opening an ontology tree in the main panel, as the new advanced search does. This could include some filters or a search box to find relevant ontologies quickly.

A common complaint of Galaxy users is that the service is bloated with tools. I think features like this are critical to making the service navigable and approachable for users.

[mvdbeek]

2. Why are we displaying hierarchically ordered data in flat lists?

because it was the easiest way to get started and it mirrored the original too panel. That doesn't mean it needs to stay that way.

I think for the tree view you'll want to split up the tool data from the "panel data" and link them together through the tool id.

So instead of this big blob of json (with things we don't even consume anymore ...):

            {
                "model_class": "Tool",
                "id": "toolshed.g2.bx.psu.edu/repos/iuc/ncbi_acc_download/ncbi_acc_download/0.2.8+galaxy0",
                "name": "NCBI Accession Download",
                "version": "0.2.8+galaxy0",
                "description": "Download sequences from GenBank/RefSeq by accession through the NCBI ENTREZ API",
                "labels": [],
                "edam_operations": [],
                "edam_topics": [],
                "hidden": "",
                "is_workflow_compatible": true,
                "xrefs": [
                    {
                        "value": "ncbi_resources",
                        "reftype": "bio.tools"
                    }
                ],
                "tool_shed_repository": {
                    "name": "ncbi_acc_download",
                    "owner": "iuc",
                    "changeset_revision": "e063168e0a81",
                    "tool_shed": "toolshed.g2.bx.psu.edu"
                },
                "config_file": "/cvmfs/main.galaxyproject.org/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/ncbi_acc_download/e063168e0a81/ncbi_acc_download/ncbi_acc_download.xml",
                "link": "/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2Fiuc%2Fncbi_acc_download%2Fncbi_acc_download%2F0.2.8%2Bgalaxy0",
                "min_width": -1,
                "target": "galaxy_main",
                "panel_section_id": "get_data",
                "panel_section_name": "Get Data",
                "form_style": "regular"
            },

you'd return 2 data structures, one with visible tools, e.g

"tools": {
  "toolshed.g2.bx.psu.edu/repos/iuc/ncbi_acc_download/ncbi_acc_download/0.2.8+galaxy0":
    "name": ...,
    "version": ...,
    "description": ...,
}

one with the various mappings:

"default_panel_view": {
  "get_data": {
    "section_name": "Get Data",
    "tools": [
       "toolshed.g2.bx.psu.edu/repos/iuc/ncbi_acc_download/ncbi_acc_download/0.2.8+galaxy0",
       "__FETCH_DATA__",
       ...
    ]
  }
  ...,
}

I think @ahmedhamidawan was interested in working on this particular aspect.

[ahmedhamidawan]

Thanks for setting this up @neoformit and thanks for detailing the structure @mvdbeek !!

So, for the tree view, is this the tree we are referring to?:

which is for the Sequence Alignment ontology here:

If so, is this the general idea we are going for (corresponding to the Sequence Alignment ontology for e.g., with the graph above in mind):

Before	After

Obviously, we could try out way better ways to implement this in terms of the UI, I'm just trying to understand if this is what we mean by the tree view here?

[neoformit]

Yes, that's what we meant by tree view thank you both for the details. Happy to chat if you want to brainstorm on this!
Awesome you're already working on something, is there another issue we can link to for this?

[ahmedhamidawan]

Yes, that's what we meant by tree view thank you both for the details. Happy to chat if you want to brainstorm on this! Awesome you're already working on something, is there another issue we can link to for this?

A discussion would be great! Yes, there is an issue for refactoring the toolbox structure: #16101, although needs to be updated with what Marius described above.

[neoformit]

Ok feel free to link that info to your issue and close this one. I think your approach looks good - your drilldown UI looks like it would represent the tree structure quite concisely in the side panel.

[ahmedhamidawan]

Ok feel free to link that info to your issue and close this one. I think your approach looks good - your drilldown UI looks like it would represent the tree structure quite concisely in the side panel.

Sounds good, maybe we can keep this one instead or keep both and then the PR will link both of them; as this one pertains more to the EDAM structure specifically, and the other one relates to the whole structure

[hechth]

@ahmedhamidawan and @mvdbeek what is the current status on this? There will be many new concepts and terms introduced to EDAM soon.

Also one thing which I saw is that currently, it is possible to choose either topics or operations. Would it be possible to have the topics as filter and then the operations? this would allow better refinement of tools and would also work better with the ontology, as some terms are more generic which are then made more specific using the related topics and the context of the tools.

[ahmedhamidawan]

Hello @hechth !

As of now, tools for given ontologies and topics are both searchable via a field in the advanced tool search (You even get a drop-down list to choose from once you have either view enabled).

Other than that, I did some work almost a year ago trying to implement sort of a nested/graph like view for EDAM in the panel, but halted work on it because most tools didn't have EDAM properties.

[hechth]

So this is on hold until more tools are annotated is what you are saying? This is a bit of a chicken and egg problem.

What would be useful is to filter by topic and then show the tools sorted by operation according to the hierarchy outlined by EDAM.

[ahmedhamidawan]

It wasn't just that not enough tools were annotated, we were also not able to agree on an appropriate UI approach on showing the nesting tools in a tree structure in the panel.

It's good to hear that there is more interest in this lately, and we can take your recommendations up for discussion as well (in the dev meeting)

[hechth]

Okay I see - it would be good to know how many of the actually used tools are annotated, because I think the fact that few tools have annotations is also related to the fact that the majority of Galaxy tools overall is not used much.

But I fully agree that there needs to be better tool annotation.