OrthoMCL - Custom tree-table for OrthoGroup page #904

bobular · 2024-03-01T21:50:24Z

Testing:

There are only a few tree files available on the back end, so only a few orthogroups work. Here's are the valid OGs at the moment:

OG6_102443
OG6_103280
OG6_105395
OG6_105810
OG6_126264
OG6_126317
OG6_126334
OG6_126431

Run ortho-site locally and access the path /app/record/group/OG6_102443
Back end config for .env is

BASE_PROXY_URL=https://qa.orthomcl.org/
LEGACY_WEB_APP_URL=${BASE_PROXY_URL}/orthomcl
PROJECT_ID=OrthoMCL

yarn && yarn nx start @veupathdb/ortho-site

To do:

UX suggestions

organism/taxon legend(s)?
zoom out to see whole tree? or maybe a modal?
tree bolder (thicker lines)
add explanation of tree
tree branch metrics? (future)

… the Mesa inline option

…cally fixed-height mode)

…de (basically fixed-height mode)" This reverts commit f215762.

dmfalke

This is looking really good. I made some comments below. Let me know what you think!

dmfalke · 2024-03-19T17:34:35Z

packages/libs/components/src/components/tidytree/TreeTable.tsx

+      <div
+        style={{
+          flexGrow: 1,
+          width: 1 /* arbitrary non-zero width seems necessary for flex */,
+        }}
+      >
        <Global
          styles={globalStyle`
 	  .DataTable {
 	    margin-bottom: 0px !important;
+            width: 80%;
 	  }
 	`}
        />
        <Mesa state={tableState} />
-      </>
+      </div>


Sorry, I'm just noticing this. I'm not sure we should use Global here, as the style will be reflected on all subsequent pages.

Instead, could you use a css prop on the parent div component (line 96), and target .DataTable in there?

<div css={{ flexGrow: 1, width: 1 /* arbitrary non-zero width seems necessary for flex */, '.DataTable': { marginBottom: '0px !important', width: '80%' } }} > <Mesa state={tableState} /> </div>

Oh yeah, that's much simpler (and no side effects) - done in 5c896a0 - thanks!

dmfalke · 2024-03-19T17:40:54Z

packages/sites/ortho-site/webapp/wdkCustomization/js/client/utils/tree.ts

Could we change these identifiers such that they include the "group" prefix? E.g., GroupTreeResponse, groupTreeResponseDecoder, etc?

Sure! fd446f8

dmfalke · 2024-04-29T11:23:43Z

It's imported here:

https://github.com/VEuPathDB/web-monorepo/blob/26807b89d8f8a131984a63f67da68a70e83309f2/packages/sites/ortho-site/webapp/wdkCustomization/js/client/wrapStoreModules.tsx

I'm pretty sure you can just delete the reduce and don't need to make any other changes.

bobular · 2024-04-30T10:30:08Z

Ah, thanks, I was missing something fundamental that I get now.

bobular · 2024-05-29T21:49:23Z

It looks like we can't avoid the "We're sorry, something went wrong." modal being rendered if the wdkService back end call returns a 404. That hook (useWdkService/useOrthoService) doesn't handle errors the same way as the EDA's usePromise.

However, we can avoid asking for a tree when there are too few sequences, thus avoiding some user-facing errors. Will push a commit soon to address this and make sure the regular table still renders.

  const numSequences = mesaRows.length;
  const treeResponse = useOrthoService(
    numSequences > 2
      ? (orthoService) => orthoService.getGroupTree(groupName)
      : () => Promise.resolve(undefined),
    [groupName, numSequences]
  );

…ange; fix some imports

dmfalke · 2024-06-03T17:26:37Z

@bobular what is the state of the backend for this? Do I need to use the diamond branch and the rm database John mentioned?

dmfalke

Overall, this looks great. I added some comments. Let me know what you think!

dmfalke · 2024-06-04T18:22:22Z

packages/libs/coreui/src/components/Mesa/Ui/TableSearch.jsx

@@ -18,7 +18,7 @@ class TableSearch extends React.PureComponent {
  clearSearchQuery() {
    const query = null;


This can be deleted

dmfalke · 2024-06-04T18:31:16Z

packages/sites/ortho-site/webapp/wdkCustomization/js/client/services.tsx

@@ -26,6 +31,12 @@ const orthoServiceWrappers = {
      method: 'get',
      path: `/group/${groupName}/layout`,
    }),
+  getGroupTree: (wdkService: WdkService) => (groupName: string) =>
+    wdkService.sendRequest(groupTreeResponseDecoder, {
+      useCache: true,


It's worth noting that this property causes the response to be stored in indexeddb, which is governed by the browsers quota policy for domains. If the database exceeds the quota, attempts to cache responses will begin to fail (don't worry, this is handled for a seamless user experience). In all likelihood, important/critical responses will already be cached, so it should not impact website load times.

In the future, we might consider using LRU (least recently used) caches for things like this, where the cache container has an identifier, and we retain up to a certain number of entries, ejecting those used least recently.

For now, this is probably fine.

Interesting. Are the cache entries keyed by the full request payload (like react-query)?

If so, this would seem to be a general WDK problem, not specific to fetching the tree (is it massively bigger than the responses from the other endpoints?)

Entries are keyed by path and, if provided, cacheId. See here. I don't know how big these newick strings can get, so it's hard to say how much of an issue it would be.

Again, I think this is probably fine, but I wanted to mention the current limitations.

dmfalke · 2024-06-04T18:41:51Z

packages/sites/ortho-site/package.json

-  ]
+  ],
+  "dependencies": {
+    "patristic": "^0.6.0"


I think this should be declared in packages/libs/components/package.json.

There's

import { parseNewick } from 'patristic';

in Sequences.tsx though.

We don't use patristic directly in components. If tidytree needs it then that's its business!

Aha, thanks!

dmfalke · 2024-06-04T18:42:06Z

packages/sites/ortho-site/webapp/wdkCustomization/js/client/types/patristic.d.ts

I think this should be moved to packages/libs/components

dmfalke · 2024-06-04T20:16:39Z

Another thing... it would be nice for the row counts to indicate when the table is filtered, as in this screenshot:

bobular · 2024-06-19T20:26:13Z

Hi all. I've made some expensive synchronous operations into asynchronous effects and added a switch to show only core proteins when #seqs > 2000

OG7_0000009 with 10k sequences is a good test case.

The remaining performance issues seem to be
a) filtering the tree based on search (which is currently synchronous) - takes a couple of seconds for a 10k group.
b) rendering

My FF still complains sometimes for the full (core+peripheral) and this seems to be a mix of Pfam architecture rendering and tree rendering. Only disabling both seems to prevent any slow-down warnings. We probably need to think a bit more about large groups and what sensible cutoffs are.

…quence_link column and enabled the link

New optional Mesa inline table behaviour using tooltips

dmfalke · 2024-06-26T04:09:37Z

packages/sites/ortho-site/webapp/wdkCustomization/js/client/records/Sequences.tsx

@@ -448,3 +456,21 @@ function createSafeSearchRegExp(input: string): RegExp {
    return new RegExp(escapedInput, 'i');
  }
 }
+
+function summarizeIDMismatch(A: string[], B: string[]) {


Maybe name this something like logIdMismatchSummary

bobular · 2024-07-02T13:44:21Z

Latest commit adds this. No dashed outline when there are PFam domains.

made a start - WIP

7ac2d3e

bobular marked this pull request as draft March 1, 2024 21:50

bobular changed the title ~~made a start - WIP~~ OrthoMCL - Custom tree-table for OrthoGroup page Mar 1, 2024

bobular added 17 commits March 4, 2024 15:55

re-render sequence table with Mesa

2f4bc1f

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

4340f10

tweak comments

3498348

WIP

dc1731f

loosely wire in TreeTable and get shimmimg working

785a9ed

add patristic and basic type support

8c3d330

made a start with controlling table row height but should investigate…

adb464a

… the Mesa inline option

added Mesa table tooltips for td contents when in 'inline' mode (basi…

f215762

…cally fixed-height mode)

use Mesa's inline option and tidy up

1abea4b

Revert "added Mesa table tooltips for td contents when in 'inline' mo…

ded7a66

…de (basically fixed-height mode)" This reverts commit f215762.

remove placeholder td title

a7c0e1b

restore whitespace in DataCell.tsx

a19369e

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

5c04215

upgrade tidytree and use new interactive option

5f908bd

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

89d17a7

table and tree order are the same

ea9283d

added basic checkboxes

76bfe05

dmfalke reviewed Mar 19, 2024

View reviewed changes

bobular added 9 commits March 19, 2024 22:41

WIP clustal form

cdc9d49

that's hopefully clustal sorted - but can't test with local ortho-site

1495c07

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

57f6a10

improve 'must select two proteins' verbiage

8831695

de-complicate .DataTable margin-bottom override

5c896a0

rename TreeResponse to GroupTreeResponse

fd446f8

add PFam domain architecture column

a36c6ad

ugly search working as a demo on description column

f489066

add warning banner for data issue and tweak pfam column heading

c06178d

Merge branch 'main' into orthogroup-tree-table

8a27952

bobular marked this pull request as ready for review May 29, 2024 13:33

remove code that collapsed the main section of the orthogroup page

c0cb043

no longer request trees for 1 or 2 sequences; fix RowCounter props ch…

3e47e2c

…ange; fix some imports

dmfalke reviewed Jun 4, 2024

View reviewed changes

bobular added 9 commits June 4, 2024 21:36

fudge to sort table based on dodgy colon-containing full_ids

40eacc1

make tree darker and make tree/table check more robust

8ff24e2

provide PFam descriptions in domain cartoon tooltips

fcdddbd

sort out row counts

032c479

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

9f3d563

add better console logging for tree/table mismatches

7bf58f4

made tree-table comparison for console more robust

e6904a1

add inlineUseTooltips options prop but not working fully yet

06f4959

attempted better handling of very large groups

9ecec3d

bobular added 3 commits June 21, 2024 11:50

looks good now

30da70a

removed full_id column (from display only) and added Accession aka se…

443528c

…quence_link column and enabled the link

Merge pull request #1115 from VEuPathDB/mesa-inline-changes

e7abcb9

New optional Mesa inline table behaviour using tooltips

dmfalke reviewed Jun 26, 2024

View reviewed changes

dmfalke and others added 3 commits June 28, 2024 11:52

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

9567b62

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

f33f711

Explicit empty PFam legend

5e04d5f

bobular and others added 2 commits July 2, 2024 14:57

Merge branch 'main' into orthogroup-tree-table

5a49184

Merge remote-tracking branch 'origin/main' into orthogroup-tree-table

0cbb71d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OrthoMCL - Custom tree-table for OrthoGroup page #904

OrthoMCL - Custom tree-table for OrthoGroup page #904

bobular commented Mar 1, 2024 •

edited

Loading

dmfalke left a comment

dmfalke Mar 19, 2024

bobular Mar 20, 2024 •

edited

Loading

dmfalke Mar 19, 2024

bobular Mar 20, 2024

dmfalke commented Apr 29, 2024

bobular commented Apr 30, 2024

bobular commented May 29, 2024 •

edited

Loading

dmfalke commented Jun 3, 2024

dmfalke left a comment

dmfalke Jun 4, 2024

dmfalke Jun 4, 2024

bobular Jun 4, 2024

dmfalke Jun 5, 2024

dmfalke Jun 4, 2024

bobular Jun 4, 2024

bobular Jun 4, 2024

dmfalke Jun 4, 2024

dmfalke Jun 4, 2024

dmfalke commented Jun 4, 2024

bobular commented Jun 19, 2024

dmfalke Jun 26, 2024

bobular commented Jul 2, 2024

		@@ -18,7 +18,7 @@ class TableSearch extends React.PureComponent {
		clearSearchQuery() {
		const query = null;

OrthoMCL - Custom tree-table for OrthoGroup page #904

Are you sure you want to change the base?

OrthoMCL - Custom tree-table for OrthoGroup page #904

Conversation

bobular commented Mar 1, 2024 • edited Loading

dmfalke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bobular Mar 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmfalke commented Apr 29, 2024

bobular commented Apr 30, 2024

bobular commented May 29, 2024 • edited Loading

dmfalke commented Jun 3, 2024

dmfalke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmfalke commented Jun 4, 2024

bobular commented Jun 19, 2024

Choose a reason for hiding this comment

bobular commented Jul 2, 2024

bobular commented Mar 1, 2024 •

edited

Loading

bobular Mar 20, 2024 •

edited

Loading

bobular commented May 29, 2024 •

edited

Loading