feat: Add dendrogram #6511

hoxbro · 2025-02-18T16:41:59Z

resolves #6501 (main part of)

TODO:

~~Adjoint plot support responsive~~ (will be done with: Improve responsive in adjoint layout for Bokeh backend #6527)
Adjoint plot weird height issue (problem: shared kdims in height axis: hv.Curve([]) << hv.Path([(0, 0), (1, 1)]) << hv.Path([(0, 0), (200, 200)]))
Add tests
Add examples
Don't need to use Dendrogram(zip(...)) for creating the data
Add simple dendrogram for mpl ~~and plotly~~

Screencast.From.2025-02-24.17-41-32.mp4

Details

import random

import pandas as pd
import panel as pn

import holoviews as hv

hv.extension("bokeh")
hv.opts.defaults(
    hv.opts.HeatMap(width=700, height=400, border=0, tools=["hover"]),
)
random.seed(1)


df = pd.DataFrame(
    [(i, chr(65 + j), random.random()) for j in range(10) for i in range(5)],
    columns=["z", "x", "y"]
)
dataset = hv.Dataset(df, vdims="y")  # TODO: Should not be needed

for adjoint_dims in (["x", "z"], ["x"], ["z"]):
    dendro = hv.operation.dendrogram(dataset, adjoint_dims=adjoint_dims, main_dim="y")
    pn.panel(dendro).servable()

codecov · 2025-02-19T15:07:20Z

Codecov Report

Attention: Patch coverage is 99.18699% with 2 lines in your changes missing coverage. Please review.

Project coverage is 88.85%. Comparing base (bb77ab0) to head (58de7aa).
Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
holoviews/operation/element.py	95.55%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6511      +/-   ##
==========================================
+ Coverage   88.79%   88.85%   +0.06%     
==========================================
  Files         323      324       +1     
  Lines       68961    69226     +265     
==========================================
+ Hits        61235    61513     +278     
+ Misses       7726     7713      -13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

droumis · 2025-02-24T12:36:46Z

@hoxbro , is this ready for review?

hoxbro · 2025-03-05T21:01:18Z

holoviews/operation/element.py

+
+    adjoint_dims = param.List(item_type=str) # , bounds=(1, 2))
+
+    main_element = param.ClassSelector(default=HeatMap, class_=Dataset, instantiate=False, is_instance=False)


Maybe also expose some scipy setting for dendrogram and linkage here.

I think optimal_ordering is a good start, ref.

hoxbro · 2025-03-05T21:04:03Z

holoviews/plotting/bokeh/__init__.py

+    xaxis=None,
+    yaxis=None,
+    show_grid=False,
+    show_title=False,
+    show_frame=False,
+    border=0,
+    default_tools=[],


I am not sure if a blank plot is the best default.

probably fine for now, and easy enough to restore

hoxbro · 2025-03-06T07:09:03Z

holoviews/plotting/bokeh/path.py

+            data[side_dim] = list(0.5 / data_adj.min() * data_adj)
+        else:
+            main_src = main.renderers[0].data_source.data
+            data_adj, data_main = np.asarray(data[side_dim]), np.asarray(main_src.get(main_dim, main_src.get(f"{main_dim}s")))


This is not great. Is there a better way to extract data_main?

hoxbro · 2025-03-06T15:44:17Z

holoviews/operation/element.py

+        else:
+            main = self.p.main_element(dataset.sort(sort_dims).reindex(element_kdims[:2]), vdims=vdims)
+
+        for dim in map(str, main.kdims[::-1]):


Likely, this is a bit sensitive to invert_axes.

hoxbro · 2025-03-07T05:13:08Z

examples/reference/elements/matplotlib/Dendrogram.ipynb

+   "source": [
+    "from holoviews.operation import dendrogram\n",
+    "\n",
+    "hv.operation.dendrogram(heatmap, adjoint_dims=[\"z\"], main_dim=\"y\")"


Something is wrong with the right plot in the matplotlib backend, e.g., it does not show up:

Though there seems to be a general problem with it

Can confirm, I see the same.

examples/reference/elements/bokeh/Dendrogram.ipynb

examples/reference/elements/matplotlib/Dendrogram.ipynb

droumis

Are we saving hover support for a future PR?

droumis · 2025-03-20T19:51:47Z

holoviews/element/path.py

+        return super().__new__(cls)
+
+    def __init__(self, x, y=None, kdims=None, vdims=None, **params):
+        data = x if y is None else zip(x, y)  # strict=True


Suggested change

data = x if y is None else zip(x, y) # strict=True

if y is not None and len(x) != len(y):

raise ValueError("x and y must have the same length.")

data = x if y is None else zip(x, y) # strict=True when 3.9 is dropped

Explicit strict enforcement for now?

I plan to drop 3.9 in the next release (1.21.0), so I will let it slide even though this is correct.

droumis · 2025-03-20T20:07:41Z

holoviews/plotting/bokeh/__init__.py

+    xaxis=None,
+    yaxis=None,
+    show_grid=False,
+    show_title=False,
+    show_frame=False,
+    border=0,
+    default_tools=[],


probably fine for now, and easy enough to restore

philippjfr · 2025-03-24T10:23:45Z

holoviews/operation/element.py

+    operation is typically used to visualize hierarchical clustering of the
+    data.
+    """
+


Maybe add adjoined as a parameter and if it's false just return the Dendrogram itself.

Added in 58de7aa

hoxbro · 2025-03-24T14:21:29Z

Are we saving hover support for a future PR?

Yes.

hoxbro added 2 commits February 18, 2025 11:52

feat: Add dendrogram

4b500d1

Link adjoined dendogram

159acf5

hoxbro force-pushed the dendrogram branch from 07da019 to 159acf5 Compare February 18, 2025 16:46

Move adjoint logic to DendrogramPlot

093347d

hoxbro and others added 3 commits February 19, 2025 18:29

Merge branch 'main' into dendrogram

4490279

cleanup

2de401d

Merge branch 'main' into dendrogram

24a4209

hoxbro force-pushed the dendrogram branch from 6f0911c to 24a4209 Compare February 20, 2025 13:09

hoxbro added 7 commits February 24, 2025 14:21

Merge branch 'main' into dendrogram

2f1b0b9

Add _update_adjoined

90c208f

Add operation, though still WIP

77525e3

Handle one dimension on xaxis

b62b15b

Add main_element and use inputed element type if not dataset

38c66b7

Remove unnessary calculation

19b3a97

fix height issue in dendrograms

bba335f

This was referenced Feb 28, 2025

Investigate dendrograms holoviz-topics/hv-anndata#12

Open

Improve responsive in adjoint layout for Bokeh backend #6527

Open

hoxbro and others added 4 commits March 3, 2025 13:31

unique two kdims

9779adf

Merge branch 'main' into dendrogram

5f375a3

clean up duplicate code

e266195

Switch to x, y instead of data

f3496f5

hoxbro commented Mar 5, 2025

View reviewed changes

Make y optional

14bf28a

hoxbro force-pushed the dendrogram branch from f7f23f4 to 7fc6410 Compare March 6, 2025 07:13

hoxbro added 4 commits March 6, 2025 10:09

Add basic tests

1fbbd4c

Add adjoint plot main 1kdim

8b3d020

Handle case where data source is ys

7669423

Add some operation tests

27d27b4

update operation to automatic detect which dim it should use

167da22

hoxbro force-pushed the dendrogram branch from 7fc6410 to 167da22 Compare March 6, 2025 09:09

hoxbro added 3 commits March 6, 2025 10:13

fix merge

7dc1624

add shared axis test

5da82c3

Add basic dendrogram plotting support for mpl

de982a5

hoxbro commented Mar 6, 2025

View reviewed changes

hoxbro added 2 commits March 6, 2025 16:25

add examples

3ce491e

hide axis

6302043

hoxbro marked this pull request as ready for review March 6, 2025 15:29

Merge branch 'main' into dendrogram

836ac29

hoxbro commented Mar 6, 2025

View reviewed changes

Update docstring and enable optimal_ordering

dee2c51

hoxbro commented Mar 7, 2025

View reviewed changes

hoxbro added 2 commits March 13, 2025 07:53

Merge branch 'main' into dendrogram

9c458c9

Merge branch 'main' into dendrogram

6957d2a

droumis requested review from philippjfr, droumis and cvaske and removed request for cvaske March 18, 2025 16:16

droumis reviewed Mar 20, 2025

View reviewed changes

philippjfr reviewed Mar 24, 2025

View reviewed changes

philippjfr approved these changes Mar 24, 2025

View reviewed changes

hoxbro added 2 commits March 24, 2025 15:00

apply notebook suggestions

e30dd9e

add adjoined parameter

58de7aa

holoviz deleted a comment from review-notebook-app bot Mar 24, 2025

hoxbro enabled auto-merge (squash) March 24, 2025 14:24

hoxbro merged commit eaab13e into main Mar 24, 2025
16 checks passed

hoxbro deleted the dendrogram branch March 24, 2025 14:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add dendrogram #6511

feat: Add dendrogram #6511

hoxbro commented Feb 18, 2025 •

edited by droumis

Loading

codecov bot commented Feb 19, 2025 •

edited

Loading

droumis commented Feb 24, 2025

hoxbro Mar 5, 2025 •

edited

Loading

hoxbro Mar 5, 2025

droumis Mar 20, 2025

hoxbro Mar 6, 2025

hoxbro Mar 6, 2025

hoxbro Mar 7, 2025

droumis Mar 20, 2025

droumis left a comment

droumis Mar 20, 2025

hoxbro Mar 24, 2025

droumis Mar 20, 2025

philippjfr Mar 24, 2025

hoxbro Mar 24, 2025

hoxbro commented Mar 24, 2025


		adjoint_dims = param.List(item_type=str) # , bounds=(1, 2))

		main_element = param.ClassSelector(default=HeatMap, class_=Dataset, instantiate=False, is_instance=False)

-        data = x if y is None else zip(x, y)  # strict=True
+        if y is not None and len(x) != len(y):
+            raise ValueError("x and y must have the same length.")
+        data = x if y is None else zip(x, y)  # strict=True when 3.9 is dropped

feat: Add dendrogram #6511

feat: Add dendrogram #6511

Conversation

hoxbro commented Feb 18, 2025 • edited by droumis Loading

codecov bot commented Feb 19, 2025 • edited Loading

Codecov Report

droumis commented Feb 24, 2025

hoxbro Mar 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

droumis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hoxbro commented Mar 24, 2025

hoxbro commented Feb 18, 2025 •

edited by droumis

Loading

codecov bot commented Feb 19, 2025 •

edited

Loading

hoxbro Mar 5, 2025 •

edited

Loading