Minimal viable version of the movement plugin for napari. (#450)

* initialise napari plugin development * Create skeleton for napari plugin with collapsible widgets (#218) * initialise napari plugin development * initialise napari plugin development * create skeleton for napari plugin with collapsible widgets * add basic widget smoke tests and allow headless testing * do not depend on napari from pip * include napari option in install instructions * make meta_widget module private * pin atlasapi version to avoid unnecessary dependencies * pin napari >= 0.4.19 from conda-forge * switched to pip install of napari[all] * seperation of concerns in widget tests * add pytest-mock dev dependency * Qt widget for loading pose datasets as napari Points layers (#253) * initialise napari plugin development * Create skeleton for napari plugin with collapsible widgets (#218) * initialise napari plugin development * initialise napari plugin development * create skeleton for napari plugin with collapsible widgets * add basic widget smoke tests and allow headless testing * do not depend on napari from pip * include napari option in install instructions * make meta_widget module private * pin atlasapi version to avoid unnecessary dependencies * pin napari >= 0.4.19 from conda-forge * switched to pip install of napari[all] * seperation of concerns in widget tests * add pytest-mock dev dependency * initialise napari plugin development * initialise napari plugin development * initialise napari plugin development * Added loader widget for poses * update widget tests * simplify dependency on brainglobe-utils * consistent monospace formatting for movement in public docstrings * get rid of code that's only relevant for displaying Tracks * enable visibility of napari layer tooltips * renamed widget to PosesLoader * make cmap optional in set_color_by method * wrote unit tests for napari convert module * wrote unit-tests for the layer styles module * linkcheck ignore zenodo redirects * move _sample_colormap out of PointsStyle class * small refactoring in the loader widget * Expand tests for loader widget * added comments and docstrings to napari plugin tests * refactored all napari tests into separate unit test folder * added napari-video to dependencies * replaced deprecated edge_width with border_width * got rid of widget pytest fixtures * remove duplicate word from docstring * remove napari-video dependency * include napari extras in docs requirements * add test for _on_browse_clicked method * getOpenFileName returns tuple, not str * simplify poses_to_napari_tracks Co-authored-by: Chang Huan Lo <[email protected]> * [pre-commit.ci] pre-commit autoupdate (#338) updates: - [github.com/astral-sh/ruff-pre-commit: v0.6.9 → v0.7.2](astral-sh/ruff-pre-commit@v0.6.9...v0.7.2) - [github.com/pre-commit/mirrors-mypy: v1.11.2 → v1.13.0](pre-commit/mirrors-mypy@v1.11.2...v1.13.0) - [github.com/mgedmin/check-manifest: 0.49 → 0.50](mgedmin/check-manifest@0.49...0.50) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Implement `compute_speed` and `compute_path_length` (#280) * implement compute_speed and compute_path_length functions * added speed to existing kinematics unit test * rewrote compute_path_length with various nan policies * unit test compute_path_length across time ranges * fixed and refactor compute_path_length and its tests * fixed docstring for compute_path_length * Accept suggestion on docstring wording Co-authored-by: Chang Huan Lo <[email protected]> * Remove print statement from test Co-authored-by: Chang Huan Lo <[email protected]> * Ensure nan report is printed Co-authored-by: Chang Huan Lo <[email protected]> * adapt warning message match in test * change 'any' to 'all' * uniform wording across path length docstrings * (mostly) leave time range validation to xarray slice * refactored parameters for test across time ranges * simplified test for path lenght with nans * replace drop policy with ffill * remove B905 ruff rule * make pre-commit happy --------- Co-authored-by: Chang Huan Lo <[email protected]> * initialise napari plugin development * initialise napari plugin development * initialise napari plugin development * initialise napari plugin development * initialise napari plugin development * avoid redefining duplicate attributes in child dataclass * modify test case to match poses_to_napari_tracks simplification * expected_log_messages should be a subset of captured messages Co-authored-by: Chang Huan Lo <[email protected]> * fix typo Co-authored-by: Chang Huan Lo <[email protected]> * use names for Qwidgets * reorganised test_valid_poses_to_napari_tracks * parametrised layer style tests * delet integration test which was reintroduced after conflict resolution * added test about file filters * deleted obsolete loader widget file (had snuck back in due to conflict merging) * combine tests for button callouts Co-authored-by: Chang Huan Lo <[email protected]> * Simplify test_layer_style_as_kwargs Co-authored-by: Chang Huan Lo <[email protected]> --------- Co-authored-by: Chang Huan Lo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * added `movement launch` CLI entrypoint (#345) * updated dimensions order in poses_to_napari_tracks conversion function * Document napari plugin (#393) * started user guide about napari plugin * added optional dependencies for napari-video and pyvideoreader * WIP expanding on the napari plugin user guide * add section on loading poses datasets * set maximum card columns to 2 in the user guide page * corrected grammar and spelling mistakes in the plugin guide * removed automatic setting of playback fps to tracking fps * update to v3 of pyvista/setup-headless-display-action * Apply suggestions from Sofia's code review Co-authored-by: sfmig <[email protected]> * renamed napari guide and added clarifications * fps spinbox: make float, default 1, add tooltip * made image annotations more prominent * Apply suggestions from CH's code review Co-authored-by: Chang Huan Lo <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * applied more suggestions from CH's review * documented known issues * renamed napari_plugin.md to gui.md * add link to github issues tagged with GUI + bug * Use movement-github link from myst-url-scheme Co-authored-by: Chang Huan Lo <[email protected]> --------- Co-authored-by: sfmig <[email protected]> Co-authored-by: Chang Huan Lo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * use updated pytest fixtures in napari convert tests * Replace bg-utils with qt-niu (#441) * update command for testing GUI in installation guide. * updated wording in v0.1 roadmap --------- Co-authored-by: Chang Huan Lo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: sfmig <[email protected]>
neuroinformatics-unit · Mar 3, 2025 · 822a10f · 822a10f
2 parents b0676b7 + 97ead8f
commit 822a10f
Show file tree

Hide file tree

Showing 23 changed files with 1,020 additions and 5 deletions.
diff --git a/.github/workflows/test_and_deploy.yml b/.github/workflows/test_and_deploy.yml
@@ -38,6 +38,10 @@ jobs:
           python-version: "3.13"
 
     steps:
+      # these libraries enable testing on Qt on linux
+      - uses: pyvista/setup-headless-display-action@v3
+        with:
+          qt: true
       - name: Cache Test Data
         uses: actions/cache@v4
         with:

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,6 +1,7 @@
 include LICENSE
 include *.md
 include CITATION.CFF
+include movement/napari/napari.yaml
 exclude .pre-commit-config.yaml
 exclude .cruft.json
 

diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,4 +1,4 @@
--e .
+-e .[napari]
 ablog
 linkify-it-py
 myst-parser

diff --git a/docs/source/_static/napari_plugin_with_poses_as_points.png b/docs/source/_static/napari_plugin_with_poses_as_points.png
diff --git a/docs/source/community/index.md b/docs/source/community/index.md
@@ -5,6 +5,7 @@ develop a new feature, or improve the documentation.
 To help you get started, we have prepared a statement on the project's [mission and scope](target-mission),
 a [roadmap](target-roadmaps) outlining our current priorities, and a detailed [contributing guide](target-contributing).
 
+(target-get-in-touch)=
 ```{include} ../snippets/get-in-touch.md
 ```
 

diff --git a/docs/source/community/roadmaps.md b/docs/source/community/roadmaps.md
@@ -20,12 +20,12 @@ navigation, social interactions, etc.
 - __Integrate with neurophysiological data analysis tools__. We eventually aim to facilitate combined analysis of motion and neural data.
 
 ## Short-term milestone - `v0.1`
-We plan to release version `v0.1` of `movement` in early 2025, providing a minimal set of features to demonstrate the project's potential and to gather feedback from users. At minimum, it should include:
+We've released version `v0.1` of `movement` in March 2025, providing a basic set of features to demonstrate the project's potential and to gather feedback from users. Our minimum requirements for this milestone were:
 
 - [x] Ability to import pose tracks from [DeepLabCut](dlc:), [SLEAP](sleap:) and [LightningPose](lp:) into a common `xarray.Dataset` structure.
 - [x] At least one function for cleaning the pose tracks.
 - [x] Ability to compute velocity and acceleration from pose tracks.
 - [x] Public website with [documentation](target-movement).
 - [x] Package released on [PyPI](https://pypi.org/project/movement/).
 - [x] Package released on [conda-forge](https://anaconda.org/conda-forge/movement).
-- [ ] Ability to visualise pose tracks using [napari](napari:). We aim to represent pose tracks as napari [layers](napari:howtos/layers/index.html), overlaid on video frames.
+- [x] Ability to visualise pose tracks using [napari](napari:). We aim to represent pose tracks as napari [layers](napari:howtos/layers/index.html), overlaid on video frames.
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -181,6 +181,7 @@
     "https://opensource.org/license/bsd-3-clause/",  # to avoid odd 403 error
 ]
 
+
 myst_url_schemes = {
     "http": None,
     "https": None,

diff --git a/docs/source/user_guide/gui.md b/docs/source/user_guide/gui.md
@@ -0,0 +1,174 @@
+(target-gui)=
+# Graphical User Interface
+
+The `movement` graphical user interface (GUI), powered by our custom plugin for
+[napari](napari:), makes it easy to view and explore `movement`
+motion tracks. Currently, you can use it to
+visualise 2D [poses datasets](target-poses-and-bboxes-dataset)
+as points overlaid on video frames.
+
+:::{warning}
+The GUI is still in early stages of development but we are working on ironing
+out the [kinks](movement-github:issues?q=sort%3Aupdated-desc+is%3Aissue+state%3Aopen+label%3AGUI+label%3Abug).
+Please [get in touch](target-get-in-touch)
+if you find any bugs or have suggestions for improvements!
+:::
+
+The `napari` plugin is shipped with the `movement` package starting from
+version `0.1.0`.  To use it, you need to
+[install the package](target-installation) via a method that
+includes the `napari` dependency.
+
+
+## Launch the GUI
+
+To launch the `movement` GUI, type the following command in your terminal:
+
+```sh
+movement launch
+```
+
+This is equivalent to running `napari -w movement` and will open the `napari`
+window with the `movement` widget docked on the
+right-hand side, as in the [screenshot](target-widget-screenshot) below.
+
+In `napari`, data is typically loaded into [layers](napari:guides/layers.html),
+which can be reordered and toggled for visibility in the layers list panel.
+For example, keypoint data can be added as a
+[points layer](napari:howtos/layers/points.html),
+while image stacks (including videos) can be added as
+[image layers](napari:howtos/layers/image.html).
+Below, we'll explain how to do this.
+
+## Load a background layer
+
+Though this is not strictly necessary, it is usually informative to
+view the keypoints overlaid on a background that provides
+some spatial context. You can either [load the video](target-load-video)
+corresponding to the poses dataset, or a [single image](target-load-frame),
+e.g., a still frame derived from that video.
+You can do this by dragging and dropping the corresponding file onto the
+`napari` window or by using the `File > Open File(s)` menu option.
+Please read the following sections for detailed information
+and some important considerations.
+
+(target-load-video)=
+### Load a video
+
+When trying to load a video file into `napari`, you will be prompted
+via a pop-up dialog to select the reader.
+Choose the `video` reader—corresponding to the
+[`napari-video`](https://github.com/janclemenslab/napari-video)
+plugin—and click `OK`. You can optionally select to remember this reader
+for all files with the same extension.
+
+`napari-video` will load the video as an image stack with a slider
+at the bottom that you can use to navigate through frames.
+You may also use the left and right arrow keys to navigate
+frame-by-frame.
+
+Clicking on the play button will start the video playback at a default
+rate of 10 frames per second. You can adjust that by right-clicking on the
+play button or by opening the `napari > Preferences` menu
+(`File > Preferences` on Windows) and changing
+the `Playback frames per second` setting.
+
+(target-video-playback-limitations)=
+:::{admonition} Video playback limitations
+:class: warning
+
+- The video playback may freeze or stutter if you click on the slider to jump
+  to a specific frame. We recommended pausing the playback before such jumps.
+- `napari-video` may struggle to play videos at a high frame rate, depending
+  on your hardware, the video resolution and codec. If you experience
+  performance issues, such as the video freezing or skipping frames,
+  try reducing the playback frames per second or fall back to
+  using a [single image](target-load-frame) as a background.
+:::
+
+
+(target-load-frame)=
+### Load an image
+
+This usually means using a still frame extracted from the video, but in theory
+you could use any image that's in the same coordinate system as the
+tracking data. For example, you could use a schematic diagram of the arena,
+as long as it has the same width and height as the video and is
+properly aligned with the tracking data.
+
+::: {dropdown} Extracting a still frame from a video
+:color: info
+:icon: info
+
+You can use the command line tool [`ffmpeg`](https://www.ffmpeg.org/)
+to extract a still frame from a video.
+
+To extract the first frame of a video:
+
+```sh
+ffmpeg -i video.mp4 -frames:v 1 first-frame.png
+```
+
+To extract a frame at a specific time stamp (e.g. at 2 seconds):
+
+```sh
+ffmpeg -i video.mp4 -ss 00:00:02 -frames:v 1 frame-2sec.png
+```
+:::
+
+Dragging and dropping the image file onto the `napari` window
+(or opening it via the `File` menu) will load the image
+as a single 2D frame without a slider.
+
+## Load the poses dataset
+
+Now you are ready to load some pose tracks over your chosen background layer.
+
+On the right-hand side of the window you should see
+an expanded `Load poses` menu. To load pose data in napari:
+1. Select the `source software` from the dropdown menu.
+2. Set the `fps`  (frames per second) of the video the pose data refers to. Note this will only affect the units of the time variable shown when hovering over a keypoint. If the `fps` is not known, you can set it to 1, which will effectively make the time variable equal to the frame number.
+3. Select the file containing the predicted poses. The path can be directly pasted or you can use the file browser button.
+4. Click `Load`.
+
+The data should be loaded into the viewer as a
+[points layer](napari:howtos/layers/points.html).
+By default, it is added at the top of the layer list.
+
+::: {note}
+See [supported formats](target-supported-formats) for more information on
+the expected software and file formats.
+:::
+
+
+You will see a view similar to the one below:
+
+(target-widget-screenshot)=
+
+![napari widget with poses dataset loaded](../_static/napari_plugin_with_poses_as_points.png)
+
+The keypoints are represented as points, colour-coded by
+keypoint ID for single-individual datasets, or by individual ID for
+multi-individual datasets. These IDs can be also displayed as text
+next to the points by enabling the `display text` option from the
+layer controls panel.
+
+Hovering with your mouse over a point
+(with the points layer selected) will
+bring up a tooltip containing the names of the individual and keypoint,
+the point-wise confidence score (provided by the source software),
+and the time in seconds (calculated based on the frame number and
+the `fps` value you provided).
+
+Using the slider at the bottom of the window, you can move through
+the frames of the dataset, and the points and video will update
+in sync.
+
+::: {admonition} Stay tuned
+Though the display style of the points layer is currently fixed, we are
+working on adding more customisation options in future releases, such as
+enabling you to change the point size, colour, or shape.
+
+We are also working on enabling the visualisation of
+[bounding boxes datasets](target-poses-and-bboxes-dataset) in the plugin.
+:::
diff --git a/docs/source/user_guide/index.md b/docs/source/user_guide/index.md
@@ -8,7 +8,7 @@ Before you dive deeper, we highly recommend reading about the structure
 and usage of [movement datasets](movement_dataset.md), which are a central
 concept in the package.
 
-::::{grid} 1 2 2 3
+::::{grid} 1 1 2 2
 :gutter: 3
 
 :::{grid-item-card} {fas}`wrench;sd-text-primary` Installation
@@ -32,6 +32,13 @@ Load and save tracking data.
 Learn about our data structures.
 :::
 
+:::{grid-item-card} {fas}`line-chart;sd-text-primary` Graphical User Interface
+:link: gui
+:link-type: doc
+
+Use our `napari` plugin to view and explore your data interactively.
+:::
+
 ::::
 
 
@@ -42,4 +49,5 @@ Learn about our data structures.
 installation
 input_output
 movement_dataset
+gui
 ```
diff --git a/docs/source/user_guide/installation.md b/docs/source/user_guide/installation.md
@@ -20,17 +20,24 @@ Create and activate an environment with movement installed:
 conda create -n movement-env -c conda-forge movement
 conda activate movement-env
 ```
+This will install all dependencies, including [napari](napari:),
+which provides the GUI for movement.
 :::
 :::{tab-item} Pip
 Create and activate an environment with some prerequisites:
 ```sh
 conda create -n movement-env -c conda-forge python=3.12 pytables
 conda activate movement-env
 ```
-Install the latest movement release from PyPI:
+Install the core package from the latest release on PyPI:
 ```sh
 pip install movement
 ```
+If you wish to use the GUI, which additionally requires [napari](napari:),
+you should instead run:
+```sh
+pip install movement[napari]
+```
 :::
 ::::
 
@@ -45,6 +52,15 @@ movement info
 You should see a printout including the version numbers of movement
 and some of its dependencies.
 
+To test the GUI installation, you can run:
+
+```sh
+movement launch
+```
+
+This is equivalent to running `napari -w movement` and should open the `napari`
+window with the `movement` widget docked on the right-hand side.
+
 ## Update the package
 To update movement to the latest version, we recommend installing it in a new environment,
 as this prevents potential compatibility issues caused by changes in dependency versions.

diff --git a/movement/cli_entrypoint.py b/movement/cli_entrypoint.py
@@ -2,6 +2,8 @@
 
 import argparse
 import platform
+import subprocess
+import sys
 
 import numpy as np
 import pandas as pd
@@ -41,12 +43,19 @@ def main() -> None:
     """Entrypoint for the CLI."""
     parser = argparse.ArgumentParser(prog="movement")
     subparsers = parser.add_subparsers(dest="command", title="commands")
+
     # Add 'info' command
     info_parser = subparsers.add_parser(
         "info", help="output diagnostic information about the environment"
     )
     info_parser.set_defaults(func=info)
 
+    # Add 'launch' command
+    launch_parser = subparsers.add_parser(
+        "launch", help="launch the movement plugin in napari"
+    )
+    launch_parser.set_defaults(func=launch)
+
     args = parser.parse_args()
     if args.command is None:
         help_message = parser.format_help()
@@ -68,5 +77,20 @@ def info() -> None:
     )
 
 
+def launch() -> None:
+    """Launch the movement plugin in napari."""
+    try:
+        # Use sys.executable to ensure the correct Python interpreter is used
+        subprocess.run(
+            [sys.executable, "-m", "napari", "-w", "movement"], check=True
+        )
+    except subprocess.CalledProcessError as e:
+        # if subprocess.run() fails with non-zero exit code
+        print(
+            "\nAn error occurred while launching the movement plugin "
+            f"for napari:\n  {e}"
+        )
+
+
 if __name__ == "__main__":  # pragma: no cover
     main()
diff --git a/movement/napari/__init__.py b/movement/napari/__init__.py