Create a glossary of common terms

[niksirbi]

The problem

Throughout the code and the docs we refer to specialist terms such as "poses", "pose tracks", "pose estimation", "tracking".
We either resort to explaining these terms at multiple places, or we assume the the user/reader know what they mean.

Solution

As proposed by @lochhh, we could have a glossary of terms in the docs, defining these terms unambiguously. We could then reference the glossary when we need to.

We could even add a schematic to each term, which will greatly aid their understanding. We can reuse the schematics in talks etc.

Resources

This paper defines some of these terms well.

[sfmig]

Agreed terminology

• default (cartesian) coordinate system:
  • origin top left of the image,
  • x-axis is positive across columns (pointing right),
  • y-axis resulting from the right hand rule; i.e., positive across rows (pointing down).
    This convention matches DLC, napari and most image processing tools.
• position vector for keypoint i at time t: vector from the origin of the coordinate system to keypoint i $$\vec{r_i}(t) = (x_i(t), y_i(t))$$
• velocity vector for keypoint i at time t: instant velocity of keypoint i at time t $$\vec{v_i}(t) = (\dot{x}_i(t), \dot{y}_i(t))$$
• acceleration vector for keypoint i at time t: instant acceleration of keypoint i at time t $$\vec{a_i}(t) = (\ddot{x}_i(t), \ddot{y}_i(t))$$
• displacement vector for keypoint i at time t: vector representing the displacement of a keypoint i with respect to its position at the previous frame t-1 $$\vec{d_i}(t) = (x_i(t) - x_i(t-1), y_i(t) - y_i(t-1))$$

[talmo]

FWIW, here's a post on the coordinate system we adopt in SLEAP (center pixel-aligned):

talmolab/sleap#1640 (comment)

This is the same as OpenCV too.

[niksirbi]

Thanks @talmo, this is super useful!

[sfmig]

also include/link to the concept of movement dataset

[sfmig]

and the concept of bounding boxes (and tracked bounding boxes)

[lochhh]

Definitions for "forward vector", "head direction vector", "heading"

[lochhh]

@sfmig on zulip :

Note that as Niko said above, vectors in movement are free vectors, that is , their point of application is not fixed and they are just defined by their magnitude and direction. The magnitude and direction can be expressed by the x-y coordinates of the tip of the vector (assumed to be applied at the origin), or by the polar coordinates rho, phi.

[sfmig]

As pointed out by @lochhh we should also clarify any movement specific terms that we use for convenience, such as "motion tracks".
From PR #352:

In the context of movement, we refer to these trajectories as tracks: we use pose tracks to refer to the trajectories of a set of keypoints, bounding boxes' tracks to refer to the trajectories of bounding boxes' centroids, or motion tracks in the more general case.

[sfmig]

From the meeting with Mikkel:

• image coord system: make it clear in the glossary that this is the default and how it is defined (including, pixel-centred clarification).
• add a schematic of the right-hand rule in the glossary, something similar to Freya Holmer's nice schematic but further simplified.
• add a schematic of polar coordinates in the glossary.
• add pinhole camera schematic, showing the image coord system, the camera coord system, and the world coord system.
• if we implement a "transform to top-right / top-left / bottom-right / bottom-left" coordinate systems, we need to clarify what we mean in the glossary.

[niksirbi]

Meeting notes: 2024-12-13

On December 13, 2024 me and @sfmig had a brainstorming session trying to write down all the terms that we want to define in the glossary. We came up with the rough list below.

Note

We erred on the side of being exhaustive. Ultimately we may end up defining only a subset of the following terms in the glossary, because some of them may be sufficiently documented elsewhere (e.g. terms related to movement/xarray data structures are already explained in the movement datasets guide

General Computer Vision-related terms

• detection
• tracking
• pose
• pose estimation
• bounding box
• segmentation (mask)

Terms related to xarray data structures

Perhaps it's enough to just link to the xarray docs for these.

• dimensions
• coordinates
• Dataset
• DataArray
• Data variable
• Dataset/DataArray attributes

Terms related to movement data structures

• space
• time
• keypoint
• individual
• position
• confidence
• shape
• motion tracks / pose tracks / bboxes tracks / keypoint tracks
• movement dataset
  • poses dataset
  • bboxes dataset

Derived variables/quantities

• displacement
• velocity
• acceleration
• speed
• forward vector (+ head direction vector)?
• heading (angle)

Coordinate systems

Note

This section will probably end up being a standalone guide, complete with schematics such as the ones mentioned in previous comments. I'm just adding these terms here for completeness, since they have been discussed under this issue.

• Center pixel-aligned systems
• Cartesian vs Polar space
• Image / Camera / World coordinate systems
• Allocentric vs Egocentric systems
• Static vs moving reference frames?
• right hand rule
• Linear algebra:
  • linear transformations
  • origin
  • translation
  • rotation
  • scaling
  • righ-hand rule
  • (free) vector

Current status and way forward

We have started a draft PR which adds a Sphinx glossary page, and contains some in-progress definitions for a few of the above terms.

Because writing the entire glossary is too big of a task for a single person (or a single PR) I propose doing that is steps, i.e. break down the terms into arbitrary categories (similar to the one above) and add one category at a time.

Note that the work doesn't end with just populating the glossary, we also have to create cross-references from docstring and other parts of the documentation to term definitions in the glossary. For example to cross-link the mention of the term "velocity" to its definition in the Glossary, it's enough to write is as {term}`velocity` (in Myst-markdown format). This shouldn't necessarily be done for all mentions of a term, but only where it makes sense—e.g. the first time a term is mentioned in a page/docstring. So each PR should add a few terms to the glossary alongside the appropriate cross-references.

Regarding the coordinate systems, I think it should be a separate guide. I will take a crack at this. It's a pressing need because it will be necessary for us to figure out how to design the transforms module—see this discussion on Zulip.

[niksirbi]

Following the discussion during today's meeting, I've opened separate issues for the coordinate systems guide (assigned to myself) and for showing tooltips on hover (which is more of a stretch goal).

The current issue will be closed once we have a minimal glossary up (even lacking terms). Missing/additional terms can be added on an as-needed basis.