N-dimensional timelines and Parameter grid search

[eskjorg]

Imagine that you are working on developing an algorithm that has some hyperparameters that you want to tune. And you want to run your algorithm for a set of hyperparameter configurations in order to visually evaluate the results with Rerun: How should the parameters be set for the best visual result? What if you could take a cartesian product over the parameter space and do a visual grid search?

Example:
Mesh reconstruction from point cloud using Open3d's create_from_point_cloud_poisson(). (please disregard the fact that in this particular function you shouldn't actually change all parameters independently). So you want to find the parameters that creates the best looking mesh.

How I imagine this could work:

1. Have a separate timeline for each parameter.
2. Each time you run your function with a parameter setting, you set the value of all timelines
3. In Rerun later, you can change one parameter at a time by selecting its timeline and changing it.
4. Importantly, when changing from one timeline to selecting another one, the data shown doesn't change (which is what would happen today). Also, when sliding across one timeline, the time of another timeline stays unchanged, so that you can see the effect of only changing one parameter.
5. A mix of discrete and continuous timelines (int/float) should work, like in the example below.
6. And of course, if your output is not something static (like a mesh) but a sequence, you should be able to have an additional timeline that represent a timeline in the usual sense so that for each parameter setting, you could go back and forth in time.

Snippet for poisson mesh example:

from itertools import product
from typing import Final

import open3d as o3d
import rerun as rr

pcl_path: Final = "poisson_pc_front.ply"

# Parameters
depth: Final = [7, 8, 9, 10]
width: Final = [0.0, 5.0, 10.0, 15.0]
scale: Final = [1.0, 1.1, 1.2, 1.3, 1.4]
linear_fit: Final = [False, True]

if __name__ == "__main__":
    rr.init("Open3D Poisson reconstruction -- grid search")
    rr.save("poisson_grid.rrd")

    # Load point cloud
    pcd = o3d.io.read_point_cloud(str("poisson_pc_front.ply"))
    pcd.estimate_normals(
        search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=10, max_nn=30)
    )

    rr.log("pcd", rr.Points3D(positions=pcd.points))

    # Grid search
    for d, w, s, lf in product(depth, width, scale, linear_fit):
        # Run you algorithm
        mesh, densities = o3d.geometry.TriangleMesh.create_from_point_cloud_poisson(
            pcd, depth=d, width=w, scale=s, linear_fit=lf
        )  # Note: this example doesn't make sense, you can't change both depth and width at the same time

        # For each parameter, log it's value to timeline
        rr.set_time_sequence("depth", d)
        rr.set_time_seconds("width", w)
        rr.set_time_seconds("scale", s)
        rr.set_time_sequence("linear_fit", int(lf))
        rr.log(
            "mesh",
            rr.Mesh3D(
                vertex_positions=mesh.vertices,
                vertex_normals=mesh.vertex_normals,
                triangle_indices=mesh.triangles,
            ),
        )