refactor(sdk): simplify the 0.36.0 surfaces (stacked on #202)

.claude/skills/plotjuggler-plugin/references/builtin-objects.md

@@ -114,10 +114,8 @@ exactly once. Do not keep a raw pointer into anything the closure captured.
 - `ImageAnnotations.image_topic` explicitly names the base image topic; camera
   calibration correlates by **name convention** (`<ns>/camera_info` ↔
   `<ns>/image_raw`), not by a reference.
-- **Association fields ride *outside* the serialized bytes.** The annotations
-  codec deliberately does not encode `image_topic` (or a timestamp) into the
-  payload — the store keys entries by (topic, timestamp), and association is a
-  topic/metadata convention. Don't expect a codec round-trip to carry them.
+- The annotations codec carries the top-level `timestamp` (only when non-zero) and
+  `image_topic` (only when non-empty); see `docs/image_annotations_format.md`.
 
 ## Codec round-trips are not bit-exact
 

.claude/skills/plotjuggler-plugin/references/toolbox.md

@@ -80,19 +80,9 @@ PJ_TOOLBOX_PLUGIN(MyToolbox,
 
 ## Detecting host features (SDK 0.36.0)
 
-One rule, in the header comment of `pj_base/plugin_data_api.h`; never test a
-version string:
-
-- ABI service feature: a named `hasX()` on the view
-  (`DataProcessorsHostView::hasTypedRequests()`, `PlotTabHostView::hasSceneTabs()`,
-  `ToolboxHostView::hasCatalogSnapshotV2()`).
-- Flag-bit feature (`PJ_DATA_PROCESSOR_FLAG_INFER_OUTPUTS`): do not probe; an older
-  host rejects the bit. Its floor is the `hasX()` of the slot carrying it.
-- Build-dependent behaviour (Python for on_demand): probe by doing it,
-  `validateScript("on_demand", "python", "return {}")`.
-- Dialog feature (`scene_view`): `hostHas(DialogHostCapability::kEmbedsSceneViews)`
-  inside the dialog.
-- Manifest metadata (`badge`, `custom_topics_editor`): declarative, no probe.
+Never test a version string. The rule is in the header comment of
+`pj_base/plugin_data_api.h`; `pj_plugins/docs/toolbox-guide.md` → "Detecting what
+the host can do" has the table with one example per feature kind.
 
 On-demand requests: `createV2` returns `<owner>/<id>/<name>` for object and number
 outputs (read a number series by exactly that string) and the bare name for a

docs/builtin_type.md

@@ -680,19 +680,16 @@ compile time — name, shape (`FieldKind`), and type-erased get/set accessors 
 so a generic consumer (e.g. a script binder) can read or write any described
 field by name without hand-written per-type glue. A struct opts in with a
 `FieldTable<T>` specialization built from `field<&T::member>("member")` calls
-(`buffer<>()` for a packed record buffer like `PointCloud::data`); nested
+(a `kBuffer` factory such as `pointCloudDataField()` for a packed record buffer
+like `PointCloud::data`); nested
 structs and lists of described structs link to their own table through
 `FieldDescriptor::nested`, so a consumer walks an arbitrarily deep struct tree
 with one generic recursive routine. `field_table_registry.hpp` exposes
 `describe(BuiltinObjectType)` to look up a table from the runtime tag a
 `BuiltinObject` carries.
 
-`FrameTransforms`, `ImageAnnotations`, `PointCloud`, `SceneEntities`, `Image`,
-`DepthImage`, `CameraInfo`, and `VideoFrame` are tabled today
-(`frame_transforms_fields.hpp`, `image_annotations_fields.hpp`,
-`point_cloud_fields.hpp`, `scene_entities_fields.hpp`, `image_fields.hpp`,
-`depth_image_fields.hpp`, `camera_info_fields.hpp`, `video_frame_fields.hpp`;
-`describe()` in `field_table_registry.hpp` returns a table for exactly these eight). `Image::data`,
+Each tabled builtin has a `<type>_fields.hpp` header; `describe()` in
+`field_table_registry.hpp` is the authoritative list. `Image::data`,
 `DepthImage::data`, and `VideoFrame::data` each expose their packed pixel /
 bitstream bytes through a `kBuffer` descriptor the same way `PointCloud::data`
 does, with the per-record layout derived from the encoding/format string

docs/image_annotations_format.md

@@ -66,7 +66,7 @@ the renderable annotation fields and ignores the rest:
 | `ImageAnnotations.circles` | Mapped to `ImageAnnotations::circles`. |
 | `ImageAnnotations.points` | Mapped to `ImageAnnotations::points`. |
 | `ImageAnnotations.texts` | Mapped to `ImageAnnotations::texts`. |
-| Top-level `timestamp` | Serialized when non-zero, decoded into `ImageAnnotations::timestamp`. |
+| Top-level `timestamp` | Serialized when non-zero, decoded into `ImageAnnotations::timestamp`; a timestamp of 0 is indistinguishable from an absent one. |
 | `image_topic` (field 6, string) | Serialized when non-empty, decoded into `ImageAnnotations::image_topic`. |
 | Top-level `metadata` | Not serialized or decoded today. |
 | Per-annotation `timestamp` | Not serialized or decoded today. |

docs/sdk-utilities.md

@@ -60,6 +60,10 @@ CMake helpers become available through `find_package(plotjuggler_sdk)`.
 | `PJ::sdk::MediaMetadataBuilder, ObjectTopicMetadataBuilder` | Build media hints and canonical renderer metadata | `pj_base/sdk/media_metadata.hpp; pj_base/sdk/object_topic_metadata.hpp` |
 | `PJ::sdk::ObjectBytes` | Own object-read bytes through move-only RAII | `pj_base/sdk/object_bytes.hpp` |
 | `PJ::sdk::ArrowSchemaHolder, ArrowArrayHolder, ArrowStreamHolder` | Own Arrow out-params; successful writes transfer ownership, failed writes retain it | `pj_base/sdk/arrow.hpp` |
+| `PJ::sdk::FieldTable<T>, field<&T::member>, describe` | Read/write builtin struct members by name in a generic binder; lists are element-addressable | `pj_base/builtin/field_table.hpp; pj_base/builtin/field_table_registry.hpp` |
+| `PJ::sdk::PinholeIntrinsics, unprojectPixel` | Unproject rectified metric depth pixels; validate K once per image | `pj_base/builtin/depth_image_utils.hpp` |
+| `PJ::sdk::fieldCovered, PJ_HAS_TAIL_SLOT` | struct_size checks before reading an appended field or tail slot | `pj_base/sdk/plugin_data_api.hpp; pj_base/plugin_data_api.h` |
+| `PJ::sdk::DialogPluginBase::hostHas` | Gate dialog UI on a host dialog-capability bit | `pj_plugins/sdk/dialog_plugin_base.hpp` |
 | `pj_embed_file, pj_configure_plugin` | Embed UI/manifest assets and configure plugin exports/build rules | `cmake/PjPlugin.cmake` |
 
 Outside this SDK, reuse pj-official-plugins common/:

pj_base/CLAUDE.md

@@ -18,7 +18,8 @@ build details. It must not depend on Qt, `pj_datastore` or `pj_plugins`.
 - `include/pj_base/builtin/` — 18 builtin struct headers (`*.hpp`) and all 18
   wire codecs (`*_codec.hpp`). Numeric tags are stable; values 2 and 12 are
   permanently reserved. Also contains the tagged, type-erased `BuiltinObject`
-  holder and type-erased codec dispatcher.
+  holder and type-erased codec dispatcher, and the compile-time field tables
+  (`field_table.hpp`, one `<type>_fields.hpp` per tabled struct).
 - `include/pj_base/sdk/` — C++ SDK over the ABI: DataSource + Toolbox `*_plugin_base.hpp`, `service_registry.hpp`/`service_traits.hpp`, host views, Arrow RAII holders, `testing/`.
 - `include/pj_base/*_protocol.h`, `plugin_data_api.h`, `builtin_object_abi.h`, `plugin_abi_export.hpp` — the stable C-ABI surface for DataSource/MessageParser/Toolbox (the Dialog protocol header lives in `pj_plugins/dialog_protocol/`).
 - `proto/pj/` — canonical `.proto` wire contracts for the builtin types (see its README).

pj_base/include/pj_base/builtin/camera_info_fields.hpp

@@ -22,7 +22,7 @@ namespace PJ::sdk {
 /// `distortion_model`).
 template <>
 struct FieldTable<CameraInfo> {
-  static constexpr std::array<FieldDescriptor, 9> fields{
+  static constexpr std::array fields{
       field<&CameraInfo::timestamp_ns>("timestamp_ns"),
       field<&CameraInfo::frame_id>("frame_id"),
       field<&CameraInfo::width>("width"),

pj_base/include/pj_base/builtin/depth_image_fields.hpp

@@ -12,44 +12,45 @@
 
 #include "pj_base/builtin/depth_image.hpp"
 #include "pj_base/builtin/field_table.hpp"
+#include "pj_base/builtin/point_cloud.hpp"
 
 namespace PJ::sdk {
 
 namespace detail {
 
-/// Bytes-per-pixel for a `DepthImage::encoding` value, or 0 when the string
-/// is not one of the two encodings `depth_image.hpp`'s doc comment documents
-/// ("16UC1" — millimeters as uint16 — or "32FC1" — meters as float32).
-/// `DepthImage::encoding` is an open string (like `Image::encoding`), so an
-/// unrecognized value is expected, not an error.
-[[nodiscard]] constexpr uint32_t depthImageBytesPerPixel(std::string_view encoding) noexcept {
+/// Pixel datatype of a `DepthImage::encoding` value, or `kUnknown` when the
+/// string is not one of the two encodings `depth_image.hpp`'s doc comment
+/// documents ("16UC1" — millimeters as uint16 — or "32FC1" — meters as
+/// float32). `DepthImage::encoding` is an open string (like `Image::encoding`),
+/// so an unrecognized value is expected, not an error.
+[[nodiscard]] constexpr PointField::Datatype depthImageDatatype(std::string_view encoding) noexcept {
   if (encoding == "16UC1") {
-    return 2;
+    return PointField::Datatype::kUint16;
   }
   if (encoding == "32FC1") {
-    return 4;
+    return PointField::Datatype::kFloat32;
   }
-  return 0;
+  return PointField::Datatype::kUnknown;
 }
 
 }  // namespace detail
 
 /// `kBuffer` descriptor for `DepthImage::data`. Unlike `Image`, `DepthImage`
 /// has no `row_step` member, so `buffer()` derives it as `width * bpp` (no
 /// row padding representable) whenever `bpp` (from
-/// `detail::depthImageBytesPerPixel(encoding)`) is known; an unrecognized
+/// `bytesPerElement(detail::depthImageDatatype(encoding))`) is known; an unrecognized
 /// encoding reports `record_step = row_step = 0` and `bytes` alone (its full
 /// span) is the payload — same convention as `imageDataField()`
 /// (`image_fields.hpp`). `channels` always carries exactly one entry naming
-/// the encoding string. `buffer_assign()` follows the same
-/// take-ownership-and-re-anchor idiom as `pointCloudDataField()`.
+/// the encoding string. `buffer_assign()` is `detail::assignPayloadBytes`.
 [[nodiscard]] constexpr FieldDescriptor depthImageDataField(std::string_view name) {
   FieldDescriptor d{};
   d.name = name;
   d.kind = FieldKind::kBuffer;
   d.buffer = [](const void* p) -> BufferLayout {
     const auto& image = *static_cast<const DepthImage*>(p);
-    const uint32_t bpp = detail::depthImageBytesPerPixel(image.encoding);
+    const PointField::Datatype datatype = detail::depthImageDatatype(image.encoding);
+    const uint32_t bpp = bytesPerElement(datatype);
     BufferLayout layout;
     layout.bytes = image.data;
     layout.record_step = bpp;
@@ -60,16 +61,11 @@ namespace detail {
         BufferLayout::Channel{
             .name = image.encoding,
             .offset = 0,
-            .datatype = bpp == 2 ? uint8_t{4} : (bpp == 4 ? uint8_t{7} : uint8_t{0}),  // 4=uint16, 7=float32
+            .datatype = static_cast<uint8_t>(datatype),
             .count = bpp == 0 ? 0 : uint32_t{1}});
     return layout;
   };
-  d.buffer_assign = [](void* p, std::vector<uint8_t> bytes) {
-    auto& image = *static_cast<DepthImage*>(p);
-    const PayloadView view = makePayloadView(std::move(bytes));
-    image.data = view.bytes;
-    image.anchor = view.anchor;
-  };
+  d.buffer_assign = &detail::assignPayloadBytes<DepthImage>;
   return d;
 }
 
@@ -81,7 +77,7 @@ namespace detail {
 /// (`std::vector<double>`, size depends on `distortion_model`).
 template <>
 struct FieldTable<DepthImage> {
-  static constexpr std::array<FieldDescriptor, 8> fields{
+  static constexpr std::array fields{
       field<&DepthImage::width>("width"),
       field<&DepthImage::height>("height"),
       field<&DepthImage::encoding>("encoding"),

pj_base/include/pj_base/builtin/depth_image_utils.hpp

@@ -23,27 +23,49 @@
 namespace PJ {
 namespace sdk {
 
-/// Unproject a rectified pixel with positive metric depth using a conventional
-/// pinhole intrinsic matrix. Rejects singular/nonfinite intrinsics and unsupported
-/// skew/projective terms; distortion correction belongs to the caller.
-[[nodiscard]] inline std::optional<std::array<double, 3>> unprojectPixel(
-    const std::array<double, 9>& k, double u, double v, double depth_m) noexcept {
-  for (double value : k) {
-    if (!std::isfinite(value)) {
+/// Conventional pinhole intrinsics, validated once so a per-pixel loop checks only
+/// its own inputs.
+struct PinholeIntrinsics {
+  double fx = 0.0;
+  double fy = 0.0;
+  double cx = 0.0;
+  double cy = 0.0;
+
+  /// Rejects singular/nonfinite intrinsics and unsupported skew/projective terms;
+  /// distortion correction belongs to the caller.
+  [[nodiscard]] static std::optional<PinholeIntrinsics> fromK(const std::array<double, 9>& k) noexcept {
+    for (double value : k) {
+      if (!std::isfinite(value)) {
+        return std::nullopt;
+      }
+    }
+    if (k[0] <= 0 || k[4] <= 0 || k[1] != 0 || k[3] != 0 || k[6] != 0 || k[7] != 0 || k[8] != 1) {
       return std::nullopt;
     }
+    return PinholeIntrinsics{.fx = k[0], .fy = k[4], .cx = k[2], .cy = k[5]};
   }
-  if (!std::isfinite(u) || !std::isfinite(v) || !std::isfinite(depth_m) || depth_m <= 0 || k[0] <= 0 || k[4] <= 0 ||
-      k[1] != 0 || k[3] != 0 || k[6] != 0 || k[7] != 0 || k[8] != 1) {
-    return std::nullopt;
-  }
-  std::array<double, 3> result{(u - k[2]) * depth_m / k[0], (v - k[5]) * depth_m / k[4], depth_m};
-  for (double value : result) {
-    if (!std::isfinite(value)) {
+
+  /// Unproject a rectified pixel with positive metric depth.
+  [[nodiscard]] std::optional<std::array<double, 3>> unproject(double u, double v, double depth_m) const noexcept {
+    if (!std::isfinite(u) || !std::isfinite(v) || !std::isfinite(depth_m) || depth_m <= 0) {
       return std::nullopt;
     }
+    std::array<double, 3> result{(u - cx) * depth_m / fx, (v - cy) * depth_m / fy, depth_m};
+    for (double value : result) {
+      if (!std::isfinite(value)) {
+        return std::nullopt;
+      }
+    }
+    return result;
   }
-  return result;
+};
+
+/// One-pixel form of `PinholeIntrinsics::fromK(k)->unproject(u, v, depth_m)`; a loop
+/// over an image should call `fromK` once instead.
+[[nodiscard]] inline std::optional<std::array<double, 3>> unprojectPixel(
+    const std::array<double, 9>& k, double u, double v, double depth_m) noexcept {
+  const auto intrinsics = PinholeIntrinsics::fromK(k);
+  return intrinsics ? intrinsics->unproject(u, v, depth_m) : std::nullopt;
 }
 
 /// Rectification rotation. For a DepthImage with empty distortion_model

pj_base/include/pj_base/builtin/field_table.hpp

@@ -153,7 +153,8 @@ struct FieldDescriptor {
 
   /// For `kind == kList`: element count and element address — populated for
   /// both a growable list (`std::vector<E>`) and a fixed-size one
-  /// (`std::array<E, N>`).
+  /// (`std::array<E, N>`). Elements are addressable one by one: a binder should
+  /// read `list_at(i)` lazily rather than materialize the whole list per access.
   size_t (*list_size)(const void*) = nullptr;
   const void* (*list_at)(const void*, size_t) = nullptr;
 
@@ -336,6 +337,17 @@ inline constexpr bool kHasFieldTableImpl = false;
 template <class T>
 inline constexpr bool kHasFieldTableImpl<T, std::void_t<decltype(FieldTable<T>::view)>> = true;
 
+/// `buffer_assign` of a builtin whose payload is a `data` view plus its `anchor`:
+/// takes ownership of `bytes` (`makePayloadView`), `data` views them and `anchor`
+/// keeps them alive.
+template <class T>
+void assignPayloadBytes(void* p, std::vector<uint8_t> bytes) {
+  auto& object = *static_cast<T*>(p);
+  const PayloadView view = makePayloadView(std::move(bytes));
+  object.data = view.bytes;
+  object.anchor = view.anchor;
+}
+
 }  // namespace detail
 
 /// True iff `FieldTable<T>` has a specialization (i.e. `T` is describable),

pj_base/include/pj_base/builtin/frame_transforms_fields.hpp

@@ -18,7 +18,7 @@ namespace PJ::sdk {
 /// Fields of `Vector2`: `x`, `y`.
 template <>
 struct FieldTable<Vector2> {
-  static constexpr std::array<FieldDescriptor, 2> fields{
+  static constexpr std::array fields{
       field<&Vector2::x>("x"),
       field<&Vector2::y>("y"),
   };
@@ -28,7 +28,7 @@ struct FieldTable<Vector2> {
 /// Fields of `Vector3`: `x`, `y`, `z`.
 template <>
 struct FieldTable<Vector3> {
-  static constexpr std::array<FieldDescriptor, 3> fields{
+  static constexpr std::array fields{
       field<&Vector3::x>("x"),
       field<&Vector3::y>("y"),
       field<&Vector3::z>("z"),
@@ -39,7 +39,7 @@ struct FieldTable<Vector3> {
 /// Fields of `Quaternion`: `x`, `y`, `z`, `w`.
 template <>
 struct FieldTable<Quaternion> {
-  static constexpr std::array<FieldDescriptor, 4> fields{
+  static constexpr std::array fields{
       field<&Quaternion::x>("x"),
       field<&Quaternion::y>("y"),
       field<&Quaternion::z>("z"),
@@ -51,7 +51,7 @@ struct FieldTable<Quaternion> {
 /// Fields of `Pose`: `position` (Vector3), `orientation` (Quaternion).
 template <>
 struct FieldTable<Pose> {
-  static constexpr std::array<FieldDescriptor, 2> fields{
+  static constexpr std::array fields{
       field<&Pose::position>("position"),
       field<&Pose::orientation>("orientation"),
   };
@@ -62,7 +62,7 @@ struct FieldTable<Pose> {
 /// `child_frame_id` (strings), `translation` (Vector3), `rotation` (Quaternion).
 template <>
 struct FieldTable<FrameTransform> {
-  static constexpr std::array<FieldDescriptor, 5> fields{
+  static constexpr std::array fields{
       field<&FrameTransform::timestamp>("timestamp"),
       field<&FrameTransform::parent_frame_id>("parent_frame_id"),
       field<&FrameTransform::child_frame_id>("child_frame_id"),
@@ -75,7 +75,7 @@ struct FieldTable<FrameTransform> {
 /// Fields of `FrameTransforms`: `transforms` (list of `FrameTransform`).
 template <>
 struct FieldTable<FrameTransforms> {
-  static constexpr std::array<FieldDescriptor, 1> fields{
+  static constexpr std::array fields{
       field<&FrameTransforms::transforms>("transforms"),
   };
   static constexpr FieldTableView view{"FrameTransforms", Span<const FieldDescriptor>(fields)};

pj_base/include/pj_base/builtin/image_annotations_fields.hpp

@@ -18,7 +18,7 @@ namespace PJ::sdk {
 /// Fields of `Point2`: `x`, `y`.
 template <>
 struct FieldTable<Point2> {
-  static constexpr std::array<FieldDescriptor, 2> fields{
+  static constexpr std::array fields{
       field<&Point2::x>("x"),
       field<&Point2::y>("y"),
   };
@@ -28,7 +28,7 @@ struct FieldTable<Point2> {
 /// Fields of `ColorRGBA`: `r`, `g`, `b`, `a` (each `uint8_t`, read/written as `double`).
 template <>
 struct FieldTable<ColorRGBA> {
-  static constexpr std::array<FieldDescriptor, 4> fields{
+  static constexpr std::array fields{
       field<&ColorRGBA::r>("r"),
       field<&ColorRGBA::g>("g"),
       field<&ColorRGBA::b>("b"),
@@ -41,7 +41,7 @@ struct FieldTable<ColorRGBA> {
 /// `Point2`), `thickness`, `color`, `colors` (list of `ColorRGBA`), `fill_color`.
 template <>
 struct FieldTable<PointsAnnotation> {
-  static constexpr std::array<FieldDescriptor, 6> fields{
+  static constexpr std::array fields{
       field<&PointsAnnotation::topology>("topology"),   field<&PointsAnnotation::points>("points"),
       field<&PointsAnnotation::thickness>("thickness"), field<&PointsAnnotation::color>("color"),
       field<&PointsAnnotation::colors>("colors"),       field<&PointsAnnotation::fill_color>("fill_color"),
@@ -53,7 +53,7 @@ struct FieldTable<PointsAnnotation> {
 /// `color`, `fill_color`.
 template <>
 struct FieldTable<CircleAnnotation> {
-  static constexpr std::array<FieldDescriptor, 5> fields{
+  static constexpr std::array fields{
       field<&CircleAnnotation::center>("center"),         field<&CircleAnnotation::radius>("radius"),
       field<&CircleAnnotation::thickness>("thickness"),   field<&CircleAnnotation::color>("color"),
       field<&CircleAnnotation::fill_color>("fill_color"),
@@ -64,7 +64,7 @@ struct FieldTable<CircleAnnotation> {
 /// Fields of `TextAnnotation`: `position` (Point2), `font_size`, `color`, `text`.
 template <>
 struct FieldTable<TextAnnotation> {
-  static constexpr std::array<FieldDescriptor, 4> fields{
+  static constexpr std::array fields{
       field<&TextAnnotation::position>("position"),
       field<&TextAnnotation::font_size>("font_size"),
       field<&TextAnnotation::color>("color"),
@@ -77,7 +77,7 @@ struct FieldTable<TextAnnotation> {
 /// (string), `points`, `circles`, `texts` (lists of the primitive structs above).
 template <>
 struct FieldTable<ImageAnnotations> {
-  static constexpr std::array<FieldDescriptor, 5> fields{
+  static constexpr std::array fields{
       field<&ImageAnnotations::timestamp>("timestamp"), field<&ImageAnnotations::image_topic>("image_topic"),
       field<&ImageAnnotations::points>("points"),       field<&ImageAnnotations::circles>("circles"),
       field<&ImageAnnotations::texts>("texts"),