Reduction workflow for ess nmx files

[YooSunYoung]

This is a workflow for actual nexus files collected with our filewriter.

It also has a helper to ignore empty logs but I can't guarantee that it always works.
for example, if any of the transformation is empty and one of transformation is time dependent, then the whole transformation is time dependent, therefore we can't really decide the rotation matrix.

However, it's little likely to happen.

[YooSunYoung]

@aaronfinke this is the whole workflow for the real nmx nexus files.

You can use similar command line like mcstas workflow after installing this branch with pip:

pip install -e .

essnmx-reduce \
  --input_file docs/user-guide/977695_00052796.hdf \
  --output_file scipp_out.h5 \
  --verbose \
  --detector_ids 0 \
  --chunk_size 1_000

[YooSunYoung]

I didn't use sciline in this case or our generic workflow since it's a very simple workflow.

[aaronfinke]

Getting the following error when trying to process one of Doro's files:

Traceback (most recent call last):
  File "/Users/aaronfinke/miniforge3/envs/scipp/bin/essnmx-reduce", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/Users/aaronfinke/essnmx/src/ess/nmx/executables.py", line 387, in main
    reduction(
    ~~~~~~~~~^
        input_file=input_file,
        ^^^^^^^^^^^^^^^^^^^^^^
    ...<8 lines>...
        logger=logger,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/Users/aaronfinke/essnmx/src/ess/nmx/executables.py", line 287, in reduction
    toa_bin_edges = toa_bin_edges.to(unit=da.bins.coords['event_time_offset'].unit)
  File "/Users/aaronfinke/miniforge3/envs/scipp/lib/python3.13/site-packages/scipp/core/operations.py", line 293, in to
    raise ValueError("Must provide dtype or unit or both")
ValueError: Must provide dtype or unit or both

[YooSunYoung]

@aaronfinke
Hmm... strange... I tried all three files but it was okay.

I did find some bugs so I updated this branch yesterday though. Did you pull the changes?

Also, if you need to quickly have the 2d plot, here is the code snippet:

%matplotlib widget

import plopp as pp
import scippnexus as snx

with snx.File('/Users/sunyoungyoo/ESS/essnmx/testfile-20250625-192510.hdf') as f:
    dg = f['/entry/instrument/detector_panel_0/'][()]
    display(dg)
    display(dg['data'])
    da_2d = dg['data'].hist()
    da_3d = dg['data'].hist(event_time_offset=50)


display(pp.plot(da_2d, norm='log', title="2D Histogram of Detector Data"))
pp.slicer(
    da_3d,
    title="3D Histogram of Detector Data",
    norm='log',
    keep=['x_pixel_offset', 'y_pixel_offset'],
)

[YooSunYoung]

This is the hack we need to handle the empty nxlogs.

[aaronfinke]

We need to check how the filewriter is writing to HDF5. If it is writing data in chunks already, we should just use that. It's more efficient

[YooSunYoung]

Yes we talked about implementing iter_chunk on scippnexus with our team.
So I'll apply it here once it's done.

Either way, we'll allow both since even the chunk might not fit in the memory for some reason...

[YooSunYoung]

@aaronfinke I have tried this with the data that Doro gave me (/dmsc/scipp/nmx/cern_202506/testfile-20250625-192510.hdf)
and then found some bugs so fixed them...!

This one only has one detector so I didn't plot them in 3 D but indeed it'll be nice to be able to plot them in 3 D without Dials... So I'll prioritize #141 this one

[YooSunYoung]

The error was because of this:

https://scipp.github.io/development/adr/0019-binned-data-unit-dtype-python-properties.html

I added the lower pin of scipp...