Add infrastructure to filter command events to get alternate command history

[taldcroft]

Description

This PR adds infrastructure to filter command events to get alternate command history:

• New event_filter kwarg in get_cmds, get_states, get_continuity, get_observations,
  get_starcats, get_starcats_as_table functions.
• event_filter is a function that takes a command events table (e.g. from the google sheet) and
  returns a boolean mask that selects events to keep.
• Event filtering functions to facilitate the process, most notably filter_scs107_events that removes
  events normally associated with an SCS-107 run.

Adding this event filtering was not initially compatible with the caching mechanism for recent commands (within the last 30 days). Previously the cache key was just the scenario, but this did not account for changes to:

• CXOTIME_NOW current time mocking
• Change in lookback time (conf.default_lookback).

So the cache key was changed to include those along with the event_filter.

These changes ended up challenging my ability to keep straight the flow and dependencies in the core commands code, so I ended up improving the code:

• Make some dependencies explicit rather than implicit (e.g. dependence on CXOTIME_NOW).
• Renaming variables to be more clear (e.g. get_default_stop -> get_cxotime_now).
• Adding type annotations in places.
• Factoring out code where possible.

Interface impacts

• Previously the State column in a local scenario command events file was being ignored. Now it behaves like the google sheet and (if that column is present), events are filtered accordingly. I.e. only Predictive and Definitive are allowed.
• Added event_filter kwarg to get_cmds, get_states, get_continuity, get_observations,
  get_starcats, get_starcats_as_table functions.

Testing

Unit tests

• Mac (new unit tests)

(ska3) ➜  kadi git:(ignore-scs107s-arg) git rev-parse --short HEAD
fb97db3

(ska3) ➜  kadi git:(ignore-scs107s-arg) pytest                    
======================================================== test session starts ========================================================
platform darwin -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /Volumes/git
configfile: pytest.ini
plugins: hypothesis-6.125.2, doctestplus-1.3.0, anyio-4.7.0, timeout-2.3.1
collected 185 items                                                                                                                 

kadi/commands/tests/test_commands.py ..................................................................................       [ 44%]
kadi/commands/tests/test_filter_events.py ..                                                                                  [ 45%]
kadi/commands/tests/test_states.py .......................x..........................                                         [ 72%]
kadi/commands/tests/test_validate.py ...................                                                                      [ 82%]
kadi/tests/test_events.py ..........                                                                                          [ 88%]
kadi/tests/test_occweb.py ......................                                                                              [100%]

============================================= 184 passed, 1 xfailed in 94.34s (0:01:34) =============================================

Independent check of unit tests by Jean

• Linux

ska3-jeanconn-fido> git rev-parse HEAD
2bce4cfd3b80fe775ba5bacd6573c1d54ddbfc6f
ska3-jeanconn-fido> pytest
==================================================== test session starts ====================================================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /proj/sot/ska/jeanproj/git
configfile: pytest.ini
plugins: anyio-4.7.0, timeout-2.3.1
collected 185 items                                                                                                         

kadi/commands/tests/test_commands.py ................................................................................ [ 43%]
..                                                                                                                    [ 44%]
kadi/commands/tests/test_filter_events.py ..                                                                          [ 45%]
kadi/commands/tests/test_states.py .......................x..........................                                 [ 72%]
kadi/commands/tests/test_validate.py ...................                                                              [ 82%]
kadi/tests/test_events.py ..........                                                                                  [ 88%]
kadi/tests/test_occweb.py ......................                                                                      [100%]

===================================================== warnings summary ======================================================
kadi/kadi/commands/tests/test_commands.py: 88 warnings
kadi/kadi/commands/tests/test_filter_events.py: 26 warnings
kadi/kadi/commands/tests/test_states.py: 7 warnings
kadi/kadi/commands/tests/test_validate.py: 9 warnings
kadi/kadi/tests/test_occweb.py: 19 warnings
  /proj/sot/ska3/flight/lib/python3.12/site-packages/bs4/builder/_lxml.py:124: DeprecationWarning: The 'strip_cdata' option of HTMLParser() has never done anything and will eventually be removed.
    parser = parser(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================= 184 passed, 1 xfailed, 149 warnings in 222.18s (0:03:42

Functional tests

No functional testing.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[jeanconn]

The only thing overall that I note is that "filter" as a verb always feels very ambiguous to me in that it isn't clear from the outset if the filter condition is the thing you want or the thing you don't want. Here you've gone with setting the filter conditions to get the items you don't want (the "filter out" strategy) which is fine. I think all the labeling, method names, and docstrings line up with this, but I note that for me it is easy to get tripped up by the verb.

[taldcroft]

The only thing overall that I note is that "filter" as a verb always feels very ambiguous to me in that it isn't clear from the outset if the filter condition is the thing you want or the thing you don't want. Here you've gone with setting the filter conditions to get the items you don't want (the "filter out" strategy) which is fine. I think all the labeling, method names, and docstrings line up with this, but I note that for me it is easy to get tripped up by the verb.

Noted. Hopefully the docs are clear enough that the filter function returns a mask of the events that you are keeping. This is consistent with the kadi events "filter()" method. Maybe the filter_scs107_events function should have been remove_scs107_events to be more clear.