Skip to content

Commit 7323a0a

Browse files
committed
Implement out-of-core mapmaking
Using the previously implemented loader pattern, support on demand loading of observation data when accumulating hits, inverse covariance, and noise weighted maps for each observation. - Add PipelineLoader class, which implements a loader which runs a Pipeline to generate data for an observation. This can be used to simulate data on demand, one observation at a time. - Add AccumulateObservation operator, which can accumulate and cache or load per-observation map domain products. This allows building RHS map domain objects without keeping all detector data in memory, and also allows computing the RHS on smaller jobs which are I/O bound. - Change the BinMap operator to use this new accumulation technique. Use new pixel io interfaces WIP porting mapmaking utils to new accumulation Large WIP on per-obs accumulation with optional cache and load.
1 parent 4fc144d commit 7323a0a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+4553
-2003
lines changed

packaging/conda/deps.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ libtool
44
automake
55
psutil
66
cython
7+
ruamel.yaml
78
pytest
89
tomlkit
910
traitlets

setup.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,7 @@ def readme():
325325
conf["entry_points"] = {
326326
"console_scripts": [
327327
"toast_env = toast.scripts.toast_env:main",
328+
"toast_run = toast.scripts.toast_run:main",
328329
"toast_fake_focalplane = toast.scripts.toast_fake_focalplane:main",
329330
"toast_ground_schedule = toast.scripts.toast_ground_schedule:main",
330331
"toast_project_schedule = toast.scripts.toast_project_schedule:main",

src/toast/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,7 @@ install(FILES
141141
observation_dist.py
142142
observation_data.py
143143
observation_view.py
144+
observation_cache.py
144145
pointing_utils.py
145146
vis.py
146147
rng.py

src/toast/config/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
dump_config,
1010
load_config,
1111
parse_config,
12+
run_config,
1213
)
1314
from .json import dump_json
1415
from .toml import dump_toml

0 commit comments

Comments
 (0)