Code

Multiprocessing#486

Code

Draft

nkrah wants to merge 169 commits intomasterOpenGATE/opengate:masterfrom multiprocOpenGATE/opengate:multiprocCopy head branch name to clipboard

+1,468-387

Commits

Commits on Oct 9, 2024

• Correct typo in comment

  nkrah

  committed
  fe4c659

  Copy full SHA for fe4c659

• Implement DynamicGateObject.reassign_subset_of_dynamic_params()

  nkrah

  committed
  b66c665

  Copy full SHA for b66c665

• Add attribute process_index to SimulationEngine (not used yet)

  nkrah

  committed
  0456d04

  Copy full SHA for 0456d04

• First steps towards multi processing

  nkrah

  committed
  30526c8

  Copy full SHA for 30526c8

• Add test080_multiprocessing_1.py

  nkrah

  committed
  a7434e5

  Copy full SHA for a7434e5

• Add test030_dose_motion_dynamic_param_multiproc.py

  nkrah

  committed
  605192f

  Copy full SHA for 605192f

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  e4bc3a7

  Copy full SHA for e4bc3a7

Commits on Oct 13, 2024

• Implement MultiProcessingHandler classes

  nkrah

  committed
  523fdc9

  Copy full SHA for 523fdc9

• create test080_multiprocessing_handler.py

  nkrah

  committed
  cea24cf

  Copy full SHA for cea24cf

• Implement import_data_from_actor_output()

  nkrah

  committed
  ae37675

  Copy full SHA for ae37675

• Implement import_user_output_from_actor()

  nkrah

  committed
  9c74aa2

  Copy full SHA for 9c74aa2

• Implement import_user_output_from_actor in ActorBase

  nkrah

  committed
  87af53e

  Copy full SHA for 87af53e

• Change local variable name in reassign_dynamic_params_for_process()

  nkrah

  committed
  196d954

  Copy full SHA for 196d954

• Implement FinalizeSimulation() in VoxelDepositActor

  nkrah

  committed
  138a44f

  Copy full SHA for 138a44f

• Add simulation_id to SimulationOutput

  nkrah

  committed
  a796bec

  Copy full SHA for a796bec

• Implement SimulationOutput.store_output_from_simulation_engine()

  nkrah

  committed
  1375e4f

  Copy full SHA for 1375e4f

• Simplify code in run_engine()

  nkrah

  committed
  fc90573

  Copy full SHA for fc90573

• Implement SimulationMetaData class

  nkrah

  committed
  91f6b7f

  Copy full SHA for 91f6b7f

• Use SimulationMetaData in Simulation

  nkrah

  committed
  722552a

  Copy full SHA for 722552a

• Update MultiProcessingHandlerBase class

  nkrah

  committed
  c7a317b

  Copy full SHA for c7a317b

• Remove obsolete generate_run_timing_interval_map() method

  nkrah

  committed
  52bde4f

  Copy full SHA for 52bde4f

• Update imports in managers.py

  nkrah

  committed
  3d1448b

  Copy full SHA for 3d1448b

• Update run_in_process()

  nkrah

  committed
  fc0d155

  Copy full SHA for fc0d155

• Store number_of_sub_processes and start_new_process in simulation_meta_data

  nkrah

  committed
  8b43fb2

  Copy full SHA for 8b43fb2

• Introduce avoid_write_to_disk_in_subprocess kwarg in Simulation.run()

  nkrah

  committed
  a544da9

  Copy full SHA for a544da9

• Use multi_proc_handler in Simulation.run()

  nkrah

  committed
  19c9b17

  Copy full SHA for 19c9b17

• Trigger import_user_output_from_actor() after multi_proc run

  nkrah

  committed
  23fb55e

  Copy full SHA for 23fb55e

• store meta_data after run

  nkrah

  committed
  a1ce476

  Copy full SHA for a1ce476

• Get parameters from sources after multiproc run

  nkrah

  committed
  a120987

  Copy full SHA for a120987

• Remove obsolete code

  nkrah

  committed
  78bcd34

  Copy full SHA for 78bcd34

• Trigger FinalizeSimulation() at the end of a run

  nkrah

  committed
  39a45c8

  Copy full SHA for 39a45c8

• remove obsolete code

  nkrah

  committed
  7e47bec

  Copy full SHA for 7e47bec

• Add test008_dose_actor_multiproc.py

  nkrah

  committed
  b8581ce

  Copy full SHA for b8581ce

• Update test080_multiprocessing_1.py

  nkrah

  committed
  3d4fe1e

  Copy full SHA for 3d4fe1e

• Update GateObject to rename property 'warnings' in Simulation

  nkrah

  committed
  dcebecc

  Copy full SHA for dcebecc

• Update run_engine to use Simulation.meta_data

  nkrah

  committed
  c804709

  Copy full SHA for c804709

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  27b49c6

  Copy full SHA for 27b49c6

• Rename FinalizeSimulation() to EndOfMultiProcessAction()

  nkrah

  committed
  0a25f0d

  Copy full SHA for 0a25f0d

Commits on Oct 14, 2024

• Implement explicit inplace_merge_with in ItkImageDataItem

  nkrah

  committed
  c4cf4a4

  Copy full SHA for c4cf4a4

• Adapt merge_data to try to accelerate it (WIP)

  nkrah

  committed
  d94c0e4

  Copy full SHA for d94c0e4

• Implement sum_itk_images based on SimpleITK (workaround)

  nkrah

  committed
  7d919d8

  Copy full SHA for 7d919d8

• remove debug prints and clean code

  nkrah

  committed
  7bfa8b7

  Copy full SHA for 7bfa8b7

• In BaseUserInterfaceToActorOutput.__getstate__: always use return_dict = self.__dict__.copy()

  nkrah

  committed
  11f6740

  Copy full SHA for 11f6740

• Implement reset_user_output() and reset_data()

  nkrah

  committed
  1383057

  Copy full SHA for 1383057

• remove debug print

  nkrah

  committed
  6bba1b7

  Copy full SHA for 6bba1b7

• Update comment

  nkrah

  committed
  56061bc

  Copy full SHA for 56061bc

• Update test008_dose_actor_multiproc.py

  nkrah

  committed
  aebcc40

  Copy full SHA for aebcc40

• Update test009_voxels_dynamic.py (not relevant for test result)

  nkrah

  committed
  557e679

  Copy full SHA for 557e679

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  15d72cd

  Copy full SHA for 15d72cd

Commits on Oct 24, 2024

• Correct typo in comment

  nkrah

  committed
  53f5ff0

  Copy full SHA for 53f5ff0

• Implement DynamicGateObject.reassign_subset_of_dynamic_params()

  nkrah

  committed
  1463013

  Copy full SHA for 1463013

• Add attribute process_index to SimulationEngine (not used yet)

  nkrah

  committed
  b3e8f0c

  Copy full SHA for b3e8f0c

• First steps towards multi processing

  nkrah

  committed
  0b7569b

  Copy full SHA for 0b7569b

• Add test080_multiprocessing_1.py

  nkrah

  committed
  d17682f

  Copy full SHA for d17682f

• Add test030_dose_motion_dynamic_param_multiproc.py

  nkrah

  committed
  fb3cb56

  Copy full SHA for fb3cb56

• [pre-commit.ci] Automatic python and c++ formatting

  

  pre-commit-ci[bot]

  authored and

  nkrah

  committed
  aa5e710

  Copy full SHA for aa5e710

• Implement MultiProcessingHandler classes

  nkrah

  committed
  76fbfba

  Copy full SHA for 76fbfba

• create test080_multiprocessing_handler.py

  nkrah

  committed
  6973f13

  Copy full SHA for 6973f13

• Implement import_data_from_actor_output()

  nkrah

  committed
  0260522

  Copy full SHA for 0260522

• Implement import_user_output_from_actor()

  nkrah

  committed
  3b93451

  Copy full SHA for 3b93451

• Implement import_user_output_from_actor in ActorBase

  nkrah

  committed
  4d141c3

  Copy full SHA for 4d141c3

• Change local variable name in reassign_dynamic_params_for_process()

  nkrah

  committed
  f06e627

  Copy full SHA for f06e627

• Implement FinalizeSimulation() in VoxelDepositActor

  nkrah

  committed
  3ea4bc7

  Copy full SHA for 3ea4bc7

• Add simulation_id to SimulationOutput

  nkrah

  committed
  504673f

  Copy full SHA for 504673f

• Implement SimulationOutput.store_output_from_simulation_engine()

  nkrah

  committed
  76b0f86

  Copy full SHA for 76b0f86

• Simplify code in run_engine()

  nkrah

  committed
  fd799dd

  Copy full SHA for fd799dd

• Implement SimulationMetaData class

  nkrah

  committed
  3b6d94d

  Copy full SHA for 3b6d94d

• Use SimulationMetaData in Simulation

  nkrah

  committed
  c4cf0f0

  Copy full SHA for c4cf0f0

• Update MultiProcessingHandlerBase class

  nkrah

  committed
  8b28b15

  Copy full SHA for 8b28b15

• Remove obsolete generate_run_timing_interval_map() method

  nkrah

  committed
  b1f2d21

  Copy full SHA for b1f2d21

• Update imports in managers.py

  nkrah

  committed
  2a033b7

  Copy full SHA for 2a033b7

• Update run_in_process()

  nkrah

  committed
  9a6f871

  Copy full SHA for 9a6f871

• Store number_of_sub_processes and start_new_process in simulation_meta_data

  nkrah

  committed
  18b49a0

  Copy full SHA for 18b49a0

• Introduce avoid_write_to_disk_in_subprocess kwarg in Simulation.run()

  nkrah

  committed
  0a4b237

  Copy full SHA for 0a4b237

• Use multi_proc_handler in Simulation.run()

  nkrah

  committed
  805468a

  Copy full SHA for 805468a

• Trigger import_user_output_from_actor() after multi_proc run

  nkrah

  committed
  fa5ece3

  Copy full SHA for fa5ece3

• store meta_data after run

  nkrah

  committed
  af06335

  Copy full SHA for af06335

• Get parameters from sources after multiproc run

  nkrah

  committed
  002c000

  Copy full SHA for 002c000

• Remove obsolete code

  nkrah

  committed
  43ae7d2

  Copy full SHA for 43ae7d2

• Trigger FinalizeSimulation() at the end of a run

  nkrah

  committed
  78eecbb

  Copy full SHA for 78eecbb

• remove obsolete code

  nkrah

  committed
  c6b9ae1

  Copy full SHA for c6b9ae1

• Add test008_dose_actor_multiproc.py

  nkrah

  committed
  d6d5b37

  Copy full SHA for d6d5b37

• Update test080_multiprocessing_1.py

  nkrah

  committed
  53ceb61

  Copy full SHA for 53ceb61

• Update GateObject to rename property 'warnings' in Simulation

  nkrah

  committed
  38896f0

  Copy full SHA for 38896f0

• Update run_engine to use Simulation.meta_data

  nkrah

  committed
  0e9928d

  Copy full SHA for 0e9928d

• [pre-commit.ci] Automatic python and c++ formatting

  

  pre-commit-ci[bot]

  authored and

  nkrah

  committed
  0020307

  Copy full SHA for 0020307

• Rename FinalizeSimulation() to EndOfMultiProcessAction()

  nkrah

  committed
  2612857

  Copy full SHA for 2612857

• Implement explicit inplace_merge_with in ItkImageDataItem

  nkrah

  committed
  956be1c

  Copy full SHA for 956be1c

• Adapt merge_data to try to accelerate it (WIP)

  nkrah

  committed
  7bb0000

  Copy full SHA for 7bb0000

• Implement sum_itk_images based on SimpleITK (workaround)

  nkrah

  committed
  d5c7c40

  Copy full SHA for d5c7c40

• remove debug prints and clean code

  nkrah

  committed
  481dbc4

  Copy full SHA for 481dbc4

• In BaseUserInterfaceToActorOutput.__getstate__: always use return_dict = self.__dict__.copy()

  nkrah

  committed
  bffb06c

  Copy full SHA for bffb06c

• Implement reset_user_output() and reset_data()

  nkrah

  committed
  fecd1e1

  Copy full SHA for fecd1e1

• remove debug print

  nkrah

  committed
  845544e

  Copy full SHA for 845544e

• Update comment

  nkrah

  committed
  2450ef5

  Copy full SHA for 2450ef5

• Update test008_dose_actor_multiproc.py

  nkrah

  committed
  ee5b1df

  Copy full SHA for ee5b1df

• Update test009_voxels_dynamic.py (not relevant for test result)

  nkrah

  committed
  2f49d87

  Copy full SHA for 2f49d87

• Increment random_seed in run_in_process

  nkrah

  committed
  5ccba4e

  Copy full SHA for 5ccba4e

• Extend inplace_merge_with() to accept *other, i.e. an unpackaged list

  nkrah

  committed
  775ddb2

  Copy full SHA for 775ddb2

• Fix ItkImageDataItem.inplace_merge_with()

  nkrah

  committed
  ccc141d

  Copy full SHA for ccc141d

• Fix typo in error message

  nkrah

  committed
  2df161a

  Copy full SHA for 2df161a

• Update test008_dose_actor_multiproc.py

  nkrah

  committed
  5dcb80b

  Copy full SHA for 5dcb80b

• Deprecate unused kwarg 'stat' in assert_images()

  nkrah

  committed
  79788b8

  Copy full SHA for 79788b8

• Update multiproc logic in Simulation class

  nkrah

  committed
  22411d4

  Copy full SHA for 22411d4

• Merge remote-tracking branch 'origin/multiproc' into multiproc

  Show description for d21bcbf
  nkrah

  committed
  d21bcbf

  Copy full SHA for d21bcbf

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  fb36286

  Copy full SHA for fb36286

• Implement reset_data() in DataItem base class

  nkrah

  committed
  27f4204

  Copy full SHA for 27f4204

• update imports in dataitems.py

  nkrah

  committed
  994fded

  Copy full SHA for 994fded

• Implement StatisticsDataItem

  nkrah

  committed
  1e706b4

  Copy full SHA for 1e706b4

• Implement StatisticsItemContainer

  nkrah

  committed
  b17a98c

  Copy full SHA for b17a98c

• Let SimulationStatisticsActor use new ActorOutputStatisticsActor

  nkrah

  committed
  09a95c4

  Copy full SHA for 09a95c4

• Implement ActorOutputStatisticsActor

  nkrah

  committed
  a2cd755

  Copy full SHA for a2cd755

• Implement UserInterfaceToActorOutputStatisticsActor

  nkrah

  committed
  abdaacd

  Copy full SHA for abdaacd

Commits on Oct 25, 2024

• let default_suffix be an instance attribute of the ActorOutput classes rather than a class attribute.

  nkrah

  committed
  37b6bba

  Copy full SHA for 37b6bba

• Implement ActorOutputUsingDataItemContainer.reset_data()

  nkrah

  committed
  3b0e499

  Copy full SHA for 3b0e499

• Add process_cls(ActorOutputStatisticsActor)

  Show description for 6ce548c
  nkrah

  committed
  6ce548c

  Copy full SHA for 6ce548c

• Remove unused store_output_data() method

  nkrah

  committed
  fd4d7fc

  Copy full SHA for fd4d7fc

• Update test008_dose_actor_multiproc.py

  nkrah

  committed
  e1a6713

  Copy full SHA for e1a6713

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  3c4e7b5

  Copy full SHA for 3c4e7b5

Commits on Oct 28, 2024

• Update physics and sources documentation

  Show description for 51733c1
  

  dsarrut

  authored and

  nkrah

  committed
  51733c1

  Copy full SHA for 51733c1

• Add test080_multiprocessing_1.py

  nkrah

  committed
  d76b64f

  Copy full SHA for d76b64f

• [pre-commit.ci] Automatic python and c++ formatting

  

  pre-commit-ci[bot]

  authored and

  nkrah

  committed
  98752bf

  Copy full SHA for 98752bf

• Implement MultiProcessingHandler classes

  nkrah

  committed
  8217a0b

  Copy full SHA for 8217a0b

• create test080_multiprocessing_handler.py

  nkrah

  committed
  3193890

  Copy full SHA for 3193890

• Implement FinalizeSimulation() in VoxelDepositActor

  nkrah

  committed
  552fae4

  Copy full SHA for 552fae4

• Add test008_dose_actor_multiproc.py

  nkrah

  committed
  47bfa06

  Copy full SHA for 47bfa06

• [pre-commit.ci] Automatic python and c++ formatting

  

  pre-commit-ci[bot]

  authored and

  nkrah

  committed
  352ac37

  Copy full SHA for 352ac37

• Rename FinalizeSimulation() to EndOfMultiProcessAction()

  nkrah

  committed
  cb90f29

  Copy full SHA for cb90f29

• Update test008_dose_actor_multiproc.py

  nkrah

  committed
  d4d0f4b

  Copy full SHA for d4d0f4b

• [pre-commit.ci] Automatic python and c++ formatting

  

  pre-commit-ci[bot]

  authored and

  nkrah

  committed
  8cb80b8

  Copy full SHA for 8cb80b8

• Merge remote-tracking branch 'origin/master' into multiproc

  nkrah

  committed
  092dd69

  Copy full SHA for 092dd69

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  ad36e56

  Copy full SHA for ad36e56

• rename test080 on multi processing to test082_xxx

  nkrah

  committed
  216209f

  Copy full SHA for 216209f

• Add merge_root to processing.py

  nkrah

  committed
  8063a4d

  Copy full SHA for 8063a4d

• rename import_data_from_actor_output to merge_data_from_actor_output

  nkrah

  committed
  3729d25

  Copy full SHA for 3729d25

• Extend signature of import_user_output_from_actor by **kwargs

  nkrah

  committed
  fafbb32

  Copy full SHA for fafbb32

• Add process_index attribute to SimulationEngine and SimulationOutput

  nkrah

  committed
  69d9649

  Copy full SHA for 69d9649

• Implement clear_output_dir_before_run

  Show description for 33fdb50
  nkrah

  committed
  33fdb50

  Copy full SHA for 33fdb50

• Implement sub_process_registry

  nkrah

  committed
  26d6c54

  Copy full SHA for 26d6c54

• Make multi_proc_handler an instance attribute

  nkrah

  committed
  9fe4915

  Copy full SHA for 9fe4915

• Add output_dir as argument to run_in_process()

  nkrah

  committed
  e1d58bf

  Copy full SHA for e1d58bf

• Let each process store a json in the subfolder

  nkrah

  committed
  a70ba52

  Copy full SHA for a70ba52

• Capsulate code into merge_simulations_from_multiprocessing()

  nkrah

  committed
  f5f0d44

  Copy full SHA for f5f0d44

• check global variable __spec__ in run() to avoid breaking when running multiproc in interactive session

  nkrah

  committed
  c9b9b66

  Copy full SHA for c9b9b66

• Remove merge_root() and unicity(): now in ActorOutputRoot class

  nkrah

  committed
  43a8292

  Copy full SHA for 43a8292

• Implement ActorOutputRootmerge_data_from_actor_output()

  nkrah

  committed
  facc182

  Copy full SHA for facc182

• Add test019_phsp_actor_multiproc.py

  nkrah

  committed
  5d06d5b

  Copy full SHA for 5d06d5b

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  01d0ea5

  Copy full SHA for 01d0ea5

• Remove obsolete imports in processing.py

  nkrah

  committed
  245b73b

  Copy full SHA for 245b73b

Commits on Oct 29, 2024

• Add property counts to SimulationStatisticsActor

  nkrah

  committed
  aaed7a5

  Copy full SHA for aaed7a5

• remove obsolete UserInterfaceToActorOutputStatisticsActor

  nkrah

  committed
  c70cfef

  Copy full SHA for c70cfef

• Extend DataItem.__getattr__ to get attributes from self.data if possible

  nkrah

  committed
  f16c1ea

  Copy full SHA for f16c1ea

• Adapt read_stat_file_legacy() to updated stats actor

  nkrah

  committed
  3423476

  Copy full SHA for 3423476

• Adapt assert_stats() and assert_stats_json() to updated stats actor

  nkrah

  committed
  fc7322a

  Copy full SHA for fc7322a

Commits on Oct 30, 2024

• Merge sim_start_time and sim_stop_time via min/max

  nkrah

  committed
  a8f9add

  Copy full SHA for a8f9add

• Adapt assert_stats_json() to updated StatsActor

  nkrah

  committed
  2210788

  Copy full SHA for 2210788

• Improve handling of start_time and stop_time in StatisticsDataItem

  nkrah

  committed
  d482c62

  Copy full SHA for d482c62

• Update GateSimulationStatisticsActor::GetCounts()

  nkrah

  committed
  1473df0

  Copy full SHA for 1473df0

• In GateSimulationStatisticsActor: save start_time and stop_time as epoch time rather than date time str

  nkrah

  committed
  014e950

  Copy full SHA for 014e950

Commits on Oct 31, 2024

• adapt read_stat_file_json() to updated StatsActor

  nkrah

  committed
  451e7da

  Copy full SHA for 451e7da

• remove redundant code in test008_dose_actor_multiproc.py

  nkrah

  committed
  f6f348b

  Copy full SHA for f6f348b

• Merge remote-tracking branch 'origin/master' into multiproc

  nkrah

  committed
  09b4830

  Copy full SHA for 09b4830

• [pre-commit.ci] Automatic python and c++ formatting

  pre-commit-ci[bot]

  committed
  9e69420

  Copy full SHA for 9e69420

• activate ssh session on github

  nkrah

  committed
  0eb6b81

  Copy full SHA for 0eb6b81

• update test082_multiprocessing_1.py

  nkrah

  committed
  bbec500

  Copy full SHA for bbec500

• pick up simulation_id as meta_data after a simulation

  nkrah

  committed
  ec56329

  Copy full SHA for ec56329

• add explicit process_cls() for MultiProcessHandler classes

  nkrah

  committed
  039ef82

  Copy full SHA for 039ef82

• Update file names in test081_simulation_optigan_with_random_seed.py

  nkrah

  committed
  dd3068c

  Copy full SHA for dd3068c

• Adapt test006_runs.py to updated StatsActorOutput

  nkrah

  committed
  0ac5519

  Copy full SHA for 0ac5519