Code

train rf model in background thread#1178

Code

Open

bbudescu wants to merge 21 commits intoautoml:mainautoml/SMAC3:mainfrom bbudescu:async_train_rfbbudescu/SMAC3:async_train_rfCopy head branch name to clipboard

+770-84

Commits

Commits on Dec 4, 2024

• - train rf model in the background while worker processes use older version

  Show description for f5d8822
  

  Bogdan Budescu

  committed
  f5d8822

Commits on Dec 10, 2024

• work in progress: replace threads by multiprocessing, use shared memory (backed by mmap on posix-compliant OSs) for data, and queues for synchronization

  

  Bogdan Budescu

  committed
  642d8fc

Commits on Dec 11, 2024

• work in progress: keep handle to shared memory until change instead of re-opening every time

  

  Bogdan Budescu

  committed
  7492c48

• refactor: encapsulation: move multiprocessing implementation to dedicated subpackage

  

  Bogdan Budescu

  committed
  d9c3867

• delete metadata

  

  Bogdan Budescu

  committed
  6a449b9

• - refactor: encapsulation of some code in growing array class

  Show description for 413e905
  

  Bogdan Budescu

  committed
  413e905

• - refactor: encapsulation: move training data init where it can be used within background thread

  Show description for 984c59f
  

  Bogdan Budescu

  committed
  984c59f

• minor fixes in sync semantics

  

  Bogdan Budescu

  committed
  75eb34e

• minor fixes in sync

  

  Bogdan Budescu

  committed
  ed50624

Commits on Dec 12, 2024

• add flag to allow user switch to emulating the old behavior, i.e., to wait until training is done before being able to query the model to suggest a new config to try

  

  Bogdan Budescu

  committed
  65edb8d

• - fix bugs in arrays backed by shared memory

  Show description for 818ffd0
  

  Bogdan Budescu

  committed
  818ffd0

• disable resource tracking for shared memory instances to avoid warnings

  

  Bogdan Budescu

  committed
  cc89e16

• use uuids instead of incremental ids to avoid collisions when running multiple sessions in parallel, e.g., when running tests

  

  Bogdan Budescu

  committed
  880e0cc

• minor aesthetic refactors

  

  Bogdan Budescu

  committed
  8458d41

• more minor aesthetic refactors

  

  Bogdan Budescu

  committed
  ae9b821

Commits on Dec 13, 2024

• - add cleanup semantics in hierarchy of objects containing random forests to terminate the training loop process gracefully (to as high as an extent as possible)

  Show description for f05d0bf
  

  Bogdan Budescu

  committed
  f05d0bf

• - improve sync between optimization and training loops

  Show description for cb76f8c
  

  Bogdan Budescu

  committed
  cb76f8c

• add flag to switch off (concurrent) background training entirely (default to old behavior)

  

  Bogdan Budescu

  committed
  6198d8f

• fix typo that caused bug

  

  Bogdan Budescu

  committed
  d956bfb

Commits on Dec 17, 2024

• default enable background training to run tests

  

  Bogdan Budescu

  committed
  396d2f5

Commits on Dec 18, 2024

• use setproctitle, if available to rename rf trainer process, so it's more easily identifiable in monitoring tools

  

  Bogdan Budescu

  committed
  4b30167