Improve memory dimensioning for SNB training

[folmos-at-orange]

Description

Currently the dimensioning of the SNB task is too conservative as it slices the training database in various ChallengeAutoML datasets. We suspect that this could be improved by better estimating the necessary memory for the task.

Questions/Ideas

• To be studied in the ChallengeAutoML suite with a 2GB limit.

- ChallengeAutoML/Dionis : Train database was sliced. Number of slices: 2 (needs extra 119.7 MB)
- ChallengeAutoML/Flora : Train database was sliced. Number of slices: 4 (needs extra 30.7 MB)
- ChallengeAutoML/Robert : Train database was sliced. Number of slices: 2 (needs extra 10.2 MB)
- ChallengeAutoML/Tania : Train database was sliced. Number of slices: 5 (needs extra 28.0 MB)
- ChallengeAutoML/Wallis : Train database was sliced. Number of slices: 2 (needs extra 1024.0 KB)
- MTClassification/Auslan : Train database was sliced. Number of slices: 2 (needs extra 40.8 MB)
- MTClassification/MTConnect4 : Train database was sliced. Number of slices: 3 (needs extra 789.1 MB)
- MTClassification/MTConnect4Extended : Train database was sliced. Number of slices: 2 (needs extra 2.5 GB)
- MTClassification/MTPokerHandExtended : Train database was sliced. Number of slices: 6 (needs extra 214.1 MB)
- SmallInstability/AIDS10000 : Train database was sliced. Number of slices: 4 (needs extra 1.3 GB)
- TextClassification/20newsgroups : Train database was sliced. Number of slices: 2 (needs extra 2.5 MB)
- TextClassification/20newsgroups : Train database was sliced. Number of slices: 12 (needs extra 273.0 MB)
- TextClassification/RegressionWineReviews : Train database was sliced. Number of slices: 7 (needs extra 536.3 MB)
- TextClassification/RegressionWineReviews : Train database was sliced. Number of slices: 2 (needs extra 4.6 MB)
- TextClassification/RegressionWineReviews : Train database was sliced. Number of slices: 8 (needs extra 1024.0 KB)

• Make a small study on the necessary memory for the recoding class.
  • MB will implement a tool for this study

Context

• Khiops 10.5.2-b.0