Logger cannot merge runs in output data when called out of context

[nojhan]

I'm interfacing my solver with IOHexperimenter at the problem level, because I wish to use my own experiment manager. Indeed, I wish to use my own batch system to place (down to) single runs as independent jobs on an arbitrary set of machines. However, this is not feasible with the current architecture.

The core of the problem is that the Logger interface expect to be initialized once, and then to be called on different target_problem, for several runs. However, if one wish to handle jobs at the runs level, the Logger will be initialized for each run. In that setting, the Logger avoid adding the results in the previously written data folder, and create an additional folder, leading to two different folders holding two separated runs.

Unfortunately, this data structure cannot be loaded as a single experiment by IOHanalyzer.

To test the problem, merge PR #38 and run t-bbob-csvlogger twice.

The solutions I can imagine are to:

• add a state manager to Logger that would allow to load previous data structures from saved files,
• allow IOHanalyzer to parse several folders as a single experiment,
• handle this specific case with an additional data merger.

[jacobdenobel]

@nojhan Is this still relevant?

[nojhan]

Yes. There is no way to parallelize runs out of a single executable while merging the data of the logger::Analyzer in the same folder.

The use case would be to spread the runs on an HPC cluster, while having a single folder with all the data at the end.

Having a script able to merge two folders seems like the easier approach here.

[jacobdenobel]

This is available in Python, but not in yet C++.

[nojhan]

That's sufficient, given that it's needed out of executables anyway. Is it documented somewhere? Le jeu. 23 déc. 2021 à 20:38, Jacob ***@***.***> a écrit :

…

This is available in Python, but not in yet C++. — Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACFSUPQTOMJNV6ITC6NKZ3USN2RHANCNFSM4LOS64WA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jacobdenobel]

I think this method does what you need.

IOHexperimenter/ioh/__init__.py

Line 378 in 8a49d76

def merge_output_to_single_folder(self, prefix: str, target_folder: str):

If you use the Experiment class in Python you can use it. This uses Python multiprocess to parallize, so I think the beviour is similar to what you intend.