Running CARACal on Power processors

[paoloserra]

There is an opportunity to test CARACal at an Italian computing centre called CINECA. This is an exploratory step towards bigger things (eventually, SKA regional centres).

One of the CINECA machines runs Power processors, see https://www.hpc.cineca.it/hardware/marconi100 .

Can those of you with divine knowledge make a list of all steps required to install and run Stimela (and thus CARACal) on Power processors? It'd be good to understand how much work is required and by whom.

(Incidentally, CINECA includes machines with Intel processors, too. So if Power processors prove too challenging it's not the end of the world. But I'd still like to understand what's required.)

[bennahugo]

Interesting question Paolo. This will require recompilation of all software written in C/C++/GFortran as far as I can tell, as software bundled with stimela are all x86_64 compiled to run at native performance. https://developer.ibm.com/articles/compilers-for-lop-optimization-options/ Perhaps it would be useful to instead contact the sysadmin to see if they can provide x86 emulation on these machines using kvm / libvirtd? A 5-10% performance hit normally applies with thin hypervisors last time I checked, but it will be by far the easiest way forward. It is not guaranteed that the powerpc has the required instruction support to all software which may have targetted intel / amd vectorization instructions. At least that is my two cents.

…

On Fri, Dec 4, 2020 at 1:36 PM paoloserra ***@***.***> wrote: There is an opportunity to test CARACal at an Italian computing centre called CINECA. This is an exploratory step towards bigger things (eventually, SKA regional centres). One of the CINECA machines runs Power processors, see https://www.hpc.cineca.it/hardware/marconi100 . Can those of you with divine knowledge make a list of all steps required to install and run Stimela (and thus CARACal) on Power processors? It'd be good to understand how much work is required and by whom. (Incidentally, CINECA includes machines with Intel processors, too. So if Power processors prove too challenging it's not the end of the world. But I'd still like to understand what's required.) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/caracal-pipeline/caracal/issues/1295>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4RE6WQWFWPQGYUS7GOARDSTDCSPANCNFSM4UNKKDWA> .

-- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

[bennahugo]

I should include all software that uses things like casacore and further dependencies as well. As things like astropy also include compiled wcslib binaries these will also have to be natively built, so even most python packages will have to be rebuilt if not written in pure python or JIT compiled python.

…

On Fri, Dec 4, 2020 at 8:09 PM Benna Hugo ***@***.***> wrote: Interesting question Paolo. This will require recompilation of all software written in C/C++/GFortran as far as I can tell, as software bundled with stimela are all x86_64 compiled to run at native performance. https://developer.ibm.com/articles/compilers-for-lop-optimization-options/ Perhaps it would be useful to instead contact the sysadmin to see if they can provide x86 emulation on these machines using kvm / libvirtd? A 5-10% performance hit normally applies with thin hypervisors last time I checked, but it will be by far the easiest way forward. It is not guaranteed that the powerpc has the required instruction support to all software which may have targetted intel / amd vectorization instructions. At least that is my two cents. On Fri, Dec 4, 2020 at 1:36 PM paoloserra ***@***.***> wrote: > There is an opportunity to test CARACal at an Italian computing centre > called CINECA. This is an exploratory step towards bigger things > (eventually, SKA regional centres). > > One of the CINECA machines runs Power processors, see > https://www.hpc.cineca.it/hardware/marconi100 . > > Can those of you with divine knowledge make a list of all steps required > to install and run Stimela (and thus CARACal) on Power processors? It'd be > good to understand how much work is required and by whom. > > (Incidentally, CINECA includes machines with Intel processors, too. So if > Power processors prove too challenging it's not the end of the world. But > I'd still like to understand what's required.) > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <https://github.com/caracal-pipeline/caracal/issues/1295>, or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AB4RE6WQWFWPQGYUS7GOARDSTDCSPANCNFSM4UNKKDWA> > . > -- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

-- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

[paoloserra]

thanks @bennahugo

[o-smirnov]

So, this is prime fodder for the "convert to discussion" feature, just trying it out...