From fe20cc463f6743fc259baeb63827df0a9eab9538 Mon Sep 17 00:00:00 2001 From: lucaferranti Date: Mon, 25 Mar 2024 09:20:32 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20=20@=20a627f?= =?UTF-8?q?61e4140fbdb48bc140213e8a6ce7160e6c2=20=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2020-online-get-together/index.html | 1692 ++++++++--------- .../2022-online-unconference/index.html | 144 +- previews/PR445/index.html | 2 +- 3 files changed, 919 insertions(+), 919 deletions(-) diff --git a/previews/PR445/events/2020-online-get-together/index.html b/previews/PR445/events/2020-online-get-together/index.html index 78cb10ee..f5dbea4b 100644 --- a/previews/PR445/events/2020-online-get-together/index.html +++ b/previews/PR445/events/2020-online-get-together/index.html @@ -230,8 +230,6 @@

Monday, November 30th

- - @@ -248,16 +246,6 @@

Monday, November 30th

- Introduction - (Jarno Rantaharju) - - -
-

Welcome and introduction to the event and Nordic RSE.

- -
- - @@ -269,6 +257,8 @@

Monday, November 30th

+ + @@ -327,6 +317,16 @@

Monday, November 30th

+ + Introduction + (Jarno Rantaharju) + + +
+

Welcome and introduction to the event and Nordic RSE.

+ +
+ @@ -377,50 +377,6 @@

Monday, November 30th

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -538,6 +494,50 @@

Questions and comments

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -576,6 +576,56 @@

Questions and comments

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -603,6 +653,8 @@

Questions and comments

+ + @@ -730,58 +782,6 @@

Questions and comments

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -811,34 +811,6 @@

Questions and comments

- - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -989,19 +961,47 @@

Questions and comments

- - - - - - 15:30 (CET) - - - Close - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 15:30 (CET) + + + Close + + + @@ -1037,33 +1037,23 @@

Tuesday, December 1st

-
  • - MXAimbot - AI-based sample centering for macromolecular crystallography - - (5 min) - - (Isak Lindhé) - - -
    -

    What is this?

    -

    MXAimbot is a neural network based tool currently in development, designed to -relieve researchers of the task of manually and individually centering their -samples in synchrotron beamlines for macromolecular crystallography.

    -

    How does it do that?

    -

    It is a pretty simple CNN trained on a few thousand images from a camera -pointed at the loop which holds the samples. These images are annotated with -coordinates, height, and width.

    -

    Why?

    -

    Because the other two alternatives are

    -
      -
    1. Manual centering by humans, which is boring and tedious and consumes researchers valuable time.
      2. -
    2. X-ray centering, which can cause radiation-damage the crystal.
      3. -
    - -
    - -
    • + + + + + + + + + + + + + + + + + @@ -1105,21 +1095,23 @@

    Why?

  • - Aalto RSE + RSE work done at the Oslo Centre for Biostatistics and Epidemiology (5 min) - (Jarno Rantaharju) - - + (Waldir Leoncio Netto) + +
    -

    Introducing the Aalto RSE group and program.

    +

    Abstract

    +

    The Oslo Centre for Biostatistics and Epidemiology (OCBE) is a scientific center integrating the activities of the Department of Biostatistics, University of Oslo, and the Section of Biostatistics and Epidemiology, Oslo University Hospital. Our 70+ scientists work together to provide research, teaching, and advising to the scientific community.

    +

    In this lightning talk, I share a bit of my experience as the first (and currently only) hired RSE professional.

    +

    Some links for the curious

    -

    Slides: https://drive.google.com/file/d/1wf2ZFLGbwnCGzRwB07cquKauDvjv2WIu/view?usp=sharing

    @@ -1130,22 +1122,28 @@

    Why?

  • - The CodeRefinery project + MXAimbot - AI-based sample centering for macromolecular crystallography (5 min) - (Radovan Bast) - - + (Isak Lindhé) + +
    -

    In this short presentation I will discuss how we grew the CodeRefinery project -over the past 4 years and taught hundreds of students and researchers across -all disciplines in best practices in reproducible research software -engineering.

    -

    I will highlight how we transitioned from in-person workshops to online -training and the team effort which made it possible to scale the workshops to -almost 100 participants per event.

    -

    Slides: https://bit.ly/coderefinery-nordic-rse-2020

    +

    What is this?

    +

    MXAimbot is a neural network based tool currently in development, designed to +relieve researchers of the task of manually and individually centering their +samples in synchrotron beamlines for macromolecular crystallography.

    +

    How does it do that?

    +

    It is a pretty simple CNN trained on a few thousand images from a camera +pointed at the loop which holds the samples. These images are annotated with +coordinates, height, and width.

    +

    Why?

    +

    Because the other two alternatives are

    +
      +
    1. Manual centering by humans, which is boring and tedious and consumes researchers valuable time.
      2. +
    2. X-ray centering, which can cause radiation-damage the crystal.
      3. +
    @@ -1164,55 +1162,6 @@

    Why?

    - - - - - - - - - - - - - - - - - - - - - -
  • - RSE work done at the Oslo Centre for Biostatistics and Epidemiology - - (5 min) - - (Waldir Leoncio Netto) - - -
    -

    Abstract

    -

    The Oslo Centre for Biostatistics and Epidemiology (OCBE) is a scientific center integrating the activities of the Department of Biostatistics, University of Oslo, and the Section of Biostatistics and Epidemiology, Oslo University Hospital. Our 70+ scientists work together to provide research, teaching, and advising to the scientific community.

    -

    In this lightning talk, I share a bit of my experience as the first (and currently only) hired RSE professional.

    -

    Some links for the curious

    - - -
    - -
    • - - - - - - @@ -1278,6 +1227,35 @@

    Some links for the curious

    + + + + + + + + + +
  • + Aalto RSE + + (5 min) + + (Jarno Rantaharju) + + +
    +

    Introducing the Aalto RSE group and program.

    +
      +
    • Exists inside Aalto Scientific Computing
      • +
    • Supported by deparments, basic service
      • +
    • Expected to grow with longer term projects
      • +
    +

    Slides: https://drive.google.com/file/d/1wf2ZFLGbwnCGzRwB07cquKauDvjv2WIu/view?usp=sharing

    + +
    + +
    • @@ -1298,6 +1276,28 @@

    Some links for the curious

    + +
  • + The CodeRefinery project + + (5 min) + + (Radovan Bast) + + +
    +

    In this short presentation I will discuss how we grew the CodeRefinery project +over the past 4 years and taught hundreds of students and researchers across +all disciplines in best practices in reproducible research software +engineering.

    +

    I will highlight how we transitioned from in-person workshops to online +training and the team effort which made it possible to scale the workshops to +almost 100 participants per event.

    +

    Slides: https://bit.ly/coderefinery-nordic-rse-2020

    + +
    + +
    • @@ -1342,28 +1342,57 @@

    Some links for the curious

    - - - - - - - - - - - - - - - - - - - - - - +
  • + My first R package + + (15 min) + + (Tobias Busch) + + +
    +

    Writing an R package has helped me leave my comfort zone and level up my R +programming skills. The code I write as a researcher is mostly single-user and +single-use. Writing and publishing code meant for others has helped me break +old habits and acquire useful new software engineering skills. R has a +streamlined ecosystem for package development that supports understanding and +adhering to best practices. I will talk about the things I have learned while +writing my first R package, why I think writing a package should be a rite of +passage for any aspiring research software engineer, and why R is a great tool +for this.

    + + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + +
  • @@ -1401,8 +1430,6 @@

    Some links for the curious

    - - @@ -1455,33 +1482,6 @@

    Some links for the curious

    - -
  • - My first R package - - (15 min) - - (Tobias Busch) - - -
    -

    Writing an R package has helped me leave my comfort zone and level up my R -programming skills. The code I write as a researcher is mostly single-user and -single-use. Writing and publishing code meant for others has helped me break -old habits and acquire useful new software engineering skills. R has a -streamlined ecosystem for package development that supports understanding and -adhering to best practices. I will talk about the things I have learned while -writing my first R package, why I think writing a package should be a rite of -passage for any aspiring research software engineer, and why R is a great tool -for this.

    - - -
    - -
    • @@ -1556,20 +1556,6 @@

    Some links for the curious

    - - - - - - - - - - - - - - @@ -1603,6 +1589,36 @@

    Some links for the curious

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1785,26 +1801,6 @@

    Questions and comments

    - - - - - - - - - - - - - - - - - - - -
  • Computational reproducibility and the FAIR principles @@ -1835,6 +1831,10 @@

    Questions and comments

    + + + + @@ -1866,14 +1866,6 @@

    Questions and comments

    - - - - - - - -
  • Intro to the HTCondor Python API on a laptop cluster @@ -1919,26 +1911,6 @@

    References

    - - - - - - - - - - - - - - - - - - - - @@ -1970,17 +1942,80 @@

    References

    - - - - - - - - - - - +
  • + What academic RSEs could learn from startups? + + (Dmitrii Borisevich) + + +
    +

    Dear all,

    +

    Below you will find a proposal for the discussion "What academic RSE could learn from startups?" on the 1st of December, 14:00. +If you feel that you share this frustration about research software, and you would like to join the discussion session, feel free to comment on the proposal, and let us all know in advance what your experience is.

    +

    The academic world strives to perform the best research possible. The research that was done thirty years ago created a foundation for modern-day computational methods in many areas. But today many academic areas suffer a reproducibility crisis. Letters and papers are regularly published in high-impact journals about reproducibility crisis,... and nothing changes. Poor scientific software is considered one of the major causes of the crisis.

    +

    From a startup perspective, academic environments often look outdated and generally wrong. CI/CD, shared codebase, code review, Agile, and orientation to the product are seen as necessary to just survive in the startup world. At the same time, these concepts are completely unheard of or even opposed in most of the non-CS academic places. Why is it so and what can we do about it? Do we really want reproducible research, or do we only want to grumble about it?

    +

    The discussion will:

    +

    Start with discussing the experiences of the participants, +Analyze a trade-off between the benefits and the costs of reproducibility, and how it affects research, +Compare the benefits of teamwork with the academic "single researcher" mentality, and check how it affects RSE's outcome, +Discuss infrastructure and management problems, +Summarize potential solutions. +We look to meet everyone, who feels they have the same problem in their area of research.

    +

    Questions and comments

    +

    Stats: 1/2 are "senior" staff, 1/4 are PhD fellows, 1/4 are others

    +
    Problems:
    +
      +
    • Individual publication pressure +
        +
      • Publications are KPI
        • +
      • "Software won't give you a PhD"
        • +
      • Individual work is expected
        • +
      • This leads to people using their limited time towards personal research rather than developing tools and collaborating
        • +
      • "Cultural inertia" among peers and leadership doesn't help
        • +
      +
      • +
    • No clear future career and role model +
        +
      • No good role models, no understanding of how to transition from MSc/PhD to an "RSE"
        • +
      • No clear expectations how much freedom to do research an RSE should have - is RSE a researcher or employee?
        • +
      +
      • +
    • No resources and training +
        +
      • There is not enough knowledge resources and training
        • +
      • And different backgrounds need different training
        • +
      +
      • +
    +
    Solutions needed:
    +
      +
    • Promotion of team work (both RSE + "scientists" for more papers and RSE + RSE for day-to-day working and learning) +
        +
      • "In industry you may go to other people who would complement your skills"
        • +
      +
      • +
    • Adoption and enforcement of industry's technical solutions for co-developing (VCS, etc.) to enable the co-developing itself +
        +
      • Technical debt is addressed in product startups because the quality of thier product matters - doesn't quality of research matter too?
        • +
      +
      • +
    • Allocation of time for teaching and knowledge transfer +
        +
      • Remember "bus factor" - how many RSEs need to leave the group for its research to fall apart?
        • +
      +
      • +
    +

    But no one makes these solutions!

    +
    Actionable steps - what we could do as the RSE society?
    +
      +
    • Public advocacy campaing towards funders - they should fund RSE projects and put pressure on leadership!
      • +
    • Advocacy campaing towards leadership - they will benefit the most because good RSE practices are beneficial in a long run, over 2-4 years
      • +
    • More educational materials within the community
      • +
    + +
    + +
    • @@ -2053,80 +2088,45 @@

    Questions and comments

    -
  • - What academic RSEs could learn from startups? - - (Dmitrii Borisevich) - - -
    -

    Dear all,

    -

    Below you will find a proposal for the discussion "What academic RSE could learn from startups?" on the 1st of December, 14:00. -If you feel that you share this frustration about research software, and you would like to join the discussion session, feel free to comment on the proposal, and let us all know in advance what your experience is.

    -

    The academic world strives to perform the best research possible. The research that was done thirty years ago created a foundation for modern-day computational methods in many areas. But today many academic areas suffer a reproducibility crisis. Letters and papers are regularly published in high-impact journals about reproducibility crisis,... and nothing changes. Poor scientific software is considered one of the major causes of the crisis.

    -

    From a startup perspective, academic environments often look outdated and generally wrong. CI/CD, shared codebase, code review, Agile, and orientation to the product are seen as necessary to just survive in the startup world. At the same time, these concepts are completely unheard of or even opposed in most of the non-CS academic places. Why is it so and what can we do about it? Do we really want reproducible research, or do we only want to grumble about it?

    -

    The discussion will:

    -

    Start with discussing the experiences of the participants, -Analyze a trade-off between the benefits and the costs of reproducibility, and how it affects research, -Compare the benefits of teamwork with the academic "single researcher" mentality, and check how it affects RSE's outcome, -Discuss infrastructure and management problems, -Summarize potential solutions. -We look to meet everyone, who feels they have the same problem in their area of research.

    -

    Questions and comments

    -

    Stats: 1/2 are "senior" staff, 1/4 are PhD fellows, 1/4 are others

    -
    Problems:
    -
      -
    • Individual publication pressure -
        -
      • Publications are KPI
        • -
      • "Software won't give you a PhD"
        • -
      • Individual work is expected
        • -
      • This leads to people using their limited time towards personal research rather than developing tools and collaborating
        • -
      • "Cultural inertia" among peers and leadership doesn't help
        • -
      -
      • -
    • No clear future career and role model -
        -
      • No good role models, no understanding of how to transition from MSc/PhD to an "RSE"
        • -
      • No clear expectations how much freedom to do research an RSE should have - is RSE a researcher or employee?
        • -
      -
      • -
    • No resources and training -
        -
      • There is not enough knowledge resources and training
        • -
      • And different backgrounds need different training
        • -
      -
      • -
    -
    Solutions needed:
    -
      -
    • Promotion of team work (both RSE + "scientists" for more papers and RSE + RSE for day-to-day working and learning) -
        -
      • "In industry you may go to other people who would complement your skills"
        • -
      -
      • -
    • Adoption and enforcement of industry's technical solutions for co-developing (VCS, etc.) to enable the co-developing itself -
        -
      • Technical debt is addressed in product startups because the quality of thier product matters - doesn't quality of research matter too?
        • -
      -
      • -
    • Allocation of time for teaching and knowledge transfer -
        -
      • Remember "bus factor" - how many RSEs need to leave the group for its research to fall apart?
        • -
      -
      • -
    -

    But no one makes these solutions!

    -
    Actionable steps - what we could do as the RSE society?
    -
      -
    • Public advocacy campaing towards funders - they should fund RSE projects and put pressure on leadership!
      • -
    • Advocacy campaing towards leadership - they will benefit the most because good RSE practices are beneficial in a long run, over 2-4 years
      • -
    • More educational materials within the community
      • -
    - -
    - -
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2190,46 +2190,6 @@

    Wednesday, December 2nd

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -2360,6 +2320,46 @@

    Questions and comments

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2404,64 +2404,19 @@

    Questions and comments

    -
  • - R <-> Python transpiler - - (15 min) - - (Dmitrii Borisevich) - - -
    -

    Python and R are two major programming languages used for research software development and data analysis in bioinformatics. It is not a symbiotic relationship, but a cold war between the fans of both.

    -

    Different tools are available to use R in python and vice versa, but they demand learning both languages. This is not easy, and thus rarely adopted.

    -

    This talk will pitch an idea of using AST to build a transpiler between two languages and showcase a simple demo of converting code written in one language directly into another.

    -

    The talk will present:

    -
      -
    • the R vs python problem and its consequences in bioinformatics,
      • -
    • the idea of a transpiler,
      • -
    • some examples of existing transpilers,
      • -
    • a demo of R <-> python PoC transpiler.
      • -
    -
    -

    Questions and comments

    - - -
    - -
    • + + + + + + + + + + + + + @@ -2493,6 +2448,16 @@

    Questions and comments

    + + + + + + + + + + @@ -2561,29 +2526,64 @@

    Questions and comments

    - - - - - - - - - - - - - - - - - - - - - - - +
  • + R <-> Python transpiler + + (15 min) + + (Dmitrii Borisevich) + + +
    +

    Python and R are two major programming languages used for research software development and data analysis in bioinformatics. It is not a symbiotic relationship, but a cold war between the fans of both.

    +

    Different tools are available to use R in python and vice versa, but they demand learning both languages. This is not easy, and thus rarely adopted.

    +

    This talk will pitch an idea of using AST to build a transpiler between two languages and showcase a simple demo of converting code written in one language directly into another.

    +

    The talk will present:

    +
      +
    • the R vs python problem and its consequences in bioinformatics,
      • +
    • the idea of a transpiler,
      • +
    • some examples of existing transpilers,
      • +
    • a demo of R <-> python PoC transpiler.
      • +
    +
    +

    Questions and comments

    + + +
    + +
    • @@ -2656,105 +2656,6 @@

    Questions and comments

    -
  • - European Environment for Scientific Software Installations (EESSI) - - (Thomas Röblitz) - - -
    -

    We will do a presentation of the EESSI (European Environments for Scientific Software Installations) project including a demo of its current pilot software stack.

    -

    In a nutshell, EESSI develops an infrastructure/service which will eventually allow you to use the same scientific software stack on any machine (e.g., Raspberry Pi, laptop, server, cluster, cloud, supercomputer) running on various operating systems (Linux, macOS, Windows) and the software stack is built from sources and can thereby be optimised for the CPU/GPU/interconnect at your machine. Even better you don't even have to install (almost) any software package as the stack will be delivered to you via CernVM-FS a proven solution to distribute software in the WLCG (Worldwide LHC Computing Grid).

    -

    The current pilot stack can be easily tested via Singularity, supports ARM, Intel and AMD processors and includes scientific software packages such as GROMACS, OpenFOAM, bioconductor, TensorFlow as well as all their dependencies.

    -
    -

    Questions and comments

    -
      -
    • Question: Is it possible to test the whole stack, please add links? -
        -
      • Yes, see https://eessi.github.io/docs/pilot/
        • -
      • To get help, join the EESSI Slack, see https://www.eessi-hpc.org/join/
        • -
      -
      • -
    -
      -
    • Question: Will you also support AMD Rocm and AMD ecosystem overall? -
        -
      • Yes, eventually. Right now there already are optimized installations for AMD Zen2 (Rome). -OpenMPI is included and is installed on top of UCX & libfabric, so should properly support AMD Rocm interconnect, but this is currently untested.
        • -
      -
      • -
    • Comment: I like this idea as for us it is important people can use it in their laptops. -Personally I not much loosing time in setting up sw at my laptop but I see for the users it is important -to have an option to install/use it also in their lab. They like it more. -
        -
      • Yes, this could allow people to literally write a job script that just works on the HPC cluster. Same modules, same software. -(and no need to build containers, or copy them over, etc.)
        • -
      -
      • -
    • Question: This builds on existing projects so it has some content from the begining. -
        -
      • Thanks to EasyBuild we can easily provide 1000s of installations. -Right now we limit what we provide, so we can focus on solving the problems we're hitting first.
        • -
      -
      • -
    • Question: Why European in the name? -
        -
      • Because it started with European sites. -We're already thinking about changing the first E to "Easy" :) -"EESSI is the Easy Environment..."
        • -
      -
      • -
    • Question: Question: what are the possibilities to add “own dirty module”, is it like same as e.g. with EasyBuild itself? -
        -
      • You can easily install additional software on top, for example in your home directory on in /tmp, just like you can with any other software stack built with EasyBuild).
        • -
      -
      • -
    • Question: Sensitivity of central Stratum-0 component, in terms of resilience? -
        -
      • The CernVM-FS design is very robust. If the Stratum-0 dies, the only impact is that you can't add new software to the repositories. -As long as one Stratum-1 server is still alive, the software remains available (all Stratum-1 servers have a full copy of the provided software). -So it comes down to having enough Stratum-1 servers, spread across the world, in different sites and cloud providers.
        • -
      • W.r.t adding software: we plan to fully automate the workflow of adding software to the EESSI repository, such that adding software comes down to opening a pull request on GitHub. When the PR is approved by a reviewer, the software gets built automatically on all supported CPU architectures, and added to Stratum-0, fully automatically. Ideally we also have (small) test cases to verify that the installations are functional before deploying them.
        • -
      -
      • -
    • Question: You mentioned that CernVM-FS only relies on HTTP connections. Shouldn't that be HTTPS for security reasons? -
        -
      • No, switching to HTTPS has no added value in terms of security, we've discussed that with the CernVM-FS developers. -CernVM-FS has built in security checks between server and clients, so HTTPS doesn't provide any additional security (I think, should be checked in CernVM-FS documentation).
        • -
      -
      • -
    • How would this work for large jobs across multiple nodes, can a lot of network traffic to pull in the software be avoided? -
        -
      • Yes, you can set up a shared CernVM-FS cache on a shared filesystem. -If there's no internet access on the cluster workernodes, you can use a squid proxy in the cluster network (on a login node for example). -This setup has been tested with the EESSI pilot stack at the Jülich Supercomputing Centre, worked really well!
        • -
      -
      • -
    • Comment: Detection of CPU architecture is a very nice feature. This is a big issue with containers where generic binaries are often, which can have a big impact on performance. -
        -
      • Yes, indeed! Containers are also very rigid: what if you want to add additional software? -The EESSI environment is way more dynamic, easy to add software on top of it (without paying for it in terms of performance), etc.
        • -
      -
      • -
    • Comment: This would also work really well in heterogenous environments with a mix of old/new CPUs, thanks to the auto-detection mechanism. -
        -
      • Yes, very correct, this is an interesting use case!
        • -
      -
      • -
    - -
    - -
    • - - - - - - - - -
  • Lessons learned from procuring a fairly large HPC system @@ -2856,9 +2757,83 @@

    Questions and comments

    - - - +
  • + Is it possible to make code quality more important than the number of published papers in academia? + + (Sunniva Indrehus) + + +
    +

    The code quality in academia has a bad reputation. A global measure of the quality of a computational-oriented research group is typically based on the number of published papers and not a stable and well organized code. The latter is crucial for the further development of the scientific quality of the group. Is it possible to make code quality more important than the number of published papers in academia?

    +

    Interesting questions that are closely related to my title:

    +
      +
    • How to construct a sustainable workflow for groups working where someone has a user and someone a developer perspective? Where goes the line between what type of knowledge is expected from the users and the maintainers?
      • +
    • Who has the responsibility to teach the academic staff about best software practices? Is it the individual doing computational stuff he(r)self?
      • +
    • How can the studying programs at the universities speed up and keep track of the “standard” developments in the business? For instance: in 2020 everybody doing some kind of development should be aware of version control and testing.
      • +
    +

    Slides: https://drive.google.com/file/d/1lnveZ3flvTixgIDv2kwKDIhL0saXFE1b/view?usp=sharing

    +
    +

    Questions and comments

    +

    Follow-up questions:

    +
      +
    • Does it make sense to move in this direction?
      • +
    • Better coding culture and education ?
      • +
    • Incentivisation ?
      • +
    +
      +
    • code quality vs code volume: Is it worth putting effort in going open source? -> ongoing debate in many places
      • +
    • github: a representation of the university towards the world, should be showcasing the good
      • +
    • we may not be experts but we should be using the tools provided (version control, testing) +-> "you wouldn't trust an uncalibrated thermometer"
      • +
    • pressure form funding agency to produce research results and not code, hard to try to find excuses to make good code an output
      • +
    • code should be a part of research proposal, otherwise there is 'not enough time'
      • +
    • often severity of the problem is not seen by professors
      • +
    • people get by by producing adequate code and get by and get funding, that does not motivate people to do testing etc
      • +
    • professors need to know and understand the problem (often they do not do any coding (anymore) and forget)
      • +
    • later additions to 'bad code' leads to problems which get noticed -> explanation to the 'higher level' how we could now save time with better code from the beginning
      • +
    • Lots of time pressure in research projects where improving and making code reproducible is not focussed on
      • +
    • no courses on how to write research code :( or not many
      • +
    • stuff like software engineering is often one of the first courses to be dropped when money runs out. Often because 'higher levels' do not know about the importance
      • +
    • new field: not much old stuff to build on top, no real need for sharing. But not anymore the case.
      • +
    • Catchin up takes time, no time to reproduce everything
      • +
    +

    How can we find the 'paper' of coding?

    +
      +
    • github stars
      • +
    • https://joss.theoj.org/ +https://openresearchsoftware.metajnl.com/ +https://www.journals.elsevier.com/softwarex, software wrapped in paper for traditional metrics, good start but does not solve the problem
      • +
    • one citable paper for many years of developing the software? -> need to be judged on different scale
      • +
    • Standard research outputs are not the only thing that research is measured by anymore, need to 'jump on the train'
      • +
    • continuous necessity for novelty, all metrics problematic, potential metrics: how many people are using your product? -> if many people use it, it is valuable to maintain and update software, supporting a large community
      • +
    • usage metrics as a way of demonstrating impact -> hard to make funding bodies recognize that
      • +
    • in ok it is now pushed to be recognized, slowly building up now, took several years to build evidence base of usage
      • +
    • UK’s Research Excellence Framework: https://www.ref.ac.uk/
      • +
    • importance of being able to read documentation, you need to know where to look and how to do (not everyone can do that)
      • +
    • what about promoting the importance of releasing often? It is a measure of continuous effort, something that writing one-off papers doesn't do, and even small, bug-fixing patches are important IMO. The flip side is it could encourage busy-work, but I still think it's worth it.
      • +
    • all metrics can break -> wide variety of metrics for value is important
      • +
    • some people work in a field where software is not used much by other people, but may be very useful for your colleagues -> citation supports more novelty than quality
      • +
    • no tasks anymore that can be solved by one person. As a researcher, pair up with an RSE to solve a problem, paper together. win-win. can also help your career.
      • +
    • people are rewarded for bad code by keeping their job through being the only person who can actually read and work with their code, no incentive to make code better -> better long term management needed
      • +
    • today no one is indispensible, dont hire people who think they are
      • +
    • make sure multiple people can 'keep the server running', collaborate
      • +
    • -> culture change needed
      • +
    • need for basic education (version control etc) of students, code review
      • +
    • but people do not like to find out / being pointed put as having written bad code, creates high barrier, but one one need to get over
      • +
    • when sharing code, poeple will get used to it, as its part of development
      • +
    • Coderefinery as a good place to send new phds students to learn version control and how collaborative coding works -> no merge without someone else reading the code first
      • +
    • it's all research, you never know which part will become part of your codebase, turns into something big -> hard to go back later, so its important to start early with reviewing, version control etc
      • +
    • So maybe we should have some sense of "continual review" like "continuous integration"
      • +
    • writing test framework takes time but is worth it in the long run
      • +
    • do what you want in your own code,but you will need to 'act like a software engineer' when working with others
      • +
    • compared to how long it takes to make things work, test implementation does not take too much time
      • +
    • courses such as coderefinery are not appealing to some people who think of themselves knowing git* enough for current use, 'bubble workers', so they never learn how it could be also with branches in own code
      • +
    • 'onboarding package' as in how we do things here is lacking in academia, not everywhere though: example: https://scicomp.aalto.fi/aalto/welcomeresearchers/
      • +
    + +
    + +
    • @@ -2867,8 +2842,6 @@

    Questions and comments

    - - @@ -2883,8 +2856,6 @@

    Questions and comments

    - - @@ -2947,83 +2918,112 @@

    Questions and comments

    + + + + + + + + + + + + + + + +
  • - Is it possible to make code quality more important than the number of published papers in academia? + European Environment for Scientific Software Installations (EESSI) - (Sunniva Indrehus) - - + (Thomas Röblitz) + +
    -

    The code quality in academia has a bad reputation. A global measure of the quality of a computational-oriented research group is typically based on the number of published papers and not a stable and well organized code. The latter is crucial for the further development of the scientific quality of the group. Is it possible to make code quality more important than the number of published papers in academia?

    -

    Interesting questions that are closely related to my title:

    -
      -
    • How to construct a sustainable workflow for groups working where someone has a user and someone a developer perspective? Where goes the line between what type of knowledge is expected from the users and the maintainers?
      • -
    • Who has the responsibility to teach the academic staff about best software practices? Is it the individual doing computational stuff he(r)self?
      • -
    • How can the studying programs at the universities speed up and keep track of the “standard” developments in the business? For instance: in 2020 everybody doing some kind of development should be aware of version control and testing.
      • -
    -

    Slides: https://drive.google.com/file/d/1lnveZ3flvTixgIDv2kwKDIhL0saXFE1b/view?usp=sharing

    +

    We will do a presentation of the EESSI (European Environments for Scientific Software Installations) project including a demo of its current pilot software stack.

    +

    In a nutshell, EESSI develops an infrastructure/service which will eventually allow you to use the same scientific software stack on any machine (e.g., Raspberry Pi, laptop, server, cluster, cloud, supercomputer) running on various operating systems (Linux, macOS, Windows) and the software stack is built from sources and can thereby be optimised for the CPU/GPU/interconnect at your machine. Even better you don't even have to install (almost) any software package as the stack will be delivered to you via CernVM-FS a proven solution to distribute software in the WLCG (Worldwide LHC Computing Grid).

    +

    The current pilot stack can be easily tested via Singularity, supports ARM, Intel and AMD processors and includes scientific software packages such as GROMACS, OpenFOAM, bioconductor, TensorFlow as well as all their dependencies.

    Questions and comments

    -

    Follow-up questions:

      -
    • Does it make sense to move in this direction?
      • -
    • Better coding culture and education ?
      • -
    • Incentivisation ?
      • +
    • Question: Is it possible to test the whole stack, please add links? +
        +
      • Yes, see https://eessi.github.io/docs/pilot/
        • +
      • To get help, join the EESSI Slack, see https://www.eessi-hpc.org/join/
        • +
      +
      -
    • code quality vs code volume: Is it worth putting effort in going open source? -> ongoing debate in many places
      • -
    • github: a representation of the university towards the world, should be showcasing the good
      • -
    • we may not be experts but we should be using the tools provided (version control, testing) --> "you wouldn't trust an uncalibrated thermometer"
      • -
    • pressure form funding agency to produce research results and not code, hard to try to find excuses to make good code an output
      • -
    • code should be a part of research proposal, otherwise there is 'not enough time'
      • -
    • often severity of the problem is not seen by professors
      • -
    • people get by by producing adequate code and get by and get funding, that does not motivate people to do testing etc
      • -
    • professors need to know and understand the problem (often they do not do any coding (anymore) and forget)
      • -
    • later additions to 'bad code' leads to problems which get noticed -> explanation to the 'higher level' how we could now save time with better code from the beginning
      • -
    • Lots of time pressure in research projects where improving and making code reproducible is not focussed on
      • -
    • no courses on how to write research code :( or not many
      • -
    • stuff like software engineering is often one of the first courses to be dropped when money runs out. Often because 'higher levels' do not know about the importance
      • -
    • new field: not much old stuff to build on top, no real need for sharing. But not anymore the case.
      • -
    • Catchin up takes time, no time to reproduce everything
      • +
    • Question: Will you also support AMD Rocm and AMD ecosystem overall? +
        +
      • Yes, eventually. Right now there already are optimized installations for AMD Zen2 (Rome). +OpenMPI is included and is installed on top of UCX & libfabric, so should properly support AMD Rocm interconnect, but this is currently untested.
      -

      How can we find the 'paper' of coding?

      +
      • +
    • Comment: I like this idea as for us it is important people can use it in their laptops. +Personally I not much loosing time in setting up sw at my laptop but I see for the users it is important +to have an option to install/use it also in their lab. They like it more.
        -
      • github stars
        • -
      • https://joss.theoj.org/ -https://openresearchsoftware.metajnl.com/ -https://www.journals.elsevier.com/softwarex, software wrapped in paper for traditional metrics, good start but does not solve the problem
        • -
      • one citable paper for many years of developing the software? -> need to be judged on different scale
        • -
      • Standard research outputs are not the only thing that research is measured by anymore, need to 'jump on the train'
        • -
      • continuous necessity for novelty, all metrics problematic, potential metrics: how many people are using your product? -> if many people use it, it is valuable to maintain and update software, supporting a large community
        • -
      • usage metrics as a way of demonstrating impact -> hard to make funding bodies recognize that
        • -
      • in ok it is now pushed to be recognized, slowly building up now, took several years to build evidence base of usage
        • -
      • UK’s Research Excellence Framework: https://www.ref.ac.uk/
        • -
      • importance of being able to read documentation, you need to know where to look and how to do (not everyone can do that)
        • -
      • what about promoting the importance of releasing often? It is a measure of continuous effort, something that writing one-off papers doesn't do, and even small, bug-fixing patches are important IMO. The flip side is it could encourage busy-work, but I still think it's worth it.
        • -
      • all metrics can break -> wide variety of metrics for value is important
        • -
      • some people work in a field where software is not used much by other people, but may be very useful for your colleagues -> citation supports more novelty than quality
        • -
      • no tasks anymore that can be solved by one person. As a researcher, pair up with an RSE to solve a problem, paper together. win-win. can also help your career.
        • -
      • people are rewarded for bad code by keeping their job through being the only person who can actually read and work with their code, no incentive to make code better -> better long term management needed
        • -
      • today no one is indispensible, dont hire people who think they are
        • -
      • make sure multiple people can 'keep the server running', collaborate
        • -
      • -> culture change needed
        • -
      • need for basic education (version control etc) of students, code review
        • -
      • but people do not like to find out / being pointed put as having written bad code, creates high barrier, but one one need to get over
        • -
      • when sharing code, poeple will get used to it, as its part of development
        • -
      • Coderefinery as a good place to send new phds students to learn version control and how collaborative coding works -> no merge without someone else reading the code first
        • -
      • it's all research, you never know which part will become part of your codebase, turns into something big -> hard to go back later, so its important to start early with reviewing, version control etc
        • -
      • So maybe we should have some sense of "continual review" like "continuous integration"
        • -
      • writing test framework takes time but is worth it in the long run
        • -
      • do what you want in your own code,but you will need to 'act like a software engineer' when working with others
        • -
      • compared to how long it takes to make things work, test implementation does not take too much time
        • -
      • courses such as coderefinery are not appealing to some people who think of themselves knowing git* enough for current use, 'bubble workers', so they never learn how it could be also with branches in own code
        • -
      • 'onboarding package' as in how we do things here is lacking in academia, not everywhere though: example: https://scicomp.aalto.fi/aalto/welcomeresearchers/
        • +
      • Yes, this could allow people to literally write a job script that just works on the HPC cluster. Same modules, same software. +(and no need to build containers, or copy them over, etc.)
        • +
      +
      • +
    • Question: This builds on existing projects so it has some content from the begining. +
        +
      • Thanks to EasyBuild we can easily provide 1000s of installations. +Right now we limit what we provide, so we can focus on solving the problems we're hitting first.
        • +
      +
      • +
    • Question: Why European in the name? +
        +
      • Because it started with European sites. +We're already thinking about changing the first E to "Easy" :) +"EESSI is the Easy Environment..."
        • +
      +
      • +
    • Question: Question: what are the possibilities to add “own dirty module”, is it like same as e.g. with EasyBuild itself? +
        +
      • You can easily install additional software on top, for example in your home directory on in /tmp, just like you can with any other software stack built with EasyBuild).
        • +
      +
      • +
    • Question: Sensitivity of central Stratum-0 component, in terms of resilience? +
        +
      • The CernVM-FS design is very robust. If the Stratum-0 dies, the only impact is that you can't add new software to the repositories. +As long as one Stratum-1 server is still alive, the software remains available (all Stratum-1 servers have a full copy of the provided software). +So it comes down to having enough Stratum-1 servers, spread across the world, in different sites and cloud providers.
        • +
      • W.r.t adding software: we plan to fully automate the workflow of adding software to the EESSI repository, such that adding software comes down to opening a pull request on GitHub. When the PR is approved by a reviewer, the software gets built automatically on all supported CPU architectures, and added to Stratum-0, fully automatically. Ideally we also have (small) test cases to verify that the installations are functional before deploying them.
        • +
      +
      • +
    • Question: You mentioned that CernVM-FS only relies on HTTP connections. Shouldn't that be HTTPS for security reasons? +
        +
      • No, switching to HTTPS has no added value in terms of security, we've discussed that with the CernVM-FS developers. +CernVM-FS has built in security checks between server and clients, so HTTPS doesn't provide any additional security (I think, should be checked in CernVM-FS documentation).
        • +
      +
      • +
    • How would this work for large jobs across multiple nodes, can a lot of network traffic to pull in the software be avoided? +
        +
      • Yes, you can set up a shared CernVM-FS cache on a shared filesystem. +If there's no internet access on the cluster workernodes, you can use a squid proxy in the cluster network (on a login node for example). +This setup has been tested with the EESSI pilot stack at the Jülich Supercomputing Centre, worked really well!
        • +
      +
      • +
    • Comment: Detection of CPU architecture is a very nice feature. This is a big issue with containers where generic binaries are often, which can have a big impact on performance. +
        +
      • Yes, indeed! Containers are also very rigid: what if you want to add additional software? +The EESSI environment is way more dynamic, easy to add software on top of it (without paying for it in terms of performance), etc.
        • +
      +
      • +
    • Comment: This would also work really well in heterogenous environments with a mix of old/new CPUs, thanks to the auto-detection mechanism. +
        +
      • Yes, very correct, this is an interesting use case!
        • +
      +
    @@ -3103,44 +3103,6 @@

    Questions and comments

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  • How to explain containers are friends of scientists? Are they? @@ -3169,42 +3131,8 @@

    Questions and comments

    - - - - - - - - - - - - - - - - - - - -
  • - TBA - 'unconference' sessions - - (TBA) - - -
    -

    An "unconference" has events scheduled based on interest of -participants, not decided by organizers. We are leaving this time -open for ad-hoc events proposed by participants. If there is not much -interest, we will move the conclusion forward.

    - -
    - -
    • @@ -3275,6 +3203,86 @@

    Questions and comments

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • + TBA - 'unconference' sessions + + (TBA) + + +
    +

    An "unconference" has events scheduled based on interest of +participants, not decided by organizers. We are leaving this time +open for ad-hoc events proposed by participants. If there is not much +interest, we will move the conclusion forward.

    + +
    + +
    • + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • Problematic demographics within research computing and ways to rectify the problem @@ -3319,14 +3327,6 @@

    Questions and comments

    - - - - - - - - @@ -3368,6 +3368,40 @@

    Questions and comments

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -3469,40 +3503,6 @@

    Notes

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/previews/PR445/events/2022-online-unconference/index.html b/previews/PR445/events/2022-online-unconference/index.html index 43c4da31..e32cf0ff 100644 --- a/previews/PR445/events/2022-online-unconference/index.html +++ b/previews/PR445/events/2022-online-unconference/index.html @@ -213,26 +213,11 @@

    Tuesday, October 18th

    - - - Introduction - (Luca Ferranti) - - -
    -
      -
    • Welcome and introduction to the unconference format: HackMD, proposing sessions, scheduling (15min)
      • -
    • Introduction to Nordic-RSE (15min)
      • -
    - -
    - - @@ -256,6 +241,19 @@

    Tuesday, October 18th

    + + Introduction + (Luca Ferranti) + + +
    +
      +
    • Welcome and introduction to the unconference format: HackMD, proposing sessions, scheduling (15min)
      • +
    • Introduction to Nordic-RSE (15min)
      • +
    + +
    + @@ -263,6 +261,8 @@

    Tuesday, October 18th

    + + @@ -278,6 +278,8 @@

    Tuesday, October 18th

    + + @@ -323,8 +325,6 @@

    Tuesday, October 18th

    - - @@ -349,8 +349,6 @@

    Tuesday, October 18th

    Keynote: - - @@ -358,6 +356,16 @@

    Tuesday, October 18th

    + Research software development in an open science landscape: on reform, co-creation and opportunities for professional establishment + (Sanna Isabel Ulfsparre) + + +
    +

    As Europe transitions into open science and FAIR research data management, new infrastructures and possibilities for professional development emerge. Sanna Isabel Ulfsparre, librarian at Umeå university library, will talk about policy, trends and tendencies relevant to research software developers who want to further specialise in RSE.

    + +
    + + @@ -369,16 +377,6 @@

    Tuesday, October 18th

    - - Research software development in an open science landscape: on reform, co-creation and opportunities for professional establishment - (Sanna Isabel Ulfsparre) - - -
    -

    As Europe transitions into open science and FAIR research data management, new infrastructures and possibilities for professional development emerge. Sanna Isabel Ulfsparre, librarian at Umeå university library, will talk about policy, trends and tendencies relevant to research software developers who want to further specialise in RSE.

    - -
    - @@ -396,6 +394,8 @@

    Tuesday, October 18th

    + + @@ -423,17 +423,15 @@

    Tuesday, October 18th

    @@ -528,6 +528,8 @@

    Wednesday, October 19th

    @@ -625,8 +625,6 @@

    Wednesday, October 19th

    diff --git a/previews/PR445/index.html b/previews/PR445/index.html index 05917b46..db06563d 100644 --- a/previews/PR445/index.html +++ b/previews/PR445/index.html @@ -158,7 +158,7 @@
    -

    Nordic-RSE in a nutshell

    +

    Nordic-RSE in a shell