Development: Principles of engineering design for sustainable software

[kamikitty]

https://docs.google.com/presentation/d/1T1Hy1ASMQDARfkZRk0QywhXq0W69OLfmdT6L-uAW_ro/edit#slide=id.g6f37ecea3_0_0

Topic 6: Principles of engineering design for sustainable software
What do you think our topic is about? Get everyone on the same page.

Ian - categories of tools and design principles of communication to create something

• confused about scoping
• Reproducability: tools that would make it easier for people to use (put it in container)

Patrick - Tools & best practices & user wants analysis

Alex - a list of rules and principles I can apply to make next software project sustainable and so that others can continue developing and serve user community, focussing on development side

Matthias - our project has a grown a lot, new developers are changing the ways the project is working, I want
to focus on how to keep the community engaged when the project grows to fast (facilitate for people to move in between different funding models) and how to do knowledge management

Ryan - best practices and principles that help keep things going, so that it's modular, knowledge management etc. so that it doesn't bit-rot in 5 years

• standard software engineering is not a thing where I come from

Birgit - high-level KKMonSD principles and relate / break them down into best practices for software designers

Colin - core set of fundamental principles and relate to set of practices that relate to this set.

• ability to take in other dimensions
• estimation of what impacts software systems have and what cost is associated

Cast aside: "What is the difference between industry and science in developing software?"

• What kind of industry are we talking about?
• Funding schemes and incentives.
• Software developers versus scientists.
• Incentive: Publications versus software sales (quality)
• Could be used as a different lense on the rest.

Colin: There is a huge gap between SWEBOK and domain scientists.

OUTLINE: Write down and explore principles of software design, user interface design, and sustainability and application to scientific software
Here are the principles we know about good software design (brainstorm):
-- Modularity (low coupling)
-- High cohesion (individual modules make sense)
-- Compatibility
-- Knowledge Management
-- Abstraction (and finding the right level for that for applicability versus reuse; example: traveling from Earth to Moon, level of traveling from X to Y is too abstract, but traveling from Earth to B might be good)
-- Usability

The actual principles from SWEBOK
Software design principles
Abstraction,
Coupling and cohesion,
Decomposition and modularization,
Encapsulation and information hiding,
Separation of interface and implementation,
Sufficiency completeness & primitiveness,
Separation of concerns
User interface design principles
Learnability
User familiarity
Consistency
Minimal surprise
Recoverability
User guidance
User diversity

Best practices derived from those:
-- Pair programming (knowledge management)
-- Shared responsibility for code modules
-- Automated testing (rationale: regression testing, for compatibility)
-- Code reviews (rationale: quality assurance, optimization, knowledge management)
-- Develop user scenarios (usability)

Sustainability (design) principles:
Sustainability is systemic. �
... is multidimensional. �
... is interdisciplinary. �
... transcends the system’s purpose. �
... applies to both a system and its wider contexts. �
... requires action on multiple levels. �
... requires multiple timescales. �
Changing design to take into account long-term effects doesn’t automatically imply sacrifices. �
System visibility is a precondition for and enabler of sustainability design.

Tates (Kevin Tate. 2005. Sustainable Software Development: An Agile Perspective. Addison-Wesley Professional.) Principles of Sustainable Software Development

Continual Refinement of Product and Project Practices
A Working Product at All Times
A Continual Emphasis on Design
Value Defect Prevention over Defect Detection

Intel Collaborative Research Institute (http://www.cities.io)

Design for humans as part of the solution
Build for the long term – design robustly for the imperfections of the real world
Deliver the triple bottom line – economic, social and environmental value

Best practices derived from those:
-- We have to establish common terminology across disciplines to be able to make software
-- etc.
What can we take from both of those sets for scientific software?
What do the principles MEAN for scientific software? Using illustrating examples.
(What do they mean for non-software engineers?)
Usage scenarios -> Challenge: user community often disconnected outside the project of the original funding
Interdisciplinarity: different scientific disciplines have to talk to each other and they have to talk to software engineers

To incorporate the other post-its on the poster: How do the principles relate to usability and reusability and the other characteristics? What do the 5 dimensions of sustainability mean for scientific software?
Those are some of the sub-questions of what we’ll be dealing with.

Resources:

Best Practices for Scientific Computing:
http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001745
Day 2: Exploring and applying the principles

Team: Birgit, Colin, Ian, Pat, Alex, Matthias, Jeff
Barriers: Unclear where the difference to “standard software engineering” is.
Today/Next step: Agree on scenario explore principles of software design, user interface design, and sustainability and application to scientific software.
Example Scenario:
An example from the scientific domain, a system that either has been developed, is being developed, or shall be developed that shows some of the domain-typical characteristics and challenges, e.g. for scalability, usability, complexity

Application Domain: Chemistry. System that calculates the characteristics and capabilities of molecules.
Computing-intensive, heavy workstation required, parallelism built in
One of the systems chosen for the new Titan (Oakridge)
Engineering effort: software engineers developed the behind the scenes communication layer, allows to develop parallel code easily
www.nwchem-sw.org
Online modeling platform to manipulate a complex Malaria model with a UI where a biologist could enter the parameters - open source (MPLv2)
Automated biology system, e.g. microscopy and related imaging, with a user interface that limits user input to what is needed (so they can’t “mess things up”)
Tools:
workflow system that allows to plug components and wires methods together for the gravitational wave community, plug into an FFT and a power spectrum http://www.trianacode.org/ - open source
workflow system for 1 person in EU project for wiring python components together and coordinates multiple workflows - http://www.nrl.navy.mil/itd/ncs/products/wzerorpc - open source
Epidemiology / child obesity: built on a series of software services that allows the epidemiologist to analyze data across range of features and use different graphing techniques
Civil aerospace engineering equipment health monitoring: sucks in data from an asset and goes through series of different processes and compares whether there is delta against normal trends, also requires human operator to double-check those fluctuations
Integrated data view for earth science data, 1 mio LOC, 15 years of support, strong community
GSAC GPS seamless archive center: middleware for search and repository systems on legacy software - https://www.unavco.org/software/data-management/gsac/gsac.html
Ramada: middleware for atmospheric science climate world. One fundamental question behind those middlewares is how to add functionality if you want to make the framework more applicable and scalable, because user community is very diverse
DL_POLY Classic
http://www.ccp5.ac.uk/DL_POLY_CLASSIC/
Open source, general-purpose, classical molecular dynamics simulation software
Developed at Daresbury Laboratory/STFC
Serial or parallel application.
Now driven by its community of end users

Single scientist/developer
GPU-enabled magnetohydrodynamcs (MHD) code
Soft matter and biomolecules employing Fluctuating Finite Element Analysis (FFEA).

Example 1: Quantum chemistry program
Requirements Engineering
Eliciting, analyzing, specifying, and verifying and validating stakeholder goals, needs, and constraints.
RE Principles:
Requirements sources: Goals, Domain knowledge, Stakeholders, Business rules, Operational Environment, Organizational environment

Techniques / Best practices for Elicitation: Interviews, Scenarios, Prototypes, Facilitated meetings, Observation, User stories.

Application to example system:
RE resources:
Goals: Robert Harrison main developer wanted to develop modern version of “Columbus”, which is a quantum chemistry program that could conceivably compute anything a chemist might want to know.
Domain knowledge: Quantum chemistry
Stakeholders:
Champion: Used to be Robert, now head of Berkeley’s scientific computing, and some more ex-developers, philanthropic people in the funding landscape
Users
Steering committee (representing the user community)
Lab structure
Developers
Lab
Active user community that contributes (3-4 projects at a time that end up being part of a module)
Hardware Vendors that contribute code (so that it runs on their latest hardware)
Business rules: Core research funding and individual project funding
Operational environment: Stronger machines than before, new system should be scalable and make use of those stronger machines, current status 2 Mio LOC, from starting with relying only on make-files we went to GUI for input and now that got discarded again (bc of proprietary graphics library).
Organizational environment: Funding for the original version, yearly funding about 5 full time employees and (60% core funding, 40% projects)

Best practices for Elicitation - which ones of those are applied:
Interviews: collaboration with user community, go to conferences and talk to other users “I really wanna do this, can we make that happen?”
Scenarios: pretty much set because of system purpose. We develop user stories, then we break them down according to the task structure that exists in the system and then determine how and where we have to adapt that to fulfill the new requirements.
Prototypes and testing: We make prototypes and have a number of input decks we feed in where we know what should be the answer (regression tests).
Facilitated meetings: more informal via the helpdesk and feature and change request in a database (actually an email list)
Observation: in training sessions. As a postdoc I taught some companies how to use the software, and then I could take the experience from that back to develop.
User stories: see above, used instead of scenarios.

Application of Sustainability principles:
Sustainability is systemic: apply that �
... is multidimensional
Economic sustainability: system has lots of user support because it’s free, the main developer core has done a really good job in securing DoE funding. 60% core, 40% projects should remain the structure in the future.
Technical: code reuse, system organized in modules, contributors don’t have to know a lot about programming, e.g. in terms of multithreading etc.
Environmental: energy it uses for actually computing, one of the main uses is to cleaning up actinide (nuclear) waste - some of the dangerous elements are soluble, so have to be bound so they won’t go into rivers (2nd/3rd order impacts)
15 metallic chemical elements with atomic numbers from 89 to 103: Actin­ium (89), Thor­ium (90), Protac­tinium (91), Ura­nium (92), Neptu­nium (93), Pluto­nium (94), Ameri­cium (95), Curium (96), Berkel­ium (97), Califor­nium (98), Einstei­nium (99), Fer­mium (100), Mende­levium (101), Nobel­ium (102), Lawren­cium (103).
Individual sustainability: software system for research, has educational component
Social sustainability: strong user community that contributes to development
... is interdisciplinary: quantum chemists, software engineers (we had some doing the distributed low-level communication layer), user interface design, nuclear waste, environmental clean-ups of any kind, clean energy, hydrogen fuels, (some) biology.
... transcends the system’s purpose: If you can’t meet your goal ever, that’s hard to say, which is the case for this system.
... applies to both a system and its wider contexts: The community is the key, it’s the lifeblood and environment of the software. Without user and development community, there is no software system. The software system is then the catalyst for the community to be, to share their research results, to cross-fertilize. In reality, we want to sustain the community, because that’s what will sustain the system.
... requires action on multiple levels: grassroot level is the user and development community, another level is funding, then you also have to go out and “make disciples�” (at big conferences etc.) to grow the community - the equivalent of marketing in scientific software that is not charged for.
... requires multiple timescales: short-term timescales like MSc theses or PhD thesis and “getting the thing to run and do what you want”, then those might or might not be embedded in larger projects, and then even larger time scales in follow-up projects, some refactoring in between and further along the way, and long-term development.
Changing design to take into account long-term effects doesn’t automatically imply sacrifices. �There is a good bit of modularity but also modularity with a purpose for being reusable.
System visibility is a precondition for and enabler of sustainability design. We provide users access to the current development status, to feature requests and issues (so they can participate in development), and partially to where and what for the system is being used. That last part could be extended and made better accessible for growing the user community and better funding acquisition.

https://software-carpentry.org/

Jonas Gamalielsson and Björn Lundell. (2014). Sustainability of Open Source software communities beyond a fork: How and why has the LibreOffice project evolved? Journal of Systems and Software, 89, pp: 128–145

Design (including UI):

Implementation:

Testing:

Deployment:

Maintenance:

Evolution:

Joel Spolsky (“The Joel test”)

Example 2: Online modeling platform to manipulate a complex Malaria model with a UI where a biologist could enter the parameters - open source (MPLv2)
Background: It was a funded project with 5 FTEs. Original idea came from biologists who wanted to do field research (very time consuming), so they were looking at other fields that used models to save time and do more targeted field research. They also pitched that to the funding agencies in terms of the opportunities for fighting malaria etc. Very broad scope, every single problem that could be tackled. Small domain expert user community (<5) combined with a large undergraduate population. Requirement for a user interface to support ease of use. Grant funded for 1 year: time pressures to delivery or lose funding.
Project stopped after a year, so during the 2nd year it was kind of a zombie we were trying to maintain even though we don’t have funding anymore.
Usage Scenario - Example user story: I go to your system and pick a place in Africa and get all the information you have in there. And then I can add my own information about species or whatever else I want to the database.

[Academic] Environmental factors play a significant role in affecting sustainability.

Prototyping approach based on use cases.

Application of sustainability principles

Sustainability is systemic. �-
... is multidimensional. �-
... is interdisciplinary. -
... transcends the system’s purpose. -
... applies to both a system and its wider contexts. -
... requires action on multiple levels. -
... requires multiple timescales. -
Changing design to take into account long-term effects doesn’t automatically imply sacrifices. -
System visibility is a precondition for and enabler of sustainability design. -

Funding and active user communities are the two crucial drivers for (free) scientific software systems.
Make the leverage points stand out for :
Enabling the user community to contribute
Describe in a tutorial how to do that
Make contributions possible for nightly / weekend contributors

We identified 3 funding models for engineering scientific software systems:
Long-term funded system with actively contributing user community
Short-term funded project
Unfunded System that just gets worked on by student after student (or postdoc)
Long term community maintained purely open source, e.g. Apache Company baked (hdf group)

Current status: We are working on this on Google Docs and will post significant updates here.