Proposal for the introduction of extensions

.github/ISSUE_TEMPLATE/petab-extensions.md

@@ -0,0 +1,26 @@
+---
+name: PEtab Extension
+about: Suggest a new extension for PEtab core
+title: ''
+labels: file format
+assignees: ''
+
+---
+
+**Name of the Extension**
+Please make sure that the extension is named matches the regular expressin `^[a-zA-Z_]\w*$`.
+
+**Which problem would you like to address?**
+A clear and concise description of which use case you want to address and, if applicable, why the current specifications do not fulfill your requirements.
+
+**Describe the solution you would like**
+A clear and concise description of the changes you want to propose. Please describe any additional fields / files you would want to add, including allowed inputs and implications.
+
+**Describe why this should not be implemented by changes to PEtab core**
+A clear and concise description in what way the proposed changes introduce features that are orthogonal to the PEtab core specification.
+
+**List the extension library that implements validation checks**
+A link to the website or github repository that accompanies the proposed extension.
+
+**List two toolboxes that support the proposed standard **
+A link to the website or github repository that contains the software that implements support for the standard.

CHANGELOG.md

@@ -14,6 +14,13 @@ available at https://github.com/PEtab-dev/libpetab-python/.
 * Update tutorial.rst (#512)
 * Update how-to-cite (Closes #432) (#509)
 
+## 0.2 series
+
+### 0.2.0
+
+* Specify how PEtab functionality can be expanded through extensions.
+* YAML files are now required for the specification of PEtab problems
+
 ## 0.1 series
 
 ### 0.1.14

README.md

@@ -125,6 +125,8 @@ will have to:
 
 1. Create a parameter table.
 
+1. Create a yaml file that list the model and all of the tables above.
+
 If you are using Python, some handy functions of the
 [PEtab library](https://github.com/PEtab-dev/libpetab-python/) can help
 you with that. This includes also a PEtab validator called `petablint` which

doc/development.rst

@@ -192,6 +192,59 @@ Upon a new release, the PEtab editors ensure that
 * the new version of the specifications is deposited at Zenodo
 * the new release is announced on the PEtab mailing list
 
+PEtab Extensions
+----------------
+
+An elaborate, monolithic format would make it difficult to understand and
+implement support for PEtab, leading to a steep learning curve and discouraging
+support in new toolboxes. To address this issue the PEtab format is modular and
+also permits modifications through extensions that complement the core standard.
+This modular specification evens the learning curve and provides toolbox
+developers with more guidance on which features to implement to maximize
+support for real world applications. Moreover, such modular extensions
+facilitate and promote the use of specialized tools for specific, non-parameter
+estimation tasks such as visualization.
+
+Requirements for new extensions:
+
+* Specifications in PEtab extensions take precedence over PEtab core, i.e., they
+can ease or refine format restrictions imposed by PEtab core.
+* PEtab extensions should extend PEtab core with new orthogonal features or
+tasks, i.e., they should not make trivial changes to PEtab core.
+* PEtab extensions must be named according to ^[a-zA-Z_]\w*$
+* PEtab extensions must be versioned using semantic versioning.
+* PEtab extensions required for interpretation of a problem specification must
+be specified in the in the PEtab-YAML files
+* There are at least two tools that support the proposed extension
+* The authors provide a library that implements validation checks for the
+proposed format.
+
+Developers are free to develop any PEtab extension. To become an official
+PEtab extension, it needs to go through the following process.
+
+1. The developers write a proposal describing the motivation and specification
+of the extension, following the respective issue template provided in this
+repository.
+1. The proposal submitted as an issue in this repository.
+1. The technical specification and documentation of the extension is submitted
+as pull request in this repository that references the respective issue.
+1. The developers submit a pull request that adds test cases to validate support
+for the extension https://github.com/PEtab-dev/petab_test_suite that references
+the issue in this repository.
+
+The PEtab editors jointly decide whether an extension meets the requirements
+described here. In case of a positive evaluation, they announce a poll for the
+acceptance as official extension to the PEtab forum. All members of the PEtab
+community are eligible to vote. If at least 50% of the votes are in favor,
+the extension is accepted and the respective pull requests with specifications,
+documentation and test cases are merged.
+
+It is encouraged that extensions are informally discussed with the community
+before initiating the process of becoming an official extension. Such
+discussions can be conducted through the communication channels mentioned
+above.
+
+
 Versioning of the PEtab format
 ------------------------------
 

doc/documentation_data_format.rst

@@ -58,6 +58,9 @@ and
 - A parameter file specifying optimization parameters and related information
   [TSV]
 
+- A grouping file that list all of the files and provides additional information
+  including employed extensions [YAML]
+
 - (optional) A simulation file, which has the same format as the measurement
   file, but contains model simulations [TSV]
 
@@ -697,7 +700,7 @@ To link the SBML model, measurement table, condition table, etc. in an
 unambiguous way, we use a `YAML <https://yaml.org/>`_ file.
 
 This file also allows specifying a PEtab version (as the format is not unlikely
-to change in the future).
+to change in the future) and employed PEtab extensions.
 
 Furthermore, this can be used to describe parameter estimation problems
 comprising multiple models (more details below).

doc/tutorial.rst

@@ -19,9 +19,9 @@ details of the model are not relevant for the purpose of this tutorial.
 For more details, we refer to the original publication.
 
 A PEtab problem consists of 1) an SBML model of a biological system, 2)
-condition, observable and measurement definitions, and 3) the
-specification of the parameters. We will show how to generate the
-respective files in the following.
+condition, observable and measurement definitions, 3) the
+specification of the parameters and 4) a configuration file that lists all of
+these files. We will show how to generate the respective files in the following.
 
 1. The model
 ++++++++++++
@@ -324,11 +324,10 @@ https://petab.readthedocs.io/en/latest/documentation_data_format.html#visualizat
 5. YAML file
 ++++++++++++
 
-To group the previously mentioned PEtab files, a YAML file can be used,
-defining which files constitute a PEtab problem. While being optional,
-this makes it easier to import a PEtab problem into tools, and allows
-reusing files for different PEtab problems. This file has the following
-format (``Boehm_JProteomeRes2014.yaml``):
+To group the previously mentioned PEtab files, a YAML file must be used,
+defining which files constitute a PEtab problem. This makes it easier to import
+a PEtab problem into tools, and allows reusing files for different PEtab
+problems. This file has the following format (``Boehm_JProteomeRes2014.yaml``):
 
 .. code-block:: yaml