From e83e51dcd89256403bb787c3d9a46e4ee8d04a9e Mon Sep 17 00:00:00 2001 From: Daniel Himmelstein Date: Sat, 6 Oct 2018 10:01:27 -0400 Subject: [PATCH 1/6] CSL: display="block" every other group Merges https://github.com/greenelab/manubot-rootstock/pull/134 Previous blocks were causing a blank line after the author line before the journal-date-URL line in markdown output. Placing blocks at every other line seems to be the solution required for pandoc-citeproc / pandoc to properly place the newlines. --- build/assets/style.csl | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/build/assets/style.csl b/build/assets/style.csl index 756c758..07ba262 100644 --- a/build/assets/style.csl +++ b/build/assets/style.csl @@ -51,7 +51,7 @@ - + @@ -60,7 +60,7 @@ - + From 830cdc088dee2a61452bd56d990e0277daa0a9af Mon Sep 17 00:00:00 2001 From: Daniel Himmelstein Date: Mon, 8 Oct 2018 15:12:33 -0400 Subject: [PATCH 2/6] Update USAGE.md & environment on 2018-10-08 Merges https://github.com/greenelab/manubot-rootstock/pull/135 Closes https://github.com/greenelab/manubot/issues/59 * USAGE.md: suggested citation & acknowledgments Mostly copied from https://github.com/greenelab/manubot/blob/9d97ec347882bcd85ab6aee7a3b4734105ebfc15/README.md * Update environment on 2018-10-08 Updates to https://github.com/greenelab/manubot@9d97ec347882bcd85ab6aee7a3b4734105ebfc15 which is slightly past the manubot v0.2.0 release. jsonschema & jsonref should have been added to the environment previousely since they are required by manubot for JSON schema validation, but were not added. Do not upgrade to Python 3.7 due to collections DeprecationWarnings in several packages. Fixed warning in https://github.com/gazpachoking/jsonref/pull/26, but several remain. * Travis CI: update miniconda to 4.5.11 --- .travis.yml | 2 +- USAGE.md | 17 +++++++++++++++++ build/environment.yml | 10 ++++++---- 3 files changed, 24 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index 60cc09a..29420f4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -5,7 +5,7 @@ branches: only: - master before_install: - - wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh + - wget https://repo.continuum.io/miniconda/Miniconda3-4.5.11-Linux-x86_64.sh --output-document miniconda.sh - bash miniconda.sh -b -p $HOME/miniconda - source $HOME/miniconda/etc/profile.d/conda.sh diff --git a/USAGE.md b/USAGE.md index df02493..ddcdc07 100644 --- a/USAGE.md +++ b/USAGE.md @@ -167,3 +167,20 @@ For additional examples, check out existing manuscripts that use the Manubot (so + The Manubot 2018 Development Proposal ([source](https://github.com/greenelab/manufund-2018), [manuscript](https://greenelab.github.io/manufund-2018/)) If you are using the Manubot, feel free to submit a pull request to add your manuscript to the list above. + +## Citing Manubot + +To cite the Manubot project or for more information on its design and history, see `@url:https://greenelab.github.io/meta-review/`: + +> **Open collaborative writing with Manubot**
+Daniel S. Himmelstein, David R. Slochower, Venkat S. Malladi, Casey S. +Greene, Anthony Gitter
+_Manubot Preprint_ (2018) + +## Acknowledgments + +We would like to thank the contributors and funders whose support makes the Manubot project possible. +Specifically, Manubot development has been financially supported by: + +- the **Alfred P. Sloan Foundation** in [Grant G-2018-11163](https://sloan.org/grant-detail/8501) to [**@dhimmel**](https://github.com/dhimmel). +- the **Gordon & Betty Moore Foundation** ([**@DDD-Moore**](https://github.com/DDD-Moore)) in [Grant GBMF4552](https://www.moore.org/grant-detail?grantId=GBMF4552) to [**@cgreene**](https://github.com/cgreene). diff --git a/build/environment.yml b/build/environment.yml index a77ba34..cc38436 100644 --- a/build/environment.yml +++ b/build/environment.yml @@ -6,15 +6,17 @@ dependencies: - conda-forge::cffi=1.11.5 - conda-forge::ghp-import=0.5.5 - conda-forge::jinja2=2.10 + - conda-forge::jsonschema=2.6.0 - conda-forge::pandas=0.23.4 - - conda-forge::pandoc=2.2.2 - - conda-forge::python=3.6.6 - - conda-forge::pyyaml=3.12 + - conda-forge::pandoc=2.3.1 + - conda-forge::python=3.6.5 + - conda-forge::pyyaml=3.13 - conda-forge::requests=2.19.1 - conda-forge::watchdog=0.8.3 - pip: - errorhandler==2.0.1 - - git+https://github.com/greenelab/manubot@66a6efc6f4b84153a813aa423ec00725ed1417c5 + - git+https://github.com/greenelab/manubot@9d97ec347882bcd85ab6aee7a3b4734105ebfc15 + - jsonref==0.2 - opentimestamps-client==0.6.0 - opentimestamps==0.4.0 - pandoc-eqnos==1.3.0 From 6bb804c018d40f0ef4e7f79fcfa5600827f112ca Mon Sep 17 00:00:00 2001 From: Anthony Gitter Date: Tue, 9 Oct 2018 11:20:08 -0500 Subject: [PATCH 3/6] Case sensitive environment variables in setup instructions Merges https://github.com/greenelab/manubot-rootstock/pull/138 Closes https://github.com/greenelab/manubot-rootstock/issues/137 * Case sensitive environment variables * Update phrasing borrowing dhimmel ideas * Change account to username --- SETUP.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/SETUP.md b/SETUP.md index 554cabd..22a4f06 100644 --- a/SETUP.md +++ b/SETUP.md @@ -9,10 +9,12 @@ Setup is supported on Linux and macOS, but [**not on Windows**](https://github.c First, you must configure two environment variables (`OWNER` and `REPO`). These variables specify the GitHub repository for the manuscript (i.e. `https://github.com/OWNER/REPO`). +Make sure that the case of `OWNER` matches how your username is displayed on GitHub. +In general, assume that all commands in this setup are case-sensitive. **Edit the following commands with your manuscript's information:** ```sh -# GitHub account (change from greenelab) +# GitHub username (change from greenelab) OWNER=greenelab # Repository name (change from manubot-rootstock) REPO=manubot-rootstock From dcfe402544f90e0047a247cdd8d01a2bca08df54 Mon Sep 17 00:00:00 2001 From: Daniel Himmelstein Date: Tue, 23 Oct 2018 15:23:20 -0400 Subject: [PATCH 4/6] Bugfix environment updates on 2018-10-23 Merges https://github.com/greenelab/manubot-rootstock/pull/140 Closes https://github.com/greenelab/manubot-rootstock/issues/136 Updates Manubot to fix empty date-parts issue Fix opentimestamps incompatible pinned versions --- build/environment.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/build/environment.yml b/build/environment.yml index cc38436..73171db 100644 --- a/build/environment.yml +++ b/build/environment.yml @@ -15,16 +15,16 @@ dependencies: - conda-forge::watchdog=0.8.3 - pip: - errorhandler==2.0.1 - - git+https://github.com/greenelab/manubot@9d97ec347882bcd85ab6aee7a3b4734105ebfc15 + - git+https://github.com/greenelab/manubot@4e6a0f6d28220264c0d7892c732cb68c3e97c07a - jsonref==0.2 - opentimestamps-client==0.6.0 - - opentimestamps==0.4.0 + - opentimestamps==0.3.0 - pandoc-eqnos==1.3.0 - pandoc-fignos==1.3.0 - pandoc-tablenos==1.3.0 - pandoc-xnos==1.1.1 - pybase62==0.4.0 - pysha3==1.0.2 - - python-bitcoinlib==0.10.1 + - python-bitcoinlib==0.9.0 - requests-cache==0.4.13 - weasyprint==0.42.3 From 895924e6a8c693a0ee110ec410b153b7ceb09bc2 Mon Sep 17 00:00:00 2001 From: Daniel Himmelstein Date: Tue, 30 Oct 2018 15:28:24 -0400 Subject: [PATCH 5/6] Enable raw citations & small USAGE updates Merges https://github.com/greenelab/manubot-rootstock/pull/141 * Update USAGE with current URLs & recs Use HTTPS URLs where possible Switch SVG embedding solution from rawgit.com, which is shutting down, to the native GitHub solution. https://twitter.com/dhimmel/status/1049361799244664834 * Enable raw citations Update USAGE.md with raw citation example. Also update citations in delete-me.md. --- USAGE.md | 44 +++++++++++++++++++++++++++-------------- build/environment.yml | 2 +- content/02.delete-me.md | 4 ++-- 3 files changed, 32 insertions(+), 18 deletions(-) diff --git a/USAGE.md b/USAGE.md index ddcdc07..5bd3507 100644 --- a/USAGE.md +++ b/USAGE.md @@ -1,18 +1,18 @@ # Manubot usage guidelines -This repository uses the [Manubot](https://github.com/greenelab/manubot) to automatically produce a manuscript from its source. +This repository uses [Manubot](https://github.com/greenelab/manubot-rootstock) to automatically produce a manuscript from the source in the [`content`](content) directory. ## Manubot markdown Manuscript text should be written in markdown files, which should be located in [`content`](content) with a digit prefix for ordering (e.g. `01.`, `02.`, etc.) and a `.md` extension. -For basic formatting, check out the [CommonMark Help](http://commonmark.org/help/) page for an introduction to the formatting options provided by standard markdown. -In addition, manubot supports an extended version of markdown, tailored for scholarly writing, which includes [Pandoc's Markdown](http://pandoc.org/MANUAL.html#pandocs-markdown) and the extensions discussed below. +For basic formatting, check out the [CommonMark Help](https://commonmark.org/help/) page for an introduction to the formatting options provided by standard markdown. +In addition, Manubot supports an extended version of markdown, tailored for scholarly writing, which includes [Pandoc's Markdown](https://pandoc.org/MANUAL.html#pandocs-markdown) and the extensions discussed below. Within a paragraph in markdown, single newlines are interpreted as whitespace (same as a space). A paragraph's source does not need to contain newlines. However, "one paragraph per line" makes the git diff less precise, leading to less granular review commenting, and makes conflicts more likely. -Therefore, we recommend using [semantic linefeeds](http://rhodesmill.org/brandon/2012/one-sentence-per-line/ "Semantic Linefeeds. Brandon Rhodes. 2012") — newlines between sentences. +Therefore, we recommend using [semantic linefeeds](https://rhodesmill.org/brandon/2012/one-sentence-per-line/ "Semantic Linefeeds. Brandon Rhodes. 2012") — newlines between sentences. We have found that "one sentence per line" is preferable to "word wrap" or "one paragraph per line". ### Tables @@ -30,7 +30,7 @@ Table: Caption for this example table. {#tbl:example-id} Support for table numbering and citation is provided by [`pandoc-tablenos`](https://github.com/tomduck/pandoc-tablenos). Above, `{#tbl:example-id}` sets the table ID, which creates an HTML anchor and allows citing the table like `@tbl:example-id`. -For easy creation of markdown tables, check out the [Tables Generator](http://www.tablesgenerator.com/markdown_tables) webapp. +For easy creation of markdown tables, check out the [Tables Generator](https://www.tablesgenerator.com/markdown_tables) webapp. ### Figures @@ -45,11 +45,11 @@ This figure can be cited in the text using `@fig:example-id`. In context, a figure citation may look like: `Figure {@fig:example-id}B shows …`. For images created by the manuscript authors that are hosted elsewhere on GitHub, we recommend using a [versioned](https://help.github.com/articles/getting-permanent-links-to-files/) GitHub URL to embed figures, thereby preserving exact image provenance. -When embedding SVG images hosted on GitHub, passing the URL through [RawGit](https://rawgit.com/) is necessary. -An example of a URL that has been passed through RawGit is: +When embedding SVG images hosted on GitHub, it's necessary to append `?sanitize=true` to the `raw.githubusercontent.com` URL. +For example: ``` -https://cdn.rawgit.com/greenelab/scihub/572d6947cb958e797d1a07fdb273157ad9154273/figure/citescore.svg +https://raw.githubusercontent.com/greenelab/scihub/572d6947cb958e797d1a07fdb273157ad9154273/figure/citescore.svg?sanitize=true ``` Figures placed in the [`content/images`](content/images) directory can be embedded using their relative path. @@ -59,19 +59,19 @@ For example, we embed an [ORCID](https://orcid.org/) icon inline using: ![ORCID icon](images/orcid.svg){height="13px"} ``` -The bracketed text following the image declaration is interpreted by Pandoc's [`link_attributes`](http://pandoc.org/MANUAL.html#extension-link_attributes) extension. +The bracketed text following the image declaration is interpreted by Pandoc's [`link_attributes`](https://pandoc.org/MANUAL.html#extension-link_attributes) extension. For example, the following will override the figure number to be "S1" and set the image width to 5 inches: ```md {#fig:supplement tag="S1" width="5in"} ``` -We recommend always specifying the width of SVG images (even if just `width="100%"`), since otherwise SVGs may not render properly in the [WeasyPrint](http://weasyprint.org/) PDF export. +We recommend always specifying the width of SVG images (even if just `width="100%"`), since otherwise SVGs may not render properly in the [WeasyPrint](https://weasyprint.org/) PDF export. ### Citations -Manubot supports Pandoc [citations](http://pandoc.org/MANUAL.html#citations) via `pandoc-citeproc`. -However, Manubot performs automated citation processing and metadata retrieval on raw citations. +Manubot supports Pandoc [citations](https://pandoc.org/MANUAL.html#citations) via `pandoc-citeproc`. +However, Manubot performs automated citation processing and metadata retrieval on in-text citations. Therefore, citations must be of the following form: `@source:identifier`, where `source` is one of the options described below. When choosing which source to use for a citation, we recommend the following order: @@ -80,6 +80,8 @@ When choosing which source to use for a citation, we recommend the following ord 3. PubMed ID, cite like `@pmid:26158728`. 4. _arXiv_ ID, cite like `@arxiv:1508.06576v2`. 5. URL / webpage, cite like `@url:http://openreview.net/pdf?id=Sk-oDY9ge`. +6. For references that do not have any of the persistent identifiers above, use a raw citation like `@raw:old-manuscript`. +Metadata for raw citations must be provided manually. Cite multiple items at once like: @@ -109,8 +111,8 @@ The Manubot workflow requires the bibliographic details for references (the set The Manubot attempts to automatically retrieve metadata and generate valid citeproc JSON for references, which is exported to `output/references.json`. However, in some cases the Manubot fails to retrieve metadata or generates incorrect or incomplete citeproc metadata. Errors are most common for `url` references. -For these references, you can manually specify the correct citeproc in [`content/manual-references.json`](content/manual-references.json), which will override the automatically generated reference data. -To do so, create a new citeproc record that contains the field `"standard_citation"` with the appropriate reference identifier as its value. +For these references, you can manually specify the correct CSL Data in [`content/manual-references.json`](content/manual-references.json), which will override the automatically generated reference data. +To do so, create a new CSL JSON Item that contains the field `"standard_citation"` with the appropriate reference identifier as its value. The identifier can be obtained from the `standard_citation` column of `citations.tsv`, which is located in the `output` branch or in the `output` subdirectory of local builds. As an example, `manual-references.json` contains: @@ -118,11 +120,23 @@ As an example, `manual-references.json` contains: "standard_citation": "url:https://github.com/greenelab/manubot-rootstock" ``` +The metadata for `raw` citations must be provided in `manual-references.json` or an error will occur. +For example, to cite `@raw:private-message` in a manuscript, a corresponding CSL Item in `manual-references.json` is required, such as: + +```json +{ + "type": "personal_communication", + "standard_citation": "raw:private-message", + "title": "Personal communication with Doctor X" +} +``` + +All references provided in `manual-references.json` must provide values for the `type` and `standard_citation` fields. For guidance on what CSL JSON should be like for different document types, refer to [these examples](https://github.com/aurimasv/zotero-import-export-formats/blob/a51c342e66bebd97b73a7230047b801c8f7bb690/CSL%20JSON.json). ## Manuscript metadata -[`content/metadata.yaml`](content/metadata.yaml) contains manuscript metadata that gets passed through to Pandoc, via a [`yaml_metadata_block`](http://pandoc.org/MANUAL.html#extension-yaml_metadata_block). +[`content/metadata.yaml`](content/metadata.yaml) contains manuscript metadata that gets passed through to Pandoc, via a [`yaml_metadata_block`](https://pandoc.org/MANUAL.html#extension-yaml_metadata_block). `metadata.yaml` should contain the manuscript `title`, `authors` list, `keywords`, and `lang` ([language tag](https://www.w3.org/International/articles/language-tags/ "W3C: Language tags in HTML and XML")). Additional metadata, such as `date`, will automatically be created by the Manubot. Manubot uses the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) specified in [`build.sh`](build/build.sh) for setting the manuscript's date. diff --git a/build/environment.yml b/build/environment.yml index 73171db..77d7a50 100644 --- a/build/environment.yml +++ b/build/environment.yml @@ -15,7 +15,7 @@ dependencies: - conda-forge::watchdog=0.8.3 - pip: - errorhandler==2.0.1 - - git+https://github.com/greenelab/manubot@4e6a0f6d28220264c0d7892c732cb68c3e97c07a + - git+https://github.com/greenelab/manubot@a008126b39e3bd4b80ebaa5af9f9fa2f30b3a670 - jsonref==0.2 - opentimestamps-client==0.6.0 - opentimestamps==0.3.0 diff --git a/content/02.delete-me.md b/content/02.delete-me.md index ba79516..6484a4e 100644 --- a/content/02.delete-me.md +++ b/content/02.delete-me.md @@ -6,10 +6,10 @@ The Manubot is a system for automating scholarly publishing. Content is written in [Pandoc Markdown](http://pandoc.org/MANUAL.html#pandocs-markdown) source files. See [`USAGE.md`](https://github.com/greenelab/manubot-rootstock/blob/master/USAGE.md) for more information on how to use the Manubot. -The Manubot project began with the [Deep Review](https://github.com/greenelab/deep-review), where it was used to compose a highly-collaborative review article [@doi:10.1101/142760]. +The Manubot project began with the [Deep Review](https://github.com/greenelab/deep-review), where it was used to compose a highly-collaborative review article [@doi:10.1098/rsif.2017.0387]. Other manuscripts that were created with Manubot include: -+ The Sci-Hub Coverage Study ([GitHub](https://github.com/greenelab/scihub-manuscript), [HTML manuscript](https://greenelab.github.io/scihub-manuscript/)) ++ The Sci-Hub Coverage Study ([GitHub](https://github.com/greenelab/scihub-manuscript), [HTML manuscript](https://greenelab.github.io/scihub-manuscript/)) [@doi:10.7554/eLife.32822] + Michael Zietz's Report for the Vagelos Scholars Program ([GitHub](https://github.com/zietzm/Vagelos2017), [HTML manuscript](https://zietzm.github.io/Vagelos2017/)) [@doi:10.6084/m9.figshare.5346577] If you notice a problem with Manubot, it's best to submit an upstream fix to the appropriate repository: From 135f45193f2bd1f6ce2aa95e06973d19073286d5 Mon Sep 17 00:00:00 2001 From: Daniel Himmelstein Date: Fri, 2 Nov 2018 13:23:22 -0400 Subject: [PATCH 6/6] Ignore OS specific files Merges https://github.com/greenelab/manubot-rootstock/pull/142 --- .gitignore | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.gitignore b/.gitignore index 6befc7c..ec87fe3 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,17 @@ __pycache__/ # Misc temporary files *.bak +# Operating system specific files + +## Linux +*~ +.Trash-* + +## macOS +.DS_Store +._* +.Trashes + +## Windows +Thumbs.db +[Dd]esktop.ini