Error when building update citations

[sterding]

Hi,

My site looks fine when in preview with Docker. However, I got the following error after pushing to the GitHub: https://github.com/sterding/donglab.org/actions/runs/6681986579/job/18156659299#step:6:160

Could you please help?

What I did was, I first add a search term in the pubmed.yaml and after the citations are generated in citations.yaml, I then copied the references from there to source.yaml and removed the search term in pubmed.yaml. Not sure if this is the recommended way to manage references. The reason I did like that is, I need to manually edit some reference and also added some details for some of them. I feel it's better managed in source.yaml, so that I don't need pubmed.yaml to retrieve again every time. Please advise.

Thanks,
-Xianjun

[vincerubinetti]

The run that you linked shows the error in the logs:

Looking at your repo, your orcid and pubmed yaml files are blank, which I believe gets parsed as null. The cite process expects these to be lists of dicts. You should delete those files completely.

---

As for your method, that's indeed not the recommended way to do things, but I could see why you might want to do that in the specific case of PubMed, because those search terms often return a lot of false positives and require manual selection.

However, if you're getting all the right sources from PubMed and just need to tweak the details, you can just keep your orcid.yaml and pubmed.yaml files filled out, then use this feature to manually correct details on certain citations in sources.yaml:
https://greene-lab.gitbook.io/lab-website-template-docs/basics/citations#manual-override
https://greene-lab.gitbook.io/lab-website-template-docs/basics/citations#orcid

Because the cite process merges duplicate sources by id, you can also use the manual override method above to manually correct sources returned from metasources. Example:

Also FYI, all network calls are cached by the template, so you don't have to worry about making too many repeated API calls.

[sterding]

Thanks so much, Vincent, for your professional help!! To remove the false positive from pubmed term search, if I take the manual-override method you suggested, while keeping the pubmed.yaml file, can I set something like `remove: true` in source.yaml to manually remove the false positive? - id: pubmed:37723137 remove: true Thanks, -Xianjun

…

On Oct 29, 2023, at 11:18 AM, Vincent Rubinetti ***@***.***> wrote: The run that you linked shows the error in the logs: <https://user-images.githubusercontent.com/8326331/278879408-8b97bcd4-63e6-4844-8325-db1eb6c145dc.png> Looking at your repo, your orcid and pubmed yaml files are blank, which I believe gets parsed as null. The cite process expects these to be lists of dicts. You should delete those files completely. As for your method, that's indeed not the recommended way to do things, but I could see why you might want to do that in the specific case of PubMed, because those search terms often return a lot of false positives and require manual selection. However, if you're getting all the right sources from PubMed and just need to tweak the details, you can just just use this feature to manually correct details on a certain citation: https://greene-lab.gitbook.io/lab-website-template-docs/basics/citations#manual-override <https://greene-lab.gitbook.io/lab-website-template-docs/basics/citations#manual-override> Also FYI, all network calls are cached by the template <https://greene-lab.gitbook.io/lab-website-template-docs/basics/citations#cache>, so you don't have to worry about making too many repeated API calls. — Reply to this email directly, view it on GitHub <#221 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFGSHR4TN6VJSUWUJ73GGDYBZXS3AVCNFSM6AAAAAA6UU5V7OVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TIMJWGI3TM>. You are receiving this because you authored the thread.

[vincerubinetti]

At the moment there isn't a way built into the template to do that, and I'm hesitant to add it because 1) it'd be a dedicated keyword for a pretty niche use case and 2) pubmed isn't t he recommended way to do metasources, for this very reason that it's brittle and has false positives.

However, you can try adding this to _cite/cite.py right after _id = get_safe(source, "id", "").strip():

    # source id
    _id = get_safe(source, "id", "").strip()

    # remove entry
    if get_safe(source, "remove", False) == True:
      continue

I may add something like this to the template but comment it out.

[sterding]

One more follow-up question about citation: 1. I tried to add `| markdownify` in the citation.publisher section in the _include/citation.html, e.g. `{{ citation.publisher | markdownify | default: "[no publisher info]" }}`, to allow me to use markdown in the publisher title, but it won’t work. It actually add a newline to the publisher now. e.g. Before: Nature genetics · 06 May 2021 · pubmed:33958783 After: Nature genetics · 06 May 2021 · pubmed:33958783 2. How can I capitalize every word of the publisher automatically? e.g. Nature genetics —> Nature Genetics. Thanks, -Xianjun

…

On Oct 29, 2023, at 6:10 PM, Xianjun Dong ***@***.***> wrote: Thanks so much, Vincent, for your professional help!! To remove the false positive from pubmed term search, if I take the manual-override method you suggested, while keeping the pubmed.yaml file, can I set something like `remove: true` in source.yaml to manually remove the false positive? - id: pubmed:37723137 remove: true Thanks, -Xianjun > On Oct 29, 2023, at 11:18 AM, Vincent Rubinetti ***@***.***> wrote: > > > The run that you linked shows the error in the logs: > > > Looking at your repo, your orcid and pubmed yaml files are blank, which I believe gets parsed as null. The cite process expects these to be lists of dicts. You should delete those files completely. > > As for your method, that's indeed not the recommended way to do things, but I could see why you might want to do that in the specific case of PubMed, because those search terms often return a lot of false positives and require manual selection. > > However, if you're getting all the right sources from PubMed and just need to tweak the details, you can just just use this feature to manually correct details on a certain citation: > https://greene-lab.gitbook.io/lab-website-template-docs/basics/citations#manual-override > > Also FYI, all network calls are cached by the template, so you don't have to worry about making too many repeated API calls. > > — > Reply to this email directly, view it on GitHub, or unsubscribe. > You are receiving this because you authored the thread. >

[vincerubinetti]

For the capitalization, you can add this to _styles/citation.scss:

.citation-details {
  text-transform: capitalize;
}

This will capitalize the whole row though, including a DOI. To avoid that, you'd need more complex changes. I'll add this change to #209.

[vincerubinetti]

For the markdownify question, see this comment: #217 (comment)

[sterding]

One last picky question re the citation: If I want to make the publisher name bold and italic systemically, how would I do that? I noticed that in citation.html, you combine publisher, date, and id into one citation-details class, so I guess there is no way to only italic/bold the publisher part in css. I am less familiar with the grammar of the following part. Is that python in html? Or javascript? How can I learn to change it to add bold/italic feature there? I can learn it myself if you could point some introduction. Thanks {{ citation.publisher | markdownify | remove: "<p>" | remove: "</p>" | default: "[no publisher info]" }}

…

On Oct 29, 2023, at 7:00 PM, Vincent Rubinetti ***@***.***> wrote: For the markdownify question, see this comment: #217 (comment) <#217 (comment)> — Reply to this email directly, view it on GitHub <#221 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFGSHWBMVOOG2CKSZRUFR3YB3NZZAVCNFSM6AAAAAA6UU5V7OVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TIMJYGIYDK>. You are receiving this because you authored the thread.

[vincerubinetti]

There is some background knowledge in the docs: https://greene-lab.gitbook.io/lab-website-template-docs/advanced/background-knowledge That is not python, that is Jekyll/Liquid, and it gets run when your site is built, unlike javascript which runs when the site is viewed by a visitor. markdownify is a liquid filter, which you can lookup more about.

In the citation component (_includes/citation.html), there is a "citation details" <div> row with the publisher, date, and id. The fix is to wrap each of those 3 things in <span></span>'s, add uniq classes to them like class="citation-publisher", then add CSS styles in _styles/citation.scss like .citation-publisher { text-transform: capitalize; }. But if you just want to bold or italicize, and you've already implemented markdownify, and you're already manually correcting the publisher... then you can just do _**Nature Genetics**_.

Or if you want a more immediate and less flexible solution (for the capitalization problem), you can use add more liquid filters after the markdownify like this: split: " " | capitalize | join: " ". This essentially hard codes the capitalization into the HTML generated at build time.

[sterding]

Thank you for the wonderful tips!! It works.

…

On Oct 29, 2023, at 7:50 PM, Vincent Rubinetti ***@***.***> wrote: There is some background knowledge in the docs: https://greene-lab.gitbook.io/lab-website-template-docs/advanced/background-knowledge <https://greene-lab.gitbook.io/lab-website-template-docs/advanced/background-knowledge> That is not python, that is Jekyll/Liquid, and it gets run when your site is built, unlike javascript which runs when the site is viewed by a visitor. markdownify is a liquid filter, which you can lookup more about. In the citation component (_includes/citation.html), there is a "citation details" <div> row with the publisher, date, and id. The fix is to wrap each of those 3 things in <span></span>'s, add uniq classes to them like class="citation-publisher", then add CSS styles in _styles/citation.scss like .citation-publisher { text-transform: capitalize; }. But if you just want to bold or italicize, and you've already implemented markdownify, and you're already manually correcting the publisher... then you can just do _**Nature Genetics**_. — Reply to this email directly, view it on GitHub <#221 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFGSHSXD2O3PL7SNJV2YILYB3TTBAVCNFSM6AAAAAA6UU5V7OVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TIMJYGM3TQ>. You are receiving this because you authored the thread.