Using a CMS for backend data?

[yawaramin]

I would like to propose using a content management system for ocaml.org. A couple of contenders:

• https://pocketbase.io/ - new and minimalistic, uses SQLite DB file, single binary, easy to deploy
• https://strapi.io/ - popular headless CMS, uses Node.js

I personally have been trying out PocketBase and think its minimalism would be a pretty good fit. We would be able to define an actual data model using its dashboard, set up user accounts who could either log on or programmatically access the HTTP API, and normalize and validate all data in the system.

Here's an example. In the 'Books' dataset, there are some authors, like Anil Madhavapeddy, who are also credited as authors in the 'Papers' dataset. With a normalized data model in a database, we could have both of these refer to the same 'author' entity. Then we could add more fields to the authors dataset, like GitHub, Twitter, website, bio. This would make it much easier to implement an 'Authors' (or 'People') section of the website in the future, and have a single unambiguous link to the unique person from all their books, presentations, and papers.

Another example: in papers.yml, right now the first paper has an author KC Sivaramakrishnan, and the second one has K. C. Sivaramakrishnan. Of course they are the same person, but with a proper CMS we would have the opportunity to de-duplicate them during entry, or change the initials style in one place and have it take effect everywhere automatically.

Bonus: using an SQLite would give us, with minimal effort, access to its fulltext search capabilities. We could literally define triggers to update the FTS table(s) whenever any of the data are changed. It would be able to power a 'universal' search.

[cuihtlauac]

Thanks @yawaramin for starting this discussion, this is great.

Setting aside the CMS matter for a second, the book and papers scenario you are using is very interesting, we have an open issue on the support of BibTeX for papers and books: #648 It is the academic standard. Having BibTeX aside or instead of Yaml is fairly widely accepted as a needed improvement.

How would a CMS help toward this goal?

[yawaramin]

Hi, if I understand correctly that issue is for accepting submissions for adding new papers and books (and I suppose other materials?) in BibTeX format. The general process I had been thinking of for data additions and changes would involve some amount of automation and some amount of manual work.

For example, if we had a web form for submitting a new BibTeX entry, we could use Talaria to parse it and a web client to send it to PocketBase. We could have an additional field in the papers collection, public, that would be manually switched on before the paper would be actually displayed on the website. And this part would need to be done by a human, to prevent mistakes or intentionally malicious content from being publicly displayed.

Alternatively, if we don't want new submissions to go straight into the CMS to prevent potential spamming or DOS attacks, we could have the backend send an email to a private group address with the details of each submission and a set of curl commands that can be copy-pasted and run locally in order to programmatically upload it to the CMS (alternatively, log in to the PocketBase UI and manually enter it).

You will notice that I am proposing a change from our current process of sending GitHub PRs to add data. I feel reducing reliance on GitHub is a positive, however there are possible workflows with PRs as well, e.g. set up a script to run as a daily (or hourly, etc.) job to process all new data submitted into the repo and programmatically send them into PocketBase. This script can be made to parse BibTeX entries as well as YAML format entries.

[yawaramin]

Another thought–should the ocaml.org backend really grow a partial bibliography management engine? It seems out of scope for a programming language website (even if the language is important in academia like OCaml). Others have already built bibliography and scholarly reference citation systems. Maybe it would be fine to upload and show basic info on our site, like say authors and title, and link to an appropriate Google Scholar entry to let people get more detailed information, like citations in BibTex or other formats? E.g.,

Pottier & Remy, 'The Essence of ML Type Inference'

[sabine]

Indeed, we have to very closely watch scope here. IMO, anything we add should either

1. improve the developer experience - i.e. make it easier for people to successfully write OCaml programs, or
2. increase the likelihood of adoption of OCaml in businesses and personal projects, or
3. encourage people to grow the ecosystem by publishing packages and/or writing documentation.

It is true that there are reasonable existing sites for scholarly references and it is also fine for OCaml to showcase its own academic roots by highlighting the most influential and useful papers relating to it. I do not think we should cross-reference data relating to publications or books because there are so many construction sites that could make a direct impact on growing the ecosystem.

A positive aspect regarding the current approach of data management is that we entirely avoid a whole host of complexity. It's true that using a relational database would enable us to better cross-reference all kinds of data. At the time being, managing the data manually is transparent and straightforward: there is no need to manage database migrations and we can be sure that, when the code compiles, the HTML pages that embed the data will render successfully.

It's possible that a time will come where we find that managing data via pull requests and the lack of consistent cross-referencing is too cumbersome. And that would be the right time to add a content management system (or a database) behind ocaml.org.