Remove plone/meta / pre-commit from templates

[ericof]

I would like to initiate a discussion regarding the removal of plone/meta from the existing templates. This proposal is not due to any issue with plone/meta itself but rather its reliance on pre-commit, which does not and will not support mono repositories. Given that cookieplone-templates incorporates the mono repository use case, maintaining two distinct configurations in the long term would introduce additional complexity.

Background

plone/meta & pre-commit

Since the decision to adopt pre-commit was made during the 2023 Alpine City Sprint, the community has gradually transitioned non-core codebases to utilize plone/meta. Examples of this implementation can be observed in the following repositories:

• collective.blog
• collective.bluesky
• collective.mastodon
• collective.opentelemetry
• collective.ploneintranet
• pas.plugins.oidc

Additionally, plone/meta was incorporated into our legacy cookiecutter templates and subsequently into cookieplone-templates.

For codebases that contain a single Python package, this approach has proven to be effective.

Mono Repositories

A significant portion of new projects within the Plone community now follow a mono repository structure, typically comprising a Python backend and a React frontend. This structure presents a challenge with pre-commit its monorepo standpoint:

1. Retain plone/meta and attempt to adapt pre-commit to support the mono repository use case.
2. Disregard plone/meta and maintain a separate set of QA tools.

During last year's Beethoven Sprint 2024, following some research, we opted for the first approach and implemented a plugin named sub-pre-commit. It was understood at the time that this approach required pre-commit to remain pinned to a stable version, as this was the only way to ensure compatibility with sub-pre-commit.

Following this decision, pre-commit and sub-pre-commit were added to cookieplone-templates, and all new projects since have been structured accordingly.

Reasons for Reconsideration

After nearly a year of working with this setup, it has become evident that maintaining support for mono repositories under the current approach is proving to be too fragile and does not offer sufficient benefits:

• plone/meta does not provide significant advantages, as its focus remains on Python packages located at the root of the repository.
• pre-commit does not offer the intended benefits, as running make check within a subdirectory often leads to unexpected failures.
• The dependency on pre-commit remains fixed at version 3.7.1, despite version [4.1.0](https://github.com/ddanier/sub-pre-commit/commit/68316dfbe1863166307a4a65bf73c62cfcf89c25) already being available.

Based on my experience in building and maintaining mono repositories for distributions (portalbrasil-intranet, kitconcept.intranet), as well as for customer projects, we have consistently moved away from plone/meta and pre-commit over time. This decision has been driven by the challenges outlined above, which have made these tools less suitable for long-term maintenance in mono repository environments.

Challenges of Transitioning Away

• The need to establish, document, and maintain a different set of QA tools tailored to Plone codebases.
• The overhead of having different toolsets
• cookieplone-templates not being used to create new core packages

Alternative path

As an alternative solution we should improve plone/meta and address the following points:

• Have a configuration tailored for mono repositories, and document it properly
• Get rid of pre-commit:
  • Replace it with some other tool (gookme, lefthook..)
  • Replace it with make commands + CI

Next steps

• Use this issue to list more points in favor and against plone/meta + pre-commit
• Have a call during Alpine City Sprint to discuss it
• Implement whatever is decided

c/c @sneridagh, @fredvd, @davisagli, @mauritsvanrees, @gforcada, @pbauer, @MrTango

[1letter]

Thats the reason and way, why it is so exhausting to find new devs for Plone. We change our toolset so often... But perhaps i'm too old.

[ericof]

@1letter I see your perspective, but I’d like to offer a different viewpoint. There are many aspects to consider, but for now, let’s focus on the toolset.

I believe it’s important to recognize that using a toolset that is not widely adopted can make it more challenging to attract new contributors. Exploring new approaches can often be more beneficial in the long run.

For example, no matter how much I appreciated and relied on Buildout, onboarding new developers—especially those coming from Django or FastAPI—was quite difficult. Similarly, continuing to use ZPT in an ecosystem dominated by Jinja, React, and Vue, or resisting the shift toward containers and object storage, would have made integration and collaboration more challenging.

The goal of this discussion is to simplify our current QA processes. For those who actively maintain recent Plone projects and contribute as core developers, the existing setup has introduced significant mental overhead, which we need to address.

That said, if you are maintaining a single legacy project that will eventually be upgraded to the latest version of Plone, you can certainly continue using the workflow you are accustomed to.

[fredvd]

Thank you for your write up of the current status @ericof . I added a few words in 2 places for smoother reading and posterity, see edit dropdown.

Question: we are talking now about project mono repo's, finding a way out of the pre-commit issue will also benefit future plone 'addon' packages advancements, where we want a monorepo for the volto frontend and a possible python backend in one repo. Now we have 2 repo's in this situation.

[davisagli]

Points in favor of keeping plone/meta but evolving it to support monorepos better:

• It is useful to have a tool for making existing repositories conform to a standard way of doing things (not just creating a new repository in a standard way)
• It's nice to have consistency in the approaches between creating a new repository and onboarding an existing one to the standard ways.

Points in favor of doing something else:

• plone/meta brings consistency at the cost of some complexity in understanding how to use it. You have to edit files that control how other files are generated, instead of just editing files.

I'm not sure what I prefer personally. I would like things to be as simple as possible for both new users and existing users, but it's not obviously what path achieves that.

[ericof]

Question: we are talking now about project mono repo's, finding a way out of the pre-commit issue will also benefit future plone 'addon' packages advancements, where we want a monorepo for the volto frontend and a possible python backend in one repo. Now we have 2 repo's in this situation.

The new distributions with Volto + Backend are a special case of addon packages, and this is where we should move in the long run for packages like collective.bookmarks, volto-form, etc

[stevepiercy]

• plone/meta brings consistency at the cost of some complexity in understanding how to use it. You have to edit files that control how other files are generated, instead of just editing files.

As a first-timer to plone/meta, this bit me hard. So I rewrote its documentation with fancy GitHub Markdown admonitions, and added a page to Plone 6 Documentation to give it more visibility.

• https://6.docs.plone.org/developer-guide/standardize-python-project-configuration.html
• https://github.com/plone/meta?tab=readme-ov-file#plonemeta

I don't know if it will still bite the next first-timer, but now I think it won't go as deep.

That said, almost all first-timers won't develop or maintain Plone packages. IMO doing something else would miss the target audience—specifically, experienced developers of Plone Python packages—of what we're trying to do as an organization.

[gforcada]

As it seems that my opinion is of some interest:

tl;dr: drop pre-commit on cookieplone.

Longer version:

plone/meta is a copy&paste&adapt from zopefoundation/meta. zopefoundation/meta aim is to bring consistency across +100 repositories in the zopefoundation GitHub organization (and likewise we are trying to do the same on plone GitHub organization.

Consistency is key on an aging software stack, specially if not a lot of people are looking at it on a daily basis. This really shows off when changes across all repositories have to happen: newer python/Plone versions, update GHA, etc etc.

cookieplone-templates OTOH, builds monorepos and is meant for projects not standalone addons (correct me if I'm wrong), so plone.meta core ideas have a lot of sense, but not the implementation per se maybe.

What would be cool/nice is that the python/backend/addon part of what cookieplone-templates generates keeps the developer experience (if we all agree on that part) from what plone.meta enforces on core.

Right now that developer experience boils down to the tox environments that it generates. If we ever switch to nox/mxmake/whatever, then adapting to that.

This would reduce mental friction when working on a plone core addon (plone.app.registry) and then working on your project (collective.you-name-it / my-fancy-project-with-frontend-and-backend).

Bottom line though, and as the tl;dr said: drop pre-commit if it's painful to maintain. If you can replace it with another tool that can be called by tox -e format and tox -e lint that would be great 👍🏾