[docs/component-stability.md] Add criteria for graduating between stability levels #11864
Conversation
mx-psi
requested review from
atoulme,
codeboten,
bogdandrutu,
dmitryax and
TylerHelmuth
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #11864 +/- ##
=======================================
Coverage 91.62% 91.62%
=======================================
Files 447 447
Lines 23731 23731
=======================================
Hits 21743 21743
Misses 1613 1613
Partials 375 375 ☔ View full report in Codecov by Sentry. |
Co-authored-by: Christos Markou <[email protected]>
| ## Beta to stable | ||
|
|
||
| To graduate any signal from beta to stable on a component: | ||
| 1. The component MUST have at least three active code owners. |
There was a problem hiding this comment.
I think there should be a commitment from codeowners that there is a SLA for first response on bug issues.
The commitment should be measured in days.
There was a problem hiding this comment.
This raises some questions for me including:
- What happens when people go on vacation/have a kid/[insert activity here that leads to a prolonged period of absence]?
- What happens if people don't follow this SLA? Typically an SLA means that you pay if you don't meet a certain standard, how do you "pay" here?
There was a problem hiding this comment.
I think we shouldn't make it too hard to have community components.
I think vendor components and widely used components will not have any issue to follow the guidelines.
When we think about components that add value to the overall project, but may not be interesting/priority to vendors, they may struggle to get folks involved on it, and that would disqualify them from being moved to stable.
I do understand that we need to provide a way to ensure maintainability of stable components, but maybe we could draw something over the ideas of:
- Being active
- Replying to issues related to its components timely
- Fixing reported bug timely
- ...
If we have a component that is not vendor related, but maintained by 2 folks from a single employer, that wouldn't allow them to move on.
Also, let's imagine the following scenario:
- 3 folks are codeowners of a component, 2 from one company and another one from another company.
- They graduate to stable.
- A couple of months later the person that was from the other company moves to the same company of the other 2. Would the component be demoted?
I don't think it should be, if they are active and responsive in issues related to that component.
I know it is a corner case, and may never happen, but it still can.
My main point here, is actually that we shouldn't make it too hard to have community components.
I know a couple of companies that develop internal components to solve their customers' issues. It would be awesome to have a couple of those contributed back to upstream, and let the community grow together.
There is a trade-off between having more components and having fewer components that are more actively maintained. We need to be mindful of where we draw the line, but my feeling is that right now we have too many components that are not well maintained.
There's two questions to consider here:
On this PR I am focusing on (1). For doing (1), we need to focus on things that we can measure/check at the time of marking as stable. Some of the things you mention are, I feel like, important criteria for deciding if a component should be moved to unmaintained, but not to move to stable.
The point with the 'vendor diversity' is that I think it is a good predictor of component quality (more than one vendor means more focus on a wide number of use cases) and maintainability (we don't depend on a single company). Maybe it would help to do an analysis of existing components to see how difficult this is to achieve?
This PR makes no claims about when a component should be 'demoted'. Currently the only way to be demoted is to be moved to unmaintained, we have some rules about when that can happen. I personally don't think we should move from stable->beta, I think that would be confusing for end users. The way I see this it is a bit like the CNCF project status: there is no moving from graduated to incubating, only from graduated to deprecated. |
|
I split off part of this PR in #11937. PTAL at that one first |
Agree.
Makes sense.
Let's take the Connector first, from all components none of them have 3 ACTIVE codeowners.
On the other hand, I think this "rule" would bring more awareness to the components, and maybe it would also bring more codeowners to "important" components. Still not sure about the the amount of codeowners. |
After thinking further and discussing with @mx-psi, I believe this criteria is going to be the foundation for users to further engage and assume codeowner's responsibility in the components they would like to move to stable. |
…levels' section (#11937) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Split off from #11864, describes how the graduation would work without any additional criteria. Rendered diagram: ```mermaid stateDiagram-v2 state Maintained { InDevelopment --> Alpha Alpha --> Beta Beta --> Stable } InDevelopment: In Development Maintained --> Unmaintained Unmaintained --> Maintained Maintained --> Deprecated Deprecated --> Maintained: (should be rare) ``` --------- Co-authored-by: Christos Markou <[email protected]>
…levels' section (open-telemetry#11937) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Split off from open-telemetry#11864, describes how the graduation would work without any additional criteria. Rendered diagram: ```mermaid stateDiagram-v2 state Maintained { InDevelopment --> Alpha Alpha --> Beta Beta --> Stable } InDevelopment: In Development Maintained --> Unmaintained Unmaintained --> Maintained Maintained --> Deprecated Deprecated --> Maintained: (should be rare) ``` --------- Co-authored-by: Christos Markou <[email protected]>
Description
Code ownership and maintenance of components continues to be an issue, with varying levels of support across contrib. As we approach 1.0 and the ability to mark components as stable, we want to make sure that components that we deem as 'stable' have a healthy community around them. We have three datapoints that we can leverage here: how many codeowners a component has, how diverse these are in terms of employers and how actively the codeowners have been responding to issues/PRs in the recent past.
We need criteria that
Some notes:
Link to tracking issue
Fixes #11850