Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: 4.x releases #446

Open
totallyzen opened this issue Mar 7, 2024 · 16 comments
Open

RFC: 4.x releases #446

totallyzen opened this issue Mar 7, 2024 · 16 comments

Comments

@totallyzen
Copy link
Collaborator

Context

Now that we've got a release pipeline back in action, there is lots we can do to make things nicer.

I am raising this issue in order for community to gather and talk about what should be fixed as priorities and what improvements we want to add.

Known Issues

We know there are several areas we want to improve:

Note

The list above is not conclusive, please comment on this issue and show your support!

@totallyzen totallyzen changed the title RFC: What to prioritise in 4.x releases? RFC: 4.x releases Mar 7, 2024
@totallyzen totallyzen pinned this issue Mar 7, 2024
@max-pfeiffer
Copy link
Contributor

max-pfeiffer commented Mar 8, 2024

@totallyzen Hello Balint, I just created a couple of new issues:

The tests for these modules fail on my M2 MacBook. And most likely will fail for everyone else using these testcontainer modules on their machine with ARM CPU.

We already have an older pull request addressing that issue, but that one seems to be outdated and partially obsolet as a couple of issues already became fixed: #357

I would offer to work these issues as it is important to me to have working tests. Also anyone else using these testcontainer modules on their ARM machines would profit from it.

@max-pfeiffer
Copy link
Contributor

@totallyzen PS: I just checked for an M1/M2 GitHub runner. It is actually available since January 2024.

But it comes without a Docker installation and has other quirks. I just did a proof of concept with it: because of the mentioned issues I was not able to run the tests in the pipeline.

@bearrito
Copy link
Contributor

bearrito commented Mar 8, 2024

Might be more appropriate to get that in a 5.x.

AFAIK ARM runners aren't generally available yet - https://resources.github.com/devops/accelerate-your-cicd-with-arm-and-gpu-runners-in-github-actions/ . Not sure you'd want to correlate ARM with MAC OS.

In general not too difficult to get that working with github actions using a matrix - https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#using-a-matrix-strategy

@bearrito
Copy link
Contributor

bearrito commented Mar 8, 2024

  1. Revise documentation after the poetry workflow change. Most of the docs refer to the previous workflow.

  2. Correct usage/mental model of extras and community modules. I get the desire for keeping dependencies low. I think the downside of not including a client dependency is while you can verify that the container is running, you can't verify that the connection details you might return are valid for the eventual client. Some want a full uri, others perhaps host and port. Or maybe I didn't expose the correct port etc.

I'm the committer on #439 There is AFAIK on one supported python client to interact with NATS, so it seems natural to include it. I could take or leave any approach.

@max-pfeiffer
Copy link
Contributor

max-pfeiffer commented Mar 9, 2024

Might be more appropriate to get that in a 5.x.

AFAIK ARM runners aren't generally available yet - https://resources.github.com/devops/accelerate-your-cicd-with-arm-and-gpu-runners-in-github-actions/ . Not sure you'd want to correlate ARM with MAC OS.

In general not too difficult to get that working with github actions using a matrix - https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#using-a-matrix-strategy

Yes, I agree here. I was not aware of the general ARM runner roadmap of GitHub.

And yes, it's quite straight forward to achieve this with a matrix. I already did a little POC: max-pfeiffer#3

@totallyzen
Copy link
Collaborator Author

@max-pfeiffer thanks for the summary on mac! It helps a great deal indeed to know!

For the matrices, I think you'll find it that I've used them extensively in this project already.
We can easily add a PoC PR that piles up all the fixes for MacOS, Windows and ARM and I'm glad there are people who care about this! :)

For me I'm running on a vanilla Ubuntu and not an ARM machine so as you might imagine, testing can be kinda difficult. I also have a young child so time is limited sometimes. If you find me responding somewhat slowly, this is why (managing energy and time).


@bearrito Good points overall, I started on the documentation for CONTRIBUTING.md and update the issue templates so we can communicate how things fit together. I'll add all the current active members to that PR as a reviewer so you can have your say! 🙂

As for "what about auto-including clients" I think I am gravitating towards "let's add them as dev dependencies to prove the clients are working, but don't enforce it in live dependencies". This way we can show the users how we intended to use it, without enforcing the specific library (even if there is only one right now).
Deal?


I'll check with @kiview on giving you both some level of permissions so you can auto-run some tests yourself on PRs. Not write permissions, just so that your PRs get auto-approval for the GH runs. Same goes for @jankatins as he is also actively contributing with other things!

I think with your dedication and activity you have earned it.

@bearrito
Copy link
Contributor

bearrito commented Mar 9, 2024

@totallyzen Deal. I implemented what I think should be satisfactory in a new pr at https://github.com/testcontainers/testcontainers-python/pull/462/files. That could be an overall good approach to managing clients.

@alexanderankin
Copy link
Collaborator

found another good fix to get merged #457 for 4.0.x

@alexanderankin
Copy link
Collaborator

alright im unsubscribing from notifications from this repo for half a week - its too much - so many people are so busy pushing commits and updating the project with feedback. good job everyone 👍

@alexanderankin
Copy link
Collaborator

what if we only maintained semver for core, and we treated all community changes as "fix"?

@kiview
Copy link
Member

kiview commented Mar 25, 2024

Maintaining a module ecosystem in itself complex and we have explored different options for this in the past (monorepo vs multirepo, synced versioning, etc.). Since tc-python currently follows the monorepo approach for its modules, they are released as part of a synchronized release job, together with a synchronized version (even if certain modules won't contain changes themself).

For other languages we learned, that this is generally an easy process, not without its drawbacks, but straight forward to maintain.

However in this case, I agree with @alexanderankin that changes in the modules should not be able to significantly contribute to the semver version of core, since I would assume this to be counterintuitive for the majority of users.

@alexanderankin
Copy link
Collaborator

@totallyzen @santi any objection to this idea which i have proposed above:

what if we only maintained semver for core, and we treated all community changes as "fix"?

@santi
Copy link
Collaborator

santi commented Mar 30, 2024

@totallyzen @santi any objection to this idea which i have proposed above:

what if we only maintained semver for core, and we treated all community changes as "fix"?

@alexanderankin Having looked through how things are done in tc-java, it seems like they are following the approach with updating (including breaking changes) in community modules as fix/patch changes. At least that's what I gather from this example, where they make breaking changes to the mysql module, which was later released as part of the 1.19.4 patch.

So yes, you have my support. If we were to follow semver for all community module changes we would release a lot of major versions rapidly, which would be like shouting "wolf, wolf" repeatedly and users would stop thinking of that as "dangerous territory" / likely requiring a small code fix.

@totallyzen
Copy link
Collaborator Author

totallyzen commented Mar 30, 2024

👋 Gotcha! Agreed then, I'll document it in #460 to let users know that we guarantee best-effort semver for community modules.

EDIT: aaand done! 8bfc758

@alexanderankin
Copy link
Collaborator

alexanderankin commented Apr 17, 2024

just thinking aloud:

next(?) round of containers:

still figuring out environment fixes (dind (#283 still not merged, still some user pains from #388), macos ipv6 localhost nonsense, windows, i started doing some fixes in #457 and #458 but we have had 0 related user reports, which probably means this library doesn't see much use on windows)

@alexanderankin
Copy link
Collaborator

cosmos, went out in 4.7.0, cockroachdb also released. for 4.8.0:

fixed dind tests in #622 - going to hopefully also fix dind there and then scylla, BQ, and the others from my last comment - #446 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants