Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the prod-playbooks #379

Merged
merged 10 commits into from
Nov 26, 2024

Conversation

pwalczysko
Copy link
Member

@pwalczysko pwalczysko commented Nov 15, 2024

This PR is replacing #378

The idea is to slim down the prod-playbooks repo to a more manageable size of 1-2 playbooks which are offered to community as examples.

The plan (to be executed in 2 stages) is to

  1. adjust and add the ome-demoserver.yml playbook and test it via molecule
  2. adjust and add the training server playbook and test it via molecule

This PR is tackling step 1. above:

  • Delete all the playbooks except the ome-demoserver.yml
  • Update the ome-demoserver.yml playbook for Rocky Linux 9 / RHEL 9 as it is used now in our private repo in production
  • Add appropriate requirements.yml file
  • Drop the site.yml logic as it is too complex for just 1-2 playbooks and does not help the users when attempting to run/use this repo imho
  • Unify the templates with the private repo ones -> this involves update of omero-web-config-for-webapps.j2 (tagsearch and webtagging)
  • Add full test sequence (including create, converge..., i.e. the install of OMERO.server and OMERO.web as per ome-demoserver.yml playbook) to the molecule/ome-demoserver/molecule.yml
  • Adjust the README to reflect the new purpose and scope of this repo
  • Make the changes neccessary for the molecule test above to pass, including
    - create a prepare.yml for install of cron
    - add the inventory paragraph to molecule.ymlto pass the correct variable values to the testing framework
    - make sure the roles during the test are installed where the molecule machinery can find them (change in ansible.cfg)
    - run the dependency step before the first destroy step, as otherwise the destroy step crashes

I have also verified that the playbook ome-demoserver.yml is working fine on the production demo server in check mode when run from this repo exactly as prescribed now in the README cmds.

cc @khaledk2 @sbesson @jburel
-

@pwalczysko pwalczysko requested review from sbesson and jburel November 15, 2024 14:23
@pwalczysko pwalczysko mentioned this pull request Nov 15, 2024
- name: ome.omero_web
version: 4.0.1

- src: ome.java
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these expected to be examples or the actual playbooks used for deploying the production services?
In the latter case, I think you might want to keep the version rather than installing the latest role by default

Copy link
Member Author

@pwalczysko pwalczysko Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin the versions of roles:

The drawback of this is that the versions will become stale (that happened already).
The advantage of this - I am not sure about the disadvantages implied, maybe we should discuss this ? I thought our roles are pretty well maintained and we do not want to introduce anything to them which is not compliant with our prod playbooks deployment ?

Copy link
Member

@sbesson sbesson Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The drawback of this is that the versions will become stale (that happened already).

That's correct, if these are stored, these versions need to be maintained, that's not different from any other libraries with dependencies.

The advantage of this - I am not sure, maybe we should discuss this ?

Primarily a reproducibility argument. If a new version of the role with backwards-incompatible changes is made, this approach means installing the roles and running the playbook from the exact same commit could result in a different outcome.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 25533d4


At the moment, only the `ome-demoserver.yml` playbook is available here. This is a playbook for OMERO.demo server on https://demo.openmicroscopy.org OMERO.server and OMERO.web. You can read more about this [OMERO.demo server on our website](https://www.openmicroscopy.org/explore/).

We are in the process of adding more OME team's production playbooks here.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the intent of this repository to only hold OMERO production playbooks moving forward?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the intent of this repository to only hold OMERO production playbooks moving forward?

That was my initial intent when planning this PR and discussing the tactics previously with @jburel .
What else would you like to hold here @sbesson ?
The idea is to minimize and simplify wherever possible.
But happy to discuss about better strategies.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have anything different to propose. I was mostly enquiring as I saw the omero subfolder was maintained and this was historically used for separating the OMERO playbooks from the others.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, omero folder is maintained to get out of the conandrum of molecule not finding the roles folder. Maybe I could rename the folder to playbooks ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in efd4d1f

@pwalczysko pwalczysko force-pushed the only-demo-on-prod-playbooks branch from ecb6a70 to c1461fe Compare November 18, 2024 12:05
@dominikl
Copy link
Member

👍 For the demo server email changes!

@pwalczysko pwalczysko requested a review from sbesson November 20, 2024 19:12
@pwalczysko
Copy link
Member Author

The fixes were pushed, the new changes are implemented too.

Copy link
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review is quite tricky here as the scope of the changes is quite big (>4K lines deleted, complete overhaul). Focusing exclusively on the diff for the non-deleted files, the changes to the playbook and the template files are fairly minimal. The requirements files have been reordered and there is no easy way to review the version bump but the Molecule scenarios are upgraded to run on Rocky Linux 9 and passing.

Assuming this PR represents the state of the current deployment of the various OMERO production servers at UoD, my recommendation would certainly be to merge this sooner rather than later.

@pwalczysko
Copy link
Member Author

@jburel Are you happy for me to merge this please ?

strategy:
# Keep running so we can see if other tests pass
fail-fast: false
matrix:
scenario: ${{fromJson(needs.list-scenarios.outputs.matrix)}}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actions/setup-python bump to v5

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in fee1fde

@pwalczysko
Copy link
Member Author

Merging as discussed at the meeting. thanks All.

@pwalczysko pwalczysko merged commit cbd713d into ome:master Nov 26, 2024
2 checks passed
@pwalczysko pwalczysko deleted the only-demo-on-prod-playbooks branch November 26, 2024 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants