Adding tests for Services

[lloydrichards]

There was a comment in #4232 from @Bilge (here) that was a bit brutal, but also touches on something important that could be worth discussing:

How to we make the increasingly large number of services maintainable in the long run through testing?

I feel this would be a great addition to the Add a new service template to Coolify documentation on how to properly test the services we're adding and be able to check their continual functionality over time (since many are not version locked).

Does anyone have any suggestions on how we could test this? And maybe then wrap it up in a github action for CI validation 🤔

[Bilge]

Thanks for raising this important discussion. The way I see it, there's two possible paths to take:

1. Test using the API.
2. Test by automating clicks with a UI testing framework like Selenium or Cypress.

My preference would be for the former, because I think UI tests can be a little too brittle. Of course, in a perfect world we would have both, but the API approach seems like the best starting point.

[peaklabs-dev]

I totally agree that we need testing. I will refactor a lot of code hopefully next week 😅. And then we will add parser test first, which parses the compose files, as that is the most important part. Services tests will probably follow shortly after. Also during the refactoring I will have to delete a lot of services that are super small or not really official, so we will only keep the big and official ones, the other services should not live in the main repo and be added using custom services URL in the future so we keep it the services managable.

[lloydrichards]

Awesome to hear this is on the radar! looking forward to seeing how the new service api will work 👀

[Bilge]

The service API isn't new, lol.