Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(TKC-1581): add mechanism to build Kubernetes resources for the TestWorkflow #5096

Merged

Conversation

rangoo94
Copy link
Member

@rangoo94 rangoo94 commented Mar 3, 2024

Pull request description

  • Add mechanism to build (and deploy) Kubernetes resources for the TestWorkflows
  • Add API endpoint for running the TestWorkflows
    • Almost dummy - it doesn't return any data yet, and it's not persisting data
  • Add basic processors for some operations
    • For the moment, only operations that do not require an additional toolkit: nested steps, content files, and running the containers
  • Understands groups, conditions, optional steps, and all other syntaxes

Checklist (choose whats happened)

  • breaking change! (describe)
  • tested locally
  • tested on cluster
  • added new dependencies
  • updated the docs
  • added a test

Fixes

Copy link

netlify bot commented Mar 3, 2024

Deploy Preview for testkube-docs-preview canceled.

Name Link
🔨 Latest commit 560588c
🔍 Latest deploy log https://app.netlify.com/sites/testkube-docs-preview/deploys/65e599e50e4b5e0008817792

@codecov-commenter
Copy link

codecov-commenter commented Mar 3, 2024

Codecov Report

Attention: Patch coverage is 45.70948% with 968 lines in your changes are missing coverage. Please review.

Project coverage is 31.22%. Comparing base (9bfec74) to head (560588c).
Report is 1 commits behind head on develop.

Files Patch % Lines
...rkflowstcl/testworkflowprocessor/mock_container.go 0.00% 262 Missing ⚠️
...stworkflowstcl/testworkflowprocessor/mock_stage.go 0.00% 187 Missing ⚠️
...lowstcl/testworkflowprocessor/mock_intermediate.go 0.00% 128 Missing ⚠️
...stworkflowstcl/testworkflowprocessor/operations.go 31.00% 67 Missing and 2 partials ⚠️
...estworkflowstcl/testworkflowprocessor/container.go 71.17% 56 Missing and 10 partials ⚠️
...estworkflowstcl/testworkflowprocessor/processor.go 71.35% 38 Missing and 19 partials ⚠️
...tworkflowstcl/testworkflowprocessor/initprocess.go 76.92% 26 Missing and 4 partials ⚠️
...rkflowstcl/testworkflowprocessor/mock_processor.go 0.00% 28 Missing ⚠️
...workflowstcl/testworkflowprocessor/intermediate.go 71.59% 23 Missing and 2 partials ⚠️
internal/common/common.go 0.00% 23 Missing ⚠️
... and 9 more
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #5096      +/-   ##
===========================================
+ Coverage    30.47%   31.22%   +0.75%     
===========================================
  Files          315      333      +18     
  Lines        28381    30070    +1689     
===========================================
+ Hits          8648     9390     +742     
- Misses       18899    19801     +902     
- Partials       834      879      +45     
Flag Coverage Δ
integrationtests 5.71% <1.17%> (-0.27%) ⬇️
unittests 27.33% <45.70%> (+0.98%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

)

const (
defaultImage = "busybox:1.36.1"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't it be configurable parameter?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default is already configurable on the TestWorkflow level, as you may set top-level container: { image: "another-default-image" }. We could add some environment variable on API Server level to default the default though, if you want 🙂 I'm not sure if it's worth too, considering that we will likely have some global TestWorkflowTemplate, that will be automatically added to all of the TestWorkflows, where it could be configured (with top-level container settings, as above).

When it comes to the other properties, like internal or data directory, I'm considering that we could add in the future option to pass another property to the TestWorkflow, like internal: { dataDirectory: {path string, volumeName string} } or something like that

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, let's see, if we have any questions about it

_, err = s.Clientset.CoreV1().ConfigMaps(s.Namespace).Create(context.Background(), &item, metav1.CreateOptions{})
if err != nil {
return s.BadRequest(c, errPrefix, "creating configmap", err)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we clear them after job finish?

Copy link
Member Author

@rangoo94 rangoo94 Mar 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will, but not at this point. The TKC-1651 ticket is for building the Controller, which will be handling the Job/Pod/Containers events, logs, and the cleanup 👍 I'm working on that in this very moment 🙂

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great

@rangoo94 rangoo94 merged commit 285642b into develop Mar 4, 2024
11 checks passed
@rangoo94 rangoo94 deleted the dawid/feat/TKC-1581-testworkflows-build-kubernetes-resources branch March 4, 2024 10:17
rangoo94 added a commit that referenced this pull request Mar 14, 2024
…estWorkflow (#5096)

* feat(TKC-1581): adjust schema for TestWorkflow CRD adjustments
* chore(TKC-1581): update testkube-operator
* feat(TKC-1581): add initial mechanism for building Kubernetes objects from TestWorkflow
* feat(TKC-1581): add basic API endpoint for execution TestWorkflows
* chore(TKC-1581): clean code and add tests
* feat(TKC-1581): support "delay" step clause
* chore(TKC-1581): move init process arg names to constants
rangoo94 added a commit that referenced this pull request Mar 14, 2024
…estWorkflow (#5096)

* feat(TKC-1581): adjust schema for TestWorkflow CRD adjustments
* chore(TKC-1581): update testkube-operator
* feat(TKC-1581): add initial mechanism for building Kubernetes objects from TestWorkflow
* feat(TKC-1581): add basic API endpoint for execution TestWorkflows
* chore(TKC-1581): clean code and add tests
* feat(TKC-1581): support "delay" step clause
* chore(TKC-1581): move init process arg names to constants
exu pushed a commit that referenced this pull request Apr 2, 2024
…estWorkflow (#5096)

* feat(TKC-1581): adjust schema for TestWorkflow CRD adjustments
* chore(TKC-1581): update testkube-operator
* feat(TKC-1581): add initial mechanism for building Kubernetes objects from TestWorkflow
* feat(TKC-1581): add basic API endpoint for execution TestWorkflows
* chore(TKC-1581): clean code and add tests
* feat(TKC-1581): support "delay" step clause
* chore(TKC-1581): move init process arg names to constants
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants