Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [TKC-2556] test workflow running context #5926

Draft
wants to merge 33 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
29ba776
feat: running context model for test workflows
vsukhin Jul 4, 2024
a0ed48a
feat: add running context to test workflow execution models
vsukhin Jul 4, 2024
7fd49ad
fix: unit test
vsukhin Jul 4, 2024
6911023
Merge commit '7da22fdf8a5f96cf3cf4606e138bceffed17c3ba' into vsukhin/…
vsukhin Jul 5, 2024
c3b438e
fix: dep update
vsukhin Jul 5, 2024
ee741fb
feat: map test workflow running context
vsukhin Jul 5, 2024
6e49ecb
feat: pass running context to execution
vsukhin Jul 5, 2024
d54ca82
fix: running context for test workflows
vsukhin Jul 8, 2024
2b7b2b6
fix: add test workflow running context model
vsukhin Jul 8, 2024
66c39d5
fix: additional enum for test workflow running context
vsukhin Jul 8, 2024
8e467d0
fix: add one more enum
vsukhin Jul 9, 2024
48042d7
fix: mapping for test workflow running context
vsukhin Jul 9, 2024
c5d9641
fix: mapping for cd events for test workflow running context
vsukhin Jul 9, 2024
6b537c2
fix: show test workflow running context
vsukhin Jul 9, 2024
581d826
feat: get full psth
vsukhin Jul 9, 2024
b1b25cc
fix: add parent ids for test workflow running context
vsukhin Jul 10, 2024
235dfb7
fix: format running context ui
vsukhin Jul 10, 2024
e751657
fux: add to mock
vsukhin Jul 10, 2024
e6f9a62
fix: remove unused var
vsukhin Jul 10, 2024
eb5d7d2
Merge branch 'main' into vsukhin/feature/test-workflow-running-context
vsukhin Oct 4, 2024
60e5ebd
fix: dep update
vsukhin Oct 7, 2024
88866c4
Merge branch 'main' into vsukhin/feature/test-workflow-running-context
vsukhin Oct 7, 2024
6e5d1e3
fix: merge
vsukhin Oct 7, 2024
b68824d
fix: change running context model
vsukhin Oct 7, 2024
b5052ef
fix: use reference instead of array
vsukhin Oct 7, 2024
f459b3b
fix: dep update
vsukhin Oct 8, 2024
d972b1a
fix: additional running contexts
vsukhin Oct 8, 2024
3269f3d
fix: add tcl for testworkflow mapper
vsukhin Oct 8, 2024
f91bb1b
fix: trigger running context
vsukhin Oct 16, 2024
6f2ac47
feat: move runnng context to tcl
vsukhin Oct 16, 2024
c3a077a
Merge branch 'main' into vsukhin/feature/test-workflow-running-context
vsukhin Oct 16, 2024
9dc607f
fixL move mapper to tcl
vsukhin Oct 16, 2024
d4825eb
fix: remove test
vsukhin Oct 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions api/v1/testkube.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7158,10 +7158,80 @@ components:
- testsuite
- testtrigger
- scheduler
- testworkflow
context:
type: string
description: Context value depending from its type

TestWorkflowRunningContext:
description: running context for test workflow execution
type: object
required:
- interface
- actor
properties:
interface:
$ref: "#/components/schemas/TestWorkflowRunningContextInterface"
actor:
$ref: "#/components/schemas/TestWorkflowRunningContextActor"

TestWorkflowRunningContextInterface:
description: running context interface for test workflow execution
type: object
required:
- type
properties:
name:
type: string
description: interface name
type:
$ref: "#/components/schemas/TestWorkflowRunningContextInterfaceType"

TestWorkflowRunningContextActor:
description: running context actor for test workflow execution
type: object
required:
- type
properties:
name:
type: string
description: actor name
username:
type: string
description: actor username
email:
type: string
description: actor email
executionId:
type: string
description: test workflow execution id
executionPath:
type: string
description: all test workflow execution ids starting from the root
type:
$ref: "#/components/schemas/TestWorkflowRunningContextActorType"

TestWorkflowRunningContextInterfaceType:
description: supported interfaces for test workflow running context
type: string
enum:
- cli
- ui
- api
- ci/cd
- internal

TestWorkflowRunningContextActorType:
description: supported actors for test workflow running context
type: string
enum:
- cron
- testrigger
- user
- testworkflow
- testworkflowexecution
- program

Webhook:
description: CRD based webhook data
type: object
Expand Down Expand Up @@ -8019,6 +8089,14 @@ components:
default: false
tags:
$ref: "#/components/schemas/TestWorkflowTagValue"
runningContext:
description: running context for the test workflow execution (Pro edition only)
$ref: "#/components/schemas/TestWorkflowRunningContext"
parentExecutionIds:
type: array
description: parent execution ids
items:
type: string

TestWorkflowWithExecution:
type: object
Expand Down Expand Up @@ -8112,6 +8190,9 @@ components:
- false
tags:
$ref: "#/components/schemas/TestWorkflowTagValue"
runningContext:
description: running context for the test workflow execution (Pro edition only)
$ref: "#/components/schemas/TestWorkflowRunningContext"
required:
- id
- name
Expand Down Expand Up @@ -8146,6 +8227,9 @@ components:
$ref: "#/components/schemas/TestWorkflowSummary"
tags:
$ref: "#/components/schemas/TestWorkflowTagValue"
runningContext:
description: running context for the test workflow execution (Pro edition only)
$ref: "#/components/schemas/TestWorkflowRunningContext"
required:
- id
- name
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,28 @@ func printPrettyOutput(ui *ui.UI, execution testkube.TestWorkflowExecution) {
ui.NL()
ui.Warn("Tags: ", testkube.MapToString(execution.Tags))
}
if execution.RunningContext != nil {
ui.Warn("Running context: ")
ctx := execution.RunningContext
if ctx.Interface_ != nil {
ui.Warn("Interface: ")
ui.Warn(" Name: ", ctx.Interface_.Name)
if ctx.Interface_.Type_ != nil {
ui.Warn(" Type: ", string(*ctx.Interface_.Type_))
}
}
if ctx.Actor != nil {
ui.Warn("Actor: ")
ui.Warn(" Name: ", ctx.Actor.Name)
ui.Warn(" Username: ", ctx.Actor.Username)
ui.Warn(" Email: ", ctx.Actor.Email)
ui.Warn(" Execution id: ", ctx.Actor.ExecutionId)
ui.Warn(" Execution path: ", ctx.Actor.ExecutionPath)
if ctx.Actor.Type_ != nil {
ui.Warn(" Type: ", string(*ctx.Actor.Type_))
}
}
}
if execution.Result != nil && execution.Result.Status != nil {
ui.Warn("Status: ", string(*execution.Result.Status))
if !execution.Result.QueuedAt.IsZero() {
Expand Down
19 changes: 19 additions & 0 deletions cmd/kubectl-testkube/commands/testworkflows/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ import (
"github.com/kubeshop/testkube/cmd/kubectl-testkube/commands/tests"
"github.com/kubeshop/testkube/cmd/kubectl-testkube/commands/testworkflows/renderer"
"github.com/kubeshop/testkube/cmd/testworkflow-init/instructions"
intcommon "github.com/kubeshop/testkube/internal/common"
apiclientv1 "github.com/kubeshop/testkube/pkg/api/v1/client"
"github.com/kubeshop/testkube/pkg/api/v1/testkube"
"github.com/kubeshop/testkube/pkg/telemetry"
"github.com/kubeshop/testkube/pkg/testworkflows/testworkflowprocessor/constants"
"github.com/kubeshop/testkube/pkg/ui"
)
Expand Down Expand Up @@ -63,11 +65,28 @@ func NewRunTestWorkflowCmd() *cobra.Command {
ui.ExitOnError("getting client", err)

name := args[0]
runContext := telemetry.GetCliRunContext()
interfaceType := testkube.CLI_TestWorkflowRunningContextInterfaceType
if runContext == "others|local" {
runContext = ""
interfaceType = testkube.CICD_TestWorkflowRunningContextInterfaceType
}
execution, err := client.ExecuteTestWorkflow(name, testkube.TestWorkflowExecutionRequest{
Name: executionName,
Config: config,
DisableWebhooks: disableWebhooks,
Tags: tags,
RunningContext: &testkube.TestWorkflowRunningContext{
Interface_: &testkube.TestWorkflowRunningContextInterface{
Name: runContext,
Type_: intcommon.Ptr(interfaceType),
},
Actor: &testkube.TestWorkflowRunningContextActor{
Type_: intcommon.Ptr(testkube.USER_TestWorkflowRunningContextActorType),
Username: "",
Email: "",
},
},
})
if err != nil {
// User friendly Open Source operation error
Expand Down
20 changes: 19 additions & 1 deletion cmd/tcl/testworkflow-toolkit/commands/execute.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"encoding/json"
"fmt"
"os"
"strings"
"sync"
"time"

Expand Down Expand Up @@ -72,7 +73,7 @@ func buildTestExecution(test testworkflowsv1.StepExecuteTest, async bool) (func(

exec, err := c.ExecuteTest(test.Name, test.ExecutionRequest.Name, client.ExecuteTestOptions{
RunningContext: &testkube.RunningContext{
Type_: "testworkflow",
Type_: string(testkube.RunningContextTypeTestWorkflow),
Context: fmt.Sprintf("%s/executions/%s", env.WorkflowName(), env.ExecutionId()),
},
IsVariablesFileUploaded: test.ExecutionRequest.IsVariablesFileUploaded,
Expand Down Expand Up @@ -165,13 +166,30 @@ func buildWorkflowExecution(workflow testworkflowsv1.StepExecuteWorkflow, async
ui.Errf("failed to decode tags: %s: %s", workflow.Name, err.Error())
}

parentIds := []string{env.ExecutionId()}
if env.Config().Execution.ParentIds != "" {
parentIds = append(strings.Split(env.Config().Execution.ParentIds, "/"), parentIds...)
}

var exec testkube.TestWorkflowExecution
for i := 0; i < CreateExecutionRetryOnFailureMaxAttempts; i++ {
exec, err = c.ExecuteTestWorkflow(workflow.Name, testkube.TestWorkflowExecutionRequest{
Name: workflow.ExecutionName,
Config: testworkflows.MapConfigValueKubeToAPI(workflow.Config),
DisableWebhooks: env.ExecutionDisableWebhooks(),
Tags: tags,
RunningContext: &testkube.TestWorkflowRunningContext{
Interface_: &testkube.TestWorkflowRunningContextInterface{
Type_: common.Ptr(testkube.API_TestWorkflowRunningContextInterfaceType),
},
Actor: &testkube.TestWorkflowRunningContextActor{
Name: workflow.Name,
ExecutionId: env.ExecutionId(),
ExecutionPath: strings.Join(parentIds, "/"),
Type_: common.Ptr(testkube.TESTWORKFLOW_TestWorkflowRunningContextActorType),
},
},
ParentExecutionIds: parentIds,
})
if err == nil {
break
Expand Down
1 change: 1 addition & 0 deletions cmd/tcl/testworkflow-toolkit/spawn/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ func CreateExecutionMachine(prefix string, index int64) (string, expressions.Mac
"scheduledAt": env.ExecutionScheduledAt().UTC().Format(constants.RFC3339Millis),
"disableWebhooks": env.ExecutionDisableWebhooks(),
"tags": env.ExecutionTags(),
"parentIds": env.Config().Execution.ParentIds,
})
}

Expand Down
1 change: 1 addition & 0 deletions cmd/testworkflow-toolkit/env/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ type envExecutionConfig struct {
DisableWebhooks bool `envconfig:"TK_DWH"`
Tags string `envconfig:"TK_TAG"`
Labels string `envconfig:"TK_LBL"`
ParentIds string `envconfig:"TK_PAR"`
}

type envSystemConfig struct {
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ require (
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
github.com/kelseyhightower/envconfig v1.4.0
github.com/kubepug/kubepug v1.7.1
github.com/kubeshop/testkube-operator v1.15.2-beta1.0.20240916120302-48b0ceed8c5d
github.com/kubeshop/testkube-operator v1.15.2-beta1.0.20241008095446-1943acbe384c
github.com/minio/minio-go/v7 v7.0.47
github.com/montanaflynn/stats v0.6.6
github.com/moogar0880/problems v0.1.1
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -393,8 +393,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kubepug/kubepug v1.7.1 h1:LKhfSxS8Y5mXs50v+3Lpyec+cogErDLcV7CMUuiaisw=
github.com/kubepug/kubepug v1.7.1/go.mod h1:lv+HxD0oTFL7ZWjj0u6HKhMbbTIId3eG7aWIW0gyF8g=
github.com/kubeshop/testkube-operator v1.15.2-beta1.0.20240916120302-48b0ceed8c5d h1:ZwsRzKUJ+pMC2pF47L/O06Vq8vOYZe17GO3N2rL9lCk=
github.com/kubeshop/testkube-operator v1.15.2-beta1.0.20240916120302-48b0ceed8c5d/go.mod h1:P47tw1nKQFufdsZndyq2HG2MSa0zK/lU0XpRfZtEmIk=
github.com/kubeshop/testkube-operator v1.15.2-beta1.0.20241008095446-1943acbe384c h1:gBWVDIYWFdjDM7kH6jczx0CyDIekhBZn5uOzTjJXtf0=
github.com/kubeshop/testkube-operator v1.15.2-beta1.0.20241008095446-1943acbe384c/go.mod h1:P47tw1nKQFufdsZndyq2HG2MSa0zK/lU0XpRfZtEmIk=
github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w=
github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4=
Expand Down
13 changes: 13 additions & 0 deletions internal/app/api/v1/testworkflows.go
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,19 @@ func (s *TestkubeAPI) ExecuteTestWorkflowHandler() fiber.Handler {
var errs []error

request.TestWorkflowExecutionName = strings.Clone(c.Query("testWorkflowExecutionName"))
if request.RunningContext == nil {
request.RunningContext = &testkube.TestWorkflowRunningContext{
Interface_: &testkube.TestWorkflowRunningContextInterface{
Type_: common.Ptr(testkube.API_TestWorkflowRunningContextInterfaceType),
},
Actor: &testkube.TestWorkflowRunningContextActor{
Name: "",
Username: "",
Email: "",
Type_: common.Ptr(testkube.PROGRAM_TestWorkflowRunningContextActorType),
},
}
}
concurrencyLevel := scheduler.DefaultConcurrencyLevel
workerpoolService := workerpool.New[testworkflowsv1.TestWorkflow, testkube.TestWorkflowExecutionRequest,
testkube.TestWorkflowExecution](concurrencyLevel)
Expand Down
18 changes: 10 additions & 8 deletions pkg/api/v1/testkube/model_running_context_extended.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@ package testkube
type RunningContextType string

const (
RunningContextTypeUserCLI RunningContextType = "user-cli"
RunningContextTypeUserUI RunningContextType = "user-ui"
RunningContextTypeTestSuite RunningContextType = "testsuite"
RunningContextTypeTestTrigger RunningContextType = "testtrigger"
RunningContextTypeScheduler RunningContextType = "scheduler"
RunningContextTypeTestExecution RunningContextType = "testexecution"
RunningContextTypeTestSuiteExecution RunningContextType = "testsuiteexecution"
RunningContextTypeEmpty RunningContextType = ""
RunningContextTypeUserCLI RunningContextType = "user-cli"
RunningContextTypeUserUI RunningContextType = "user-ui"
RunningContextTypeTestSuite RunningContextType = "testsuite"
RunningContextTypeTestWorkflow RunningContextType = "testworkflow"
RunningContextTypeTestTrigger RunningContextType = "testtrigger"
RunningContextTypeScheduler RunningContextType = "scheduler"
RunningContextTypeTestExecution RunningContextType = "testexecution"
RunningContextTypeTestSuiteExecution RunningContextType = "testsuiteexecution"
RunningContextTypeTestWorkflowExecution RunningContextType = "testworkflowexecution"
RunningContextTypeEmpty RunningContextType = ""
)
5 changes: 3 additions & 2 deletions pkg/api/v1/testkube/model_test_workflow_execution.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ type TestWorkflowExecution struct {
// test workflow execution name started the test workflow execution
TestWorkflowExecutionName string `json:"testWorkflowExecutionName,omitempty"`
// whether webhooks on the execution of this test workflow are disabled
DisableWebhooks bool `json:"disableWebhooks,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
DisableWebhooks bool `json:"disableWebhooks,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
RunningContext *TestWorkflowRunningContext `json:"runningContext,omitempty"`
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ type TestWorkflowExecutionRequest struct {
// test workflow execution name started the test workflow execution
TestWorkflowExecutionName string `json:"testWorkflowExecutionName,omitempty"`
// whether webhooks on the execution of this test workflow are disabled
DisableWebhooks bool `json:"disableWebhooks,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
DisableWebhooks bool `json:"disableWebhooks,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
RunningContext *TestWorkflowRunningContext `json:"runningContext,omitempty"`
// parent execution ids
ParentExecutionIds []string `json:"parentExecutionIds,omitempty"`
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ type TestWorkflowExecutionSummary struct {
// when the execution has been scheduled to run
ScheduledAt time.Time `json:"scheduledAt,omitempty"`
// when the execution result's status has changed last time (queued, passed, failed)
StatusAt time.Time `json:"statusAt,omitempty"`
Result *TestWorkflowResultSummary `json:"result,omitempty"`
Workflow *TestWorkflowSummary `json:"workflow"`
Tags map[string]string `json:"tags,omitempty"`
StatusAt time.Time `json:"statusAt,omitempty"`
Result *TestWorkflowResultSummary `json:"result,omitempty"`
Workflow *TestWorkflowSummary `json:"workflow"`
Tags map[string]string `json:"tags,omitempty"`
RunningContext *TestWorkflowRunningContext `json:"runningContext,omitempty"`
}
16 changes: 16 additions & 0 deletions pkg/api/v1/testkube/model_test_workflow_running_context.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* Testkube API
*
* Testkube provides a Kubernetes-native framework for test definition, execution and results
*
* API version: 1.0.0
* Contact: [email protected]
* Generated by: Swagger Codegen (https://github.com/swagger-api/swagger-codegen.git)
*/
package testkube

// running context for test workflow execution
type TestWorkflowRunningContext struct {
Interface_ *TestWorkflowRunningContextInterface `json:"interface"`
Actor *TestWorkflowRunningContextActor `json:"actor"`
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Testkube API
*
* Testkube provides a Kubernetes-native framework for test definition, execution and results
*
* API version: 1.0.0
* Contact: [email protected]
* Generated by: Swagger Codegen (https://github.com/swagger-api/swagger-codegen.git)
*/
package testkube

// running context actor for test workflow execution
type TestWorkflowRunningContextActor struct {
// actor name
Name string `json:"name,omitempty"`
// actor username
Username string `json:"username,omitempty"`
// actor email
Email string `json:"email,omitempty"`
// test workflow execution id
ExecutionId string `json:"executionId,omitempty"`
// all test workflow execution ids starting from the root
ExecutionPath string `json:"executionPath,omitempty"`
Type_ *TestWorkflowRunningContextActorType `json:"type"`
}
Loading
Loading