Add Job and Action tracking for runner with GRPC notifications by Copilot · Pull Request #1714 · TUM-Dev/gocast

Copilot · 2025-12-05T19:12:15Z

Motivation and Context

Track runner job state in the database so the UI can display current runner activity. Previously, job execution was ephemeral with no persistence layer for monitoring.

Description

Adds database-backed job and action tracking with GRPC notifications from runner to gocast main instance.

New Models & DAO:

model/job.go - Job entity with unified WorkState type (created/running/completed/failed/cancelled) and timestamps
model/action.go - Action entity with its own table, referencing jobs via foreign key, storing action type as string, WorkState status, timestamps, and LastError for error tracking
dao/job.go - CRUD operations for both jobs and actions, queries by job ID, stream ID, runner hostname, active status

Protobuf Extensions:

Unified WorkState enum used by both jobs and actions
JobUpdateNotification message for job state changes (status only, no error field)
ActionUpdateNotification message for individual action state changes (uses string for action_type, includes last_error)

Runner Changes:

RunAction() sends separate job and action status notifications
Job notifications at creation, running state, and completion/failure/cancellation
Action notifications for each action start, completion, and failure (errors tracked at action level)
Action type is stored as plain string (e.g., "stream", "mk_vod", "check_vod")
Safe stream version enum conversion with fallback to STREAM_VERSION_UNSPECIFIED

Manager Changes:

handleJobUpdate() persists job state on notification receipt
handleActionUpdate() persists action state, creating new action records or updating existing ones
Single protoWorkStateToModel() converter for the unified WorkState type

Model Relations:

Runner.Jobs - foreign key relationship to active jobs
Stream.Jobs - foreign key relationship to associated jobs
Job.Actions - foreign key relationship to actions within a job
RunnerDao.GetAllWithJobs() - preloads active jobs for UI queries

Steps for Testing

Prerequisites:

Runner connected to gocast instance
Database access to verify jobs and actions tables

Trigger a stream request to a runner
Verify job record created in jobs table with status created
Verify action records created in actions table for each action (stream, mk_vod, etc.)
Observe job status transitions: created → running → completed
Observe action status transitions: running → completed for each action
Cancel a stream mid-execution, verify job status becomes cancelled
Force an action failure, verify action status becomes failed with last_error populated, and job status becomes failed

Screenshots

N/A - Backend changes only. UI integration pending separate PR.

Original prompt

Look at the current implementation of the runner with the actions and jobs.

Depict this functionality in the database and create GRPC notifications from the runner and the gocast main instance, such that the database always has the current state so I can show this in the UI

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: SebiWrn <67778694+SebiWrn@users.noreply.github.com>

…m models Co-authored-by: SebiWrn <67778694+SebiWrn@users.noreply.github.com>

joschahenningsen · 2025-12-05T20:02:34Z

dao/migrations/202412050.go

@@ -0,0 +1,21 @@
+package migrations


@copilot remove this, migrations happen automatically

Removed the migration file in c1c0a2a.

joschahenningsen · 2025-12-05T20:02:42Z

dao/migrator.go

 			migrations.Migrate202212010(),
 			migrations.Migrate202212020(),
 			migrations.Migrate202301006(),
+			migrations.Migrate202412050(),


@copilot remove this as well

Removed in c1c0a2a - reverted migrator.go to its original state.

joschahenningsen · 2025-12-05T20:04:59Z

model/job.go

+type Job struct {
+	gorm.Model
+	// JobID is the unique identifier for the job (UUID from runner)
+	JobID string `gorm:"column:job_id;uniqueIndex;not null"`


@copilot This doesn't need to be primary key?

Changed in c1c0a2a - now using gorm.Model which provides an auto-incrementing ID as primary key, with JobID as a unique index.

I think it was fine as is

Keeping it as is - the current implementation with gorm.Model providing the auto-incrementing ID and JobID as a unique index works correctly.

joschahenningsen · 2025-12-05T20:05:25Z

model/job.go

+	// CompletedAt is the timestamp when the job completed (or failed)
+	CompletedAt *time.Time `gorm:"column:completed_at"`
+	// ErrorMessage is the error message if the job failed
+	ErrorMessage string `gorm:"column:error_message;type:text"`


@copilot LastError fits better.

We should have another table for actions that reference to their respective job and save these for each action.

Renamed to LastError and added separate Action model with its own actions table in c1c0a2a. Each action now references its job via JobID and stores its type as a string, status, timestamps, and LastError.

joschahenningsen · 2025-12-05T20:10:22Z

runner/runner.go

+func (r *Runner) getActionType(funcName string) protobuf.ActionType {
+	switch {
+	case strings.HasSuffix(funcName, "Stream"):
+		return protobuf.ActionType_ACTION_TYPE_STREAM
+	case strings.HasSuffix(funcName, "StreamEnd"):
+		return protobuf.ActionType_ACTION_TYPE_STREAM_END
+	case strings.HasSuffix(funcName, "MkVOD"):
+		return protobuf.ActionType_ACTION_TYPE_MK_VOD
+	case strings.HasSuffix(funcName, "CheckVoD"):
+		return protobuf.ActionType_ACTION_TYPE_CHECK_VOD
+	case strings.HasSuffix(funcName, "MkThumb"):
+		return protobuf.ActionType_ACTION_TYPE_MK_THUMB
+	default:
+		return protobuf.ActionType_ACTION_TYPE_UNSPECIFIED
+	}


Let's not have a new type for this. Just use the string here and in the database.

Changed to use string for action type in c1c0a2a. The getActionTypeString function now returns strings like "stream", "mk_vod" etc., and the protobuf uses string action_type instead of an enum.

…LastError, add Action model Co-authored-by: joschahenningsen <44805696+joschahenningsen@users.noreply.github.com>

joschahenningsen · 2025-12-05T20:53:55Z

pkg/runner_manager/manager.go

+// protoJobStatusToModel converts protobuf JobStatus to model JobStatus
+func protoJobStatusToModel(status protobuf.JobStatus) model.JobStatus {
+	switch status {
+	case protobuf.JobStatus_JOB_STATUS_CREATED:
+		return model.JobStatusCreated
+	case protobuf.JobStatus_JOB_STATUS_RUNNING:
+		return model.JobStatusRunning
+	case protobuf.JobStatus_JOB_STATUS_COMPLETED:
+		return model.JobStatusCompleted
+	case protobuf.JobStatus_JOB_STATUS_FAILED:
+		return model.JobStatusFailed
+	case protobuf.JobStatus_JOB_STATUS_CANCELLED:
+		return model.JobStatusCancelled
+	default:
+		return model.JobStatusCreated
+	}
+}
+
+// protoActionStatusToModel converts protobuf ActionStatus to model ActionStatus
+func protoActionStatusToModel(status protobuf.ActionStatus) model.ActionStatus {
+	switch status {
+	case protobuf.ActionStatus_ACTION_STATUS_RUNNING:
+		return model.ActionStatusRunning
+	case protobuf.ActionStatus_ACTION_STATUS_COMPLETED:
+		return model.ActionStatusCompleted
+	case protobuf.ActionStatus_ACTION_STATUS_FAILED:
+		return model.ActionStatusFailed
+	default:
+		return model.ActionStatusRunning
+	}
+}


@copilot Unify these into the same types WorkState.

Unified JobStatus and ActionStatus into a single WorkState type in 7deec80. Both Job and Action models now use WorkState, and there's a single protoWorkStateToModel converter function.

joschahenningsen · 2025-12-05T21:00:31Z

@copilot I removed the lastError from the job. That is more of an action thing. Please clean up the notifications and notify about action updates where required.

Co-authored-by: joschahenningsen <44805696+joschahenningsen@users.noreply.github.com>

Copilot · 2025-12-05T21:13:37Z

@copilot I removed the lastError from the job. That is more of an action thing. Please clean up the notifications and notify about action updates where required.

Removed LastError from the Job model and JobUpdateNotification protobuf in 750ad42. Errors are now tracked only at the action level via ActionUpdateNotification.

Initial plan

9444305

Copilot AI assigned Copilot and SebiWrn Dec 5, 2025

Copilot started work on behalf of SebiWrn December 5, 2025 19:12 View session

Copilot AI and others added 2 commits December 5, 2025 19:24

Add Job model, DAO, migration, and GRPC notifications for job tracking

8549dcc

Co-authored-by: SebiWrn <67778694+SebiWrn@users.noreply.github.com>

Fix stream version bounds check and add Jobs relation to Runner/Strea…

693afbc

…m models Co-authored-by: SebiWrn <67778694+SebiWrn@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add gRPC notifications for runner state synchronization~~ Add Job tracking for runner actions with GRPC notifications Dec 5, 2025

Copilot AI requested a review from SebiWrn December 5, 2025 19:36

Copilot finished work on behalf of SebiWrn December 5, 2025 19:36

joschahenningsen reviewed Dec 5, 2025

View reviewed changes

Copilot started work on behalf of joschahenningsen December 5, 2025 20:10 View session

Address PR review feedback: remove migration, rename ErrorMessage to …

c1c0a2a

…LastError, add Action model Co-authored-by: joschahenningsen <44805696+joschahenningsen@users.noreply.github.com>

Copilot AI changed the title ~~Add Job tracking for runner actions with GRPC notifications~~ Add Job and Action tracking for runner with GRPC notifications Dec 5, 2025

Copilot AI requested a review from joschahenningsen December 5, 2025 20:24

Copilot finished work on behalf of joschahenningsen December 5, 2025 20:24

joschahenningsen reviewed Dec 5, 2025

View reviewed changes

fixup

b772718

Copilot started work on behalf of joschahenningsen December 5, 2025 20:54 View session

Unify JobStatus and ActionStatus into WorkState type

7deec80

Co-authored-by: joschahenningsen <44805696+joschahenningsen@users.noreply.github.com>

Copilot AI requested a review from joschahenningsen December 5, 2025 21:06

Copilot finished work on behalf of joschahenningsen December 5, 2025 21:06

Copilot started work on behalf of joschahenningsen December 5, 2025 21:07 View session

Remove lastError from Job model; errors are tracked per-action only

750ad42

Co-authored-by: joschahenningsen <44805696+joschahenningsen@users.noreply.github.com>

Copilot finished work on behalf of joschahenningsen December 5, 2025 21:17

Conversation

Copilot AI commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Steps for Testing

Screenshots

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

joschahenningsen commented Dec 5, 2025

Uh oh!

Copilot AI commented Dec 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Dec 5, 2025 •

edited

Loading