feat: get job run lineage summary function #456
Conversation
ahmadnaufal
force-pushed
the
feat/get-upstream-runs-contract
branch
from
3e910ef to
668b5ba
Compare
| jobWithLineage := queue[0] | ||
| queue = queue[1:] | ||
|
|
||
| if job, ok := jobsByName[jobWithLineage.JobName]; ok { |
There was a problem hiding this comment.
if jobsByName generated by traversing the lineage, this check is not required
| upstream.JobRuns = make(map[string]*scheduler.JobRunSummary) | ||
| } | ||
|
|
||
| upstreamJob, exists := jobsByName[upstream.JobName] |
| jobRunSummary := &scheduler.JobRunSummary{ | ||
| ScheduledAt: expectedRunSchedule, | ||
| } | ||
| upstream.JobRuns[scheduleKey] = jobRunSummary |
There was a problem hiding this comment.
does it mean we compute the job run lineage for each generated runs? if that so, this would be expensive
| jobNames = append(jobNames, jobName) | ||
| } | ||
|
|
||
| jobsByName, err := r.jobRepo.FindByNames(ctx, jobNames) |
There was a problem hiding this comment.
this can be called once, instead of recalling it again in 2 function enrichWithJobDetails and getJobsAndProjects
|
@ahmadnaufal GetJobRunLineage(Job, Schedule, JobUpstreams, PresetMap) JobRunLineage JobRunLineage JobSummary |
| return lineage, nil | ||
| } | ||
|
|
||
| func (r *LineageResolver) buildLineageStructure(ctx context.Context, jobSchedules []*scheduler.JobSchedule) ([]*scheduler.JobLineageSummary, error) { |
There was a problem hiding this comment.
note:
- avoid having function which returns partial information. We can use a different struct which only contain the relation, and use it for our next function, and so on until we get the JobLineageSummary
- if possible, improve the naming
|
closing this to use the new MR instead: #458 |
No description provided.