Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v2][storage] Create v2 query service to operate on otlp data model #6343

Merged
merged 42 commits into from
Dec 31, 2024
Merged
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ad0de07
Create Query Service V2 To Operate on OTEL Data Model
mahadzaryab1 Dec 11, 2024
24278bb
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 24, 2024
41f1d51
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 24, 2024
a33ab6c
Address Build Failures And Feedback
mahadzaryab1 Dec 24, 2024
ffad629
Fix ArchiveTrace And GetTraces To Operate On Iterators
mahadzaryab1 Dec 24, 2024
1d721ac
Change Adjuster To Work On Seq
mahadzaryab1 Dec 24, 2024
a6df503
Fix Error Capture
mahadzaryab1 Dec 24, 2024
60a875a
Address Feedback From PR Review
mahadzaryab1 Dec 25, 2024
d31a5e5
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 25, 2024
2c7a848
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 29, 2024
d32ba48
Run Linter
mahadzaryab1 Dec 29, 2024
ecab080
Update Query Service To Perform Adjustments
mahadzaryab1 Dec 30, 2024
9ee2d38
Fix Signature of ArchiveTrace Function
mahadzaryab1 Dec 30, 2024
57ef9e7
Add Some Unit Tests
mahadzaryab1 Dec 30, 2024
40d9a68
Add Missing License
mahadzaryab1 Dec 30, 2024
32491f5
Adjust Archive Trace
mahadzaryab1 Dec 30, 2024
3d6af5f
Aggregate Traces
mahadzaryab1 Dec 30, 2024
ab95ab7
Remove TODO Comment
mahadzaryab1 Dec 30, 2024
6737ca9
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 30, 2024
005d8d7
Use SpanIter
mahadzaryab1 Dec 30, 2024
7171785
Add Proceed Check
mahadzaryab1 Dec 30, 2024
e5a4b1b
Address Feedback From PR Review
mahadzaryab1 Dec 30, 2024
f887d09
Create Receive Traces Helper Function
mahadzaryab1 Dec 30, 2024
959d2d4
Add Unit Tests For Get Traces
mahadzaryab1 Dec 30, 2024
d468249
Fix Lint
mahadzaryab1 Dec 30, 2024
f6d157e
Add Unit Tests For Find Traces
mahadzaryab1 Dec 30, 2024
d2f1c95
Add Test For Archive Trace Writer Error
mahadzaryab1 Dec 31, 2024
6f69d63
Add Test For Archive Trace Writer Success
mahadzaryab1 Dec 31, 2024
9250285
Write Test For Get Traces In Archive Storage
mahadzaryab1 Dec 31, 2024
1e0c5b3
Move Tests To Separate Package To Simplify Naming
mahadzaryab1 Dec 31, 2024
c14fc4f
Fix Comment
mahadzaryab1 Dec 31, 2024
73fbb19
Add Test For Error In Get Trace
mahadzaryab1 Dec 31, 2024
5020f89
Use Flatten With Errors
mahadzaryab1 Dec 31, 2024
e29eafe
Add Test For Error In Get Trace Reader
mahadzaryab1 Dec 31, 2024
2c716bf
Cleanup Tests
mahadzaryab1 Dec 31, 2024
28e5fd0
Combine Tests For Brevity
mahadzaryab1 Dec 31, 2024
5ae23f2
Fix Lint
mahadzaryab1 Dec 31, 2024
a79bcac
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 31, 2024
5636427
Move Query Service To V2 Package
mahadzaryab1 Dec 31, 2024
529bcda
Fix Lint
mahadzaryab1 Dec 31, 2024
806a265
Drop V2 Suffix
mahadzaryab1 Dec 31, 2024
f163d55
Merge branch 'main' into query-service-v2
mahadzaryab1 Dec 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions cmd/query/app/querysvc/query_service_v2.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
// // Copyright (c) 2019 The Jaeger Authors.
// // SPDX-License-Identifier: Apache-2.0

package querysvc

import (
"context"
"errors"
"time"

"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/ptrace"
"go.uber.org/zap"

"github.com/jaegertracing/jaeger/model"
"github.com/jaegertracing/jaeger/model/adjuster"
"github.com/jaegertracing/jaeger/storage/spanstore"
"github.com/jaegertracing/jaeger/storage_v2/depstore"
"github.com/jaegertracing/jaeger/storage_v2/tracestore"
)

// TODO: Remove query_service.go and rename query_service_v2.go
// to query_service.go once all components have been migrated to
// operate on the OTEL data model.
var errNoArchiveSpanStorageV2 = errors.New("archive span storage was not configured")

const (
defaultMaxClockSkewAdjustV2 = time.Second
)
mahadzaryab1 marked this conversation as resolved.
Show resolved Hide resolved

// QueryServiceOptions has optional members of QueryService
type QueryServiceOptionsV2 struct {
ArchiveTraceReader tracestore.Reader
ArchiveTraceWriter tracestore.Writer
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved
Adjuster adjuster.Adjuster
}
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved

// StorageCapabilities is a feature flag for query service
type StorageCapabilitiesV2 struct {
mahadzaryab1 marked this conversation as resolved.
Show resolved Hide resolved
ArchiveStorage bool `json:"archiveStorage"`
// TODO: Maybe add metrics Storage here
// SupportRegex bool
// SupportTagFilter bool
Copy link
Collaborator Author

@mahadzaryab1 mahadzaryab1 Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yurishkuro do you have the context for these items? do we still want to do them?

}

// QueryService contains span utils required by the query-service.
type QueryServiceV2 struct {
traceReader tracestore.Reader
dependencyReader depstore.Reader
options QueryServiceOptionsV2
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved
}

// NewQueryService returns a new QueryService.
func NewQueryServiceV2(traceReader tracestore.Reader, dependencyReader depstore.Reader, options QueryServiceOptions) *QueryService {
qsvc := &QueryService{
traceReader: traceReader,
dependencyReader: dependencyReader,
options: options,
}

if qsvc.options.Adjuster == nil {
qsvc.options.Adjuster = adjuster.Sequence(StandardAdjusters(defaultMaxClockSkewAdjustV2)...)
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved
}
return qsvc

Check warning on line 64 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L54-L64

Added lines #L54 - L64 were not covered by tests
}
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved

// GetTrace is the queryService implementation of tracestore.Reader.GetTrace
mahadzaryab1 marked this conversation as resolved.
Show resolved Hide resolved
func (qs QueryServiceV2) GetTrace(ctx context.Context, traceID pcommon.TraceID) (ptrace.Traces, error) {
trace, err := qs.traceReader.GetTrace(ctx, traceID)
if errors.Is(err, spanstore.ErrTraceNotFound) {
mahadzaryab1 marked this conversation as resolved.
Show resolved Hide resolved
if qs.options.ArchiveTraceReader == nil {
return ptrace.NewTraces(), err
}
trace, err = qs.options.ArchiveTraceReader.GetTrace(ctx, traceID)

Check warning on line 74 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L68-L74

Added lines #L68 - L74 were not covered by tests
}
return trace, err

Check warning on line 76 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L76

Added line #L76 was not covered by tests
}

// GetServices is the queryService implementation of tracestore.Reader.GetServices
func (qs QueryServiceV2) GetServices(ctx context.Context) ([]string, error) {
return qs.traceReader.GetServices(ctx)

Check warning on line 81 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L80-L81

Added lines #L80 - L81 were not covered by tests
}

// GetOperations is the queryService implementation of tracestore.Reader.GetOperations
func (qs QueryServiceV2) GetOperations(
ctx context.Context,
query tracestore.OperationQueryParameters,
) ([]tracestore.Operation, error) {
return qs.traceReader.GetOperations(ctx, query)

Check warning on line 89 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L88-L89

Added lines #L88 - L89 were not covered by tests
}

// FindTraces is the queryService implementation of tracestore.Reader.FindTraces
func (qs QueryServiceV2) FindTraces(ctx context.Context, query tracestore.TraceQueryParameters) ([]ptrace.Traces, error) {
return qs.traceReader.FindTraces(ctx, query)

Check warning on line 94 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L93-L94

Added lines #L93 - L94 were not covered by tests
}

// ArchiveTrace is the queryService utility to archive traces.
func (qs QueryServiceV2) ArchiveTrace(ctx context.Context, traceID pcommon.TraceID) error {
if qs.options.ArchiveTraceWriter == nil {
return errNoArchiveSpanStorageV2
}
trace, err := qs.GetTrace(ctx, traceID)
if err != nil {
return err
}
return qs.options.ArchiveTraceWriter.WriteTraces(ctx, trace)

Check warning on line 106 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L98-L106

Added lines #L98 - L106 were not covered by tests
}

// Adjust applies adjusters to the trace.
func (qs QueryServiceV2) Adjust(trace *model.Trace) (*model.Trace, error) {
return qs.options.Adjuster.Adjust(trace)

Check warning on line 111 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L110-L111

Added lines #L110 - L111 were not covered by tests
}
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved

// GetDependencies implements depstore.Reader.GetDependencies
func (qs QueryServiceV2) GetDependencies(ctx context.Context, endTs time.Time, lookback time.Duration) ([]model.DependencyLink, error) {
return qs.dependencyReader.GetDependencies(ctx, depstore.QueryParameters{
StartTime: endTs.Add(-lookback),
EndTime: endTs,
})

Check warning on line 119 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L115-L119

Added lines #L115 - L119 were not covered by tests
}

// GetCapabilities returns the features supported by the query service.
func (qs QueryServiceV2) GetCapabilities() StorageCapabilities {
return StorageCapabilities{
ArchiveStorage: qs.options.hasArchiveStorage(),
}

Check warning on line 126 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L123-L126

Added lines #L123 - L126 were not covered by tests
mahadzaryab1 marked this conversation as resolved.
Show resolved Hide resolved
}

// InitArchiveStorage tries to initialize archive storage reader/writer if storage factory supports them.
func (opts *QueryServiceOptionsV2) InitArchiveStorage(
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved
archiveReader tracestore.Reader,
archiveWriter tracestore.Writer,
logger *zap.Logger) {
opts.ArchiveTraceReader = archiveReader
opts.ArchiveTraceWriter = archiveWriter

Check warning on line 135 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L133-L135

Added lines #L133 - L135 were not covered by tests
}
yurishkuro marked this conversation as resolved.
Show resolved Hide resolved

// hasArchiveStorage returns true if archive storage reader/writer are initialized.
func (opts *QueryServiceOptionsV2) hasArchiveStorage() bool {
return opts.ArchiveTraceReader != nil && opts.ArchiveTraceWriter != nil

Check warning on line 140 in cmd/query/app/querysvc/query_service_v2.go

View check run for this annotation

Codecov / codecov/patch

cmd/query/app/querysvc/query_service_v2.go#L139-L140

Added lines #L139 - L140 were not covered by tests
}
Loading