Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Traces received from k6 but not real user traffic #99

Open
agardnerIT opened this issue Aug 17, 2023 · 2 comments
Open

Traces received from k6 but not real user traffic #99

agardnerIT opened this issue Aug 17, 2023 · 2 comments

Comments

@agardnerIT
Copy link
Collaborator

agardnerIT commented Aug 17, 2023 •
edited
Loading

I can see traces in Jaeger from the k6 traffic but no traces when I hit the demo app via a browser.

image

Flag value changes to the CR are respected and hte application responds accordingly - so the flags are working.

Impact: Observability not app functionality.

The OFO controller manager has these logs:

2023-08-17T00:18:39Z	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": "127.0.0.1:8080"}
2023-08-17T00:18:39Z	INFO	controller-runtime.builder	skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called	{"GVK": "core.openfeature.dev/v1alpha1, Kind=FeatureFlagConfiguration"}
2023-08-17T00:18:39Z	INFO	controller-runtime.builder	skip registering a validating webhook, object does not implement admission.Validator or WithValidator wasn't called	{"GVK": "core.openfeature.dev/v1alpha1, Kind=FeatureFlagConfiguration"}
2023-08-17T00:18:39Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/convert"}
2023-08-17T00:18:39Z	INFO	controller-runtime.builder	Conversion webhook enabled	{"GVK": "core.openfeature.dev/v1alpha1, Kind=FeatureFlagConfiguration"}
2023-08-17T00:18:39Z	INFO	controller-runtime.builder	skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called	{"GVK": "core.openfeature.dev/v1alpha1, Kind=FlagSourceConfiguration"}
2023-08-17T00:18:39Z	INFO	controller-runtime.builder	skip registering a validating webhook, object does not implement admission.Validator or WithValidator wasn't called	{"GVK": "core.openfeature.dev/v1alpha1, Kind=FlagSourceConfiguration"}
2023-08-17T00:18:39Z	INFO	controller-runtime.builder	Conversion webhook enabled	{"GVK": "core.openfeature.dev/v1alpha1, Kind=FlagSourceConfiguration"}
2023-08-17T00:18:39Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/mutate-v1-pod"}
2023-08-17T00:18:39Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/validate-v1alpha1-featureflagconfiguration"}
2023-08-17T00:18:39Z	INFO	setup	starting manager
2023-08-17T00:18:39Z	INFO	setup	restoring flagd-kubernetes-sync cluster role binding subjects from current cluster state
2023-08-17T00:18:39Z	INFO	controller-runtime.webhook.webhooks	Starting webhook server
2023-08-17T00:18:39Z	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
2023-08-17T00:18:39Z	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
2023-08-17T00:18:39Z	INFO	controller-runtime.certwatcher	Updated current TLS certificate
2023-08-17T00:18:39Z	INFO	controller-runtime.webhook	Serving webhook server	{"host": "", "port": 9443}
2023-08-17T00:18:39Z	INFO	controller-runtime.certwatcher	Starting certificate watcher
I0817 00:18:39.360130       1 leaderelection.go:248] attempting to acquire leader lease open-feature-operator-system/131bf64c.openfeature.dev...
I0817 00:18:39.375002       1 leaderelection.go:258] successfully acquired lease open-feature-operator-system/131bf64c.openfeature.dev
2023-08-17T00:18:39Z	DEBUG	events	open-feature-operator-controller-manager-5db85c65b4-4vc4n_fb61fa13-7901-4865-808c-66f8766d5b2e became leader	{"type": "Normal", "object": {"kind":"Lease","namespace":"open-feature-operator-system","name":"131bf64c.openfeature.dev","uid":"d23c05f6-9e92-4230-8bfd-b7e7e2804e4f","apiVersion":"coordination.k8s.io/v1","resourceVersion":"1767"}, "reason": "LeaderElection"}
2023-08-17T00:18:39Z	INFO	Starting EventSource	{"controller": "featureflagconfiguration", "controllerGroup": "core.openfeature.dev", "controllerKind": "FeatureFlagConfiguration", "source": "kind source: *v1alpha1.FeatureFlagConfiguration"}
2023-08-17T00:18:39Z	INFO	Starting EventSource	{"controller": "featureflagconfiguration", "controllerGroup": "core.openfeature.dev", "controllerKind": "FeatureFlagConfiguration", "source": "kind source: *v1.ConfigMap"}
2023-08-17T00:18:39Z	INFO	Starting Controller	{"controller": "featureflagconfiguration", "controllerGroup": "core.openfeature.dev", "controllerKind": "FeatureFlagConfiguration"}
2023-08-17T00:18:39Z	INFO	Starting EventSource	{"controller": "flagsourceconfiguration", "controllerGroup": "core.openfeature.dev", "controllerKind": "FlagSourceConfiguration", "source": "kind source: *v1alpha1.FlagSourceConfiguration"}
2023-08-17T00:18:39Z	INFO	Starting Controller	{"controller": "flagsourceconfiguration", "controllerGroup": "core.openfeature.dev", "controllerKind": "FlagSourceConfiguration"}
2023-08-17T00:18:39Z	INFO	Starting workers	{"controller": "featureflagconfiguration", "controllerGroup": "core.openfeature.dev", "controllerKind": "FeatureFlagConfiguration", "worker count": 1}
2023-08-17T00:49:57Z	DEBUG	controller-runtime.webhook.webhooks	wrote response	{"webhook": "/validate-v1alpha1-featureflagconfiguration", "code": 200, "reason": "", "UID": "2c0fd772-20e4-45ce-b874-0f5e4fcac8b4", "allowed": true}
2023-08-17T00:49:57Z	INFO	FeatureFlagConfiguration Controller	ReconcilingFeatureFlagConfiguration
2023-08-17T00:49:57Z	INFO	FeatureFlagConfiguration Controller	Finished Reconciling FeatureFlagConfiguration
2023/08/17 00:49:57 http: TLS handshake error from 10.244.0.1:61265: EOF
2023-08-17T00:49:57Z	INFO	FlagSourceConfiguration Controller	Searching for FlagSourceConfiguration
2023-08-17T00:49:57Z	INFO	FlagSourceConfiguration Controller	Finished Reconciling FlagSourceConfiguration
2023/08/17 00:49:57 http: TLS handshake error from 10.244.0.1:31466: EOF
2023/08/17 00:49:57 http: TLS handshake error from 10.244.0.1:1572: EOF

The flagd sidecar in the demo-app pod has these logs (looks successful):

		 ______   __       ________   _______    ______		/_____/\ /_/\     /_______/\ /______/\  /_____/\		\::::_\/_\:\ \    \::: _  \ \\::::__\/__\:::_ \ \		 \:\/___/\\:\ \    \::(_)  \ \\:\ /____/\\:\ \ \ \		  \:::._\/ \:\ \____\:: __  \ \\:\\_  _\/ \:\ \ \ \		   \:\ \    \:\/___/\\:.\ \  \ \\:\_\ \ \  \:\/.:| |		    \_\/     \_____\/ \__\/\__\/ \_____\/   \____/_/
2023-08-17T00:33:18.661Z	info	cmd/start.go:117	flagd version: v0.6.3 (bf74c5a3fc1b52108c2b057b3680576a90f31453), built at: 2023-08-04	{"component": "start"}2023-08-17T00:33:18.662Z	info	flag-evaluation/connect_service.go:199	metrics and probes listening at 8014	{"component": "service"}2023-08-17T00:33:18.662Z	info	kubernetes/kubernetes_sync.go:110	starting kubernetes sync notifier for resource: demo/featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}2023-08-17T00:33:18.663Z	info	flag-evaluation/connect_service.go:179	Flag Evaluation listening at [::]:8030	{"component": "service"}2023-08-17T00:33:18.695Z	info	kubernetes/kubernetes_sync.go:218	kube sync notifier event: add: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:34:18.699Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:35:18.699Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:36:18.700Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}2023-08-17T00:37:18.701Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:38:18.701Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:39:18.704Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:39:28.274Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:40:18.706Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:41:18.707Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:42:18.708Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:43:18.709Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:44:18.709Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:45:18.712Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:46:18.713Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:47:18.713Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:48:18.713Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:49:18.716Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:49:57.757Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:50:18.717Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:51:18.718Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:52:18.719Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:53:18.719Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
1023-08-17T00:54:18.720Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:55:18.720Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}
2023-08-17T00:56:18.723Z	info	kubernetes/kubernetes_sync.go:224	kube sync notifier event: update: demo featureflagconfiguration-sample	{"component": "sync", "sync": "kubernetes"}

The demo-app container itself has these logs:

Connecting to flagD at localhost:8030
Example app listening on port 3000
@agardnerIT agardnerIT changed the title Traces received from k6 but not browser Traces received from k6 but not real user traffic Aug 17, 2023
@agardnerIT
Copy link
Collaborator Author

Can verify that a curl to http://demoapp.demo from a pod within the demo namespace does produce a trace. But a request from a browser (via the port-forwarded browser) doesn't.

Why?

@agardnerIT agardnerIT mentioned this issue Aug 17, 2023
Open
@agardnerIT
Copy link
Collaborator Author

image

Single request using port-forward means a single request to the backend - but no traces

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@agardnerIT