Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vanflow: Duplicate message delivery per distinct router path #1365

Open
c-kruse opened this issue Jan 4, 2024 · 3 comments
Open

Vanflow: Duplicate message delivery per distinct router path #1365

c-kruse opened this issue Jan 4, 2024 · 3 comments
Labels
with-skupper Involves skupper-router usage through skupper

Comments

@c-kruse
Copy link

c-kruse commented Jan 4, 2024

As discussed in the Skupper Team g chat, I am observing repeated delivery of vanflow messages. This was uncovered by an unrelated skupper collector bug. Thanks for following along @ted-ross! I suspect this issue is in the router, but I didn't know where to start trying to reproduce it without the control plane.

To reproduce:

  • Set up Skupper version 1.5.1 running in three namespaces a, b and c and link all three.
skupper token create --namespace a --uses 2 token-a.yaml

skupper link create --namespace b token-a.yaml
skupper token create --namespace b token-b.yaml

skupper link create --namespace c token-a.yaml
skupper link create --namespace c token-b.yaml
  • Deploy this rough draft vanflow logging utility to each namespace to tail the logs. source.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vanflow-logging
  labels:
    app: vanflow-logging
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vanflow-logging
  template:
    metadata:
      labels:
        app: vanflow-logging
    spec:
      containers:
      - image: quay.io/ckruse/vanflow-tool:1.0-alpha3
        imagePullPolicy: Always
        name: main
        securityContext:
          runAsNonRoot: true
        volumeMounts:
        - mountPath: /etc/messaging/
          name: skupper-local-client
        args:
        - '-messaging-config=/etc/messaging/connect.json'
        - '-enable-flow-records=false'
        - 'log'
      volumes:
      - name: skupper-local-client
        secret:
          defaultMode: 420
          secretName: skupper-local-client
  • Tail the logs of the vanflow logging utilities as you interact with the cluster. Messages originating in the A namespace are logged once from the A namespace but twice from the B and C namespaces.
@jiridanek jiridanek added the with-skupper Involves skupper-router usage through skupper label Jan 4, 2024
@jiridanek
Copy link
Contributor

This is probably a workaround for this?

@c-kruse
Copy link
Author

c-kruse commented Jan 5, 2024

This is probably a workaround for this?

* [Fixes phantom process records observed in the console UI skupper#1341](https://github.com/skupperproject/skupper/pull/1341)

They are related @jiridanek - but I'd hesitate to call either a workaround. The flow collector should be idempotent (fix in skupper/pull/1341), and from what I understand vanflow messages should be delivered only once to each consumer with some reliability regardless of van topology (the topic brought up in this issue.)

@Karen-Schoener
Copy link
Collaborator

Hi, I'm interested in investigating this ticket. If there's any concern / issue with me investigating this ticket, please do not hesitate to let me know. Thanks, Karen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
with-skupper Involves skupper-router usage through skupper
Projects
None yet
Development

No branches or pull requests

3 participants