Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] gRPC自定义http2头信息Packet类型能采集到eBPF无法采集到,重启deepflow-server后无法Packet类型也无法采集到 #8237

Open
2 of 3 tasks
yang1992 opened this issue Sep 27, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@yang1992
Copy link

yang1992 commented Sep 27, 2024

Search before asking

  • I had searched in the issues and found no similar feature requirement.

DeepFlow Component

Agent

What you expected to happen

自定义http2头部信息,期望采集到l7_flow_log表对应的attribute_names和attribute_values字段,采集结果出现如下现象:

kprobe模式下:

1.grpc服务重启时间晚于deepflow-server,协议响应状态都是Success
Packet:使用 AF_PACKET + cBPF 采集的网卡流量数据可以采集到xxx-data,eBPF:只有很短一段时间能采集到,后面就不能采集到xxx-data,此刻重启grpc服务后Packet类型又可以采集到
2.grpc服务重启时间早于deepflow-server,协议响应状态都是Success
Packet采集不到xxx-data, eBPF也采集不到xxx-data
@yinjiping @sharang

How to reproduce

agent-group-config.yaml配置文件新增非http标准定义静态表header头信息,用于采集grpc返回结果错误码、错误信息:

l7-protocol-advanced-features:
      extra-log-fields:
        http2:
        - field-name: "xxx-data

使用grpc的UnaryServerInterceptor,invoke grpc请求,拦截返回response和err,拼凑成xxx-data,通过header返回:

func ExtractUnaryServerErrorInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo,
	handler grpc.UnaryHandler) (interface{}, error) {

	resp, err := handler(ctx, req)
	setTrailer(ctx, info, req, resp, err)
	return resp, err
}
func setTrailer(ctx context.Context, info *grpc.UnaryServerInfo, req interface{}, resp interface{}, err error) {
	//...... //
       //获取拦截返回response和err,拼凑成data
	md := metadata.Pairs("xxx-data", data)
	grpc.SetTrailer(ctx, md)
}

DeepFlow version

Name: deepflow-server community edition
Branch: v6.5
CommitID: 8555e64
RevCount: 10792
Compiler: go version go1.21.12 linux/amd64
CompileTime: 2024-08-01 15:36:18

Defaulted container "deepflow-agent" out of: deepflow-agent, configure-sysctl (init)
10793-74069ad272adad8fb53259b8d973830c0fdf7a58
Name: deepflow-agent community edition
Branch: v6.5
CommitId: 74069ad
RevCount: 10793
Compiler: rustc 1.77.1 (7cf61ebde 2024-03-27)
CompileTime: 2024-08-02 03:12:26

DeepFlow agent list

ID NAME TYPE CTRL_IP CTRL_MAC STATE GROUP EXCEPTIONS REVISION UPGRADE_REVISION
1 master-c57a7-0-V1 K8S_VM 10.107.19.118 fe:fc:fe:a4:5a:c0 NORMAL default v6.5 10793
2 master-15d62-0-V4 K8S_VM 10.2.24.51 fe:fc:fe:97:6d:f0 NORMAL default v6.5 10793
3 master-b17e6-2-V3 K8S_VM 10.2.24.53 fe:fc:fe:2c:18:f9 NORMAL default v6.5 10793
4 master-41874-1-V2 K8S_VM 10.2.24.52 fe:fc:fe:0c:2a:26 NORMAL default v6.5 10793

Kubernetes CNI

NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
calico-system calico-node 1 1 1 1 1 kubernetes.io/os=linux 12d
kube-system kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 12d
kube-system open-local-agent 1 1 1 1 1 12d

[root@master-15d62-0 fancy]# calicoctl version
Client Version: v3.26.1
Git commit: b1d192c95
Cluster Version: v3.26.1
Cluster Type: typha,kdd,k8s,operator,bgp,kubeadm

Operation-System/Kernel version

4.18.0-372.32.1.90.po1.x86_64

Anything else

http2动态表里面的header不能采集到,换成静态表里面不常用的header避免冲突,是否就没有上述问题了

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@yang1992 yang1992 added the bug Something isn't working label Sep 27, 2024
@yang1992 yang1992 changed the title [BUG] gRPC自定义http2头信息Packet类型能采集到eBPF无法采集到,重启deepflow-server后无法采Packet也无法采集到 [BUG] gRPC自定义http2头信息Packet类型能采集到eBPF无法采集到,重启deepflow-server后无法Packet类型也无法采集到 Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant