Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Flaky Test]: TestFakeInputSuite/TestManager_Chunk– component missed 3 check-ins #5980

Open
AndersonQ opened this issue Nov 8, 2024 · 1 comment
Labels
flaky-test Unstable or unreliable test cases. Team:Elastic-Agent Label for the Agent team

Comments

@AndersonQ
Copy link
Member

AndersonQ commented Nov 8, 2024

Failing test case

TestFakeInputSuite/TestManager_Chunk

Error message

the fake component takes too long to check in

Build

https://buildkite.com/elastic/elastic-agent/builds/13872#01930777-c7d8-4532-9110-854550ba2fbd

OS

Mac

Stacktrace and notes

It seems CI might be too slow and cause the fake component to take too long to check in and therefore the agent counts them as missed.

Increasing the Checkin timeout should do the trick:

=== FAIL: pkg/component/runtime TestFakeInputSuite/TestManager_Chunk (184.61s)
2024-11-07 18:38:39 CEST	{"log.level":"info","@timestamp":"2024-11-07T09:18:38.968-0800","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.NewManager","file.name":"runtime/manager.go","file.line":175},"message":"GRPC comms socket listening at localhost:0","address":"localhost:0","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	{"log.level":"info","@timestamp":"2024-11-07T09:18:38.985-0800","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*Manager).Run","file.name":"runtime/manager.go","file.line":238},"message":"Starting grpc control protocol listener on port 0 with max_message_size 8388608","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	{"log.level":"debug","@timestamp":"2024-11-07T09:19:38.123-0800","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*Manager).update","file.name":"runtime/manager.go","file.line":823},"message":"Starting component \"fake-default\"","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	{"log.level":"info","@timestamp":"2024-11-07T09:19:41.339-0800","log.logger":"component.runtime.fake-default","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*Manager).CheckinV2","file.name":"runtime/manager.go","file.line":704},"message":"control checkin v2 protocol has chunking enabled","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	{"log.level":"debug","@timestamp":"2024-11-07T09:20:41.250-0800","log.logger":"component.runtime.fake-default","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*commandRuntime).Run","file.name":"runtime/command.go","file.line":243},"message":"Last check-in was: 2024-11-07T09:20:06.344941-08:00, now is: 2024-11-07T09:20:41.249142-08:00. The diff 34.904198389s is higher than allowed 30s.","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	{"log.level":"debug","@timestamp":"2024-11-07T09:21:11.249-0800","log.logger":"component.runtime.fake-default","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*commandRuntime).Run","file.name":"runtime/command.go","file.line":243},"message":"Last check-in was: 2024-11-07T09:20:06.344941-08:00, now is: 2024-11-07T09:21:11.249165-08:00. The diff 1m4.904219566s is higher than allowed 30s.","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	{"log.level":"debug","@timestamp":"2024-11-07T09:21:41.251-0800","log.logger":"component.runtime.fake-default","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*commandRuntime).Run","file.name":"runtime/command.go","file.line":243},"message":"Last check-in was: 2024-11-07T09:20:06.344941-08:00, now is: 2024-11-07T09:21:41.251023-08:00. The diff 1m34.906076144s is higher than allowed 30s.","ecs.version":"1.6.0"}
2024-11-07 18:38:39 CEST	    manager_fake_input_test.go:2894:
2024-11-07 18:38:39 CEST	        	Error Trace:	/Users/admin/builds/bk-agent-prod-orka-1730997132055584208/elastic/elastic-agent/pkg/component/runtime/manager_fake_input_test.go:2894
2024-11-07 18:38:39 CEST	        	Error:      	Received unexpected error:
2024-11-07 18:38:39 CEST	        	            	component failed: Failed: pid '10899' missed 3 check-ins and will be killed
2024-11-07 18:38:39 CEST	        	Test:       	TestFakeInputSuite/TestManager_Chunk
2024-11-07 18:38:39 CEST	{"log.level":"debug","@timestamp":"2024-11-07T09:21:42.389-0800","log.origin":{"function":"github.com/elastic/elastic-agent/pkg/component/runtime.(*Manager).update","file.name":"runtime/manager.go","file.line":796},"message":"Stopping component \"fake-default\"","ecs.version":"1.6.0"}
@AndersonQ AndersonQ added flaky-test Unstable or unreliable test cases. Team:Elastic-Agent Label for the Agent team labels Nov 8, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Unstable or unreliable test cases. Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

No branches or pull requests

2 participants