Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure Readiness Probe Accuracy for devserver #4541

Open
DevineLiu opened this issue Jan 3, 2025 · 0 comments · May be fixed by #4544
Open

Ensure Readiness Probe Accuracy for devserver #4541

DevineLiu opened this issue Jan 3, 2025 · 0 comments · May be fixed by #4544

Comments

@DevineLiu
Copy link

DevineLiu commented Jan 3, 2025

eg:

func main() {
	flag.Parse()
	var c config.Config
	conf.MustLoad(*configFile, &c)
	// When executing MustNewServer, the dev HTTP server starts
	server := rest.MustNewServer(c.RestConf)
	defer server.Stop()
	// Sleeping for 100 seconds is used to simulate scenarios like loading cache data
	// During this 100-second sleep, if you curl http://xx.xxx.xx:xxx/ready, it will return 200 OK
	// However, at this point, the REST server is not fully ready,
	// as health.AddProbe(healthManager) has not been executed
	time.Sleep(100 * time.Second)
	ctx := svc.NewServiceContext(&c)
	handler.RegisterHandlers(server, ctx)
	fmt.Printf("Starting server at %s:%d...\n", c.Host, c.Port)
	// health.AddProbe(healthManager) is executed only in start()
	// I think health.AddProbe(healthManager) should be moved into NewServer
	server.Start()
}

I think health.AddProbe should be added in NewServer, and then the server should be marked as ready in Start.
Otherwise, the /ping endpoint of the devserver will be ready even when the rest service is not ready.

A typical case is when we use the devserver /ping endpoint as the readiness probe for Kubernetes pods.
If the rest server is not actually ready, traffic will be routed to the service prematurely.

ref:

health.AddProbe(s.healthManager)

health.AddProbe(healthManager)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant