-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replaced internal services with pod watching. #39
Conversation
Before this PR, weavelets communicated with one another via a Kubernetes service. This had two problems. First, a weavelet would try to contact the service before any backing pods started, which led to crashes. Second, a weavelet would contact the service to get an IP address and then use this IP address for all future calls, effectively bypassing our load balancing entirely. This PR fixes the issues by directly watching pods and getting their IP addresses rather than using a service. I believe the internal services we generate are now not needed, but I need to double check and will delete it in a future PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great. Will let Robert review.
I also fixed a small bug when detecting if a change happened when a pod is deleted, and I merged two goroutines into one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Srdjan!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Thanks Michael.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Robert!
Before this PR, weavelets communicated with one another via a Kubernetes service. This had two problems. First, a weavelet would try to contact the service before any backing pods started, which led to crashes. Second, a weavelet would contact the service to get an IP address and then use this IP address for all future calls, effectively bypassing our load balancing entirely.
This PR fixes the issues by directly watching pods and getting their IP addresses rather than using a service. I believe the internal services we generate are now not needed, but I need to double check and will delete it in a future PR.