Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mem leak fix when new upstreams are added #7

Open
wants to merge 44 commits into
base: master
Choose a base branch
from

Conversation

cinquemb
Copy link

@cinquemb cinquemb commented Sep 8, 2020

In order to avoid append to the existing ngx_conf_t pool, this will create a temp ngx_conf_t pool in refresh_upstream, swap old one out and delete if upstreams change.

This also implements some of the changes similar to what https://github.com/GUI/nginx-upstream-dynamic-servers/pull/33/files (no need for exit_procces, using ngx_http_upstream_init_round_robin instead of init), but avoids having to count refs to pools to address the leak.

Still leaks mem, but way less now.

Sample after ~12 hours with 2 upstreams, and about 82 different nginx servers:

$ sudo cat /proc/2048/smaps > AfterMemInc.txt
$ diff -u BeforeMemInc.txt AfterMemInc.txt 
--- BeforeMemInc.txt	2020-09-08 12:13:22.750709681 +0000
+++ AfterMemInc.txt	2020-09-08 23:52:53.499870017 +0000
@@ -94,14 +94,14 @@
 Size:              13984 kB
 KernelPageSize:        4 kB
 MMUPageSize:           4 kB
-Rss:               13804 kB
-Pss:                8476 kB
+Rss:               13816 kB
+Pss:                8992 kB
 Shared_Clean:          0 kB
-Shared_Dirty:      10656 kB
+Shared_Dirty:       9648 kB
 Private_Clean:         0 kB
-Private_Dirty:      3148 kB
-Referenced:         5596 kB
-Anonymous:         13804 kB
+Private_Dirty:      4168 kB
+Referenced:         9160 kB
+Anonymous:         13816 kB
 LazyFree:              0 kB
 AnonHugePages:         0 kB
 ShmemPmdMapped:        0 kB
@@ -113,18 +113,18 @@
 THPeligible:		0
 ProtectionKey:         0
 VmFlags: rd wr mr mw me ac sd 
-55b92e53a000-55b92f1cc000 rw-p 00000000 00:00 0                          [heap]
-Size:              12872 kB
+55b92e53a000-55b937e4a000 rw-p 00000000 00:00 0                          [heap]
+Size:             156736 kB
 KernelPageSize:        4 kB
 MMUPageSize:           4 kB
-Rss:               12116 kB
-Pss:               12116 kB
+Rss:              140172 kB
+Pss:              140172 kB
 Shared_Clean:          0 kB
 Shared_Dirty:          0 kB
 Private_Clean:         0 kB
-Private_Dirty:     12116 kB
-Referenced:        12116 kB
-Anonymous:         12116 kB
+Private_Dirty:    140172 kB
+Referenced:       140172 kB
+Anonymous:        140172 kB
 LazyFree:              0 kB
 AnonHugePages:         0 kB
 ShmemPmdMapped:        0 kB

@cinquemb
Copy link
Author

Update after ~3 days with latest changes with no restarts in between:

$ sudo cat /proc/8787/smaps > AfterMemInc.txt
$ diff -u BeforeMemInc.txt AfterMemInc.txt  | head -n 100
--- BeforeMemInc.txt	2020-09-11 07:53:07.415237986 +0000
+++ AfterMemInc.txt	2020-09-14 02:17:49.481545858 +0000
@@ -94,14 +94,14 @@
 Size:              14064 kB
 KernelPageSize:        4 kB
 MMUPageSize:           4 kB
-Rss:               14044 kB
-Pss:                9146 kB
+Rss:               14060 kB
+Pss:                9314 kB
 Shared_Clean:          0 kB
-Shared_Dirty:       9796 kB
+Shared_Dirty:       9492 kB
 Private_Clean:         0 kB
-Private_Dirty:      4248 kB
-Referenced:         9432 kB
-Anonymous:         14044 kB
+Private_Dirty:      4568 kB
+Referenced:        10272 kB
+Anonymous:         14060 kB
 LazyFree:              0 kB
 AnonHugePages:         0 kB
 ShmemPmdMapped:        0 kB
@@ -113,18 +113,18 @@
 THPeligible:		0
 ProtectionKey:         0
 VmFlags: rd wr mr mw me ac sd 
-556bd4375000-556bd67b7000 rw-p 00000000 00:00 0                          [heap]
-Size:              37128 kB
+556bd4375000-556be6437000 rw-p 00000000 00:00 0                          [heap]
+Size:             295688 kB
 KernelPageSize:        4 kB
 MMUPageSize:           4 kB
-Rss:               33060 kB
-Pss:               33060 kB
+Rss:              273152 kB
+Pss:              273152 kB
 Shared_Clean:          0 kB
 Shared_Dirty:          0 kB
 Private_Clean:         0 kB
-Private_Dirty:     33060 kB
-Referenced:        33060 kB
-Anonymous:         33060 kB
+Private_Dirty:    273152 kB
+Referenced:       273152 kB
+Anonymous:        273152 kB
 LazyFree:              0 kB
 AnonHugePages:         0 kB
 ShmemPmdMapped:        0 kB
@@ -136,6 +136,180004 @@

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant