Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

int instead of size_t in HEAP_MEM #1982

Open
oleksandrSydorenkoJ opened this issue Sep 9, 2024 · 1 comment
Open

int instead of size_t in HEAP_MEM #1982

oleksandrSydorenkoJ opened this issue Sep 9, 2024 · 1 comment
Assignees
Labels
bug Something isn't working epic:HPD
Milestone

Comments

@oleksandrSydorenkoJ
Copy link

Describe the bug
Actual for core and indexer nodes
Related to https://github.com/skalenetwork/internal-support/issues/931

Versions
skalenetwork/schain:3.19.1

Preconditions:
Skale chain Medium type with 16 active nodes
Enabled MTM
Connected skale-proxy to the Skale chain
Connected at least 1 indexer node to the Skale Chain

To Reproduce

  1. Run load tests on 1 chain for 3 days
  2. Wait until the Skaled HEAP_MEM exceeds the 2137605504 bytes
  3. Check skaled logs

Expected behavior
Skaled should print the current HEAP_MEM size in the log according to the actual usage.

Actual state:
Skaled crashes with OOM when HEAP_MEM exceeds the size of 2137605504.

[2024-09-07 23:01:41.820] [8:main] [info] 1460627:RUSAGE:USER_CPU:82694:896343:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:2136553264:
[2024-09-07 23:02:41.348] [8:main] [info] 1460647:RUSAGE:USER_CPU:82704:47259:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2141510352:
[2024-09-07 23:03:40.929] [8:main] [info] 1460667:RUSAGE:USER_CPU:82712:804585:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:2147011456:
[2024-09-07 23:04:33.644] [8:main] [info] 1460687:RUSAGE:USER_CPU:82721:152020:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2133852704:
[2024-09-07 23:05:35.895] [8:main] [info] 1460707:RUSAGE:USER_CPU:82731:263291:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2143199120:
[2024-09-07 23:06:34.832] [8:main] [info] 1460727:RUSAGE:USER_CPU:82742:42802:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2133314416:
[2024-09-07 23:07:27.557] [8:main] [info] 1460747:RUSAGE:USER_CPU:82749:227899:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2129199312:
[2024-09-07 23:08:24.426] [8:main] [info] 1460767:RUSAGE:USER_CPU:82756:505229:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2129500560:
[2024-09-07 23:09:22.378] [8:main] [info] 1460787:RUSAGE:USER_CPU:82766:534775:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2132925968:
[2024-09-07 23:10:19.730] [8:main] [info] 1460807:RUSAGE:USER_CPU:82777:12142:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2116138928:
[2024-09-07 23:11:18.786] [8:main] [info] 1460827:RUSAGE:USER_CPU:82786:233081:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2127419952:
[2024-09-07 23:12:13.705] [8:main] [info] 1460847:RUSAGE:USER_CPU:82795:301516:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2130717328:
[2024-09-07 23:13:11.449] [8:main] [info] 1460867:RUSAGE:USER_CPU:82804:21806:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2118161120:
[2024-09-07 23:14:07.450] [8:main] [info] 1460887:RUSAGE:USER_CPU:82812:413539:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2123569008:
[2024-09-07 23:15:06.004] [8:main] [info] 1460907:RUSAGE:USER_CPU:82823:79012:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2116206400:
[2024-09-07 23:16:02.783] [8:main] [info] 1460927:RUSAGE:USER_CPU:82831:595735:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2111923632:
[2024-09-07 23:16:58.166] [8:main] [info] 1460947:RUSAGE:USER_CPU:82839:435290:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2100281200:
[2024-09-07 23:17:50.239] [8:main] [info] 1460967:RUSAGE:USER_CPU:82846:12259:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2104603488:
[2024-09-07 23:18:43.550] [8:main] [info] 1460987:RUSAGE:USER_CPU:82853:733131:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2090585952:
[2024-09-07 23:19:41.414] [8:main] [info] 1461007:RUSAGE:USER_CPU:82862:406490:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2090125616:
[2024-09-07 23:20:30.644] [8:main] [info] 1461027:RUSAGE:USER_CPU:82870:81147:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2099123824:
[2024-09-07 23:20:40.989] [8:main] [info] 1461047:RUSAGE:USER_CPU:82877:472221:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2106375392:
[2024-09-07 23:21:00.233] [8:main] [info] 1461067:RUSAGE:USER_CPU:82885:18391:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2091413712:
[2024-09-07 23:21:12.708] [8:main] [info] 1461087:RUSAGE:USER_CPU:82894:72138:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2082410896:
[2024-09-07 23:21:33.132] [8:main] [info] 1461107:RUSAGE:USER_CPU:82903:47809:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2089827616:
[2024-09-07 23:21:43.216] [8:main] [info] 1461127:RUSAGE:USER_CPU:82912:103523:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2073349312:
[2024-09-07 23:21:53.437] [8:main] [info] 1461147:RUSAGE:USER_CPU:82921:597710:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2054117520:
[2024-09-07 23:22:05.944] [8:main] [info] 1461167:RUSAGE:USER_CPU:82931:753048:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2127760960:
[2024-09-07 23:22:15.721] [8:main] [info] 1461187:RUSAGE:USER_CPU:82940:721089:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2134320576:
[2024-09-07 23:22:26.443] [8:main] [info] 1461207:RUSAGE:USER_CPU:82950:655641:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2112446576:
[2024-09-07 23:22:34.788] [8:main] [info] 1461227:RUSAGE:USER_CPU:82958:128740:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2102600704:
[2024-09-07 23:22:47.830] [8:main] [info] 1461247:RUSAGE:USER_CPU:82968:846830:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2140733296:
[2024-09-07 23:23:13.829] [8:main] [info] 1461267:RUSAGE:USER_CPU:82976:415155:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:2134827056:
[2024-09-07 23:23:59.178] [8:main] [info] 1461287:RUSAGE:USER_CPU:82985:606926:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2146329744:
[2024-09-07 23:24:35.791] [8:main] [info] 1461307:RUSAGE:USER_CPU:82994:859963:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2139914560:
[2024-09-07 23:25:24.913] [8:main] [info] 1461327:RUSAGE:USER_CPU:83006:419935:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:-2147226704:
[2024-09-07 23:25:56.387] [8:main] [info] 1461347:RUSAGE:USER_CPU:83015:405461:PROCESS_RSS:3465996:SWAPPED_MEM:0:HEAP_MEM:2137478368:
@dimalit
Copy link
Collaborator

dimalit commented Sep 20, 2024

It's already int and that is the problem: sizeof(int)==4. And it comes from struct mallinfo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working epic:HPD
Projects
Status: No status
Development

No branches or pull requests

4 participants