[DNM] Revert memory leak fixes #603

Job	Run time
ci-data	4s
Compile native gem for aarch64-linux	1m 39s
Compile native gem for aarch64-linux-musl	1m 42s
Compile native gem for arm64-darwin	1m 50s
Compile native gem for x64-mingw-ucrt	1m 50s
Compile native gem for x86_64-darwin	1m 47s
Compile native gem for x86_64-linux	5m 11s
Compile native gem for x86_64-linux-musl	2m 1s
	16m 4s

Provide feedback