You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been facing super fast memory draining when using multithread + timeout-decorator in one of the methods of my class.
Solved by puting this line at the end of cancel method of _Timeout class of the lib:
def cancel(self):
"""Terminate any possible execution of the embedded function."""
if self.__process.is_alive():
self.__process.terminate()
self.__process.join() # LINE INSERTED
_raise_exception(timeout_exception, exception_message)
a .join() is necessary otherwise the spawned processes that timeouted via timeout_decorator were never releasing their memory after terminating, as pointed in this topic
EDIT 1
The above cancel() routine was giving me deadlock when calling .join()
upgrade to this next one, which will prevent them:
def cancel(self):
"""Terminate any possible execution of the embedded function."""
if self.__process.is_alive():
self.__process.terminate()
self.__process.join(0.1)
if not self.__queue.empty():
self.__queue.get()
if self.__process.is_alive():
self.__process.terminate()
self.__process.join()
_raise_exception(self.__timeout_exception, self.__exception_message)
EDIT 2
Ultimate better last solution found (no mem leaks/zombies/deadlocks)
def cancel(self):
"""Terminate any possible execution of the embedded function."""
while self.__process.is_alive():
self.__process.terminate()
self.__process.join(0.1)
_raise_exception(self.__timeout_exception, self.__exception_message)
EDIT 3 (no memory leaks/zombies/deadlocks) also force killing if non-responsive child process
import os
def cancel(self):
"""Terminate any possible execution of the embedded function."""
if self.__process.is_alive():
self.__process.terminate()
self.__process.join(0.1)
if self.__process.is_alive():
os.kill(self.__process.pid, signal.SIGKILL)
self.__process.join()
_raise_exception(self.__timeout_exception, self.__exception_message)
The text was updated successfully, but these errors were encountered:
owneroxxor
changed the title
Memory leakage when multithreading
Memory leakage when timeouting
Feb 23, 2019
I've been facing super fast memory draining when using multithread + timeout-decorator in one of the methods of my class.
Solved by puting this line at the end of cancel method of _Timeout class of the lib:
a .join() is necessary otherwise the spawned processes that timeouted via timeout_decorator were never releasing their memory after terminating, as pointed in this topic
EDIT 1
The above cancel() routine was giving me deadlock when calling .join()
upgrade to this next one, which will prevent them:
EDIT 2
Ultimate better last solution found (no mem leaks/zombies/deadlocks)
EDIT 3 (no memory leaks/zombies/deadlocks) also force killing if non-responsive child process
The text was updated successfully, but these errors were encountered: