Trace memory usage with logging #2708
Conversation
|
Build failed. ✔️ pre-commit SUCCESS in 1m 51s |
|
Build failed. ✔️ pre-commit SUCCESS in 1m 47s |
| def log_memory_stats(self) -> None: | ||
| gc.collect() | ||
| memory_usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss | ||
| logger.error(f"Memory usage: {memory_usage} (kb)") |
There was a problem hiding this comment.
do we need this as error level? It will end up in Sentry iirc
There was a problem hiding this comment.
No, I forgot to change it...
majamassarini
force-pushed
the
trace-memory-usage-in-stage
branch
2 times, most recently
from
93cc3a4 to
57f514f
Compare
|
Build failed. ✔️ pre-commit SUCCESS in 1m 47s |
majamassarini
force-pushed
the
trace-memory-usage-in-stage
branch
from
57f514f to
7b2593d
Compare
|
Build failed. ✔️ pre-commit SUCCESS in 1m 50s |
majamassarini
force-pushed
the
trace-memory-usage-in-stage
branch
from
7b2593d to
9509218
Compare
|
Build failed. ✔️ pre-commit SUCCESS in 1m 51s |
|
recheck |
Co-authored-by: Matej Focko <[email protected]>
majamassarini
force-pushed
the
trace-memory-usage-in-stage
branch
from
9509218 to
b61b539
Compare
|
recheck |
|
Build succeeded. ✔️ pre-commit SUCCESS in 1m 57s |
|
Build succeeded (gate pipeline). ✔️ pre-commit SUCCESS in 1m 49s |
Sadly
py-spywas only a time profiler and I wasn't able to find something that does not need changes in the code.I followed mainly this guide and this celery issue suggestions.
Deep profiling will be done just in stg, quicker resource check also in prod.