can not trace log_func_args when import torch

[yihong0618]

os: ubuntu20.04 and MacOS m1 both test

cat foo.py

import torch

def bar():
    return 3


bar()

run this command the viztracer will hang

viztracer --log_func_args   a.py

use py-spy to check the stack will see as follows:

seems loop in getattr (torch/_ops.py:480)

....
....
....
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __getattr__ (torch/_ops.py:480)
    __repr__ (torch/_ops.py:434)
    __init__ (torch/_ops.py:422)
    __getattr__ (torch/_ops.py:581)
    <module> (torch/fx/node.py:35)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/fx/graph.py:2)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/fx/graph_module.py:8)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/fx/__init__.py:83)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/ao/quantization/utils.py:12)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/ao/quantization/observer.py:15)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/ao/quantization/fake_quantize.py:8)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/ao/quantization/__init__.py:3)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:992)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:992)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/quantization/quantize.py:10)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (torch/quantization/__init__.py:1)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    _handle_fromlist (<frozen importlib._bootstrap>:1078)
    <module> (torch/__init__.py:1288)
    _call_with_frames_removed (<frozen importlib._bootstrap>:241)
    exec_module (<frozen importlib._bootstrap_external>:883)
    _load_unlocked (<frozen importlib._bootstrap>:688)
    _find_and_load_unlocked (<frozen importlib._bootstrap>:1006)
    _find_and_load (<frozen importlib._bootstrap>:1027)
    <module> (a.py:2)
    run_code (viztracer/main.py:334)
    run_command (viztracer/main.py:417)
    run (viztracer/main.py:301)
    main (viztracer/main.py:566)
    <module> (viztracer:8)

[yihong0618]

after some search the root cause seems from PyObject_Repr(PyDict_GetItem(locals, name));
the repr and getattar caused the recursive.
but I am not sure it needs fix on viztracer or pytorch side.

@gaogaotiantian

[gaogaotiantian]

It seems like using getattr on that object will cause infinite recursion? If this could be reproduced without viztracer, maybe it’s not a viztracer issue. Is there a note in torch for this? That users should not do it?

[yihong0618]

not exactly I did some search and no relate issues on pytorch.
but in cpython I found this one python/cpython#86075 similar but not the same thing.
I will try use bare sys.settrace to test later