"dbt configuration is invalid : cannot pickle '_thread.RLock' object"

[DavidfNZ]

Expected behavior

Lineage should work: should be displayed in the lineage tab.

Actual behavior

The error "dbt configuration is invalid : cannot pickle '_thread.RLock' object" is repeatedly appearing in the Lineage tab in VScode, so lineage has stopped working completely.

Steps To Reproduce

Open existing dbt project that has codebase going back for several years, click on any model.

This was working fine for the last 10 months on the same codebase with previous versions of dbt Power User, seems to be broken only recently.

Log output/Screenshots

Operating System

Windows 10

dbt version

1.6.6

dbt Adapter

unsure

dbt Power User version

0.41.0

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[anandgupta42]

@DavidfNZ Can you run dbt clean, followed by dbt deps and then restart vscode?

[alangner]

We are observing the same problem, but not just since 0.41. It was kind of always there, but occurs only from time to time.
We are using 3 different dbt projects and it can happen in any of them. Our workaround is to just restart VSCode and usually it is ok afterwards.
I havn't been able to find out what causes this.

[alangner]

@anandgupta42 , just saw it again, here is the log output:

DBTProject:An error occurred while trying to refresh the project "data_platform" at file:///data/data-platform configuration [{"exception":{"type":{"name":"Exception","module":"builtins"},"message":"cannot pickle '_thread.RLock' object","args":["cannot pickle '_thread.RLock' object"],"format":["Exception: cannot pickle '_thread.RLock' object

"]},"traceback":{"lineno":101,"strack":["  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/node_python_bridge.py\", line 108, in <module>
    response = dict(type='exception', value=format_exception(t, e, tb))
"],"format":["  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/node_python_bridge.py\", line 101, in <module>
    _exec(_compile(data['code'], '<input>', 'exec'), _locals)
","  File \"<input>\", line 1, in <module>
","  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/dbt_core_integration.py\", line 401, in init_project
    raise Exception(str(e))
"]},"format":["Traceback (most recent call last):
","  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/dbt_core_integration.py\", line 396, in init_project
    self.create_parser()
","  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/dbt_core_integration.py\", line 428, in create_parser
    self.dbt = project_parser.load()
               ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/site-packages/dbt/parser/manifest.py\", line 435, in load
    self.parse_project(
","  File \"/opt/conda/lib/python3.12/site-packages/dbt/parser/manifest.py\", line 733, in parse_project
    parser.parse_file(block)
","  File \"/opt/conda/lib/python3.12/site-packages/dbt/parser/base.py\", line 484, in parse_file
    self.parse_node(file_block)
","  File \"/opt/conda/lib/python3.12/site-packages/dbt/parser/base.py\", line 445, in parse_node
    self.render_update(node, config)
","  File \"/opt/conda/lib/python3.12/site-packages/dbt/parser/models.py\", line 317, in render_update
    model_parser_copy = self.partial_deepcopy()
                        ^^^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/site-packages/dbt/parser/models.py\", line 462, in partial_deepcopy
    return ModelParser(deepcopy(self.project), self.manifest, deepcopy(self.root_project))
                       ^^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
","  File \"/opt/conda/lib/python3.12/copy.py\", line 151, in deepcopy
    rv = reductor(4)
         ^^^^^^^^^^^
","TypeError: cannot pickle '_thread.RLock' object
","
During handling of the above exception, another exception occurred:
","Traceback (most recent call last):
","  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/node_python_bridge.py\", line 101, in <module>
    _exec(_compile(data['code'], '<input>', 'exec'), _locals)
","  File \"<input>\", line 1, in <module>
","  File \"/root/.vscode-server/extensions/innoverio.vscode-dbt-power-user-0.41.4/dist/dbt_core_integration.py\", line 401, in init_project
    raise Exception(str(e))
","Exception: cannot pickle '_thread.RLock' object
"]}]

A simple restart of VSCode is usually helping to get rid of it. A dbt clean is not needed for us, but even when we tried it, we saw no change in how often this issue returns.