AttributeError: Can't pickle local object 'check_correctness.<locals>.unsafe_execute'

[tianzhaotju]

When I run "evaluate_functional_correctness sample.jsonl --problem_file=problem.jsonl", it has the following problem.

Can u help me? thx
Detail log.

---

Reading samples...
1it [00:00, 2118.34it/s]
Running test suites...
0%| | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/Users/user/opt/anaconda3/envs/py38/bin/evaluate_functional_correctness", line 33, in
sys.exit(load_entry_point('human-eval', 'console_scripts', 'evaluate_functional_correctness')())
File "/Users/user/opt/anaconda3/envs/py38/bin/evaluate_functional_correctness", line 25, in importlib_load_entry_point
return next(matches).load()
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 843, in exec_module
File "", line 219, in _call_with_frames_removed
File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 28, in
sys.exit(main())
File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 25, in main
fire.Fire(entry_point)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 20, in entry_point
results = evaluate_functional_correctness(sample_file, k, n_workers, timeout, problem_file)
File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluation.py", line 75, in evaluate_functional_correctness
result = future.result()
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/execution.py", line 73, in check_correctness
p.start()
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'check_correctness..unsafe_execute'

[zhouyangen]

你可以参考18楼的回复

[emrgnt-cmplxty]

Seeing the same error, what is the fix?

[Fazziekey]

Same problem

[Fazziekey]

@tianzhaotju @emrgnt-cmplxty I have fixed this problem in my repo with pull request #30

[maximw99]

Hello Devs, i have encountered this issue too. Well for me the suggested fix does not work at all. My workaround is to use a Linux Distro under WSL-2 or just by itself. For me it straight clicks, and everything works fine. I am not sure why the problem is present only on my windows system. Hope I could help.

[jckpn]

Also getting this error on macOS Sonoma, but it runs fine in a Windows VM.