FIX: Three Bugs in async E2B code sandbox

[rasdani]

The previous implementation and #484 on async E2B sandbox have three bugs.

First in this line, verification_info is actually a list. See the list comprehension right above.

open-r1/src/open_r1/rewards.py

Line 373 in 9890a8d

rewards = run_async_from_sync(scripts, verification_info["language"])

Here also screenshot from pdb:

Fixed by collecting languages first and from then on passing languages: list[str] around.

Second bug is language was not passed to run_script.

open-r1/src/open_r1/rewards.py

Line 418 in 9890a8d

tasks = [run_script(sbx, script) for script in scripts]

Even though it expects language.

open-r1/src/open_r1/rewards.py

Line 430 in 9890a8d

async def run_script(sbx, script: str, language: str) -> float:

Third bug: see comment below.

I also added more specific error handling, as exceptions of these three bugs were handle too generally.

[rasdani]

Code rewards after fix. Previously got none.

[rasdani]

Even though we set a sandbox timeout of 30, sbx.run_code() still runs into its own default timeout of 300s.

Here's how I timed it:

async def run_script(sbx, script: str, language: str) -> float:
    try:
        start_time = time.time()
        execution = await sbx.run_code(script, language=language)
        end_time = time.time()
        print(f"Script execution time: {end_time - start_time} seconds")
    except e2b.TimeoutException as e:
        end_time = time.time()
        print(f"Script execution time: {end_time - start_time} seconds")
        print(f"TimeoutException from E2B executor: {e}")
        is_running = await sbx.is_running()
        print(f"{is_running=}")
        breakpoint()
        is_running = await sbx.is_running()
        print(f"{is_running=}")
        return 0.0
    except Exception as e:
        print(f"Error from E2B executor: {e}")
        return 0.0
    try:
        return float(execution.text)
    except (TypeError, ValueError):
        return 0.0

[rasdani]

On a more general note:

I trained with open-r1/verifiable-coding-problems-python_decontaminated. It has a long tail of examples with a lot of test cases (see plots in my new dataset).

Even though evaluate_code() limits time for subprocesses to 5s, I still created a derived dataset limiting test cases to at most six.
rasdani/verifiable-coding-problems-python_decontaminated_fewer_test_cases

Since script execution is the bottleneck, one can think about checking code with ruff first before executing it.
Unfortunately Python does not have a compiler like Rust, otherwise something like this would be feasible.

When I ran in to execution timeouts, I also noticed this was due to long code slop. Maybe discarding unreasonably long completions could also help.

[rasdani]

Also "problem_statement" key in open-r1/verifiable-coding-problems-python

does not match with

open-r1/src/open_r1/grpo.py

Line 189 in 9890a8d

prompt.append({"role": "user", "content": example["problem"]})

so recipes/Qwen2.5-1.5B-Instruct/grpo/config_demo_code.yaml is currently broken.

open-r1/verifiable-coding-problems-python_decontaminated works just fine though.