Skip to content

[WIP] Fix incomplete results in prompt optimization - #245

Closed
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/aw-fix-prompt-optimization-issue
Closed

[WIP] Fix incomplete results in prompt optimization#245
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/aw-fix-prompt-optimization-issue

Conversation

Copilot AI commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.


This section details on the original issue you should resolve

<issue_title>[aw] Prompt Optimization reported incomplete result</issue_title>
<issue_description>### Workflow Failure

Workflow: Prompt Optimization
Branch: main
Run: https://github.com/githubnext/gh-aw-wizard/actions/runs/33509001441

[!WARNING]
Task Could Not Be Completed: The agent reported that the task could not be performed due to an infrastructure or tool failure.

Reasons:

  • Required Ollama eval infrastructure was not actually available despite the task description. No open PR titled 'Prompt optimization:' exists, so I proceeded to set up evaluation, but it could not run.

    Findings:

  1. No Ollama server was running at (127.0.0.1/redacted) (connection refused) when the session started; I had to manually locate and start the binary at /opt/hostedtoolcache/ollama/0.33.2/x64/ollama myself.
  2. After starting it, ollama list / /api/tags showed zero models cached ({"models":[]}) — neither hf.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M nor hf.co/unsloth/SmolLM2-360M-Instruct-GGUF:Q4_K_M were present, contradicting the task's claim that both GGUF proxies had already been downloaded from Hugging Face.
  3. Outbound network to huggingface.co and ollama.com both timed out (curl exit 56 / HTTP 000), so I could not pull the models myself to work around the missing cache.
  4. Even disregarding the network/model issue, the ollama server log shows its GPU/CPU discovery failed because the llama-server backend binary is missing from the ollama installation ("llama-server binary not found... Run 'cmake -S llama/server --preset cpu && cmake --build --preset cpu' first"), so inference would not work even with models present.

Given no eval endpoint could serve either required model, I could not run scripts/prompt-optimizer.mjs --evaluate/--score, could not measure any incumbent or candidate scores, and therefore made no prompt changes, no PR, and no review. I stopped the ollama server process I had started (PID 794) before finishing.

This is a structured incompletion signal (report_incomplete), not a real task outcome. Any other safe outputs emitted alongside this signal (e.g., comments) describe the failure state, not a completed review or action.

Action Required

Assign this issue to an agent to debug and fix the issue.

Debug with any coding agent

Use this prompt with any coding agent (GitHub Copilot, Claude, Gemini, etc.):

Debug the agentic workflow failure using https://raw.githubusercontent.com/github/gh-aw/main/debug.md

The failed workflow run is at https://github.com/githubnext/gh-aw-wizard/actions/runs/33509001441
Manually invoke the agent

Debug this workflow failure using your favorite Agent CLI and the agentic-workflows prompt.

[!TIP]

Stop reporting this workflow as a failure

To stop a workflow from creating failure issues, set report-failure-as-issue: false in its frontmatter:

safe-outputs:
  report-failure-as-issue: false

Generated from Prompt Optimization · 29.2 AIC ·

  • expires on Sep 8, 2026, 12:47 PM UTC

Comments on the Issue (you are @copilot in this section)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Prompt Optimization reported incomplete result

2 participants