Repository navigation
fix(openai): anchor PLACEHOLDER_RE so valid summaries mentioning "placeholder" are kept - #74
Conversation
…ceholder" are kept
🤖 Augment PR SummarySummary: This PR narrows the OpenAI sentence placeholder filter to avoid discarding substantive summaries.
Technical notes: requestSentence returns null for filtered responses, and its callers record attempted timestamps while only persisting non-null descriptions.
🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
| // string instead of admitting they found nothing. Reject these the same way as a | ||
| // non-answer, rather than saving them as if they were a real result. | ||
| const PLACEHOLDER_RE = /placeholder|^(no|not) (specific |concrete |publicly )?(information|data|details) (is |was |)?(available|found)|^(i )?(could not|couldn't|was unable to) find/i; | ||
| const PLACEHOLDER_RE = /^placeholder\b|^(no|not) (specific |concrete |publicly )?(information|data|details) (is |was |)?(available|found)|^(i )?(could not|couldn't|was unable to) find/i; |
There was a problem hiding this comment.
This still drops valid 5–10-word answers that start with placeholder, such as Placeholder logo was replaced with a real project image; both callers then mark the item fetched without saving it or retrying. That sentence is permitted by the prompts and is the same substantive-summary case this change intends to preserve.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
There was a problem hiding this comment.
Addressed in 646347f: narrowed the leading-"placeholder" stand-in match to a bare token or one-word label so substantive sentences starting with "placeholder" (e.g. "Placeholder logo was replaced with a real project image") are kept; the regression tests now cover that and still reject bare/label stand-ins.
Narrow the leading-"placeholder" stand-in match to a bare token or one-word label so a real 5-10-word summary beginning with "placeholder" (e.g. "Placeholder logo was replaced with a real project image") is no longer dropped. Both callers mark the row fetched without retrying, so the dropped summary would be lost permanently.
1 participant
Detail bug report: View on Detail
Bug
requestSentence(src/openai/requestSentence.ts) filters eachsubmit_sentencetool-call answer throughPLACEHOLDER_REbefore returning it to the PR-summarization and repository-research tasks. The first alternative ofPLACEHOLDER_REwas the bare, unanchored wordplaceholder, soRegExp.testmatched the substring anywhere in any sentence — discarding legitimate substantive summaries that merely mention a placeholder (e.g. "Replaced the placeholder logo with a real project image").Both callers always set the
*FetchedAttimestamp but only persist the description when non-null, so a misclassified answer was routed into the intended "give up on this one" lane: the row left the retry queue with no summary saved and no retry attempted.Fix
Anchored the first alternative to
^placeholder\bso only a sentence that starts with the stand-in token is rejected:The two
^(…)branches were already anchored and are unchanged; substantive summaries that mention a placeholder mid-sentence are now kept. The\bretains the stand-in catch ("placeholder", "Placeholder description") while not matching a longer word such as "placeholders" — intentional, since the stand-in is terse and leading.Testing
src/openai/requestSentence.test.ts, covering thePLACEHOLDER_REkeep/drop matrix (substantive "placeholder" summaries kept; leading stand-ins and the "no information"/"couldn't find" branches still rejected),NON_ANSWER_REsanity, and the\btradeoff. Regression-proof: reverting the regex to the unanchored form makes the 5 bug-fix assertions (plus the\b-tradeoff assertion) fail; the fix returns them to passing.npx tsc --skipLibCheck --noEmit), and build (npm run build) all pass; the siblingopenai/format.test.tsandautomation/aiScoreStandups.test.tssuites still pass.fetchPullRequestvia the require cache. These confirmed thatgeneratePrDescriptionsandresearchRepositoryDescriptionsnow persist substantive "placeholder" sentences (previously dropped tonull), leading stand-ins are still dropped and marked terminal, and the PR caller still retries on error. Not included in the PR to keep the change focused on the one-line fix.@typescript-eslint/parser@3is incompatible withtypescript@5and emits a0:0 DeprecationErroron every.tsfile (confirmed against an untouched file). There is nolintscript and CI does not enforce lint; the change introduces no new lint issues.Automatic Fixes PRs can be configured here.