fix: prevent planner from generating invalid verification types (#1388) (#1529)

The planner agent was generating invalid verification types like `code_review`
because the prompt didn't explicitly constrain allowed types or document all
valid options.

Changes:
- Add explicit warning in planner.md that ONLY 6 types are valid
- Document all 6 verification types (was missing `none`)
- Add guidance: use `manual` for code review, `command` for tests
- Add `manual` and `none` verification instructions to coder.md
- Add handlers for `e2e`, `manual`, `none` in checklist_generator.py
- Add missing schema fields: `command`, `expected`, `instructions`
- Mark `component` as legacy (kept for backward compatibility)

Fixes #1388

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Andy <119136210+AndyMik90@users.noreply.github.com>
This commit is contained in:
kaigler
2026-01-27 02:51:55 -06:00
committed by GitHub
parent e9680e5119
commit 94d941333b
4 changed files with 42 additions and 5 deletions
+17 -1
View File
@@ -146,6 +146,22 @@ class ChecklistGenerator:
f"Test in browser: {verification.get('scenario', 'Check functionality')}"
)
elif ver_type == "command":
reminders.append(f"Run command: {verification.get('run', '')}")
reminders.append(
f"Run command: {verification.get('run', verification.get('command', ''))}"
)
elif ver_type == "e2e":
steps = verification.get("steps", [])
if steps:
reminders.append(
f"E2E verification: {len(steps)} steps to complete"
)
else:
reminders.append("E2E verification required")
elif ver_type == "manual":
reminders.append(
f"Manual check: {verification.get('instructions', 'Verify manually')}"
)
elif ver_type == "none":
pass # No reminder needed
return reminders
+13
View File
@@ -727,6 +727,19 @@ curl -X [method] [url] -H "Content-Type: application/json" -d '[body]'
# Use combination of API calls and browser automation
```
**Manual Verification:**
```
# For verification.type = "manual"
# Read the instructions field and perform the described check
# Mark subtask complete only after manual verification passes
```
**No Verification:**
```
# For verification.type = "none"
# No verification required - mark subtask complete after implementation
```
### FIX BUGS IMMEDIATELY
**If verification fails: FIX IT NOW.**
+7 -2
View File
@@ -365,13 +365,18 @@ Use ONLY these values for the `type` field in phases:
### Verification Types
**CRITICAL: ONLY these 6 verification types are valid. Any other type will cause validation failure.**
| Type | When to Use | Format |
|------|-------------|--------|
| `command` | CLI verification | `{"type": "command", "command": "...", "expected": "..."}` |
| `command` | CLI verification, running tests | `{"type": "command", "command": "...", "expected": "..."}` |
| `api` | REST endpoint testing | `{"type": "api", "method": "GET/POST", "url": "...", "expected_status": 200}` |
| `browser` | UI rendering checks | `{"type": "browser", "url": "...", "checks": [...]}` |
| `e2e` | Full flow verification | `{"type": "e2e", "steps": [...]}` |
| `manual` | Requires human judgment | `{"type": "manual", "instructions": "..."}` |
| `manual` | Human judgment, code review | `{"type": "manual", "instructions": "..."}` |
| `none` | No verification needed | `{"type": "none"}` |
**DO NOT invent types like `code_review`, `component`, `test`, `lint`, `build`. Use `manual` for human review, `command` for running tests.**
### Special Subtask Types
+5 -2
View File
@@ -72,21 +72,24 @@ IMPLEMENTATION_PLAN_SCHEMA = {
"required_fields": ["type"],
"optional_fields": [
"run",
"command",
"expected",
"url",
"method",
"expect_status",
"expect_contains",
"scenario",
"steps",
"instructions",
],
"verification_types": [
"command",
"api",
"browser",
"component",
"component", # Legacy - consider deprecating (use "command" with test)
"e2e",
"manual",
"none",
"e2e",
],
},
}