Files
Aperant/apps/backend/runners/github/services/sdk_utils.py
T
Andy bfc232825b feat(pr-review): evidence-based validation and trigger-driven exploration (#1593)
* gsd update

* docs: define v1 requirements with holistic PR understanding

29 requirements across 6 categories:
- Holistic PR Understanding (5) - context synthesis and passing
- Validation Pipeline (3) - finding-validator for all reviews
- Schema Enforcement (5) - VerificationEvidence required
- Prompt Improvements (6) - understand intent, evidence requirements
- Code Simplification (6) - remove programmatic filters
- Measurement (4) - 5 PRs to validate

Key addition: Pass gathered context (related files, import graph) to specialists.
Currently gathered but unused.

* feat(01-01): add Phase 0 synthesis instruction to orchestrator prompt

- Add 'Phase 0: Understand the PR Holistically' section before Phase 1
- Include PR UNDERSTANDING output format (intent, critical changes, risk areas, files to verify)
- Add explicit gate: 'Only AFTER completing Phase 0, proceed to Phase 1'
- Add 'Understand First' principle to Key Principles section

Covers: CONTEXT-01, CONTEXT-05

* feat(01-01): add related files and import graph to orchestrator prompt

- Add related files section categorizing tests vs dependencies/callers
- Add import graph section showing what files import/are imported by changed files
- Limit to 30 related files (15 tests, 15 deps) and 20 import entries
- Include actionable guidance for using the context

Covers: CONTEXT-02, CONTEXT-03

* feat(01-02): add investigation context to specialist agent descriptions

- security-reviewer: check related files for affected callers, verify tests
- quality-reviewer: check related files for pattern consistency
- logic-reviewer: check callers/dependents for broken assumptions
- codebase-fit-reviewer: use related files to understand existing patterns
- finding-validator: check related files for missed mitigations
- ai-triage-reviewer: unchanged (doesn't need related file guidance)

CONTEXT-04: Specialists now know which files to investigate beyond the diff

* feat(01-02): add specialist-specific delegation guidance to related files section

- Updated header: "Pass relevant files to specialists when delegating"
- Added per-specialist guidance for security, logic, quality, codebase-fit
- Added example delegation showing how to include related files in task

Orchestrator now knows HOW to pass investigation context to each specialist type

* feat(02-01): add VerificationEvidence class and update finding models

- Add VerificationEvidence class with required code_examined, line_range_examined, verification_method fields
- Add required verification field to BaseFinding
- Add required verification field to ParallelOrchestratorFinding
- Add is_impact_finding boolean field to ParallelOrchestratorFinding (default False)
- Add checked_for_handling_elsewhere boolean field to ParallelOrchestratorFinding (default False)
- Mark old evidence field as DEPRECATED in both BaseFinding and ParallelOrchestratorFinding

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(02-01): add tests for schema enforcement and verification evidence

- Add TestVerificationEvidence class with 5 tests for VerificationEvidence model
- Add TestParallelOrchestratorFindingVerification class with 6 tests for verification requirement
- Add TestVerificationSchemaGeneration class with 2 tests for JSON schema generation
- Update existing TestSecurityFinding and TestDeepAnalysisFinding to include verification field
- Import VerificationEvidence, ParallelOrchestratorFinding, BaseFinding in test imports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(03-02): add 'What the Diff Is For' section to orchestrator

- Reframe diff as question to investigate, not document to nitpick
- Add 3 questions to answer before delegation
- Include 'Delegate with Context' guidance
- Position after Phase 0, before Phase 1

* feat(03-01): add Understand Intent phase to all specialist prompts

- Add Phase 1: Understand the PR Intent to security, logic, quality, codebase_fit agents
- Force AI to understand PR purpose before searching for issues
- Prevents flagging intentional design decisions as bugs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(03-02): enhance delegation guidance with context requirements

- Add Context-Rich Delegation section with 3 requirements
- Include PR intent summary, specific concerns, files of interest
- Show anti-pattern vs good pattern comparison
- Update example delegation with specific verification items

* feat(03-01): add Evidence Requirements and Valid Outputs sections

- Add Evidence Requirements section documenting VerificationEvidence schema
- Document code_examined, line_range_examined, verification_method fields
- Document is_impact_finding and checked_for_handling_elsewhere fields
- Add Valid Outputs section allowing no-issues as valid output
- Document invalid outputs (forced issues, theoretical edge cases)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(03-01): update output format examples with verification object

- Add verification object with code_examined, line_range_examined, verification_method
- Add is_impact_finding and checked_for_handling_elsewhere fields
- Use domain-appropriate verification_method values per agent
- Security: direct_code_inspection for injection examples
- Logic: direct_code_inspection for off-by-one and race conditions
- Quality: direct_code_inspection + cross_file_trace for duplication
- Codebase fit: cross_file_trace for reinvention, direct_code_inspection for naming

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(04-01): add _verify_line_numbers() method

- Pre-filter findings with invalid line numbers before AI validation
- Cache file line counts to avoid re-reading same file
- Reject findings where line > file length
- Log each rejection with finding ID and reason
- Conservative: allow findings if file read fails

* feat(04-02): add hypothesis-validation structure to finding validator

- Add "Hypothesis-Validation Structure (MANDATORY)" section with 4 steps
- Define TRUE/FALSE conditions for hypothesis testing
- Include worked example showing confirmed_valid conclusion path
- Include counter-example showing dismissed_false_positive path
- Reference structure from Investigation Process section

* feat(04-01): add _validate_findings() method

- Import FindingValidationResponse from pydantic_models
- Create finding-validator agent client with pr_finding_validator type
- Build validation prompt with findings JSON and changed files
- Filter findings by validation_status:
  - confirmed_valid: keep with validation evidence
  - dismissed_false_positive: exclude from results
  - needs_human_review: keep with [NEEDS REVIEW] prefix
- Fail-safe: return original findings on any error
- Log validation statistics

* feat(04-01): wire validation pipeline into review() method

- Stage 1: Line verification after cross-validation (cheap pre-filter)
- Stage 2: AI validation for findings that pass line check
- Update programmatic filter loop to use validated_by_ai
- Log validation statistics at each stage
- Uses project_root (worktree or fallback) for file access

* refactor(05-01): remove evidence filter and confidence routing from review()

- Remove _validate_finding_evidence() call from loop
- Remove _apply_confidence_routing() call
- Simplify loop to only check scope
- Replace routed_findings with direct validated_findings assignment

* feat(05-02): remove false positive patterns from validator

- Remove VAGUE_PATTERNS constant (10 patterns)
- Remove GENERIC_PATTERNS constant (6 patterns)
- Remove _is_false_positive() method (44 lines)
- Remove _is_false_positive call from _is_valid()
- Remove TestFalsePositiveDetection class (4 tests)
- Update test_low_severity_higher_threshold to use actionability score

REMOVE-04: VAGUE_PATTERNS, GENERIC_PATTERNS deleted
REMOVE-05: _is_false_positive() method deleted

* refactor(05-01): remove redundant functions and simplify scope check

- Remove ConfidenceTier enum (no longer used)
- Remove _validate_finding_evidence function (schema enforces evidence)
- Remove _apply_confidence_routing method (validation is binary)
- Remove 'from enum import Enum' import
- Simplify _is_finding_in_scope to use schema field is_impact_finding
  instead of keyword detection

* fix(pr-review): add Task tool to orchestrator configs for SDK subagents

The pr_orchestrator_parallel and pr_followup_parallel agents need the
Task tool in their tools list to invoke SDK subagents (security-reviewer,
logic-reviewer, etc.). Without Task, the SDK cannot spawn subagents,
resulting in "Agent type not found" errors.

Also fixes test_integration_phase4.py to set is_impact_finding as an
attribute rather than constructor arg, since PRReviewFinding doesn't
have this field (it's on ParallelOrchestratorFinding Pydantic model).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): add explicit Task tool invocation syntax for specialist agents

The orchestrator was using the built-in general-purpose agent instead of
our custom specialist agents (security-reviewer, logic-reviewer, etc.)
because the prompt described agents but didn't show explicit Task tool
invocation syntax.

Changes:
- Add "CRITICAL: How to Invoke Specialist Agents" section with exact
  subagent_type values in a reference table
- Add Task tool invocation format with example syntax
- Add example showing parallel invocation of multiple specialists
- Add explicit "DO NOT USE" section warning against general-purpose
- Update example delegation to use Task tool syntax instead of prose
- Add example validation invocation for finding-validator

This ensures Claude uses our custom specialists instead of defaulting
to the built-in general-purpose agent.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(pr-review): implement evidence-based validation and trigger-driven exploration

Major enhancements to the PR review system:

**Evidence-Based Validation:**
- Shift from confidence-based to evidence-based finding validation
- All findings now require VerificationEvidence with code_examined, line_range_examined
- finding-validator validates ALL findings (CRITICAL through LOW) before output
- Add dismissed_findings array for transparency - users see what was investigated

**Trigger-Driven Exploration (6 Semantic Triggers):**
- OUTPUT CONTRACT CHANGED - function returns different value/type/structure
- INPUT CONTRACT CHANGED - parameters added/removed/reordered
- BEHAVIORAL CONTRACT CHANGED - same I/O but different internal behavior
- SIDE EFFECT CONTRACT CHANGED - observable effects added/removed
- FAILURE CONTRACT CHANGED - error handling changed
- NULL/UNDEFINED CONTRACT CHANGED - null handling changed

Orchestrator detects triggers in Phase 1 and passes them to specialists
with explicit "TRIGGER:", "EXPLORATION REQUIRED:", "Stop when:" instructions.

**Implementation Changes:**
- Add _PRDebugLogger for comprehensive agent communication logging
- Add CI status integration to verdict logic (failing CI blocks merge)
- Extract with_working_dir() to shared agent_utils.py module
- Inject working directory into all subagent prompts
- Bump SDK requirement to >=0.1.22 for custom subagent support

**Frontend:**
- Tighten AUTH_FAILURE_PATTERNS to avoid false positives on AI auth discussion
- Update tests for new pattern requirements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): wait for both queued AND in_progress CI checks

Previously, the CI wait logic only blocked on "in_progress" checks,
but not "queued" checks. This meant if a CI check (like CodeRabbit)
was queued but not yet running, the review would start immediately
and report "CI is pending" - which would be stale by the time the
contributor sees it.

Now we wait for ALL checks to reach "completed" status before
starting the review, ensuring the CI status in our review is accurate.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove duplicate .planning entries from .gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove docs/ from git tracking (already in .gitignore)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): propagate is_impact_finding field to allow impact findings

The is_impact_finding field was defined in ParallelOrchestratorFinding
but never propagated to PRReviewFinding, causing ALL impact findings
(findings about callers/affected files outside the PR's changed files)
to be incorrectly filtered out as "not in scope".

Changes:
- Add is_impact_finding field to PRReviewFinding dataclass
- Extract and pass is_impact_finding in _create_finding_from_structured()
- Add to to_dict() and from_dict() for serialization

This enables the trigger-driven exploration feature to actually work,
allowing the review to report issues in files affected by contract changes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): add Task tool invocation syntax to followup orchestrator

The follow-up review orchestrator was missing explicit Task tool
invocation syntax and examples. The AI didn't know HOW to invoke
the specialist agents (resolution-verifier, finding-validator, etc.),
causing resolution checking to never happen.

Added:
- Exact agent names table (subagent_type values)
- Task tool invocation format with examples
- Complete follow-up review workflow with Task calls
- DO NOT USE section (avoid general-purpose, Explore, Plan)
- Decision matrix for when to invoke each agent
- Explicit Task tool calls in Phase 2 workflow

This matches the main orchestrator prompt which has extensive
Task tool examples and works correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): propagate is_impact_finding in follow-up reviewer

Applied the same is_impact_finding propagation fix to the follow-up
reviewer that was already applied to the main orchestrator reviewer.

Fixes:
1. Add is_impact_finding field to ParallelFollowupFinding Pydantic model
2. Propagate is_impact_finding when creating PRReviewFinding for new findings
3. Copy is_impact_finding from original finding for unresolved findings

Without this fix, impact findings (about callers/affected files outside
the PR's changed files) would be incorrectly filtered as "not in scope"
during follow-up reviews.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(auth): enhance keychain service integration with config directory support

Added functionality to support profile-specific credentials by introducing a hash-based service name for macOS Keychain and updating Windows credential retrieval to utilize a provided config directory. This ensures that tokens are fetched from the correct profile-specific storage locations, improving credential management across different environments.

Changes include:
- New functions for calculating config directory hashes and generating keychain service names.
- Updated `get_token_from_keychain` and related functions to accept an optional config directory argument.
- Enhanced logging for better debugging when no token is found.

This aligns the backend credential handling with the frontend's expectations for profile-specific storage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 14:34:08 +01:00

742 lines
33 KiB
Python

"""
SDK Stream Processing Utilities
================================
Shared utilities for processing Claude Agent SDK response streams.
This module extracts common SDK message processing patterns used across
parallel orchestrator and follow-up reviewers.
"""
from __future__ import annotations
import logging
import os
from collections.abc import Callable
from typing import Any
try:
from .io_utils import safe_print
except (ImportError, ValueError, SystemError):
from core.io_utils import safe_print
logger = logging.getLogger(__name__)
# Check if debug mode is enabled
DEBUG_MODE = os.environ.get("DEBUG", "").lower() in ("true", "1", "yes")
# ── TEMPORARY: Per-PR full agent communication logger (v2) ────────────
# Writes every message to .auto-claude/github/pr/debug_logs/<context>_<ts>.log
# Remove after measurement phase is complete.
import datetime as _dt
import json as _json
from pathlib import Path as _Path
# Derive project root dynamically from this file's location
# sdk_utils.py is at: apps/backend/runners/github/services/sdk_utils.py
# So project root is 5 levels up
_PROJECT_ROOT = _Path(__file__).resolve().parent.parent.parent.parent.parent
_PR_LOG_DIR = _PROJECT_ROOT / ".auto-claude" / "github" / "pr" / "debug_logs"
class _PRDebugLogger:
"""Writes full agent communication to a log file for review.
Improvements (v2):
- System prompt and agent definitions logged at session start
- No truncation on thinking, text, tool input, or tool results
- No duplicate logging (single structured dump per message)
- Empty/whitespace content shown via repr()
- Agent attribution via subagent_tool_ids mapping
"""
def __init__(self, context_name: str, model: str | None = None):
self._f = None
self._subagent_tool_ids: dict[str, str] = {} # tool_id -> agent_name
try:
_PR_LOG_DIR.mkdir(parents=True, exist_ok=True)
ts = _dt.datetime.now().strftime("%Y%m%d_%H%M%S")
self.path = _PR_LOG_DIR / f"{context_name}_{ts}.log"
self._f = open(self.path, "w", encoding="utf-8")
self._write(
f"=== {context_name} Session Started at {_dt.datetime.now().isoformat()} ==="
)
if model:
self._write(f"Model: {model}")
self._write("")
except OSError as e:
# Failed to create directory or open file - logging disabled
logger.warning(f"PR debug logger disabled: {e}")
self.path = None
def _write(self, text: str):
# Skip logging if file handle was not created successfully
if self._f is None:
return
try:
self._f.write(text + "\n")
self._f.flush()
except (OSError, ValueError) as e:
# File write failed (file closed, disk full, etc.) - disable logging
logger.warning(f"PR debug logger write failed: {e}")
self._f = None
# ── Session preamble loggers ──────────────────────────────────────
def log_system_prompt(self, prompt: str):
"""Log the full system prompt (no truncation)."""
self._write(f"\n{'#' * 80}")
self._write("# SYSTEM PROMPT (full orchestrator instructions + PR context)")
self._write(f"# Length: {len(prompt)} chars")
self._write(f"{'#' * 80}")
self._write(prompt)
self._write(f"{'#' * 80}\n")
def log_agent_definitions(self, agents: dict):
"""Log all specialist agent definitions (prompts, tools, descriptions)."""
self._write(f"\n{'#' * 80}")
self._write(f"# AGENT DEFINITIONS ({len(agents)} specialists)")
self._write(f"{'#' * 80}")
for name, defn in agents.items():
self._write(f"\n--- Agent: {name} ---")
self._write(f" description: {getattr(defn, 'description', 'N/A')}")
self._write(f" model: {getattr(defn, 'model', 'N/A')}")
self._write(f" tools: {getattr(defn, 'tools', 'N/A')}")
prompt = getattr(defn, "prompt", "")
self._write(f" prompt ({len(prompt)} chars):")
self._write(prompt)
self._write(f"{'#' * 80}\n")
# ── Agent attribution ─────────────────────────────────────────────
def set_subagent_mapping(self, mapping: dict[str, str]):
"""Update the tool_id -> agent_name mapping for attribution."""
self._subagent_tool_ids = mapping
def _get_agent_label(self, tool_id: str) -> str:
"""Return agent label if this tool_id belongs to a known subagent."""
agent = self._subagent_tool_ids.get(tool_id)
return f" [Agent:{agent}]" if agent else ""
# ── Per-message logger (single structured dump) ───────────────────
def log_message(self, msg_count: int, msg_type: str, msg: object):
self._write(f"\n{'=' * 80}")
self._write(f"--- Message #{msg_count} [{msg_type}] ---")
self._write(f"{'=' * 80}")
self._dump_raw(msg)
def _dump_raw(self, msg: object, indent: int = 0):
"""Dump full raw message content recursively — NO truncation."""
prefix = " " * indent
# Content blocks
if hasattr(msg, "content"):
content = msg.content
if isinstance(content, list):
self._write(f"{prefix}[content] ({len(content)} blocks):")
for i, block in enumerate(content):
block_type = type(block).__name__
self._write(f"{prefix} [{i}] {block_type}:")
self._dump_block(block, indent + 2)
elif isinstance(content, str):
if not content or content.isspace():
self._write(
f"{prefix}[content] (string, {len(content)} chars): {repr(content)}"
)
else:
self._write(f"{prefix}[content] (string, {len(content)} chars):")
self._write(content)
else:
self._write(f"{prefix}[content] ({type(content).__name__}):")
self._write(f"{prefix} {str(content)}")
# Role / type
if hasattr(msg, "role"):
self._write(f"{prefix}[role] {msg.role}")
if hasattr(msg, "type") and not hasattr(msg, "content"):
self._write(f"{prefix}[type] {msg.type}")
# Structured output
if hasattr(msg, "structured_output") and msg.structured_output:
self._write(f"{prefix}[structured_output]:")
try:
self._write(_json.dumps(msg.structured_output, indent=2, default=str))
except Exception:
self._write(f"{prefix} {str(msg.structured_output)}")
# Result message fields
if hasattr(msg, "subtype"):
self._write(f"{prefix}[subtype] {msg.subtype}")
if hasattr(msg, "is_error"):
self._write(f"{prefix}[is_error] {msg.is_error}")
if hasattr(msg, "duration_ms"):
self._write(f"{prefix}[duration_ms] {msg.duration_ms}")
if hasattr(msg, "session_id"):
self._write(f"{prefix}[session_id] {msg.session_id}")
# Catch-all for messages without content blocks
for attr in ("text", "thinking", "name", "id", "input", "tool_use_id"):
if hasattr(msg, attr) and not hasattr(msg, "content"):
val = getattr(msg, attr)
if val is not None:
self._write(f"{prefix}[{attr}] {str(val)}")
def _dump_block(self, block: object, indent: int = 0):
"""Dump a single content block — NO truncation."""
prefix = " " * indent
block_type = getattr(block, "type", type(block).__name__)
if block_type in ("text", "TextBlock") and hasattr(block, "text"):
text = block.text
if not text or text.isspace():
self._write(f"{prefix}[text] ({len(text)} chars): {repr(text)}")
else:
self._write(f"{prefix}[text] ({len(text)} chars):")
self._write(text)
elif block_type in ("thinking", "ThinkingBlock") and hasattr(block, "thinking"):
text = block.thinking or getattr(block, "text", "")
self._write(f"{prefix}[thinking] ({len(text)} chars):")
self._write(text)
elif block_type in ("tool_use", "ToolUseBlock"):
tool_name = getattr(block, "name", "unknown")
tool_id = getattr(block, "id", "unknown")
tool_input = getattr(block, "input", {})
agent_label = self._get_agent_label(tool_id)
self._write(f"{prefix}[tool_use] {tool_name} (id={tool_id}){agent_label}")
try:
self._write(_json.dumps(tool_input, indent=2, default=str))
except Exception:
self._write(str(tool_input))
elif block_type in ("tool_result", "ToolResultBlock"):
tool_id = getattr(block, "tool_use_id", "unknown")
is_error = getattr(block, "is_error", False)
result = getattr(block, "content", "")
if isinstance(result, list):
result = " ".join(str(getattr(c, "text", c)) for c in result)
status = "ERROR" if is_error else "OK"
agent_label = self._get_agent_label(tool_id)
self._write(
f"{prefix}[tool_result] (tool_id={tool_id}) {status}{agent_label}"
)
self._write(str(result))
else:
# Unknown block type — dump everything we can
self._write(f"{prefix}[{block_type}] (raw dump):")
for attr in dir(block):
if not attr.startswith("_"):
try:
val = getattr(block, attr)
if not callable(val):
self._write(f"{prefix} {attr}: {str(val)}")
except Exception:
pass
# ── Structured output (standalone, for final result) ──────────────
def log_structured_output(self, output: dict):
self._write("[STRUCTURED_OUTPUT]")
try:
self._write(_json.dumps(output, indent=2, default=str))
except Exception:
self._write(str(output))
# ── Session close ─────────────────────────────────────────────────
def close(self, summary: dict):
self._write("\n=== Session Ended ===")
self._write(f"Messages: {summary.get('msg_count', '?')}")
self._write(f"Agents invoked: {summary.get('agents_invoked', [])}")
self._write(f"Error: {summary.get('error')}")
self._write(f"Log file: {self.path}")
if self._f is not None:
try:
self._f.close()
except OSError as e:
logger.warning(f"PR debug logger close failed: {e}")
# ── END TEMPORARY ──────────────────────────────────────────────────────
def _short_model_name(model: str | None) -> str:
"""Convert full model name to a short display name for logs.
Examples:
claude-sonnet-4-5-20250929 -> sonnet-4.5
claude-opus-4-5-20251101 -> opus-4.5
claude-3-5-sonnet-20241022 -> sonnet-3.5
"""
if not model:
return "unknown"
model_lower = model.lower()
# Handle new model naming (claude-{model}-{version}-{date})
if "opus-4-5" in model_lower or "opus-4.5" in model_lower:
return "opus-4.5"
if "sonnet-4-5" in model_lower or "sonnet-4.5" in model_lower:
return "sonnet-4.5"
if "haiku-4" in model_lower:
return "haiku-4"
# Handle older model naming (claude-3-5-{model})
if "3-5-sonnet" in model_lower or "3.5-sonnet" in model_lower:
return "sonnet-3.5"
if "3-5-haiku" in model_lower or "3.5-haiku" in model_lower:
return "haiku-3.5"
if "3-opus" in model_lower:
return "opus-3"
if "3-sonnet" in model_lower:
return "sonnet-3"
if "3-haiku" in model_lower:
return "haiku-3"
# Fallback: return last part before date (if matches pattern)
parts = model.split("-")
if len(parts) >= 2:
# Try to find model type (opus, sonnet, haiku)
for i, part in enumerate(parts):
if part.lower() in ("opus", "sonnet", "haiku"):
return part.lower()
return model[:20] # Truncate if nothing else works
def _get_tool_detail(tool_name: str, tool_input: dict[str, Any]) -> str:
"""Extract meaningful detail from tool input for user-friendly logging.
Instead of "Using tool: Read", show "Reading sdk_utils.py"
Instead of "Using tool: Grep", show "Searching for 'pattern'"
"""
if tool_name == "Read":
file_path = tool_input.get("file_path", "")
if file_path:
# Extract just the filename for brevity
filename = file_path.split("/")[-1] if "/" in file_path else file_path
return f"Reading {filename}"
return "Reading file"
if tool_name == "Grep":
pattern = tool_input.get("pattern", "")
if pattern:
# Truncate long patterns
pattern_preview = pattern[:40] + "..." if len(pattern) > 40 else pattern
return f"Searching for '{pattern_preview}'"
return "Searching codebase"
if tool_name == "Glob":
pattern = tool_input.get("pattern", "")
if pattern:
return f"Finding files matching '{pattern}'"
return "Finding files"
if tool_name == "Bash":
command = tool_input.get("command", "")
if command:
# Show first part of command
cmd_preview = command[:50] + "..." if len(command) > 50 else command
return f"Running: {cmd_preview}"
return "Running command"
if tool_name == "Edit":
file_path = tool_input.get("file_path", "")
if file_path:
filename = file_path.split("/")[-1] if "/" in file_path else file_path
return f"Editing {filename}"
return "Editing file"
if tool_name == "Write":
file_path = tool_input.get("file_path", "")
if file_path:
filename = file_path.split("/")[-1] if "/" in file_path else file_path
return f"Writing {filename}"
return "Writing file"
# Default fallback for unknown tools
return f"Using tool: {tool_name}"
async def process_sdk_stream(
client: Any,
on_thinking: Callable[[str], None] | None = None,
on_tool_use: Callable[[str, str, dict[str, Any]], None] | None = None,
on_tool_result: Callable[[str, bool, Any], None] | None = None,
on_text: Callable[[str], None] | None = None,
on_structured_output: Callable[[dict[str, Any]], None] | None = None,
context_name: str = "SDK",
model: str | None = None,
system_prompt: str | None = None,
agent_definitions: dict | None = None,
) -> dict[str, Any]:
"""
Process SDK response stream with customizable callbacks.
This function handles the common pattern of:
- Tracking thinking blocks
- Tracking tool invocations (especially Task/subagent calls)
- Tracking tool results
- Collecting text output
- Extracting structured output (per official Python SDK pattern)
Args:
client: Claude SDK client with receive_response() method
on_thinking: Callback for thinking blocks - receives thinking text
on_tool_use: Callback for tool invocations - receives (tool_name, tool_id, tool_input)
on_tool_result: Callback for tool results - receives (tool_id, is_error, result_content)
on_text: Callback for text output - receives text string
on_structured_output: Callback for structured output - receives dict
context_name: Name for logging (e.g., "ParallelOrchestrator", "ParallelFollowup")
model: Model name for logging (e.g., "claude-sonnet-4-5-20250929")
system_prompt: Full system prompt sent to the agent (logged at session start)
agent_definitions: Dict of agent name -> AgentDefinition (logged at session start)
Returns:
Dictionary with:
- result_text: Accumulated text output
- structured_output: Final structured output (if any)
- agents_invoked: List of agent names invoked via Task tool
- msg_count: Total message count
- subagent_tool_ids: Mapping of tool_id -> agent_name
- error: Error message if stream processing failed (None on success)
"""
result_text = ""
structured_output = None
agents_invoked = []
msg_count = 0
stream_error = None
# Track subagent tool IDs to log their results
subagent_tool_ids: dict[str, str] = {} # tool_id -> agent_name
completed_agent_tool_ids: set[str] = set() # tool_ids of completed agents
# TEMPORARY: per-PR debug file logger
_dbg = _PRDebugLogger(context_name, model=model)
# Log session preamble: system prompt and agent definitions
if system_prompt:
_dbg.log_system_prompt(system_prompt)
if agent_definitions:
_dbg.log_agent_definitions(agent_definitions)
safe_print(f"[{context_name}] Processing SDK stream...")
if DEBUG_MODE:
safe_print(f"[DEBUG {context_name}] Awaiting response stream...")
# Track activity for progress logging
last_progress_log = 0
PROGRESS_LOG_INTERVAL = 10 # Log progress every N messages
try:
async for msg in client.receive_response():
try:
msg_type = type(msg).__name__
msg_count += 1
_dbg.log_message(msg_count, msg_type, msg)
# Log progress periodically so user knows AI is working
if msg_count - last_progress_log >= PROGRESS_LOG_INTERVAL:
if subagent_tool_ids:
pending = len(subagent_tool_ids) - len(completed_agent_tool_ids)
if pending > 0:
safe_print(
f"[{context_name}] Processing... ({msg_count} messages, {pending} agent{'s' if pending > 1 else ''} working)"
)
else:
safe_print(
f"[{context_name}] Processing... ({msg_count} messages)"
)
else:
safe_print(
f"[{context_name}] Processing... ({msg_count} messages)"
)
last_progress_log = msg_count
if DEBUG_MODE:
# Log every message type for visibility
msg_details = ""
if hasattr(msg, "type"):
msg_details = f" (type={msg.type})"
safe_print(
f"[DEBUG {context_name}] Message #{msg_count}: {msg_type}{msg_details}"
)
# Track thinking blocks
if msg_type == "ThinkingBlock" or (
hasattr(msg, "type") and msg.type == "thinking"
):
thinking_text = getattr(msg, "thinking", "") or getattr(
msg, "text", ""
)
if thinking_text:
safe_print(
f"[{context_name}] AI thinking: {len(thinking_text)} chars"
)
if DEBUG_MODE:
# Show first 200 chars of thinking
preview = thinking_text[:200].replace("\n", " ")
safe_print(
f"[DEBUG {context_name}] Thinking preview: {preview}..."
)
# Invoke callback
if on_thinking:
on_thinking(thinking_text)
# Track subagent invocations (Task tool calls)
if msg_type == "ToolUseBlock" or (
hasattr(msg, "type") and msg.type == "tool_use"
):
tool_name = getattr(msg, "name", "")
tool_id = getattr(msg, "id", "unknown")
tool_input = getattr(msg, "input", {})
if DEBUG_MODE:
safe_print(
f"[DEBUG {context_name}] Tool call: {tool_name} (id={tool_id})"
)
if tool_name == "Task":
# Extract which agent was invoked
agent_name = tool_input.get("subagent_type", "unknown")
agents_invoked.append(agent_name)
# Track this tool ID to log its result later
subagent_tool_ids[tool_id] = agent_name
_dbg.set_subagent_mapping(subagent_tool_ids)
# Log with model info if available
model_info = f" [{_short_model_name(model)}]" if model else ""
safe_print(
f"[{context_name}] Invoking agent: {agent_name}{model_info}"
)
# Log delegation prompt for debugging trigger system
delegation_prompt = tool_input.get("prompt", "")
if delegation_prompt:
# Show first 300 chars of delegation prompt
prompt_preview = delegation_prompt[:300]
if len(delegation_prompt) > 300:
prompt_preview += "..."
safe_print(
f"[{context_name}] Delegation prompt for {agent_name}: {prompt_preview}"
)
elif tool_name != "StructuredOutput":
# Log meaningful tool info (not just tool name)
tool_detail = _get_tool_detail(tool_name, tool_input)
safe_print(f"[{context_name}] {tool_detail}")
# Invoke callback for all tool uses
if on_tool_use:
on_tool_use(tool_name, tool_id, tool_input)
# Track tool results
if msg_type == "ToolResultBlock" or (
hasattr(msg, "type") and msg.type == "tool_result"
):
tool_id = getattr(msg, "tool_use_id", "unknown")
is_error = getattr(msg, "is_error", False)
result_content = getattr(msg, "content", "")
# Handle list of content blocks
if isinstance(result_content, list):
result_content = " ".join(
str(getattr(c, "text", c)) for c in result_content
)
# Check if this is a subagent result
if tool_id in subagent_tool_ids:
agent_name = subagent_tool_ids[tool_id]
completed_agent_tool_ids.add(tool_id) # Mark agent as completed
status = "ERROR" if is_error else "complete"
result_preview = (
str(result_content)[:600].replace("\n", " ").strip()
)
safe_print(
f"[Agent:{agent_name}] {status}: {result_preview}{'...' if len(str(result_content)) > 600 else ''}"
)
else:
# Show tool completion for visibility (not gated by DEBUG)
status = "ERROR" if is_error else "done"
# Show brief preview of result for context
result_preview = (
str(result_content)[:100].replace("\n", " ").strip()
)
if result_preview:
safe_print(
f"[{context_name}] Tool result [{status}]: {result_preview}{'...' if len(str(result_content)) > 100 else ''}"
)
# Invoke callback
if on_tool_result:
on_tool_result(tool_id, is_error, result_content)
# Collect text output and check for tool uses in content blocks
if msg_type == "AssistantMessage" and hasattr(msg, "content"):
for block in msg.content:
block_type = type(block).__name__
# Check for tool use blocks within content
if (
block_type == "ToolUseBlock"
or getattr(block, "type", "") == "tool_use"
):
tool_name = getattr(block, "name", "")
tool_id = getattr(block, "id", "unknown")
tool_input = getattr(block, "input", {})
if tool_name == "Task":
agent_name = tool_input.get("subagent_type", "unknown")
if agent_name not in agents_invoked:
agents_invoked.append(agent_name)
subagent_tool_ids[tool_id] = agent_name
# Log with model info if available
model_info = (
f" [{_short_model_name(model)}]"
if model
else ""
)
safe_print(
f"[{context_name}] Invoking agent: {agent_name}{model_info}"
)
elif tool_name != "StructuredOutput":
# Log meaningful tool info (not just tool name)
tool_detail = _get_tool_detail(tool_name, tool_input)
safe_print(f"[{context_name}] {tool_detail}")
# Invoke callback
if on_tool_use:
on_tool_use(tool_name, tool_id, tool_input)
# Collect text - must check block type since only TextBlock has .text
block_type = type(block).__name__
if block_type == "TextBlock" and hasattr(block, "text"):
result_text += block.text
# Always print text content preview (not just in DEBUG_MODE)
text_preview = block.text[:500].replace("\n", " ").strip()
if text_preview:
safe_print(
f"[{context_name}] AI response: {text_preview}{'...' if len(block.text) > 500 else ''}"
)
# Invoke callback
if on_text:
on_text(block.text)
# ================================================================
# STRUCTURED OUTPUT CAPTURE (Single, consolidated location)
# Per official Python SDK docs: https://platform.claude.com/docs/en/agent-sdk/structured-outputs
# The Python pattern is: if hasattr(message, 'structured_output')
# ================================================================
# Check for error_max_structured_output_retries first (SDK validation failed)
is_result_msg = msg_type == "ResultMessage" or (
hasattr(msg, "type") and msg.type == "result"
)
if is_result_msg:
subtype = getattr(msg, "subtype", None)
if DEBUG_MODE:
safe_print(
f"[DEBUG {context_name}] ResultMessage: subtype={subtype}"
)
if subtype == "error_max_structured_output_retries":
# SDK failed to produce valid structured output after retries
logger.warning(
f"[{context_name}] Claude could not produce valid structured output "
f"after maximum retries - schema validation failed"
)
safe_print(
f"[{context_name}] WARNING: Structured output validation failed after retries"
)
if not stream_error:
stream_error = "structured_output_validation_failed"
# Capture structured output from ANY message that has it
# This is the official Python SDK pattern - check hasattr()
if hasattr(msg, "structured_output") and msg.structured_output:
# Only capture if we don't already have it (avoid duplicates)
if structured_output is None:
structured_output = msg.structured_output
_dbg.log_structured_output(msg.structured_output)
safe_print(f"[{context_name}] Received structured output")
if on_structured_output:
on_structured_output(msg.structured_output)
elif DEBUG_MODE:
# In debug mode, note that we skipped a duplicate
safe_print(
f"[DEBUG {context_name}] Skipping duplicate structured output"
)
# Check for tool results in UserMessage (subagent results come back here)
if msg_type == "UserMessage" and hasattr(msg, "content"):
for block in msg.content:
block_type = type(block).__name__
# Check for tool result blocks
if (
block_type == "ToolResultBlock"
or getattr(block, "type", "") == "tool_result"
):
tool_id = getattr(block, "tool_use_id", "unknown")
is_error = getattr(block, "is_error", False)
result_content = getattr(block, "content", "")
# Handle list of content blocks
if isinstance(result_content, list):
result_content = " ".join(
str(getattr(c, "text", c)) for c in result_content
)
# Check if this is a subagent result
if tool_id in subagent_tool_ids:
agent_name = subagent_tool_ids[tool_id]
completed_agent_tool_ids.add(
tool_id
) # Mark agent as completed
status = "ERROR" if is_error else "complete"
result_preview = (
str(result_content)[:600].replace("\n", " ").strip()
)
safe_print(
f"[Agent:{agent_name}] {status}: {result_preview}{'...' if len(str(result_content)) > 600 else ''}"
)
# Invoke callback
if on_tool_result:
on_tool_result(tool_id, is_error, result_content)
except (AttributeError, TypeError, KeyError) as msg_error:
# Log individual message processing errors but continue
logger.warning(
f"[{context_name}] Error processing message #{msg_count}: {msg_error}"
)
if DEBUG_MODE:
safe_print(
f"[DEBUG {context_name}] Message processing error: {msg_error}"
)
# Continue processing subsequent messages
except BrokenPipeError:
# Pipe closed by parent process - expected during shutdown
stream_error = "Output pipe closed"
logger.debug(f"[{context_name}] Output pipe closed by parent process")
except Exception as e:
# Log stream-level errors
stream_error = str(e)
logger.error(f"[{context_name}] SDK stream processing failed: {e}")
safe_print(f"[{context_name}] ERROR: Stream processing failed: {e}")
if DEBUG_MODE:
safe_print(f"[DEBUG {context_name}] Session ended. Total messages: {msg_count}")
safe_print(f"[{context_name}] Session ended. Total messages: {msg_count}")
result = {
"result_text": result_text,
"structured_output": structured_output,
"agents_invoked": agents_invoked,
"msg_count": msg_count,
"subagent_tool_ids": subagent_tool_ids,
"error": stream_error,
}
_dbg.close(result)
safe_print(f"[{context_name}] Full debug log: {_dbg.path}")
return result