Files
Aperant/apps/backend/runners/github/models.py
T
Andy bfc232825b feat(pr-review): evidence-based validation and trigger-driven exploration (#1593)
* gsd update

* docs: define v1 requirements with holistic PR understanding

29 requirements across 6 categories:
- Holistic PR Understanding (5) - context synthesis and passing
- Validation Pipeline (3) - finding-validator for all reviews
- Schema Enforcement (5) - VerificationEvidence required
- Prompt Improvements (6) - understand intent, evidence requirements
- Code Simplification (6) - remove programmatic filters
- Measurement (4) - 5 PRs to validate

Key addition: Pass gathered context (related files, import graph) to specialists.
Currently gathered but unused.

* feat(01-01): add Phase 0 synthesis instruction to orchestrator prompt

- Add 'Phase 0: Understand the PR Holistically' section before Phase 1
- Include PR UNDERSTANDING output format (intent, critical changes, risk areas, files to verify)
- Add explicit gate: 'Only AFTER completing Phase 0, proceed to Phase 1'
- Add 'Understand First' principle to Key Principles section

Covers: CONTEXT-01, CONTEXT-05

* feat(01-01): add related files and import graph to orchestrator prompt

- Add related files section categorizing tests vs dependencies/callers
- Add import graph section showing what files import/are imported by changed files
- Limit to 30 related files (15 tests, 15 deps) and 20 import entries
- Include actionable guidance for using the context

Covers: CONTEXT-02, CONTEXT-03

* feat(01-02): add investigation context to specialist agent descriptions

- security-reviewer: check related files for affected callers, verify tests
- quality-reviewer: check related files for pattern consistency
- logic-reviewer: check callers/dependents for broken assumptions
- codebase-fit-reviewer: use related files to understand existing patterns
- finding-validator: check related files for missed mitigations
- ai-triage-reviewer: unchanged (doesn't need related file guidance)

CONTEXT-04: Specialists now know which files to investigate beyond the diff

* feat(01-02): add specialist-specific delegation guidance to related files section

- Updated header: "Pass relevant files to specialists when delegating"
- Added per-specialist guidance for security, logic, quality, codebase-fit
- Added example delegation showing how to include related files in task

Orchestrator now knows HOW to pass investigation context to each specialist type

* feat(02-01): add VerificationEvidence class and update finding models

- Add VerificationEvidence class with required code_examined, line_range_examined, verification_method fields
- Add required verification field to BaseFinding
- Add required verification field to ParallelOrchestratorFinding
- Add is_impact_finding boolean field to ParallelOrchestratorFinding (default False)
- Add checked_for_handling_elsewhere boolean field to ParallelOrchestratorFinding (default False)
- Mark old evidence field as DEPRECATED in both BaseFinding and ParallelOrchestratorFinding

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(02-01): add tests for schema enforcement and verification evidence

- Add TestVerificationEvidence class with 5 tests for VerificationEvidence model
- Add TestParallelOrchestratorFindingVerification class with 6 tests for verification requirement
- Add TestVerificationSchemaGeneration class with 2 tests for JSON schema generation
- Update existing TestSecurityFinding and TestDeepAnalysisFinding to include verification field
- Import VerificationEvidence, ParallelOrchestratorFinding, BaseFinding in test imports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(03-02): add 'What the Diff Is For' section to orchestrator

- Reframe diff as question to investigate, not document to nitpick
- Add 3 questions to answer before delegation
- Include 'Delegate with Context' guidance
- Position after Phase 0, before Phase 1

* feat(03-01): add Understand Intent phase to all specialist prompts

- Add Phase 1: Understand the PR Intent to security, logic, quality, codebase_fit agents
- Force AI to understand PR purpose before searching for issues
- Prevents flagging intentional design decisions as bugs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(03-02): enhance delegation guidance with context requirements

- Add Context-Rich Delegation section with 3 requirements
- Include PR intent summary, specific concerns, files of interest
- Show anti-pattern vs good pattern comparison
- Update example delegation with specific verification items

* feat(03-01): add Evidence Requirements and Valid Outputs sections

- Add Evidence Requirements section documenting VerificationEvidence schema
- Document code_examined, line_range_examined, verification_method fields
- Document is_impact_finding and checked_for_handling_elsewhere fields
- Add Valid Outputs section allowing no-issues as valid output
- Document invalid outputs (forced issues, theoretical edge cases)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(03-01): update output format examples with verification object

- Add verification object with code_examined, line_range_examined, verification_method
- Add is_impact_finding and checked_for_handling_elsewhere fields
- Use domain-appropriate verification_method values per agent
- Security: direct_code_inspection for injection examples
- Logic: direct_code_inspection for off-by-one and race conditions
- Quality: direct_code_inspection + cross_file_trace for duplication
- Codebase fit: cross_file_trace for reinvention, direct_code_inspection for naming

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(04-01): add _verify_line_numbers() method

- Pre-filter findings with invalid line numbers before AI validation
- Cache file line counts to avoid re-reading same file
- Reject findings where line > file length
- Log each rejection with finding ID and reason
- Conservative: allow findings if file read fails

* feat(04-02): add hypothesis-validation structure to finding validator

- Add "Hypothesis-Validation Structure (MANDATORY)" section with 4 steps
- Define TRUE/FALSE conditions for hypothesis testing
- Include worked example showing confirmed_valid conclusion path
- Include counter-example showing dismissed_false_positive path
- Reference structure from Investigation Process section

* feat(04-01): add _validate_findings() method

- Import FindingValidationResponse from pydantic_models
- Create finding-validator agent client with pr_finding_validator type
- Build validation prompt with findings JSON and changed files
- Filter findings by validation_status:
  - confirmed_valid: keep with validation evidence
  - dismissed_false_positive: exclude from results
  - needs_human_review: keep with [NEEDS REVIEW] prefix
- Fail-safe: return original findings on any error
- Log validation statistics

* feat(04-01): wire validation pipeline into review() method

- Stage 1: Line verification after cross-validation (cheap pre-filter)
- Stage 2: AI validation for findings that pass line check
- Update programmatic filter loop to use validated_by_ai
- Log validation statistics at each stage
- Uses project_root (worktree or fallback) for file access

* refactor(05-01): remove evidence filter and confidence routing from review()

- Remove _validate_finding_evidence() call from loop
- Remove _apply_confidence_routing() call
- Simplify loop to only check scope
- Replace routed_findings with direct validated_findings assignment

* feat(05-02): remove false positive patterns from validator

- Remove VAGUE_PATTERNS constant (10 patterns)
- Remove GENERIC_PATTERNS constant (6 patterns)
- Remove _is_false_positive() method (44 lines)
- Remove _is_false_positive call from _is_valid()
- Remove TestFalsePositiveDetection class (4 tests)
- Update test_low_severity_higher_threshold to use actionability score

REMOVE-04: VAGUE_PATTERNS, GENERIC_PATTERNS deleted
REMOVE-05: _is_false_positive() method deleted

* refactor(05-01): remove redundant functions and simplify scope check

- Remove ConfidenceTier enum (no longer used)
- Remove _validate_finding_evidence function (schema enforces evidence)
- Remove _apply_confidence_routing method (validation is binary)
- Remove 'from enum import Enum' import
- Simplify _is_finding_in_scope to use schema field is_impact_finding
  instead of keyword detection

* fix(pr-review): add Task tool to orchestrator configs for SDK subagents

The pr_orchestrator_parallel and pr_followup_parallel agents need the
Task tool in their tools list to invoke SDK subagents (security-reviewer,
logic-reviewer, etc.). Without Task, the SDK cannot spawn subagents,
resulting in "Agent type not found" errors.

Also fixes test_integration_phase4.py to set is_impact_finding as an
attribute rather than constructor arg, since PRReviewFinding doesn't
have this field (it's on ParallelOrchestratorFinding Pydantic model).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): add explicit Task tool invocation syntax for specialist agents

The orchestrator was using the built-in general-purpose agent instead of
our custom specialist agents (security-reviewer, logic-reviewer, etc.)
because the prompt described agents but didn't show explicit Task tool
invocation syntax.

Changes:
- Add "CRITICAL: How to Invoke Specialist Agents" section with exact
  subagent_type values in a reference table
- Add Task tool invocation format with example syntax
- Add example showing parallel invocation of multiple specialists
- Add explicit "DO NOT USE" section warning against general-purpose
- Update example delegation to use Task tool syntax instead of prose
- Add example validation invocation for finding-validator

This ensures Claude uses our custom specialists instead of defaulting
to the built-in general-purpose agent.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(pr-review): implement evidence-based validation and trigger-driven exploration

Major enhancements to the PR review system:

**Evidence-Based Validation:**
- Shift from confidence-based to evidence-based finding validation
- All findings now require VerificationEvidence with code_examined, line_range_examined
- finding-validator validates ALL findings (CRITICAL through LOW) before output
- Add dismissed_findings array for transparency - users see what was investigated

**Trigger-Driven Exploration (6 Semantic Triggers):**
- OUTPUT CONTRACT CHANGED - function returns different value/type/structure
- INPUT CONTRACT CHANGED - parameters added/removed/reordered
- BEHAVIORAL CONTRACT CHANGED - same I/O but different internal behavior
- SIDE EFFECT CONTRACT CHANGED - observable effects added/removed
- FAILURE CONTRACT CHANGED - error handling changed
- NULL/UNDEFINED CONTRACT CHANGED - null handling changed

Orchestrator detects triggers in Phase 1 and passes them to specialists
with explicit "TRIGGER:", "EXPLORATION REQUIRED:", "Stop when:" instructions.

**Implementation Changes:**
- Add _PRDebugLogger for comprehensive agent communication logging
- Add CI status integration to verdict logic (failing CI blocks merge)
- Extract with_working_dir() to shared agent_utils.py module
- Inject working directory into all subagent prompts
- Bump SDK requirement to >=0.1.22 for custom subagent support

**Frontend:**
- Tighten AUTH_FAILURE_PATTERNS to avoid false positives on AI auth discussion
- Update tests for new pattern requirements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): wait for both queued AND in_progress CI checks

Previously, the CI wait logic only blocked on "in_progress" checks,
but not "queued" checks. This meant if a CI check (like CodeRabbit)
was queued but not yet running, the review would start immediately
and report "CI is pending" - which would be stale by the time the
contributor sees it.

Now we wait for ALL checks to reach "completed" status before
starting the review, ensuring the CI status in our review is accurate.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove duplicate .planning entries from .gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove docs/ from git tracking (already in .gitignore)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): propagate is_impact_finding field to allow impact findings

The is_impact_finding field was defined in ParallelOrchestratorFinding
but never propagated to PRReviewFinding, causing ALL impact findings
(findings about callers/affected files outside the PR's changed files)
to be incorrectly filtered out as "not in scope".

Changes:
- Add is_impact_finding field to PRReviewFinding dataclass
- Extract and pass is_impact_finding in _create_finding_from_structured()
- Add to to_dict() and from_dict() for serialization

This enables the trigger-driven exploration feature to actually work,
allowing the review to report issues in files affected by contract changes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): add Task tool invocation syntax to followup orchestrator

The follow-up review orchestrator was missing explicit Task tool
invocation syntax and examples. The AI didn't know HOW to invoke
the specialist agents (resolution-verifier, finding-validator, etc.),
causing resolution checking to never happen.

Added:
- Exact agent names table (subagent_type values)
- Task tool invocation format with examples
- Complete follow-up review workflow with Task calls
- DO NOT USE section (avoid general-purpose, Explore, Plan)
- Decision matrix for when to invoke each agent
- Explicit Task tool calls in Phase 2 workflow

This matches the main orchestrator prompt which has extensive
Task tool examples and works correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pr-review): propagate is_impact_finding in follow-up reviewer

Applied the same is_impact_finding propagation fix to the follow-up
reviewer that was already applied to the main orchestrator reviewer.

Fixes:
1. Add is_impact_finding field to ParallelFollowupFinding Pydantic model
2. Propagate is_impact_finding when creating PRReviewFinding for new findings
3. Copy is_impact_finding from original finding for unresolved findings

Without this fix, impact findings (about callers/affected files outside
the PR's changed files) would be incorrectly filtered as "not in scope"
during follow-up reviews.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(auth): enhance keychain service integration with config directory support

Added functionality to support profile-specific credentials by introducing a hash-based service name for macOS Keychain and updating Windows credential retrieval to utilize a provided config directory. This ensures that tokens are fetched from the correct profile-specific storage locations, improving credential management across different environments.

Changes include:
- New functions for calculating config directory hashes and generating keychain service names.
- Updated `get_token_from_keychain` and related functions to accept an optional config directory argument.
- Enhanced logging for better debugging when no token is found.

This aligns the backend credential handling with the frontend's expectations for profile-specific storage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 14:34:08 +01:00

1076 lines
38 KiB
Python

"""
GitHub Automation Data Models
=============================
Data structures for GitHub automation features.
Stored in .auto-claude/github/pr/ and .auto-claude/github/issues/
All save() operations use file locking to prevent corruption in concurrent scenarios.
"""
from __future__ import annotations
import json
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from pathlib import Path
try:
from .file_lock import locked_json_update, locked_json_write
except (ImportError, ValueError, SystemError):
from file_lock import locked_json_update, locked_json_write
class ReviewSeverity(str, Enum):
"""Severity levels for PR review findings."""
CRITICAL = "critical"
HIGH = "high"
MEDIUM = "medium"
LOW = "low"
class ReviewCategory(str, Enum):
"""Categories for PR review findings."""
SECURITY = "security"
QUALITY = "quality"
STYLE = "style"
TEST = "test"
DOCS = "docs"
PATTERN = "pattern"
PERFORMANCE = "performance"
VERIFICATION_FAILED = "verification_failed" # NEW: Cannot verify requirements/paths
REDUNDANCY = "redundancy" # NEW: Duplicate code/logic detected
class ReviewPass(str, Enum):
"""Multi-pass review stages."""
QUICK_SCAN = "quick_scan"
SECURITY = "security"
QUALITY = "quality"
DEEP_ANALYSIS = "deep_analysis"
STRUCTURAL = "structural" # Feature creep, architecture, PR structure
AI_COMMENT_TRIAGE = "ai_comment_triage" # Verify other AI tool comments
class MergeVerdict(str, Enum):
"""Clear verdict for whether PR can be merged."""
READY_TO_MERGE = "ready_to_merge" # No blockers, good to go
MERGE_WITH_CHANGES = "merge_with_changes" # Minor issues, fix before merge
NEEDS_REVISION = "needs_revision" # Significant issues, needs rework
BLOCKED = "blocked" # Critical issues, cannot merge
# Constants for branch-behind messaging (DRY - used across multiple reviewers)
BRANCH_BEHIND_BLOCKER_MSG = (
"Branch Out of Date: PR branch is behind the base branch and needs to be updated"
)
BRANCH_BEHIND_REASONING = (
"Branch is out of date with base branch. Update branch first - "
"if no conflicts arise, you can merge. If merge conflicts arise, "
"resolve them and run follow-up review again."
)
# =============================================================================
# Verdict Helper Functions (testable logic extracted from orchestrator)
# =============================================================================
def verdict_from_severity_counts(
critical_count: int = 0,
high_count: int = 0,
medium_count: int = 0,
low_count: int = 0,
) -> MergeVerdict:
"""
Determine merge verdict based on finding severity counts.
This is the canonical implementation of severity-to-verdict mapping.
Extracted here so it can be tested directly and reused.
Args:
critical_count: Number of critical severity findings
high_count: Number of high severity findings
medium_count: Number of medium severity findings
low_count: Number of low severity findings
Returns:
MergeVerdict based on severity levels
"""
if critical_count > 0:
return MergeVerdict.BLOCKED
elif high_count > 0 or medium_count > 0:
return MergeVerdict.NEEDS_REVISION
# Low findings or no findings -> ready to merge
return MergeVerdict.READY_TO_MERGE
def apply_merge_conflict_override(
verdict: MergeVerdict,
has_merge_conflicts: bool,
) -> MergeVerdict:
"""
Apply merge conflict override to verdict.
Merge conflicts always result in BLOCKED, regardless of other verdicts.
Args:
verdict: The current verdict
has_merge_conflicts: Whether PR has merge conflicts
Returns:
BLOCKED if conflicts exist, otherwise original verdict
"""
if has_merge_conflicts:
return MergeVerdict.BLOCKED
return verdict
def apply_branch_behind_downgrade(
verdict: MergeVerdict,
merge_state_status: str,
) -> MergeVerdict:
"""
Apply branch-behind status downgrade to verdict.
BEHIND status downgrades READY_TO_MERGE and MERGE_WITH_CHANGES to NEEDS_REVISION.
BLOCKED verdict is preserved (not downgraded).
Args:
verdict: The current verdict
merge_state_status: The merge state status (e.g., "BEHIND", "CLEAN")
Returns:
Downgraded verdict if behind, otherwise original
"""
if merge_state_status == "BEHIND":
if verdict in (MergeVerdict.READY_TO_MERGE, MergeVerdict.MERGE_WITH_CHANGES):
return MergeVerdict.NEEDS_REVISION
return verdict
def apply_ci_status_override(
verdict: MergeVerdict,
failing_count: int = 0,
pending_count: int = 0,
) -> MergeVerdict:
"""
Apply CI status override to verdict.
Failing CI -> BLOCKED (only for READY_TO_MERGE or MERGE_WITH_CHANGES verdicts)
Pending CI -> NEEDS_REVISION (only for READY_TO_MERGE or MERGE_WITH_CHANGES verdicts)
BLOCKED and NEEDS_REVISION verdicts are preserved as-is.
Args:
verdict: The current verdict
failing_count: Number of failing CI checks
pending_count: Number of pending CI checks
Returns:
Updated verdict based on CI status
"""
if failing_count > 0:
if verdict in (MergeVerdict.READY_TO_MERGE, MergeVerdict.MERGE_WITH_CHANGES):
return MergeVerdict.BLOCKED
elif pending_count > 0:
if verdict in (MergeVerdict.READY_TO_MERGE, MergeVerdict.MERGE_WITH_CHANGES):
return MergeVerdict.NEEDS_REVISION
return verdict
def verdict_to_github_status(verdict: MergeVerdict) -> str:
"""
Map merge verdict to GitHub review overall status.
Args:
verdict: The merge verdict
Returns:
GitHub review status: "approve", "comment", or "request_changes"
"""
if verdict == MergeVerdict.BLOCKED:
return "request_changes"
elif verdict == MergeVerdict.NEEDS_REVISION:
return "request_changes"
elif verdict == MergeVerdict.MERGE_WITH_CHANGES:
return "comment"
else:
return "approve"
class AICommentVerdict(str, Enum):
"""Verdict on AI tool comments (CodeRabbit, Cursor, Greptile, etc.)."""
CRITICAL = "critical" # Must be addressed before merge
IMPORTANT = "important" # Should be addressed
NICE_TO_HAVE = "nice_to_have" # Optional improvement
TRIVIAL = "trivial" # Can be ignored
FALSE_POSITIVE = "false_positive" # AI was wrong
ADDRESSED = "addressed" # Valid issue that was fixed in a subsequent commit
class TriageCategory(str, Enum):
"""Issue triage categories."""
BUG = "bug"
FEATURE = "feature"
DOCUMENTATION = "documentation"
QUESTION = "question"
DUPLICATE = "duplicate"
SPAM = "spam"
FEATURE_CREEP = "feature_creep"
class AutoFixStatus(str, Enum):
"""Status for auto-fix operations."""
# Initial states
PENDING = "pending"
ANALYZING = "analyzing"
# Spec creation states
CREATING_SPEC = "creating_spec"
WAITING_APPROVAL = "waiting_approval" # P1-3: Human review gate
# Build states
BUILDING = "building"
QA_REVIEW = "qa_review"
# PR states
PR_CREATED = "pr_created"
MERGE_CONFLICT = "merge_conflict" # P1-3: Conflict resolution needed
# Terminal states
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled" # P1-3: User cancelled
# Special states
STALE = "stale" # P1-3: Issue updated after spec creation
RATE_LIMITED = "rate_limited" # P1-3: Waiting for rate limit reset
@classmethod
def terminal_states(cls) -> set[AutoFixStatus]:
"""States that represent end of workflow."""
return {cls.COMPLETED, cls.FAILED, cls.CANCELLED}
@classmethod
def recoverable_states(cls) -> set[AutoFixStatus]:
"""States that can be recovered from."""
return {cls.FAILED, cls.STALE, cls.RATE_LIMITED, cls.MERGE_CONFLICT}
@classmethod
def active_states(cls) -> set[AutoFixStatus]:
"""States that indicate work in progress."""
return {
cls.PENDING,
cls.ANALYZING,
cls.CREATING_SPEC,
cls.BUILDING,
cls.QA_REVIEW,
cls.PR_CREATED,
}
def can_transition_to(self, new_state: AutoFixStatus) -> bool:
"""Check if transition to new_state is valid."""
valid_transitions = {
AutoFixStatus.PENDING: {
AutoFixStatus.ANALYZING,
AutoFixStatus.CANCELLED,
},
AutoFixStatus.ANALYZING: {
AutoFixStatus.CREATING_SPEC,
AutoFixStatus.FAILED,
AutoFixStatus.CANCELLED,
AutoFixStatus.RATE_LIMITED,
},
AutoFixStatus.CREATING_SPEC: {
AutoFixStatus.WAITING_APPROVAL,
AutoFixStatus.BUILDING,
AutoFixStatus.FAILED,
AutoFixStatus.CANCELLED,
AutoFixStatus.STALE,
},
AutoFixStatus.WAITING_APPROVAL: {
AutoFixStatus.BUILDING,
AutoFixStatus.CANCELLED,
AutoFixStatus.STALE,
},
AutoFixStatus.BUILDING: {
AutoFixStatus.QA_REVIEW,
AutoFixStatus.FAILED,
AutoFixStatus.CANCELLED,
AutoFixStatus.RATE_LIMITED,
},
AutoFixStatus.QA_REVIEW: {
AutoFixStatus.PR_CREATED,
AutoFixStatus.BUILDING, # Fix loop
AutoFixStatus.FAILED,
AutoFixStatus.CANCELLED,
},
AutoFixStatus.PR_CREATED: {
AutoFixStatus.COMPLETED,
AutoFixStatus.MERGE_CONFLICT,
AutoFixStatus.FAILED,
},
AutoFixStatus.MERGE_CONFLICT: {
AutoFixStatus.BUILDING, # Retry after conflict resolution
AutoFixStatus.FAILED,
AutoFixStatus.CANCELLED,
},
AutoFixStatus.STALE: {
AutoFixStatus.ANALYZING, # Re-analyze with new issue content
AutoFixStatus.CANCELLED,
},
AutoFixStatus.RATE_LIMITED: {
AutoFixStatus.PENDING, # Resume after rate limit
AutoFixStatus.CANCELLED,
},
# Terminal states - no transitions
AutoFixStatus.COMPLETED: set(),
AutoFixStatus.FAILED: {AutoFixStatus.PENDING}, # Allow retry
AutoFixStatus.CANCELLED: set(),
}
return new_state in valid_transitions.get(self, set())
@dataclass
class PRReviewFinding:
"""A single finding from a PR review."""
id: str
severity: ReviewSeverity
category: ReviewCategory
title: str
description: str
file: str
line: int
end_line: int | None = None
suggested_fix: str | None = None
fixable: bool = False
# Evidence-based validation: actual code proving the issue exists
evidence: str | None = None # Actual code snippet showing the issue
verification_note: str | None = (
None # What evidence is missing or couldn't be verified
)
redundant_with: str | None = None # Reference to duplicate code (file:line)
# Finding validation fields (from finding-validator re-investigation)
validation_status: str | None = (
None # confirmed_valid, dismissed_false_positive, needs_human_review
)
validation_evidence: str | None = None # Code snippet examined during validation
validation_explanation: str | None = None # Why finding was validated/dismissed
# Cross-validation fields
# NOTE: confidence field is DEPRECATED - we use evidence-based validation, not confidence scores
# The finding-validator determines validity by examining actual code, not by confidence thresholds
confidence: float = 0.5 # DEPRECATED: No longer used for filtering
source_agents: list[str] = field(
default_factory=list
) # Which agents reported this finding
cross_validated: bool = (
False # Whether multiple agents agreed on this finding (signal, not filter)
)
# Impact finding flag - indicates this finding is about code OUTSIDE the PR's changed files
# (e.g., callers affected by contract changes). Used by _is_finding_in_scope() to allow
# findings about related files that aren't directly in the PR diff.
is_impact_finding: bool = False
def to_dict(self) -> dict:
return {
"id": self.id,
"severity": self.severity.value,
"category": self.category.value,
"title": self.title,
"description": self.description,
"file": self.file,
"line": self.line,
"end_line": self.end_line,
"suggested_fix": self.suggested_fix,
"fixable": self.fixable,
# Evidence-based validation fields
"evidence": self.evidence,
"verification_note": self.verification_note,
"redundant_with": self.redundant_with,
# Validation fields
"validation_status": self.validation_status,
"validation_evidence": self.validation_evidence,
"validation_explanation": self.validation_explanation,
# Cross-validation and confidence routing fields
"confidence": self.confidence,
"source_agents": self.source_agents,
"cross_validated": self.cross_validated,
# Impact finding flag
"is_impact_finding": self.is_impact_finding,
}
@classmethod
def from_dict(cls, data: dict) -> PRReviewFinding:
return cls(
id=data["id"],
severity=ReviewSeverity(data["severity"]),
category=ReviewCategory(data["category"]),
title=data["title"],
description=data["description"],
file=data["file"],
line=data["line"],
end_line=data.get("end_line"),
suggested_fix=data.get("suggested_fix"),
fixable=data.get("fixable", False),
# Evidence-based validation fields
evidence=data.get("evidence"),
verification_note=data.get("verification_note"),
redundant_with=data.get("redundant_with"),
# Validation fields
validation_status=data.get("validation_status"),
validation_evidence=data.get("validation_evidence"),
validation_explanation=data.get("validation_explanation"),
# Cross-validation and confidence routing fields
confidence=data.get("confidence", 0.5),
source_agents=data.get("source_agents", []),
cross_validated=data.get("cross_validated", False),
# Impact finding flag
is_impact_finding=data.get("is_impact_finding", False),
)
@dataclass
class AICommentTriage:
"""Triage result for an AI tool comment (CodeRabbit, Cursor, Greptile, etc.)."""
comment_id: int
tool_name: str # "CodeRabbit", "Cursor", "Greptile", etc.
original_comment: str
verdict: AICommentVerdict
reasoning: str
response_comment: str | None = None # Comment to post in reply
def to_dict(self) -> dict:
return {
"comment_id": self.comment_id,
"tool_name": self.tool_name,
"original_comment": self.original_comment,
"verdict": self.verdict.value,
"reasoning": self.reasoning,
"response_comment": self.response_comment,
}
@classmethod
def from_dict(cls, data: dict) -> AICommentTriage:
return cls(
comment_id=data["comment_id"],
tool_name=data["tool_name"],
original_comment=data["original_comment"],
verdict=AICommentVerdict(data["verdict"]),
reasoning=data["reasoning"],
response_comment=data.get("response_comment"),
)
@dataclass
class StructuralIssue:
"""Structural issue with the PR (feature creep, architecture, etc.)."""
id: str
issue_type: str # "feature_creep", "scope_creep", "architecture_violation", "poor_structure"
severity: ReviewSeverity
title: str
description: str
impact: str # Why this matters
suggestion: str # How to fix
def to_dict(self) -> dict:
return {
"id": self.id,
"issue_type": self.issue_type,
"severity": self.severity.value,
"title": self.title,
"description": self.description,
"impact": self.impact,
"suggestion": self.suggestion,
}
@classmethod
def from_dict(cls, data: dict) -> StructuralIssue:
return cls(
id=data["id"],
issue_type=data["issue_type"],
severity=ReviewSeverity(data["severity"]),
title=data["title"],
description=data["description"],
impact=data["impact"],
suggestion=data["suggestion"],
)
@dataclass
class PRReviewResult:
"""Complete result of a PR review."""
pr_number: int
repo: str
success: bool
findings: list[PRReviewFinding] = field(default_factory=list)
summary: str = ""
overall_status: str = "comment" # approve, request_changes, comment
review_id: int | None = None
reviewed_at: str = field(default_factory=lambda: datetime.now().isoformat())
error: str | None = None
# NEW: Enhanced verdict system
verdict: MergeVerdict = MergeVerdict.READY_TO_MERGE
verdict_reasoning: str = ""
blockers: list[str] = field(default_factory=list) # Issues that MUST be fixed
# NEW: Risk assessment
risk_assessment: dict = field(
default_factory=lambda: {
"complexity": "low", # low, medium, high
"security_impact": "none", # none, low, medium, critical
"scope_coherence": "good", # good, mixed, poor
}
)
# NEW: Structural issues and AI comment triages
structural_issues: list[StructuralIssue] = field(default_factory=list)
ai_comment_triages: list[AICommentTriage] = field(default_factory=list)
# NEW: Quick scan summary preserved
quick_scan_summary: dict = field(default_factory=dict)
# Follow-up review tracking
reviewed_commit_sha: str | None = None # HEAD SHA at time of review
reviewed_file_blobs: dict[str, str] = field(
default_factory=dict
) # filename → blob SHA at time of review (survives rebases)
is_followup_review: bool = False # True if this is a follow-up review
previous_review_id: int | None = None # Reference to the review this follows up on
resolved_findings: list[str] = field(default_factory=list) # Finding IDs now fixed
unresolved_findings: list[str] = field(
default_factory=list
) # Finding IDs still open
new_findings_since_last_review: list[str] = field(
default_factory=list
) # New issues in recent commits
# Posted findings tracking (for frontend state sync)
has_posted_findings: bool = False # True if any findings have been posted to GitHub
posted_finding_ids: list[str] = field(
default_factory=list
) # IDs of posted findings
posted_at: str | None = None # Timestamp when findings were posted
def to_dict(self) -> dict:
return {
"pr_number": self.pr_number,
"repo": self.repo,
"success": self.success,
"findings": [f.to_dict() for f in self.findings],
"summary": self.summary,
"overall_status": self.overall_status,
"review_id": self.review_id,
"reviewed_at": self.reviewed_at,
"error": self.error,
# NEW fields
"verdict": self.verdict.value,
"verdict_reasoning": self.verdict_reasoning,
"blockers": self.blockers,
"risk_assessment": self.risk_assessment,
"structural_issues": [s.to_dict() for s in self.structural_issues],
"ai_comment_triages": [t.to_dict() for t in self.ai_comment_triages],
"quick_scan_summary": self.quick_scan_summary,
# Follow-up review fields
"reviewed_commit_sha": self.reviewed_commit_sha,
"reviewed_file_blobs": self.reviewed_file_blobs,
"is_followup_review": self.is_followup_review,
"previous_review_id": self.previous_review_id,
"resolved_findings": self.resolved_findings,
"unresolved_findings": self.unresolved_findings,
"new_findings_since_last_review": self.new_findings_since_last_review,
# Posted findings tracking
"has_posted_findings": self.has_posted_findings,
"posted_finding_ids": self.posted_finding_ids,
"posted_at": self.posted_at,
}
@classmethod
def from_dict(cls, data: dict) -> PRReviewResult:
return cls(
pr_number=data["pr_number"],
repo=data["repo"],
success=data["success"],
findings=[PRReviewFinding.from_dict(f) for f in data.get("findings", [])],
summary=data.get("summary", ""),
overall_status=data.get("overall_status", "comment"),
review_id=data.get("review_id"),
reviewed_at=data.get("reviewed_at", datetime.now().isoformat()),
error=data.get("error"),
# NEW fields
verdict=MergeVerdict(data.get("verdict", "ready_to_merge")),
verdict_reasoning=data.get("verdict_reasoning", ""),
blockers=data.get("blockers", []),
risk_assessment=data.get(
"risk_assessment",
{
"complexity": "low",
"security_impact": "none",
"scope_coherence": "good",
},
),
structural_issues=[
StructuralIssue.from_dict(s) for s in data.get("structural_issues", [])
],
ai_comment_triages=[
AICommentTriage.from_dict(t) for t in data.get("ai_comment_triages", [])
],
quick_scan_summary=data.get("quick_scan_summary", {}),
# Follow-up review fields
reviewed_commit_sha=data.get("reviewed_commit_sha"),
reviewed_file_blobs=data.get("reviewed_file_blobs", {}),
is_followup_review=data.get("is_followup_review", False),
previous_review_id=data.get("previous_review_id"),
resolved_findings=data.get("resolved_findings", []),
unresolved_findings=data.get("unresolved_findings", []),
new_findings_since_last_review=data.get(
"new_findings_since_last_review", []
),
# Posted findings tracking
has_posted_findings=data.get("has_posted_findings", False),
posted_finding_ids=data.get("posted_finding_ids", []),
posted_at=data.get("posted_at"),
)
async def save(self, github_dir: Path) -> None:
"""Save review result to .auto-claude/github/pr/ with file locking."""
pr_dir = github_dir / "pr"
pr_dir.mkdir(parents=True, exist_ok=True)
review_file = pr_dir / f"review_{self.pr_number}.json"
# Atomic locked write
await locked_json_write(review_file, self.to_dict(), timeout=5.0)
# Update index with locking
await self._update_index(pr_dir)
async def _update_index(self, pr_dir: Path) -> None:
"""Update the PR review index with file locking."""
index_file = pr_dir / "index.json"
def update_index(current_data):
"""Update function for atomic index update."""
if current_data is None:
current_data = {"reviews": [], "last_updated": None}
# Update or add entry
reviews = current_data.get("reviews", [])
existing = next(
(r for r in reviews if r["pr_number"] == self.pr_number), None
)
entry = {
"pr_number": self.pr_number,
"repo": self.repo,
"overall_status": self.overall_status,
"findings_count": len(self.findings),
"reviewed_at": self.reviewed_at,
}
if existing:
reviews = [
entry if r["pr_number"] == self.pr_number else r for r in reviews
]
else:
reviews.append(entry)
current_data["reviews"] = reviews
current_data["last_updated"] = datetime.now().isoformat()
return current_data
# Atomic locked update
await locked_json_update(index_file, update_index, timeout=5.0)
@classmethod
def load(cls, github_dir: Path, pr_number: int) -> PRReviewResult | None:
"""Load a review result from disk."""
review_file = github_dir / "pr" / f"review_{pr_number}.json"
if not review_file.exists():
return None
with open(review_file, encoding="utf-8") as f:
return cls.from_dict(json.load(f))
@dataclass
class FollowupReviewContext:
"""Context for a follow-up review."""
pr_number: int
previous_review: PRReviewResult
previous_commit_sha: str
current_commit_sha: str
# Changes since last review
commits_since_review: list[dict] = field(default_factory=list)
files_changed_since_review: list[str] = field(default_factory=list)
diff_since_review: str = ""
# Comments since last review
contributor_comments_since_review: list[dict] = field(default_factory=list)
ai_bot_comments_since_review: list[dict] = field(default_factory=list)
# PR reviews since last review (formal review submissions from Cursor, CodeRabbit, etc.)
# These are different from comments - they're full review submissions with body text
pr_reviews_since_review: list[dict] = field(default_factory=list)
# Merge conflict status
has_merge_conflicts: bool = False # True if PR has conflicts with base branch
merge_state_status: str = (
"" # BEHIND, BLOCKED, CLEAN, DIRTY, HAS_HOOKS, UNKNOWN, UNSTABLE
)
# CI status - passed to AI orchestrator so it can factor into verdict
# Dict with: passing, failing, pending, failed_checks, awaiting_approval
ci_status: dict = field(default_factory=dict)
# Error flag - if set, context gathering failed and data may be incomplete
error: str | None = None
@dataclass
class TriageResult:
"""Result of triaging a single issue."""
issue_number: int
repo: str
category: TriageCategory
confidence: float # 0.0 to 1.0
labels_to_add: list[str] = field(default_factory=list)
labels_to_remove: list[str] = field(default_factory=list)
is_duplicate: bool = False
duplicate_of: int | None = None
is_spam: bool = False
is_feature_creep: bool = False
suggested_breakdown: list[str] = field(default_factory=list)
priority: str = "medium" # high, medium, low
comment: str | None = None
triaged_at: str = field(default_factory=lambda: datetime.now().isoformat())
def to_dict(self) -> dict:
return {
"issue_number": self.issue_number,
"repo": self.repo,
"category": self.category.value,
"confidence": self.confidence,
"labels_to_add": self.labels_to_add,
"labels_to_remove": self.labels_to_remove,
"is_duplicate": self.is_duplicate,
"duplicate_of": self.duplicate_of,
"is_spam": self.is_spam,
"is_feature_creep": self.is_feature_creep,
"suggested_breakdown": self.suggested_breakdown,
"priority": self.priority,
"comment": self.comment,
"triaged_at": self.triaged_at,
}
@classmethod
def from_dict(cls, data: dict) -> TriageResult:
return cls(
issue_number=data["issue_number"],
repo=data["repo"],
category=TriageCategory(data["category"]),
confidence=data["confidence"],
labels_to_add=data.get("labels_to_add", []),
labels_to_remove=data.get("labels_to_remove", []),
is_duplicate=data.get("is_duplicate", False),
duplicate_of=data.get("duplicate_of"),
is_spam=data.get("is_spam", False),
is_feature_creep=data.get("is_feature_creep", False),
suggested_breakdown=data.get("suggested_breakdown", []),
priority=data.get("priority", "medium"),
comment=data.get("comment"),
triaged_at=data.get("triaged_at", datetime.now().isoformat()),
)
async def save(self, github_dir: Path) -> None:
"""Save triage result to .auto-claude/github/issues/ with file locking."""
issues_dir = github_dir / "issues"
issues_dir.mkdir(parents=True, exist_ok=True)
triage_file = issues_dir / f"triage_{self.issue_number}.json"
# Atomic locked write
await locked_json_write(triage_file, self.to_dict(), timeout=5.0)
@classmethod
def load(cls, github_dir: Path, issue_number: int) -> TriageResult | None:
"""Load a triage result from disk."""
triage_file = github_dir / "issues" / f"triage_{issue_number}.json"
if not triage_file.exists():
return None
with open(triage_file, encoding="utf-8") as f:
return cls.from_dict(json.load(f))
@dataclass
class AutoFixState:
"""State tracking for auto-fix operations."""
issue_number: int
issue_url: str
repo: str
status: AutoFixStatus = AutoFixStatus.PENDING
spec_id: str | None = None
spec_dir: str | None = None
pr_number: int | None = None
pr_url: str | None = None
bot_comments: list[str] = field(default_factory=list)
error: str | None = None
created_at: str = field(default_factory=lambda: datetime.now().isoformat())
updated_at: str = field(default_factory=lambda: datetime.now().isoformat())
def to_dict(self) -> dict:
return {
"issue_number": self.issue_number,
"issue_url": self.issue_url,
"repo": self.repo,
"status": self.status.value,
"spec_id": self.spec_id,
"spec_dir": self.spec_dir,
"pr_number": self.pr_number,
"pr_url": self.pr_url,
"bot_comments": self.bot_comments,
"error": self.error,
"created_at": self.created_at,
"updated_at": self.updated_at,
}
@classmethod
def from_dict(cls, data: dict) -> AutoFixState:
issue_number = data["issue_number"]
repo = data["repo"]
# Construct issue_url if missing (for backwards compatibility with old state files)
issue_url = (
data.get("issue_url") or f"https://github.com/{repo}/issues/{issue_number}"
)
return cls(
issue_number=issue_number,
issue_url=issue_url,
repo=repo,
status=AutoFixStatus(data.get("status", "pending")),
spec_id=data.get("spec_id"),
spec_dir=data.get("spec_dir"),
pr_number=data.get("pr_number"),
pr_url=data.get("pr_url"),
bot_comments=data.get("bot_comments", []),
error=data.get("error"),
created_at=data.get("created_at", datetime.now().isoformat()),
updated_at=data.get("updated_at", datetime.now().isoformat()),
)
def update_status(self, status: AutoFixStatus) -> None:
"""Update status and timestamp with transition validation."""
if not self.status.can_transition_to(status):
raise ValueError(
f"Invalid state transition: {self.status.value} -> {status.value}"
)
self.status = status
self.updated_at = datetime.now().isoformat()
async def save(self, github_dir: Path) -> None:
"""Save auto-fix state to .auto-claude/github/issues/ with file locking."""
issues_dir = github_dir / "issues"
issues_dir.mkdir(parents=True, exist_ok=True)
autofix_file = issues_dir / f"autofix_{self.issue_number}.json"
# Atomic locked write
await locked_json_write(autofix_file, self.to_dict(), timeout=5.0)
# Update index with locking
await self._update_index(issues_dir)
async def _update_index(self, issues_dir: Path) -> None:
"""Update the issues index with auto-fix queue using file locking."""
index_file = issues_dir / "index.json"
def update_index(current_data):
"""Update function for atomic index update."""
if current_data is None:
current_data = {
"triaged": [],
"auto_fix_queue": [],
"last_updated": None,
}
# Update auto-fix queue
queue = current_data.get("auto_fix_queue", [])
existing = next(
(q for q in queue if q["issue_number"] == self.issue_number), None
)
entry = {
"issue_number": self.issue_number,
"repo": self.repo,
"status": self.status.value,
"spec_id": self.spec_id,
"pr_number": self.pr_number,
"updated_at": self.updated_at,
}
if existing:
queue = [
entry if q["issue_number"] == self.issue_number else q
for q in queue
]
else:
queue.append(entry)
current_data["auto_fix_queue"] = queue
current_data["last_updated"] = datetime.now().isoformat()
return current_data
# Atomic locked update
await locked_json_update(index_file, update_index, timeout=5.0)
@classmethod
def load(cls, github_dir: Path, issue_number: int) -> AutoFixState | None:
"""Load an auto-fix state from disk."""
autofix_file = github_dir / "issues" / f"autofix_{issue_number}.json"
if not autofix_file.exists():
return None
with open(autofix_file, encoding="utf-8") as f:
return cls.from_dict(json.load(f))
@dataclass
class GitHubRunnerConfig:
"""Configuration for GitHub automation runners."""
# Authentication
token: str
repo: str # owner/repo format
bot_token: str | None = None # Separate bot account token
# Auto-fix settings
auto_fix_enabled: bool = False
auto_fix_labels: list[str] = field(default_factory=lambda: ["auto-fix"])
require_human_approval: bool = True
# Permission settings
auto_fix_allowed_roles: list[str] = field(
default_factory=lambda: ["OWNER", "MEMBER", "COLLABORATOR"]
)
allow_external_contributors: bool = False
# Triage settings
triage_enabled: bool = False
duplicate_threshold: float = 0.80
spam_threshold: float = 0.75
feature_creep_threshold: float = 0.70
enable_triage_comments: bool = False
# PR review settings
pr_review_enabled: bool = False
auto_post_reviews: bool = False
allow_fix_commits: bool = True
review_own_prs: bool = False # Whether bot can review its own PRs
use_parallel_orchestrator: bool = (
True # Use SDK subagent parallel orchestrator (default)
)
# Model settings
# Note: Default uses shorthand "sonnet" which gets resolved via resolve_model_id()
# to respect environment variable overrides (e.g., ANTHROPIC_DEFAULT_SONNET_MODEL)
model: str = "sonnet"
thinking_level: str = "medium"
def to_dict(self) -> dict:
return {
"token": "***", # Never save token
"repo": self.repo,
"bot_token": "***" if self.bot_token else None,
"auto_fix_enabled": self.auto_fix_enabled,
"auto_fix_labels": self.auto_fix_labels,
"require_human_approval": self.require_human_approval,
"auto_fix_allowed_roles": self.auto_fix_allowed_roles,
"allow_external_contributors": self.allow_external_contributors,
"triage_enabled": self.triage_enabled,
"duplicate_threshold": self.duplicate_threshold,
"spam_threshold": self.spam_threshold,
"feature_creep_threshold": self.feature_creep_threshold,
"enable_triage_comments": self.enable_triage_comments,
"pr_review_enabled": self.pr_review_enabled,
"review_own_prs": self.review_own_prs,
"auto_post_reviews": self.auto_post_reviews,
"allow_fix_commits": self.allow_fix_commits,
"model": self.model,
"thinking_level": self.thinking_level,
}
def save_settings(self, github_dir: Path) -> None:
"""Save non-sensitive settings to config.json."""
github_dir.mkdir(parents=True, exist_ok=True)
config_file = github_dir / "config.json"
# Save without tokens
settings = self.to_dict()
settings.pop("token", None)
settings.pop("bot_token", None)
with open(config_file, "w", encoding="utf-8") as f:
json.dump(settings, f, indent=2)
@classmethod
def load_settings(
cls, github_dir: Path, token: str, repo: str, bot_token: str | None = None
) -> GitHubRunnerConfig:
"""Load settings from config.json, with tokens provided separately."""
config_file = github_dir / "config.json"
if config_file.exists():
with open(config_file, encoding="utf-8") as f:
settings = json.load(f)
else:
settings = {}
return cls(
token=token,
repo=repo,
bot_token=bot_token,
auto_fix_enabled=settings.get("auto_fix_enabled", False),
auto_fix_labels=settings.get("auto_fix_labels", ["auto-fix"]),
require_human_approval=settings.get("require_human_approval", True),
auto_fix_allowed_roles=settings.get(
"auto_fix_allowed_roles", ["OWNER", "MEMBER", "COLLABORATOR"]
),
allow_external_contributors=settings.get(
"allow_external_contributors", False
),
triage_enabled=settings.get("triage_enabled", False),
duplicate_threshold=settings.get("duplicate_threshold", 0.80),
spam_threshold=settings.get("spam_threshold", 0.75),
feature_creep_threshold=settings.get("feature_creep_threshold", 0.70),
enable_triage_comments=settings.get("enable_triage_comments", False),
pr_review_enabled=settings.get("pr_review_enabled", False),
review_own_prs=settings.get("review_own_prs", False),
auto_post_reviews=settings.get("auto_post_reviews", False),
allow_fix_commits=settings.get("allow_fix_commits", True),
# Note: model is stored as shorthand and resolved via resolve_model_id()
model=settings.get("model", "sonnet"),
thinking_level=settings.get("thinking_level", "medium"),
)