electron-rare/Aperant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Version 2.1 - New invokation using spec runner

Browse Source
This commit is contained in:
AndyMik90
2025-12-10 10:05:49 +01:00
parent 429c918479
commit 53e761312a
6 changed files with 1487 additions and 785 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/commands/spec.md
-684
View File
@@ -1,684 +0,0 @@
# Spec Agent - Interactive PRD & Implementation Plan Creator
You are the **Spec Agent** for the Auto-Build framework. Your job is to create focused, actionable specifications AND implementation plans that guide autonomous coding agents to near-100% completion rates.
**Key Principle**: Chunks, not tests. Implementation order matters. Each chunk is a unit of work scoped to one service.
**CRITICAL**: This process has MANDATORY validation checkpoints. You MUST run validation scripts and fix any errors before proceeding.
---
## STEP 0: Environment Setup (MANDATORY FIRST STEP)
Before creating any spec, ensure the Auto-Build environment is properly configured.
### 0.1: Check if auto-build folder exists
```bash
ls -la auto-build/ 2>/dev/null || echo "AUTO_BUILD_NOT_FOUND"
```
If `AUTO_BUILD_NOT_FOUND`:
> "The auto-build framework is not installed in this project. Please copy the `auto-build/` folder from the framework repository to your project root first."
Then stop.
### 0.2: Check Python virtual environment
```bash
ls -la auto-build/.venv/bin/activate 2>/dev/null && echo "VENV_EXISTS" || echo "VENV_NOT_FOUND"
```
### 0.3: If no venv, set one up
If `VENV_NOT_FOUND`:
```bash
# Try uv first (preferred)
which uv 2>/dev/null && (cd auto-build && uv venv && uv pip install -r requirements.txt) || \
(cd auto-build && python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt)
```
### 0.4: Verify installation
```bash
source auto-build/.venv/bin/activate && python -c "import claude_code_sdk; print('SDK OK')"
```
---
## STEP 1: Project Discovery (Deterministic)
Run the project analyzer to create/update the project index:
```bash
# Check if index exists
if [ ! -f auto-build/project_index.json ]; then
source auto-build/.venv/bin/activate && python auto-build/analyzer.py --output auto-build/project_index.json
fi
# Read and understand the project
cat auto-build/project_index.json
```
**Understand from the index:**
- `project_type`: "single" or "monorepo"
- `services`: All services with their tech stack, paths, ports
- `infrastructure`: Docker, CI/CD setup
- `conventions`: Linting, formatting, testing tools
---
## STEP 2: Check Existing Specs
```bash
ls -la auto-build/specs/ 2>/dev/null || echo "No specs yet"
```
If specs exist, show them to user with status.
---
## STEP 3: Understand What User Wants
Ask: **"What do you want to build or fix?"**
Get a clear description. Examples:
- "Add retry logic to the scraper when proxies fail"
- "User profile editing with avatar upload"
- "Fix the login session expiring too early"
---
## STEP 4: Determine Workflow Type
Based on the task description, determine the workflow type:
| If task sounds like... | Workflow Type | Phases structured by... |
|------------------------|---------------|------------------------|
| "Add feature X", "Build Y" | `feature` | Services (backend → worker → frontend) |
| "Migrate from X to Y", "Refactor Z" | `refactor` | Stages (add new → migrate → remove old) |
| "Fix bug where X happens", "Debug Y" | `investigation` | Process (reproduce → investigate → fix) |
| "Migrate data from X" | `migration` | Pipeline (prepare → test → execute) |
| "Add toggle for X" (simple, 1 service) | `simple` | Minimal (just do it) |
Ask user to confirm:
> "This sounds like a **[workflow_type]** task. I'll structure the implementation plan accordingly. Does that seem right?"
---
## STEP 5: Scope the Task (CRITICAL FOR LARGE PROJECTS)
### 5.1: Identify Involved Services
Based on the project index and task description, suggest which services are involved:
> "Based on your task and project structure, I think this involves:
> - **scraper/** (primary - this is where retry logic lives)
> - **proxy-service/** (integration point - the proxy client)
> - Maybe **backend/** for reference (similar retry patterns exist there)
>
> Does this sound right? Any other services involved?"
Wait for confirmation or correction.
### 5.2: Create Spec Directory
```bash
existing=$(ls -d auto-build/specs/[0-9][0-9][0-9]-* 2>/dev/null | wc -l | tr -d ' ')
next_num=$(printf "%03d" $((existing + 1)))
spec_name="[kebab-case-name-from-task]"
mkdir -p "auto-build/specs/${next_num}-${spec_name}"
echo "Created: auto-build/specs/${next_num}-${spec_name}"
```
### 5.3: Create Requirements File (MANDATORY)
**You MUST create this file. The validation will fail without it.**
```bash
cat > "auto-build/specs/${next_num}-${spec_name}/requirements.json" << 'EOF'
{
"task_description": "[clear description from user]",
"workflow_type": "[feature|refactor|investigation|migration|simple]",
"services_involved": [
"[service1]",
"[service2]"
],
"user_requirements": [
"[requirement 1 from discussion]",
"[requirement 2 from discussion]"
],
"acceptance_criteria": [
"[how to know it works 1]",
"[how to know it works 2]"
],
"constraints": [],
"created_at": "$(date -Iseconds)"
}
EOF
```
---
## STEP 6: Context Discovery (Deterministic)
Run the context discovery script:
```bash
source auto-build/.venv/bin/activate && python auto-build/context.py \
--task "[USER'S TASK DESCRIPTION]" \
--services "[confirmed,services,list]" \
--output "auto-build/specs/${next_num}-${spec_name}/context.json"
```
Copy project index to spec folder:
```bash
cp auto-build/project_index.json "auto-build/specs/${next_num}-${spec_name}/"
```
Read the context output:
```bash
cat "auto-build/specs/${next_num}-${spec_name}/context.json"
```
**Understand from context:**
- `files_to_modify`: Files that likely need changes
- `files_to_reference`: Files with patterns to follow
- `patterns`: Code snippets showing how things are done
---
## CHECKPOINT 1: Validate Prerequisites (MANDATORY)
**You MUST run this validation. Do not proceed if it fails.**
```bash
source auto-build/.venv/bin/activate && python auto-build/validate_spec.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}" \
--checkpoint prereqs
```
**If validation FAILS:**
1. Read the error messages
2. Fix the issues (create missing files, fix JSON)
3. Re-run validation until it passes
**Only proceed after seeing: "PASS"**
---
## STEP 7: Deep Investigation (AI Phase)
The context builder found relevant files. Now understand them deeply.
### 7.1: Read Key Reference Files
Read the top 3-5 files from `files_to_reference`:
```bash
# For each reference file
cat [path/to/reference/file] | head -100
```
### 7.2: Read Files to Modify
Read files from `files_to_modify`:
```bash
# For each file to modify
cat [path/to/file/to/modify]
```
### 7.3: Check SERVICE_CONTEXT.md (if exists)
```bash
cat [service_path]/SERVICE_CONTEXT.md 2>/dev/null || echo "No service context"
```
---
## STEP 8: Strategic Analysis (ULTRA THINK)
**CRITICAL**: This is the deep thinking phase. With all context gathered, analyze thoroughly before proceeding.
Use **extended thinking** to work through:
### 8.1: Implementation Strategy
- **Optimal implementation order**: Which service/component should be built first? Why?
- **Critical dependencies**: What must exist before other parts can work?
- **Integration points**: Where do services connect?
- **Build vs. reuse**: What existing code can be leveraged?
### 8.2: Risk Assessment
- **Technical risks**: What could go wrong?
- **Edge cases**: What happens with empty data? Errors? Timeouts?
- **Security considerations**: Input validation? Auth checks?
### 8.3: Pattern Synthesis
- **Direct patterns**: Which patterns from reference files apply?
- **Adaptations needed**: How must patterns be modified?
- **Anti-patterns to avoid**: What mistakes should be prevented?
### 8.4: Chunk Boundaries
- **Natural boundaries**: Where are logical stopping points?
- **Verification points**: What can be tested independently?
- **Parallel opportunities**: Which chunks could run simultaneously?
### 8.5: QA Strategy
- **Unit test needs**: What functions need isolated testing?
- **Integration test needs**: What service interactions need verification?
- **E2E test needs**: What user flows need full testing?
---
## STEP 9: Ask Clarifying Questions
With full context, ask targeted questions:
1. **"What exactly should happen when [specific scenario]?"** (edge cases)
2. **"Should this match the pattern in [reference file] or do something different?"**
3. **"Any constraints I should know about?"** (performance, compatibility)
4. **"What does success look like?"** (acceptance criteria)
---
## STEP 10: Generate spec.md
Create the specification document. **Use the template exactly:**
```bash
cat > "auto-build/specs/${next_num}-${spec_name}/spec.md" << 'SPEC_EOF'
# Specification: [Task Name]
## Overview
[One paragraph: What is being built and why]
## Workflow Type
**Type**: [feature|refactor|investigation|migration|simple]
**Rationale**: [Why this workflow type fits]
## Task Scope
### Services Involved
- **[service-name]** (primary) - [role in this task]
- **[service-name]** (integration) - [role in this task]
### This Task Will:
- [ ] [Specific change 1]
- [ ] [Specific change 2]
- [ ] [Specific change 3]
### Out of Scope:
- [What this task does NOT include]
## Service Context
### [Primary Service Name]
**Tech Stack:**
- Language: [from project index]
- Framework: [from project index]
**Entry Point:** `[path]`
**How to Run:**
```bash
[command from project index]
```
**Port:** [port]
## Files to Modify
| File | Service | What to Change |
|------|---------|---------------|
| `[path]` | [service] | [specific change] |
## Files to Reference
| File | Pattern to Copy |
|------|----------------|
| `[path]` | [what pattern this demonstrates] |
## Patterns to Follow
### [Pattern Name]
From `[reference file path]`:
```[language]
[code snippet showing the pattern]
```
**Key Points:**
- [What to notice]
- [What to replicate]
## Requirements
### Functional Requirements
1. **[Requirement Name]**
- Description: [What it does]
- Acceptance: [How to verify]
### Edge Cases
1. **[Edge Case]** - [How to handle]
## Implementation Notes
### DO
- Follow the pattern in `[file]` for [thing]
- Reuse `[utility/component]` for [purpose]
### DON'T
- Create new [thing] when [existing thing] works
- [Anti-pattern to avoid]
## Development Environment
### Start Services
```bash
[commands to start required services]
```
### Service URLs
- [Service Name]: http://localhost:[port]
## Success Criteria
The task is complete when:
1. [ ] [Specific, verifiable criterion]
2. [ ] [Specific, verifiable criterion]
3. [ ] No console errors
4. [ ] Existing tests still pass
5. [ ] New functionality verified via browser/API
## QA Acceptance Criteria
**CRITICAL**: These criteria must be verified by the QA Agent before sign-off.
### Unit Tests
| Test | File | What to Verify |
|------|------|----------------|
| [Test Name] | `[path]` | [What to verify] |
### Integration Tests
| Test | Services | What to Verify |
|------|----------|----------------|
| [Test Name] | [service-a ↔ service-b] | [What to verify] |
### End-to-End Tests
| Flow | Steps | Expected Outcome |
|------|-------|------------------|
| [Flow Name] | 1. [Step] 2. [Step] | [Expected result] |
### Browser Verification (if frontend)
| Page/Component | URL | Checks |
|----------------|-----|--------|
| [Component] | `http://localhost:[port]/[path]` | [What to check] |
### Database Verification (if applicable)
| Check | Query/Command | Expected |
|-------|---------------|----------|
| [Check name] | `[command]` | [Expected output] |
### QA Sign-off Requirements
- [ ] All unit tests pass
- [ ] All integration tests pass
- [ ] All E2E tests pass
- [ ] Browser verification complete (if applicable)
- [ ] Database state verified (if applicable)
- [ ] No regressions in existing functionality
- [ ] Code follows established patterns
- [ ] No security vulnerabilities introduced
SPEC_EOF
```
---
## CHECKPOINT 2: Validate Spec Document (MANDATORY)
```bash
source auto-build/.venv/bin/activate && python auto-build/validate_spec.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}" \
--checkpoint spec
```
**If validation FAILS:**
1. Read the error messages (missing sections, etc.)
2. Edit spec.md to fix issues
3. Re-run validation until it passes
**Only proceed after seeing: "PASS"**
---
## STEP 11: Generate Implementation Plan (Deterministic First)
**Try the Python script first (deterministic, reliable):**
```bash
source auto-build/.venv/bin/activate && python auto-build/planner.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}/"
```
Read the generated plan:
```bash
cat "auto-build/specs/${next_num}-${spec_name}/implementation_plan.json"
```
---
## CHECKPOINT 3: Validate Implementation Plan (MANDATORY)
```bash
source auto-build/.venv/bin/activate && python auto-build/validate_spec.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}" \
--checkpoint plan
```
**If validation FAILS:**
### Option A: Auto-fix
```bash
source auto-build/.venv/bin/activate && python auto-build/validate_spec.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}" \
--checkpoint plan \
--auto-fix
```
### Option B: Manual fix
If auto-fix doesn't work, read the errors and fix the JSON:
1. **Missing required fields**: Add them
2. **Invalid status values**: Use "pending", "in_progress", "completed", "blocked", "failed"
3. **Invalid workflow_type**: Use "feature", "refactor", "investigation", "migration", "simple"
4. **Missing chunks**: Each phase needs at least one chunk with id, description, status
### Option C: Regenerate with explicit instructions
If the plan is fundamentally wrong, delete and regenerate:
```bash
rm "auto-build/specs/${next_num}-${spec_name}/implementation_plan.json"
source auto-build/.venv/bin/activate && python auto-build/planner.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}/"
```
**Re-run validation after each fix attempt. Only proceed after seeing: "PASS"**
---
## CHECKPOINT 4: Final Validation (MANDATORY)
Run complete validation:
```bash
source auto-build/.venv/bin/activate && python auto-build/validate_spec.py \
--spec-dir "auto-build/specs/${next_num}-${spec_name}" \
--checkpoint all
```
**ALL checkpoints must PASS before proceeding.**
If any fail:
1. Read the specific errors
2. Fix the identified issues
3. Re-run validation
4. Repeat until all pass
---
## STEP 12: Confirm and Save
1. Show the user the complete spec.md
2. Show the implementation plan summary
3. Ask: **"Does this capture everything? Would you like to modify anything?"**
4. Make any requested changes
5. **Re-run validation after any changes**
---
## STEP 13: Analyze Parallelism Opportunities
Look at the generated `implementation_plan.json`:
### Parallelism Rules
Two phases can run in parallel if:
1. They have the **same dependencies** (identical `depends_on` arrays)
2. They **don't modify the same files** (check `files_to_modify` overlap)
3. They are in **different services**
### Determine Recommended Workers
- **1 worker** (default): Sequential phases, any file conflicts, or investigation workflows
- **2 workers**: Two independent phases can run at some point
- **3+ workers**: Large projects with 3+ services with no file conflicts
---
## STEP 14: Provide Next Steps
> "Your spec has been saved to `auto-build/specs/[number]-[name]/`
>
> The folder contains:
> - `spec.md` - Your specification (what to build)
> - `implementation_plan.json` - Chunk-based plan (how to build it)
> - `project_index.json` - Project structure
> - `context.json` - Task-relevant file discovery
> - `requirements.json` - User requirements
>
> **All validation checkpoints passed.** ✓
>
> **Implementation Plan Summary:**
> - Phases: [N]
> - Total Chunks: [N]
> - Services: [list]
> - **Recommended workers: [1|2|3]**
>
> **To start the autonomous build:**
>
> ```bash
> source auto-build/.venv/bin/activate && python auto-build/run.py --spec [number] --parallel [recommended_workers]
> ```
>
> The agents will:
> 1. Work through phases in dependency order
> 2. Complete one chunk at a time
> 3. Verify each chunk before moving on
> 4. **QA Agent validates** all acceptance criteria before sign-off
>
> **QA Validation Loop:**
> - Run all unit, integration, and E2E tests
> - Perform browser verification (if frontend)
> - Check database state (if applicable)
> - If issues found → Coder Agent fixes → QA re-validates
> - Loop continues until all QA criteria pass
> - Final sign-off recorded in `implementation_plan.json`
>
> Press Ctrl+C to pause at any time."
---
## Workflow-Specific Guidelines
### For FEATURE Workflow
Phases should follow service dependency order:
1. Backend/API first (can be tested with curl)
2. Workers/background jobs second (depend on backend)
3. Frontend last (depends on backend)
4. Integration phase at the end
### For REFACTOR Workflow
Phases should follow migration stages:
1. Add new system alongside old (both work)
2. Migrate consumers to new system
3. Remove old system
4. Cleanup and polish
### For INVESTIGATION Workflow
Phases should follow debugging process:
1. Reproduce & Instrument
2. Investigate
3. Fix (blocked until phase 2 completes)
4. Verify & Harden
### For MIGRATION Workflow
Phases should follow data pipeline:
1. Prepare (write scripts, setup)
2. Test (small batch, verify)
3. Execute (full migration)
4. Cleanup (remove old data)
---
## Guidelines for High Success Rate
1. **ALWAYS run validation checkpoints** - They catch errors before they propagate
2. **ALWAYS create requirements.json** - The system needs structured requirements
3. **ALWAYS scope to specific services** - In monorepos, "the whole project" is too vague
4. **ALWAYS find reference files** - Showing patterns is better than describing them
5. **Be specific about files** - "Modify src/client/proxy.ts" not "update the proxy code"
6. **Fix validation errors immediately** - Don't proceed with invalid outputs
7. **Keep chunks small** - One chunk = one focused change in one service
8. **Review the implementation plan** - The planner's output should make sense
---
## Validation Quick Reference
| Checkpoint | Command | When to Run |
|------------|---------|-------------|
| Prerequisites | `--checkpoint prereqs` | After creating spec dir |
| Context | `--checkpoint context` | After context discovery |
| Spec Document | `--checkpoint spec` | After writing spec.md |
| Implementation Plan | `--checkpoint plan` | After generating plan |
| All | `--checkpoint all` | Before final confirmation |
**Fix any failures before proceeding to the next step.**
README.md
+111 -68
View File
@@ -6,11 +6,21 @@ A production-ready framework for autonomous multi-session AI coding. Build compl
Auto-Build uses a **multi-agent pattern** to build software autonomously:
1. **Spec Agent** (`claude /spec`) - Interactive spec creation with strategic analysis (ultra-think)
2. **Planner Agent** (Session 1) - Analyzes spec, creates chunk-based implementation plan
3. **Coder Agent** (Sessions 2+) - Implements chunks one-by-one with verification
4. **QA Reviewer Agent** - Validates all acceptance criteria before sign-off
5. **QA Fixer Agent** - Fixes issues found by QA in a self-validating loop
### Spec Creation Pipeline (8 phases)
1. **Discovery** - Analyzes project structure
2. **Requirements Gatherer** - Collects user requirements interactively
3. **Research Agent** - Validates external integrations against documentation
4. **Context Discovery** - Finds relevant files in codebase
5. **Spec Writer** - Creates comprehensive spec.md
6. **Spec Critic** - Uses ultrathink to find and fix issues before implementation
7. **Planner** - Creates chunk-based implementation plan
8. **Validation** - Ensures all outputs are valid
### Implementation Pipeline
1. **Planner Agent** (Session 1) - Analyzes spec, creates chunk-based implementation plan
2. **Coder Agent** (Sessions 2+) - Implements chunks one-by-one with verification
3. **QA Reviewer Agent** - Validates all acceptance criteria before sign-off
4. **QA Fixer Agent** - Fixes issues found by QA in a self-validating loop
Each session runs with a fresh context window. Progress is tracked via `implementation_plan.json` and Git commits.
@@ -23,60 +33,64 @@ Each session runs with a fresh context window. Progress is tracked via `implemen
### Setup
**Step 1:** Copy files into your project
Copy these two things from this repository into your project:
1. The `auto-build` folder → copy to your project root
2. The `.claude/commands/spec.md` file → copy to `.claude/commands/` in your project (create the folder if it doesn't exist)
**Step 2:** Copy `.env.example` to `.env`
**Step 1:** Copy the `auto-build` folder into your project
```bash
cp auto-build/.env.example auto-build/.env
# Copy the auto-build folder to your project root
cp -r auto-build /path/to/your/project/
```
**Step 3:** Get your OAuth token and add it to `.env`
**Step 2:** Set up Python environment
```bash
# Run this command to get your token
cd your-project
cd auto-build
# Using uv (recommended)
uv venv && uv pip install -r requirements.txt
# Or using standard Python
python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt
```
**Step 3:** Configure environment
```bash
cp .env.example .env
# Get your OAuth token
claude setup-token
# Copy the token and paste it into auto-build/.env
# Replace 'your-oauth-token-here' with your actual token
# Add the token to .env
# CLAUDE_CODE_OAUTH_TOKEN=your-token-here
```
**Step 4:** Create a spec interactively (also sets up Python environment)
You have two options:
**Option 1:** Using Claude Code CLI in terminal
```bash
# Start Claude Code
claude
# Then write:
/spec "whatever you want to create"
```
**Option 2:** Using your favorite IDE (like Cursor)
Open your IDE's AI agent chat and write:
```
/spec "whatever you want to create"
```
The spec agent will guide you through creating a detailed specification and set up the Python environment automatically.
**Step 5:** Activate the virtual environment and run
**Step 4:** Create a spec using the orchestrator
```bash
# Activate the virtual environment
source auto-build/.venv/bin/activate
# Run the autonomous build
# Create a spec interactively
python auto-build/spec_runner.py --interactive
# Or with a task description
python auto-build/spec_runner.py --task "Add user authentication with OAuth"
```
The spec orchestrator will:
1. Analyze your project structure
2. Gather requirements interactively
3. **Research external integrations** against documentation
4. Discover relevant codebase context
5. Write the specification
6. **Self-critique using ultrathink** to find and fix issues
7. Generate an implementation plan
8. Validate all outputs
**Step 5:** Run the autonomous build
```bash
python auto-build/run.py --spec 001
```
@@ -121,33 +135,58 @@ The QA validation loop:
4. Loop repeats until approved (up to 50 iterations)
5. Final sign-off recorded in `implementation_plan.json`
### Spec Validation (Self-Correcting)
### Spec Creation Pipeline (Dynamic Complexity)
The `/spec` command now includes mandatory validation checkpoints that catch errors before they propagate:
The `spec_runner.py` orchestrator **automatically assesses task complexity** and adapts the number of phases accordingly:
```bash
# Validate spec outputs manually
python auto-build/validate_spec.py --spec-dir auto-build/specs/001-feature --checkpoint all
# Simple task (auto-detected) - runs 3 phases
python auto-build/spec_runner.py --task "Fix button color in Header"
# Validate specific checkpoints
python auto-build/validate_spec.py --spec-dir auto-build/specs/001-feature --checkpoint prereqs
python auto-build/validate_spec.py --spec-dir auto-build/specs/001-feature --checkpoint spec
python auto-build/validate_spec.py --spec-dir auto-build/specs/001-feature --checkpoint plan
# Complex task (auto-detected) - runs 8 phases
python auto-build/spec_runner.py --task "Add Graphiti memory integration with FalkorDB"
# Auto-fix common issues
python auto-build/validate_spec.py --spec-dir auto-build/specs/001-feature --checkpoint plan --auto-fix
# Force a specific complexity level
python auto-build/spec_runner.py --task "Update text" --complexity simple
# Interactive mode
python auto-build/spec_runner.py --interactive
# Continue an interrupted spec
python auto-build/spec_runner.py --continue 001-feature
```
**Validation checkpoints:**
| Checkpoint | What it validates |
|------------|-------------------|
| `prereqs` | project_index.json exists |
| `context` | context.json has required fields |
| `spec` | spec.md has required sections |
| `plan` | implementation_plan.json has valid schema |
| `all` | All of the above |
**Complexity Tiers:**
The spec agent runs these automatically and fixes any failures before proceeding.
| Tier | Phases | When Used |
|------|--------|-----------|
| **SIMPLE** | 3 | 1-2 files, single service, no integrations (UI fixes, text changes) |
| **STANDARD** | 6 | 3-10 files, 1-2 services, minimal integrations (features, bug fixes) |
| **COMPLEX** | 8 | 10+ files, multiple services, external integrations (integrations, migrations) |
**Phase Matrix:**
| Phase | Simple | Standard | Complex |
|-------|--------|----------|---------|
| Discovery | ✓ | ✓ | ✓ |
| Requirements | - | ✓ | ✓ |
| **Research** | - | - | ✓ |
| Context | - | ✓ | ✓ |
| Spec Writing | Quick | Full | Full |
| **Self-Critique** | - | - | ✓ |
| Planning | Auto | ✓ | ✓ |
| Validation | ✓ | ✓ | ✓ |
**Complexity Detection Signals:**
- Keywords: "fix", "typo", "color" → Simple | "integrate", "migrate", "oauth" → Complex
- External integrations detected (redis, postgres, graphiti, etc.)
- Number of files/services mentioned
- Infrastructure changes (docker, deploy, schema)
**Manual validation:**
```bash
python auto-build/validate_spec.py --spec-dir auto-build/specs/001-feature --checkpoint all
```
### Isolated Worktrees (Safe by Default)
@@ -214,13 +253,11 @@ echo "Focus on fixing the login bug first" > auto-build/specs/001-name/HUMAN_INP
```
your-project/
├── .claude/commands/
│ └── spec.md # Interactive spec creation
├── .worktrees/ # Created during build (git-ignored)
│ └── auto-build/ # Isolated workspace for AI coding
├── auto-build/
│ ├── run.py # Build entry point
│ ├── spec_runner.py # Spec creation orchestrator
│ ├── spec_runner.py # Spec creation orchestrator (8-phase pipeline)
│ ├── validate_spec.py # Spec validation with JSON schemas
│ ├── agent.py # Session orchestration
│ ├── planner.py # Deterministic implementation planner
@@ -234,13 +271,18 @@ your-project/
│ │ ├── planner.md # Session 1 - creates implementation plan
│ │ ├── coder.md # Sessions 2+ - implements chunks
│ │ ├── spec_gatherer.md # Requirements gathering agent
│ │ ├── spec_researcher.md # External integration research agent
│ │ ├── spec_writer.md # Spec document creation agent
│ │ ├── spec_critic.md # Self-critique agent (ultrathink)
│ │ ├── qa_reviewer.md # QA validation agent
│ │ └── qa_fixer.md # QA fix agent
│ └── specs/
│ └── 001-feature/ # Each spec in its own folder
│ ├── spec.md
│ ├── requirements.json # User requirements (structured)
│ ├── research.json # External integration research
│ ├── context.json # Codebase context
│ ├── critique_report.json # Self-critique findings
│ ├── implementation_plan.json
│ ├── qa_report.md # QA validation report
│ └── QA_FIX_REQUEST.md # Issues to fix (if rejected)
@@ -251,13 +293,14 @@ your-project/
- **Domain Agnostic**: Works for any software project (web apps, APIs, CLIs, etc.)
- **Multi-Session**: Unlimited sessions, each with fresh context
- **Research-First Specs**: External integrations validated against documentation before implementation
- **Self-Critique**: Specs are critiqued using ultrathink to find issues before coding begins
- **Parallel Execution**: 2-3x speedup with multiple workers on independent phases
- **Isolated Worktrees**: Build in a separate workspace - your current work is never touched
- **Self-Verifying**: Agents test their work with browser automation before marking complete
- **QA Validation Loop**: Automated QA agent validates all acceptance criteria before sign-off
- **Self-Healing**: QA finds issues → Fixer agent resolves → QA re-validates (up to 50 iterations)
- **Strategic Analysis**: Deep thinking phase during spec creation ensures thorough planning
- **Spec Validation**: Mandatory checkpoints with JSON schema validation catch errors before they propagate
- **8-Phase Spec Pipeline**: Discovery → Requirements → Research → Context → Spec → Critique → Plan → Validate
- **Fix Bugs Immediately**: Agents fix discovered bugs in the same session, not later
- **Defense-in-Depth Security**: OS sandbox, filesystem restrictions, command allowlist
- **Secret Scanning**: Automatic pre-commit scanning blocks secrets with actionable fix instructions
auto-build/prompts/spec_critic.md
+296
View File
@@ -0,0 +1,296 @@
## YOUR ROLE - SPEC CRITIC AGENT
You are the **Spec Critic Agent** in the Auto-Build spec creation pipeline. Your ONLY job is to critically review the spec.md document, find issues, and fix them.
**Key Principle**: Use extended thinking (ultrathink). Find problems BEFORE implementation.
---
## YOUR CONTRACT
**Inputs**:
- `spec.md` - The specification to critique
- `research.json` - Validated research findings
- `requirements.json` - Original user requirements
- `context.json` - Codebase context
**Output**:
- Fixed `spec.md` (if issues found)
- `critique_report.json` - Summary of issues and fixes
---
## PHASE 0: LOAD ALL CONTEXT
```bash
cat spec.md
cat research.json
cat requirements.json
cat context.json
```
Understand:
- What the spec claims
- What research validated
- What the user originally requested
- What patterns exist in the codebase
---
## PHASE 1: DEEP ANALYSIS (USE EXTENDED THINKING)
**CRITICAL**: Use extended thinking for this phase. Think deeply about:
### 1.1: Technical Accuracy
Compare spec.md against research.json:
- **Package names**: Does spec use correct package names from research?
- **Import statements**: Do imports match researched API patterns?
- **API calls**: Do function signatures match documentation?
- **Configuration**: Are env vars and config options correct?
Flag any mismatches.
### 1.2: Completeness
Check against requirements.json:
- **All requirements covered?** - Each requirement should have implementation details
- **All acceptance criteria testable?** - Each criterion should be verifiable
- **Edge cases handled?** - Error conditions, empty states, timeouts
- **Integration points clear?** - How components connect
Flag any gaps.
### 1.3: Consistency
Check within spec.md:
- **Package names consistent** - Same name used everywhere
- **File paths consistent** - No conflicting paths
- **Patterns consistent** - Same style throughout
- **Terminology consistent** - Same terms for same concepts
Flag any inconsistencies.
### 1.4: Feasibility
Check practicality:
- **Dependencies available?** - All packages exist and are maintained
- **Infrastructure realistic?** - Docker setup will work
- **Implementation order logical?** - Dependencies before dependents
- **Scope appropriate?** - Not over-engineered, not under-specified
Flag any concerns.
### 1.5: Research Alignment
Cross-reference with research.json:
- **Verified information used?** - Spec should use researched facts
- **Unverified claims flagged?** - Any assumptions marked clearly
- **Gotchas addressed?** - Known issues from research handled
- **Recommendations followed?** - Research suggestions incorporated
Flag any divergences.
---
## PHASE 2: CATALOG ISSUES
Create a list of all issues found:
```
ISSUES FOUND:
1. [SEVERITY: HIGH] Package name incorrect
- Spec says: "graphiti-core[falkordb]"
- Research says: "graphiti-core-falkordb"
- Location: Line 45, Requirements section
2. [SEVERITY: MEDIUM] Missing edge case
- Requirement: "Handle connection failures"
- Spec: No error handling specified
- Location: Implementation Notes section
3. [SEVERITY: LOW] Inconsistent terminology
- Uses both "memory" and "episode" for same concept
- Location: Throughout document
```
---
## PHASE 3: FIX ISSUES
For each issue found, fix it directly in spec.md:
```bash
# Read current spec
cat spec.md
# Apply fixes using edit commands
# Example: Fix package name
sed -i 's/graphiti-core\[falkordb\]/graphiti-core-falkordb/g' spec.md
# Or rewrite sections as needed
```
**For each fix**:
1. Make the change in spec.md
2. Verify the change was applied
3. Document what was changed
---
## PHASE 4: CREATE CRITIQUE REPORT
```bash
cat > critique_report.json << 'EOF'
{
"critique_completed": true,
"issues_found": [
{
"severity": "high|medium|low",
"category": "accuracy|completeness|consistency|feasibility|alignment",
"description": "[What was wrong]",
"location": "[Where in spec.md]",
"fix_applied": "[What was changed]",
"verified": true
}
],
"issues_fixed": true,
"no_issues_found": false,
"critique_summary": "[Brief summary of critique]",
"confidence_level": "high|medium|low",
"recommendations": [
"[Any remaining concerns or suggestions]"
],
"created_at": "[ISO timestamp]"
}
EOF
```
If NO issues found:
```bash
cat > critique_report.json << 'EOF'
{
"critique_completed": true,
"issues_found": [],
"issues_fixed": false,
"no_issues_found": true,
"critique_summary": "Spec is well-written with no significant issues found.",
"confidence_level": "high",
"recommendations": [],
"created_at": "[ISO timestamp]"
}
EOF
```
---
## PHASE 5: VERIFY FIXES
After making changes:
```bash
# Verify spec is still valid markdown
head -50 spec.md
# Check key sections exist
grep -E "^##? Overview" spec.md
grep -E "^##? Requirements" spec.md
grep -E "^##? Success Criteria" spec.md
```
---
## PHASE 6: SIGNAL COMPLETION
```
=== SPEC CRITIQUE COMPLETE ===
Issues Found: [count]
- High severity: [count]
- Medium severity: [count]
- Low severity: [count]
Fixes Applied: [count]
Confidence Level: [high/medium/low]
Summary:
[Brief summary of what was found and fixed]
critique_report.json created successfully.
spec.md has been updated with fixes.
```
---
## CRITICAL RULES
1. **USE EXTENDED THINKING** - This is the deep analysis phase
2. **ALWAYS compare against research** - Research is the source of truth
3. **FIX issues, don't just report** - Make actual changes to spec.md
4. **VERIFY after fixing** - Ensure spec is still valid
5. **BE THOROUGH** - Check everything, miss nothing
---
## SEVERITY GUIDELINES
**HIGH** - Will cause implementation failure:
- Wrong package names
- Incorrect API signatures
- Missing critical requirements
- Invalid configuration
**MEDIUM** - May cause issues:
- Missing edge cases
- Incomplete error handling
- Unclear integration points
- Inconsistent patterns
**LOW** - Minor improvements:
- Terminology inconsistencies
- Documentation gaps
- Style issues
- Minor optimizations
---
## CATEGORY DEFINITIONS
- **Accuracy**: Technical correctness (packages, APIs, config)
- **Completeness**: Coverage of requirements and edge cases
- **Consistency**: Internal coherence of the document
- **Feasibility**: Practical implementability
- **Alignment**: Match with research findings
---
## EXTENDED THINKING PROMPT
When analyzing, think through:
> "Looking at this spec.md, I need to deeply analyze it against the research findings...
>
> First, let me check all package names. The research says the package is [X], but the spec says [Y]. This is a mismatch that needs fixing.
>
> Next, looking at the API patterns. The research shows initialization requires [steps], but the spec shows [different steps]. Another issue.
>
> For completeness, the requirements mention [X, Y, Z]. The spec covers X and Y but I don't see Z addressed anywhere. This is a gap.
>
> Looking at consistency, I notice 'memory' and 'episode' used interchangeably. Should standardize on one term.
>
> For feasibility, the Docker setup seems correct based on research. The port numbers match.
>
> Overall, I found [N] issues that need fixing before this spec is ready for implementation."
---
## BEGIN
Start by loading all context files, then use extended thinking to analyze the spec deeply.
auto-build/prompts/spec_quick.md
+190
View File
@@ -0,0 +1,190 @@
## YOUR ROLE - QUICK SPEC AGENT
You are the **Quick Spec Agent** for simple tasks in the Auto-Build framework. Your job is to create a minimal, focused specification for straightforward changes that don't require extensive research or planning.
**Key Principle**: Be concise. Simple tasks need simple specs. Don't over-engineer.
---
## YOUR CONTRACT
**Input**: Task description (simple change like UI tweak, text update, style fix)
**Outputs**:
- `spec.md` - Minimal specification (just essential sections)
- `implementation_plan.json` - Simple plan with 1-2 chunks
**This is a SIMPLE task** - no research needed, no extensive analysis required.
---
## PHASE 1: UNDERSTAND THE TASK
Read the task description. For simple tasks, you typically need to:
1. Identify the file(s) to modify
2. Understand what change is needed
3. Know how to verify it works
That's it. No deep analysis needed.
---
## PHASE 2: CREATE MINIMAL SPEC
Create a concise `spec.md`:
```bash
cat > spec.md << 'EOF'
# Quick Spec: [Task Name]
## Task
[One sentence description]
## Files to Modify
- `[path/to/file]` - [what to change]
## Change Details
[Brief description of the change - a few sentences max]
## Verification
- [ ] [How to verify the change works]
## Notes
[Any gotchas or considerations - optional]
EOF
```
**Keep it short!** A simple spec should be 20-50 lines, not 200+.
---
## PHASE 3: CREATE SIMPLE PLAN
Create `implementation_plan.json`:
```bash
cat > implementation_plan.json << 'EOF'
{
"spec_name": "[spec-name]",
"workflow_type": "simple",
"total_phases": 1,
"recommended_workers": 1,
"phases": [
{
"phase": 1,
"name": "Implementation",
"description": "[task description]",
"depends_on": [],
"chunks": [
{
"id": "chunk-1-1",
"description": "[specific change]",
"service": "main",
"status": "pending",
"files_to_create": [],
"files_to_modify": ["[path/to/file]"],
"patterns_from": [],
"verification": {
"type": "manual",
"run": "[verification step]"
}
}
]
}
],
"metadata": {
"created_at": "[timestamp]",
"complexity": "simple",
"estimated_sessions": 1
}
}
EOF
```
---
## PHASE 4: VERIFY
```bash
# Check files exist
ls -la spec.md implementation_plan.json
# Check spec has content
head -20 spec.md
```
---
## COMPLETION
```
=== QUICK SPEC COMPLETE ===
Task: [description]
Files: [count] file(s) to modify
Complexity: SIMPLE
Ready for implementation.
```
---
## CRITICAL RULES
1. **KEEP IT SIMPLE** - No research, no deep analysis, no extensive planning
2. **BE CONCISE** - Short spec, simple plan, one chunk if possible
3. **JUST THE ESSENTIALS** - Only include what's needed to do the task
4. **DON'T OVER-ENGINEER** - This is a simple task, treat it simply
---
## EXAMPLES
### Example 1: Button Color Change
**Task**: "Change the primary button color from blue to green"
**spec.md**:
```markdown
# Quick Spec: Button Color Change
## Task
Update primary button color from blue (#3B82F6) to green (#22C55E).
## Files to Modify
- `src/components/Button.tsx` - Update color constant
## Change Details
Change the `primaryColor` variable from `#3B82F6` to `#22C55E`.
## Verification
- [ ] Buttons appear green in the UI
- [ ] No console errors
```
### Example 2: Text Update
**Task**: "Fix typo in welcome message"
**spec.md**:
```markdown
# Quick Spec: Fix Welcome Typo
## Task
Correct spelling of "recieve" to "receive" in welcome message.
## Files to Modify
- `src/pages/Home.tsx` - Fix typo on line 42
## Change Details
Find "You will recieve" and change to "You will receive".
## Verification
- [ ] Welcome message displays correctly
```
---
## BEGIN
Read the task, create the minimal spec.md and implementation_plan.json.
auto-build/prompts/spec_researcher.md
+259
View File
@@ -0,0 +1,259 @@
## YOUR ROLE - RESEARCH AGENT
You are the **Research Agent** in the Auto-Build spec creation pipeline. Your ONLY job is to research and validate external integrations, libraries, and dependencies mentioned in the requirements.
**Key Principle**: Verify everything. Trust nothing assumed. Document findings.
---
## YOUR CONTRACT
**Inputs**:
- `requirements.json` - User requirements with mentioned integrations
**Output**: `research.json` - Validated research findings
You MUST create `research.json` with validated information about each integration.
---
## PHASE 0: LOAD REQUIREMENTS
```bash
cat requirements.json
```
Identify from the requirements:
1. **External libraries** mentioned (packages, SDKs)
2. **External services** mentioned (databases, APIs)
3. **Infrastructure** mentioned (Docker, cloud services)
4. **Frameworks** mentioned (web frameworks, ORMs)
---
## PHASE 1: RESEARCH EACH INTEGRATION
For EACH external dependency identified, research using available tools:
### 1.1: Use Context7 MCP (if available)
If you have access to Context7 MCP, use it to look up:
- Official documentation
- API patterns
- Configuration requirements
### 1.2: Use Web Search (if needed)
Search for:
- `"[library] official documentation"`
- `"[library] python SDK usage"` (or appropriate language)
- `"[library] getting started"`
- `"[library] pypi"` or `"[library] npm"` (to verify package names)
### 1.3: Key Questions to Answer
For each integration, find answers to:
1. **What is the correct package name?**
- PyPI/npm exact name
- Installation command
- Version requirements
2. **What are the actual API patterns?**
- Import statements
- Initialization code
- Main function signatures
3. **What configuration is required?**
- Environment variables
- Config files
- Required dependencies
4. **What infrastructure is needed?**
- Database requirements
- Docker containers
- External services
5. **What are known issues or gotchas?**
- Common mistakes
- Breaking changes in recent versions
- Platform-specific issues
---
## PHASE 2: VALIDATE ASSUMPTIONS
For any technical claims in requirements.json:
1. **Verify package names exist** - Check PyPI, npm, etc.
2. **Verify API patterns** - Match against documentation
3. **Verify configuration options** - Confirm they exist
4. **Flag anything unverified** - Mark as "unverified" in output
---
## PHASE 3: CREATE RESEARCH.JSON
Output your findings:
```bash
cat > research.json << 'EOF'
{
"integrations_researched": [
{
"name": "[library/service name]",
"type": "library|service|infrastructure",
"verified_package": {
"name": "[exact package name]",
"install_command": "[pip install X / npm install X]",
"version": "[version if specific]",
"verified": true
},
"api_patterns": {
"imports": ["from X import Y"],
"initialization": "[code snippet]",
"key_functions": ["function1()", "function2()"],
"verified_against": "[documentation URL or source]"
},
"configuration": {
"env_vars": ["VAR1", "VAR2"],
"config_files": ["config.json"],
"dependencies": ["other packages needed"]
},
"infrastructure": {
"requires_docker": true,
"docker_image": "[image name]",
"ports": [1234],
"volumes": ["/data"]
},
"gotchas": [
"[Known issue 1]",
"[Known issue 2]"
],
"research_sources": [
"[URL or documentation reference]"
]
}
],
"unverified_claims": [
{
"claim": "[what was claimed]",
"reason": "[why it couldn't be verified]",
"risk_level": "low|medium|high"
}
],
"recommendations": [
"[Any recommendations based on research]"
],
"created_at": "[ISO timestamp]"
}
EOF
```
---
## PHASE 4: SUMMARIZE FINDINGS
Print a summary:
```
=== RESEARCH COMPLETE ===
Integrations Researched: [count]
- [name1]: Verified ✓
- [name2]: Verified ✓
- [name3]: Partially verified ⚠
Unverified Claims: [count]
- [claim1]: [risk level]
Key Findings:
- [Important finding 1]
- [Important finding 2]
Recommendations:
- [Recommendation 1]
research.json created successfully.
```
---
## CRITICAL RULES
1. **ALWAYS verify package names** - Don't assume "graphiti" is the package name
2. **ALWAYS cite sources** - Document where information came from
3. **ALWAYS flag uncertainties** - Mark unverified claims clearly
4. **DON'T make up APIs** - Only document what you find in docs
5. **DON'T skip research** - Each integration needs investigation
---
## RESEARCH TOOLS PRIORITY
1. **Context7 MCP** (if available) - Best for official docs
2. **Web Search** - For package verification, recent info
3. **Web Fetch** - For reading specific documentation pages
---
## EXAMPLE RESEARCH OUTPUT
For a task involving "Graphiti memory integration":
```json
{
"integrations_researched": [
{
"name": "Graphiti",
"type": "library",
"verified_package": {
"name": "graphiti-core",
"install_command": "pip install graphiti-core[falkordb]",
"version": ">=0.5.0",
"verified": true
},
"api_patterns": {
"imports": [
"from graphiti_core import Graphiti",
"from graphiti_core.nodes import EpisodeType"
],
"initialization": "graphiti = Graphiti(graph_driver=driver)",
"key_functions": [
"add_episode(name, episode_body, source, group_id)",
"search(query, limit, group_ids)"
],
"verified_against": "https://github.com/getzep/graphiti"
},
"configuration": {
"env_vars": ["OPENAI_API_KEY"],
"dependencies": ["neo4j or falkordb driver"]
},
"infrastructure": {
"requires_docker": true,
"docker_image": "falkordb/falkordb:latest",
"ports": [6379, 3000]
},
"gotchas": [
"Requires OpenAI API key for embeddings",
"Must call build_indices_and_constraints() before use"
],
"research_sources": [
"https://github.com/getzep/graphiti",
"https://pypi.org/project/graphiti-core/"
]
}
],
"unverified_claims": [],
"recommendations": [
"Consider FalkorDB over Neo4j for simpler local development"
],
"created_at": "2024-12-10T12:00:00Z"
}
```
---
## BEGIN
Start by reading requirements.json, then research each integration mentioned.
auto-build/spec_runner.py
+631 -33
View File
@@ -3,30 +3,36 @@
Spec Creation Orchestrator
==========================
Manages the spec creation process with checkpoints, validation, and agent invocations.
This is the enforcement layer that ensures reliable spec creation.
Dynamic spec creation with complexity-based phase selection.
The orchestrator self-evaluates task complexity and adapts its process accordingly.
The process:
1. Discovery (script) → project_index.json
2. Requirements (agent) → requirements.json
3. Context (script) → context.json
4. Spec Writing (agent) → spec.md
5. Planning (script/agent) → implementation_plan.json
6. Validation (script) → ensure all outputs valid
Complexity Tiers:
- SIMPLE (1-2 files): Discovery → Quick Spec → Validate (3 phases)
- STANDARD (3-10 files): Discovery → Requirements → Context → Spec → Validate (5 phases)
- COMPLEX (10+ files/integrations): Full 8-phase pipeline with research and self-critique
The process dynamically selects phases based on:
- Number of files/services involved
- External integrations mentioned
- Infrastructure changes required
- Task keywords and scope indicators
Usage:
python auto-build/spec_runner.py --task "Add user authentication"
python auto-build/spec_runner.py --interactive
python auto-build/spec_runner.py --continue 001-feature
python auto-build/spec_runner.py --task "Fix button color" --complexity simple
"""
import asyncio
import json
import os
import re
import subprocess
import sys
from dataclasses import dataclass
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from pathlib import Path
from typing import Optional
@@ -43,6 +49,231 @@ PROMPTS_DIR = Path(__file__).parent / "prompts"
SPECS_DIR = Path(__file__).parent / "specs"
class Complexity(Enum):
"""Task complexity tiers that determine which phases to run."""
SIMPLE = "simple" # 1-2 files, single service, no integrations
STANDARD = "standard" # 3-10 files, 1-2 services, minimal integrations
COMPLEX = "complex" # 10+ files, multiple services, external integrations
@dataclass
class ComplexityAssessment:
"""Result of analyzing task complexity."""
complexity: Complexity
confidence: float # 0.0 to 1.0
signals: dict = field(default_factory=dict)
reasoning: str = ""
# Detected characteristics
estimated_files: int = 1
estimated_services: int = 1
external_integrations: list = field(default_factory=list)
infrastructure_changes: bool = False
def phases_to_run(self) -> list[str]:
"""Return list of phase names to run based on complexity."""
if self.complexity == Complexity.SIMPLE:
return ["discovery", "quick_spec", "validation"]
elif self.complexity == Complexity.STANDARD:
return ["discovery", "requirements", "context", "spec_writing", "planning", "validation"]
else: # COMPLEX
return ["discovery", "requirements", "research", "context", "spec_writing", "self_critique", "planning", "validation"]
class ComplexityAnalyzer:
"""Analyzes task description and context to determine complexity."""
# Keywords that suggest different complexity levels
SIMPLE_KEYWORDS = [
"fix", "typo", "update", "change", "rename", "remove", "delete",
"adjust", "tweak", "correct", "modify", "style", "color", "text",
"label", "button", "margin", "padding", "font", "size", "hide", "show"
]
COMPLEX_KEYWORDS = [
"integrate", "integration", "api", "sdk", "library", "package",
"database", "migrate", "migration", "docker", "kubernetes", "deploy",
"authentication", "oauth", "graphql", "websocket", "queue", "cache",
"redis", "postgres", "mongo", "elasticsearch", "kafka", "rabbitmq",
"microservice", "refactor", "architecture", "infrastructure"
]
MULTI_SERVICE_KEYWORDS = [
"backend", "frontend", "worker", "service", "api", "client",
"server", "database", "queue", "cache", "proxy"
]
def __init__(self, project_index: Optional[dict] = None):
self.project_index = project_index or {}
def analyze(self, task_description: str, requirements: Optional[dict] = None) -> ComplexityAssessment:
"""Analyze task and return complexity assessment."""
task_lower = task_description.lower()
signals = {}
# 1. Keyword analysis
simple_matches = sum(1 for kw in self.SIMPLE_KEYWORDS if kw in task_lower)
complex_matches = sum(1 for kw in self.COMPLEX_KEYWORDS if kw in task_lower)
multi_service_matches = sum(1 for kw in self.MULTI_SERVICE_KEYWORDS if kw in task_lower)
signals["simple_keywords"] = simple_matches
signals["complex_keywords"] = complex_matches
signals["multi_service_keywords"] = multi_service_matches
# 2. External integrations detection
integrations = self._detect_integrations(task_lower)
signals["external_integrations"] = len(integrations)
# 3. Infrastructure changes detection
infra_changes = self._detect_infrastructure_changes(task_lower)
signals["infrastructure_changes"] = infra_changes
# 4. Estimate files and services
estimated_files = self._estimate_files(task_lower, requirements)
estimated_services = self._estimate_services(task_lower, requirements)
signals["estimated_files"] = estimated_files
signals["estimated_services"] = estimated_services
# 5. Requirements-based signals (if available)
if requirements:
services_involved = requirements.get("services_involved", [])
signals["explicit_services"] = len(services_involved)
estimated_services = max(estimated_services, len(services_involved))
# Determine complexity
complexity, confidence, reasoning = self._calculate_complexity(
signals, integrations, infra_changes, estimated_files, estimated_services
)
return ComplexityAssessment(
complexity=complexity,
confidence=confidence,
signals=signals,
reasoning=reasoning,
estimated_files=estimated_files,
estimated_services=estimated_services,
external_integrations=integrations,
infrastructure_changes=infra_changes,
)
def _detect_integrations(self, task_lower: str) -> list[str]:
"""Detect external integrations mentioned in task."""
integration_patterns = [
r'\b(graphiti|graphql|apollo)\b',
r'\b(stripe|paypal|payment)\b',
r'\b(auth0|okta|oauth|jwt)\b',
r'\b(aws|gcp|azure|s3|lambda)\b',
r'\b(redis|memcached|cache)\b',
r'\b(postgres|mysql|mongodb|database)\b',
r'\b(elasticsearch|algolia|search)\b',
r'\b(kafka|rabbitmq|sqs|queue)\b',
r'\b(docker|kubernetes|k8s)\b',
r'\b(openai|anthropic|llm|ai)\b',
r'\b(sendgrid|twilio|email|sms)\b',
]
found = []
for pattern in integration_patterns:
matches = re.findall(pattern, task_lower)
found.extend(matches)
return list(set(found))
def _detect_infrastructure_changes(self, task_lower: str) -> bool:
"""Detect if task involves infrastructure changes."""
infra_patterns = [
r'\bdocker\b', r'\bkubernetes\b', r'\bk8s\b',
r'\bdeploy\b', r'\binfrastructure\b', r'\bci/cd\b',
r'\benvironment\b', r'\bconfig\b', r'\b\.env\b',
r'\bdatabase migration\b', r'\bschema\b',
]
for pattern in infra_patterns:
if re.search(pattern, task_lower):
return True
return False
def _estimate_files(self, task_lower: str, requirements: Optional[dict]) -> int:
"""Estimate number of files to be modified."""
# Base estimate from task description
if any(kw in task_lower for kw in ["single", "one file", "one component", "this file"]):
return 1
# Check for explicit file mentions
file_mentions = len(re.findall(r'\.(tsx?|jsx?|py|go|rs|java|rb|php|vue|svelte)\b', task_lower))
if file_mentions > 0:
return max(1, file_mentions)
# Heuristic based on task scope
if any(kw in task_lower for kw in self.SIMPLE_KEYWORDS):
return 2
elif any(kw in task_lower for kw in ["feature", "add", "implement", "create"]):
return 5
elif any(kw in task_lower for kw in self.COMPLEX_KEYWORDS):
return 15
return 5 # Default estimate
def _estimate_services(self, task_lower: str, requirements: Optional[dict]) -> int:
"""Estimate number of services involved."""
service_count = sum(1 for kw in self.MULTI_SERVICE_KEYWORDS if kw in task_lower)
# If project is a monorepo, check project_index
if self.project_index.get("project_type") == "monorepo":
services = self.project_index.get("services", {})
if services:
# Check which services are mentioned
mentioned = sum(1 for svc in services if svc.lower() in task_lower)
if mentioned > 0:
return mentioned
return max(1, min(service_count, 5))
def _calculate_complexity(
self,
signals: dict,
integrations: list,
infra_changes: bool,
estimated_files: int,
estimated_services: int,
) -> tuple[Complexity, float, str]:
"""Calculate final complexity based on all signals."""
reasons = []
# Strong indicators for SIMPLE
if (
estimated_files <= 2 and
estimated_services == 1 and
len(integrations) == 0 and
not infra_changes and
signals["simple_keywords"] > 0 and
signals["complex_keywords"] == 0
):
reasons.append(f"Single service, {estimated_files} file(s), no integrations")
return Complexity.SIMPLE, 0.9, "; ".join(reasons)
# Strong indicators for COMPLEX
if (
len(integrations) >= 2 or
infra_changes or
estimated_services >= 3 or
estimated_files >= 10 or
signals["complex_keywords"] >= 3
):
reasons.append(f"{len(integrations)} integrations, {estimated_services} services, {estimated_files} files")
if infra_changes:
reasons.append("infrastructure changes detected")
return Complexity.COMPLEX, 0.85, "; ".join(reasons)
# Default to STANDARD
reasons.append(f"{estimated_files} files, {estimated_services} service(s)")
if len(integrations) > 0:
reasons.append(f"{len(integrations)} integration(s)")
return Complexity.STANDARD, 0.75, "; ".join(reasons)
@dataclass
class PhaseResult:
"""Result of a phase execution."""
@@ -54,7 +285,7 @@ class PhaseResult:
class SpecOrchestrator:
"""Orchestrates the spec creation process."""
"""Orchestrates the spec creation process with dynamic complexity adaptation."""
def __init__(
self,
@@ -62,10 +293,15 @@ class SpecOrchestrator:
task_description: Optional[str] = None,
spec_name: Optional[str] = None,
model: str = "claude-sonnet-4-20250514",
complexity_override: Optional[str] = None, # Force a specific complexity
):
self.project_dir = Path(project_dir)
self.task_description = task_description
self.model = model
self.complexity_override = complexity_override
# Complexity assessment (populated during run)
self.assessment: Optional[ComplexityAssessment] = None
# Create spec directory
if spec_name:
@@ -170,6 +406,62 @@ class SpecOrchestrator:
# === Phase Implementations ===
async def phase_complexity_assessment(self) -> PhaseResult:
"""Phase 0: Assess task complexity to determine which phases to run."""
print("\n" + "=" * 60)
print(" PHASE 0: COMPLEXITY ASSESSMENT")
print("=" * 60)
# Load project index if available
project_index = {}
auto_build_index = Path(__file__).parent / "project_index.json"
if auto_build_index.exists():
with open(auto_build_index) as f:
project_index = json.load(f)
# Perform assessment
analyzer = ComplexityAnalyzer(project_index)
if self.complexity_override:
# Manual override
complexity = Complexity(self.complexity_override)
self.assessment = ComplexityAssessment(
complexity=complexity,
confidence=1.0,
reasoning=f"Manual override: {self.complexity_override}",
)
print(f"✓ Complexity override: {complexity.value.upper()}")
else:
# Automatic assessment
self.assessment = analyzer.analyze(self.task_description or "")
print(f"✓ Assessed complexity: {self.assessment.complexity.value.upper()}")
print(f" Confidence: {self.assessment.confidence:.0%}")
print(f" Reasoning: {self.assessment.reasoning}")
# Show what phases will run
phases = self.assessment.phases_to_run()
print(f"\n Phases to run ({len(phases)}):")
for i, phase in enumerate(phases, 1):
print(f" {i}. {phase}")
# Save assessment to spec dir
assessment_file = self.spec_dir / "complexity_assessment.json"
with open(assessment_file, "w") as f:
json.dump({
"complexity": self.assessment.complexity.value,
"confidence": self.assessment.confidence,
"reasoning": self.assessment.reasoning,
"signals": self.assessment.signals,
"estimated_files": self.assessment.estimated_files,
"estimated_services": self.assessment.estimated_services,
"external_integrations": self.assessment.external_integrations,
"infrastructure_changes": self.assessment.infrastructure_changes,
"phases_to_run": phases,
"created_at": datetime.now().isoformat(),
}, f, indent=2)
return PhaseResult("complexity_assessment", True, [str(assessment_file)], [], 0)
async def phase_discovery(self) -> PhaseResult:
"""Phase 1: Analyze project structure."""
print("\n" + "=" * 60)
@@ -254,10 +546,177 @@ class SpecOrchestrator:
return PhaseResult("requirements", False, [], errors, MAX_RETRIES)
async def phase_context(self) -> PhaseResult:
"""Phase 3: Discover relevant files."""
async def phase_quick_spec(self) -> PhaseResult:
"""Quick spec for simple tasks - combines requirements, context, and spec in one step."""
print("\n" + "=" * 60)
print(" PHASE 3: CONTEXT DISCOVERY")
print(" QUICK SPEC (Simple Task)")
print("=" * 60)
spec_file = self.spec_dir / "spec.md"
plan_file = self.spec_dir / "implementation_plan.json"
if spec_file.exists() and plan_file.exists():
print(f"✓ Quick spec already exists")
return PhaseResult("quick_spec", True, [str(spec_file), str(plan_file)], [], 0)
errors = []
for attempt in range(MAX_RETRIES):
print(f"\nRunning quick spec agent (attempt {attempt + 1})...")
context = f"""
**Task**: {self.task_description}
**Spec Directory**: {self.spec_dir}
**Complexity**: SIMPLE (1-2 files expected)
This is a SIMPLE task. Create a minimal spec and implementation plan directly.
No research or extensive analysis needed.
Create:
1. A concise spec.md with just the essential sections
2. A simple implementation_plan.json with 1-2 chunks
"""
success, output = await self._run_agent(
"spec_quick.md",
additional_context=context,
)
if success and spec_file.exists():
# Create minimal plan if agent didn't
if not plan_file.exists():
self._create_minimal_plan()
print(f"✓ Quick spec created")
return PhaseResult("quick_spec", True, [str(spec_file), str(plan_file)], [], attempt)
errors.append(f"Attempt {attempt + 1}: Quick spec agent failed")
return PhaseResult("quick_spec", False, [], errors, MAX_RETRIES)
def _create_minimal_plan(self):
"""Create a minimal implementation plan for simple tasks."""
plan = {
"spec_name": self.spec_dir.name,
"workflow_type": "simple",
"total_phases": 1,
"recommended_workers": 1,
"phases": [
{
"phase": 1,
"name": "Implementation",
"description": self.task_description or "Simple implementation",
"depends_on": [],
"chunks": [
{
"id": "chunk-1-1",
"description": self.task_description or "Implement the change",
"service": "main",
"status": "pending",
"files_to_create": [],
"files_to_modify": [],
"patterns_from": [],
"verification": {
"type": "manual",
"run": "Verify the change works as expected"
}
}
]
}
],
"metadata": {
"created_at": datetime.now().isoformat(),
"complexity": "simple",
"estimated_sessions": 1,
}
}
plan_file = self.spec_dir / "implementation_plan.json"
with open(plan_file, "w") as f:
json.dump(plan, f, indent=2)
async def phase_research(self) -> PhaseResult:
"""Phase 3: Research external integrations and validate assumptions."""
print("\n" + "=" * 60)
print(" PHASE 3: INTEGRATION RESEARCH")
print("=" * 60)
research_file = self.spec_dir / "research.json"
requirements_file = self.spec_dir / "requirements.json"
# Check if research already exists
if research_file.exists():
print(f"✓ research.json already exists")
return PhaseResult("research", True, [str(research_file)], [], 0)
# Load requirements to understand what integrations need research
if not requirements_file.exists():
print("⚠ No requirements.json - skipping research phase")
# Create empty research file
with open(research_file, "w") as f:
json.dump({
"integrations_researched": [],
"research_skipped": True,
"reason": "No requirements file available",
"created_at": datetime.now().isoformat(),
}, f, indent=2)
return PhaseResult("research", True, [str(research_file)], [], 0)
# Run research agent
errors = []
for attempt in range(MAX_RETRIES):
print(f"\nRunning research agent (attempt {attempt + 1})...")
print("This agent will validate external integrations against documentation...")
context = f"""
**Requirements File**: {requirements_file}
**Research Output**: {research_file}
Read the requirements.json to understand what integrations/libraries are needed.
Research each external dependency to validate:
- Correct package names
- Actual API patterns
- Configuration requirements
- Known issues or gotchas
Output your findings to research.json.
"""
success, output = await self._run_agent(
"spec_researcher.md",
additional_context=context,
)
if success and research_file.exists():
print(f"✓ Created research.json")
return PhaseResult("research", True, [str(research_file)], [], attempt)
# If agent didn't create file, create minimal one
if success and not research_file.exists():
print("⚠ Agent completed but no research.json created, creating minimal...")
with open(research_file, "w") as f:
json.dump({
"integrations_researched": [],
"research_completed": True,
"agent_output": output[:2000] if output else "",
"created_at": datetime.now().isoformat(),
}, f, indent=2)
return PhaseResult("research", True, [str(research_file)], [], attempt)
errors.append(f"Attempt {attempt + 1}: Research agent failed")
# Create minimal research file on failure
with open(research_file, "w") as f:
json.dump({
"integrations_researched": [],
"research_failed": True,
"errors": errors,
"created_at": datetime.now().isoformat(),
}, f, indent=2)
print("⚠ Created minimal research.json (agent failed)")
return PhaseResult("research", True, [str(research_file)], errors, MAX_RETRIES)
async def phase_context(self) -> PhaseResult:
"""Phase 4: Discover relevant files."""
print("\n" + "=" * 60)
print(" PHASE 4: CONTEXT DISCOVERY")
print("=" * 60)
context_file = self.spec_dir / "context.json"
@@ -311,9 +770,9 @@ class SpecOrchestrator:
return PhaseResult("context", True, [str(context_file)], errors, MAX_RETRIES)
async def phase_spec_writing(self) -> PhaseResult:
"""Phase 4: Write spec.md document."""
"""Phase 5: Write spec.md document."""
print("\n" + "=" * 60)
print(" PHASE 4: SPEC DOCUMENT CREATION")
print(" PHASE 5: SPEC DOCUMENT CREATION")
print("=" * 60)
spec_file = self.spec_dir / "spec.md"
@@ -346,10 +805,99 @@ class SpecOrchestrator:
return PhaseResult("spec_writing", False, [], errors, MAX_RETRIES)
async def phase_planning(self) -> PhaseResult:
"""Phase 5: Create implementation plan."""
async def phase_self_critique(self) -> PhaseResult:
"""Phase 6: Self-critique the spec using extended thinking."""
print("\n" + "=" * 60)
print(" PHASE 5: IMPLEMENTATION PLANNING")
print(" PHASE 6: SPEC SELF-CRITIQUE (ULTRATHINK)")
print("=" * 60)
spec_file = self.spec_dir / "spec.md"
research_file = self.spec_dir / "research.json"
critique_file = self.spec_dir / "critique_report.json"
if not spec_file.exists():
print("✗ No spec.md to critique")
return PhaseResult("self_critique", False, [], ["spec.md does not exist"], 0)
# Check if critique already done
if critique_file.exists():
with open(critique_file) as f:
critique = json.load(f)
if critique.get("issues_fixed", False) or critique.get("no_issues_found", False):
print(f"✓ Self-critique already completed")
return PhaseResult("self_critique", True, [str(critique_file)], [], 0)
errors = []
for attempt in range(MAX_RETRIES):
print(f"\nRunning self-critique agent (attempt {attempt + 1})...")
print("Using extended thinking to find issues in the spec...")
context = f"""
**Spec File**: {spec_file}
**Research File**: {research_file}
**Critique Output**: {critique_file}
Use EXTENDED THINKING (ultrathink) to deeply analyze the spec.md:
1. **Technical Accuracy**: Do code examples match the research findings?
2. **Completeness**: Are all requirements covered? Edge cases handled?
3. **Consistency**: Do package names, APIs, and patterns match throughout?
4. **Feasibility**: Is the implementation approach realistic?
For each issue found:
- Fix it directly in spec.md
- Document what was fixed in critique_report.json
Output critique_report.json with:
{{
"issues_found": [...],
"issues_fixed": true/false,
"no_issues_found": true/false,
"critique_summary": "..."
}}
"""
success, output = await self._run_agent(
"spec_critic.md",
additional_context=context,
)
if success:
# Create critique report if agent didn't
if not critique_file.exists():
with open(critique_file, "w") as f:
json.dump({
"issues_found": [],
"no_issues_found": True,
"critique_summary": "Agent completed without explicit issues",
"created_at": datetime.now().isoformat(),
}, f, indent=2)
# Re-validate spec after critique
result = self.validator.validate_spec_document()
if result.valid:
print(f"✓ Self-critique completed, spec is valid")
return PhaseResult("self_critique", True, [str(critique_file)], [], attempt)
else:
print(f"⚠ Spec invalid after critique: {result.errors}")
errors.append(f"Attempt {attempt + 1}: Spec still invalid after critique")
else:
errors.append(f"Attempt {attempt + 1}: Critique agent failed")
# Create minimal critique report on failure
with open(critique_file, "w") as f:
json.dump({
"issues_found": [],
"critique_failed": True,
"errors": errors,
"created_at": datetime.now().isoformat(),
}, f, indent=2)
print("⚠ Self-critique failed, continuing with existing spec")
return PhaseResult("self_critique", True, [str(critique_file)], errors, MAX_RETRIES)
async def phase_planning(self) -> PhaseResult:
"""Phase 7: Create implementation plan."""
print("\n" + "=" * 60)
print(" PHASE 7: IMPLEMENTATION PLANNING")
print("=" * 60)
plan_file = self.spec_dir / "implementation_plan.json"
@@ -416,9 +964,9 @@ class SpecOrchestrator:
return PhaseResult("planning", False, [], errors, MAX_RETRIES)
async def phase_validation(self) -> PhaseResult:
"""Phase 6: Final validation."""
"""Phase 8: Final validation."""
print("\n" + "=" * 60)
print(" PHASE 6: FINAL VALIDATION")
print(" PHASE 8: FINAL VALIDATION")
print("=" * 60)
results = self.validator.validate_all()
@@ -444,7 +992,7 @@ class SpecOrchestrator:
# === Main Orchestration ===
async def run(self, interactive: bool = True) -> bool:
"""Run the full spec creation process."""
"""Run the spec creation process with dynamic phase selection."""
print("\n" + "=" * 60)
print(" SPEC CREATION ORCHESTRATOR")
print("=" * 60)
@@ -454,18 +1002,39 @@ class SpecOrchestrator:
print(f"Task: {self.task_description}")
print()
phases = [
("discovery", lambda: self.phase_discovery()),
("requirements", lambda: self.phase_requirements(interactive)),
("context", lambda: self.phase_context()),
("spec_writing", lambda: self.phase_spec_writing()),
("planning", lambda: self.phase_planning()),
("validation", lambda: self.phase_validation()),
]
# Phase 0: Always run complexity assessment first
result = await self.phase_complexity_assessment()
if not result.success:
print("✗ Complexity assessment failed")
return False
results = []
results = [result]
for phase_name, phase_fn in phases:
# Map of all available phases
all_phases = {
"discovery": lambda: self.phase_discovery(),
"requirements": lambda: self.phase_requirements(interactive),
"research": lambda: self.phase_research(),
"context": lambda: self.phase_context(),
"spec_writing": lambda: self.phase_spec_writing(),
"self_critique": lambda: self.phase_self_critique(),
"planning": lambda: self.phase_planning(),
"validation": lambda: self.phase_validation(),
"quick_spec": lambda: self.phase_quick_spec(),
}
# Get phases to run based on complexity
phases_to_run = self.assessment.phases_to_run()
print(f"\n Running {self.assessment.complexity.value.upper()} workflow ({len(phases_to_run)} phases)")
print()
for phase_name in phases_to_run:
if phase_name not in all_phases:
print(f"⚠ Unknown phase: {phase_name}, skipping")
continue
phase_fn = all_phases[phase_name]
result = await phase_fn()
results.append(result)
@@ -481,6 +1050,8 @@ class SpecOrchestrator:
print("\n" + "=" * 60)
print(" SPEC CREATION COMPLETE")
print("=" * 60)
print(f"\nComplexity: {self.assessment.complexity.value.upper()}")
print(f"Phases run: {len(phases_to_run)}")
print(f"\nSpec saved to: {self.spec_dir}")
print("\nFiles created:")
for result in results:
@@ -498,7 +1069,27 @@ def main():
import argparse
parser = argparse.ArgumentParser(
description="Orchestrate spec creation with validation"
description="Dynamic spec creation with complexity-based phase selection",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Complexity Tiers:
simple - 3 phases: Discovery → Quick Spec → Validate (1-2 files)
standard - 6 phases: Discovery → Requirements → Context → Spec → Plan → Validate
complex - 8 phases: Full pipeline with research and self-critique
Examples:
# Simple UI fix (auto-detected as simple)
python spec_runner.py --task "Fix button color in Header component"
# Force simple mode
python spec_runner.py --task "Update text" --complexity simple
# Complex integration (auto-detected)
python spec_runner.py --task "Add Graphiti memory integration with FalkorDB"
# Interactive mode
python spec_runner.py --interactive
"""
)
parser.add_argument(
"--task",
@@ -516,6 +1107,12 @@ def main():
type=str,
help="Continue an existing spec",
)
parser.add_argument(
"--complexity",
type=str,
choices=["simple", "standard", "complex"],
help="Override automatic complexity detection",
)
parser.add_argument(
"--project-dir",
type=Path,
@@ -545,6 +1142,7 @@ def main():
task_description=args.task,
spec_name=args.continue_spec,
model=args.model,
complexity_override=args.complexity,
)
try: