Files
Aperant/tests/agents/test_agent_configs.py
T
StillKnotKnown ed93df698b test: improve backend agent test coverage to 94% (#1779)
* fix: add mock reset fixtures and resolve async iterator mock issues

- Add pytest_runtest_setup and pytest_runtest_teardown hooks to reset
  shared module-level mocks between tests
- Add module-specific mock reset fixtures for test_qa_fixer and
  test_qa_reviewer to prevent test interference
- Fix async iterator mock for receive_response to properly return
  an AsyncIteratorMock instance
- Update test_qa_fixer.py and test_qa_reviewer.py with proper mock
  setup for isolated test execution

* docs(agents): add CLAUDE.md documentation for agents module

Documents the agents module architecture including:
- Module components (coder, planner, session, memory_manager, base)
- Single-agent architecture without external parallelism
- Subagent architecture clarification

* Revert "docs(agents): add CLAUDE.md documentation for agents module"

This reverts commit bf1ddd7da08f2f34352d11a5d823da981f1a98bb.

* chore: update gitignore to allow agents/tests/

* fix(tests): resolve mock isolation and path permission issues

- Fix test_tool_concurrency_error_detection by patching where functions
  are used (qa.fixer) instead of where they're defined
- Add Path.exists/is_dir/glob mocks to avoid permission errors on
  nonexistent directories in test_validation_strategy.py
- Add helper function clean_project_index_files() to reduce code
  duplication in prereqs_validator tests
- Add comprehensive tests for spec validation validators
  (context, prereqs, spec_document)
- Fix similar mock/path issues in test_qa_reviewer.py,
  test_service_orchestrator.py, test_ci_discovery.py,
  test_prompt_generator.py, test_security_scanner.py

All 2103 tests now pass.

* fix(tests): remove unused imports and fix double assignment

- Remove unused 'patch' import from validator test files
- Remove unused 'pytest' import where not needed
- Fix double assignment typo in test_error_message_includes_filename

* fix(tests): move agents tests to tests/agents/ directory

- Move test_agent_architecture.py, test_agent_configs.py, and
  test_agent_flow.py from apps/backend/agents/tests/ to tests/agents/
- Fix path resolution to work from new location
- Remove gitignore exception for agents/tests/ (no longer needed)

This resolves the issue where tests were not included in the PR
because they were in an untracked location.

* fix(tests): simplify conftest.py mock management

- Remove redundant pytest_runtest_teardown and pytest_runtest_call hooks
  (autouse fixtures in test files already handle mock reset)
- Add prompts_pkg.project_context to potentially mocked modules list
- Remove prompts_pkg from test_qa_fixer entry (not used there)

This reduces maintenance burden by having mock reset in one place.

* refactor(tests): consolidate duplicate mock setup into shared helper

- Create tests/qa_test_helpers.py with shared mock infrastructure:
  - AsyncIteratorMock and ReceiveResponseMock classes
  - setup_qa_mocks(), cleanup_qa_mocks(), reset_qa_mocks() functions
  - Mock response creation helpers
  - Accessor functions for mock objects
- Refactor test_qa_fixer.py to use shared helpers
- Reduces ~80 lines of duplicated code per test file
- Fixes potential mock binding issues by using accessor functions

This addresses code quality issues identified in PR review:
- Duplicate mock setup between test_qa_fixer.py and test_qa_reviewer.py
- Duplicated _AsyncIteratorMock class across files

* refactor(tests): consolidate test_qa_reviewer.py with shared helpers

- Refactor test_qa_reviewer.py to use shared qa_test_helpers
- Remove ~170 lines of duplicated mock setup and helper functions
- Fix unused imports in test_qa_fixer.py (json, sys, MagicMock, etc.)
- Fix rate limit error detection tests to patch where functions are used
- Consolidate duplicated _create_*_response helper methods to module level

Addresses CodeQL warnings about unused imports and reduces code
duplication between test_qa_fixer.py and test_qa_reviewer.py.

* fix(tests): remove unused Path import from test_qa_reviewer.py

* fix(tests): address all PR review findings

PR Review Fixes:
- Remove unused create_mock_qa_approved_response/rejected_response functions
- Guard against overwriting _original_modules on second setup_qa_mocks() call
- Clear _original_modules in cleanup_qa_mocks() to prevent stale state
- Add prompts_pkg.project_context to test_qa_reviewer preserved_mocks in conftest
- Convert asyncio.run() pattern to native async tests in test_agent_flow.py
- Remove redundant @pytest.mark.asyncio decorators (asyncio_mode=auto)
- Remove unused pytest import from qa_test_helpers.py
- Fix structural duplication by keeping fixtures in test files

Code Quality:
- Removed ~100 lines of duplicated/unused code
- Consistent async test patterns across all QA test files
- Proper mock state management to prevent test pollution

* fix(tests): save original modules individually in setup_qa_mocks

The boolean guard `setup_done` prevented saving original modules on
subsequent calls with different parameters. When setup_qa_mocks was
called first with include_prompts_pkg=False, then with True, the
prompts_pkg modules were never saved to _original_modules. During
cleanup, these unsaved modules were deleted from sys.modules instead
of being restored, causing ModuleNotFoundError in subsequent tests.

Now checks each module individually before mocking, ensuring all
originals are saved across multiple setup calls.

* fix(tests): address all PR review findings including low priority

- Fix path in test_no_subtask_worker_config (parent.parent.parent)
- Add guard to prevent double setup in setup_qa_mocks()
- Don't clear _original_modules in cleanup to fix multi-module cleanup

* fix(tests): address PR review follow-up findings

- Fix module-level mock setup ordering dependency: now tracks
  include_prompts_pkg config and allows incremental setup when
  test_qa_fixer.py (False) is imported before test_qa_reviewer.py (True)
- Remove unused asyncio import from test_agent_flow.py
- Replace os.chdir() with monkeypatch.chdir() in prereqs validator
  tests for safe parallel test execution

---------

Co-authored-by: StillKnotKnown <stillknotknown@users.noreply.github.com>
2026-02-12 15:06:25 +01:00

285 lines
10 KiB
Python

"""
Tests for AGENT_CONFIGS registry and related functions.
Tests the phase-aware tool and MCP server configuration system
that provides granular control over what tools/servers are available
during each execution phase.
"""
import os
import pytest
# Set up path for imports
import sys
from pathlib import Path
# Add backend to path
backend_path = Path(__file__).parent.parent.parent / "apps" / "backend"
sys.path.insert(0, str(backend_path))
class TestAgentConfigs:
"""Tests for AGENT_CONFIGS registry."""
def test_all_agent_types_have_required_fields(self):
"""Every agent config should have tools, mcp_servers, auto_claude_tools, thinking_default."""
from agents.tools_pkg.models import AGENT_CONFIGS
required_fields = ["tools", "mcp_servers", "auto_claude_tools", "thinking_default"]
for agent_type, config in AGENT_CONFIGS.items():
for field in required_fields:
assert field in config, f"Agent type '{agent_type}' missing field '{field}'"
def test_known_agent_types_exist(self):
"""Key agent types from PRD should exist."""
from agents.tools_pkg.models import AGENT_CONFIGS
expected_types = [
# Spec phases
"spec_gatherer",
"spec_researcher",
"spec_writer",
"spec_critic",
# Build phases
"planner",
"coder",
# QA phases
"qa_reviewer",
"qa_fixer",
# Utility phases
"insights",
"merge_resolver",
"commit_message",
"pr_reviewer",
]
for agent_type in expected_types:
assert agent_type in AGENT_CONFIGS, f"Expected agent type '{agent_type}' not found"
def test_thinking_defaults_are_valid(self):
"""All thinking_default values should be valid levels."""
from agents.tools_pkg.models import AGENT_CONFIGS
from phase_config import THINKING_BUDGET_MAP
valid_levels = set(THINKING_BUDGET_MAP.keys())
for agent_type, config in AGENT_CONFIGS.items():
level = config.get("thinking_default")
assert level in valid_levels, f"Agent '{agent_type}' has invalid thinking_default: {level}"
def test_tools_are_lists(self):
"""All tool configurations should be lists."""
from agents.tools_pkg.models import AGENT_CONFIGS
for agent_type, config in AGENT_CONFIGS.items():
assert isinstance(config["tools"], list), f"Agent '{agent_type}' tools should be list"
assert isinstance(
config["auto_claude_tools"], list
), f"Agent '{agent_type}' auto_claude_tools should be list"
assert isinstance(
config["mcp_servers"], list
), f"Agent '{agent_type}' mcp_servers should be list"
class TestGetAgentConfig:
"""Tests for get_agent_config() function."""
def test_returns_config_for_known_type(self):
"""Should return config dict for known agent types."""
from agents.tools_pkg.models import get_agent_config
config = get_agent_config("coder")
assert isinstance(config, dict)
assert "tools" in config
assert "mcp_servers" in config
def test_raises_for_unknown_type(self):
"""Should raise ValueError for unknown agent types."""
from agents.tools_pkg.models import get_agent_config
with pytest.raises(ValueError) as excinfo:
get_agent_config("nonexistent_agent_type")
assert "Unknown agent type" in str(excinfo.value)
assert "nonexistent_agent_type" in str(excinfo.value)
class TestGetRequiredMcpServers:
"""Tests for get_required_mcp_servers() function."""
def test_spec_gatherer_has_no_mcp_servers(self):
"""spec_gatherer should not require any MCP servers."""
from agents.tools_pkg.models import get_required_mcp_servers
servers = get_required_mcp_servers("spec_gatherer")
assert servers == []
def test_spec_researcher_has_context7(self):
"""spec_researcher should require context7 for docs lookup."""
from agents.tools_pkg.models import get_required_mcp_servers
servers = get_required_mcp_servers("spec_researcher")
assert "context7" in servers
def test_coder_has_context7_and_auto_claude(self):
"""coder should require context7 and auto-claude."""
from agents.tools_pkg.models import get_required_mcp_servers
servers = get_required_mcp_servers("coder")
assert "context7" in servers
assert "auto-claude" in servers
def test_linear_optional_not_included_by_default(self):
"""Linear should not be included unless linear_enabled=True."""
from agents.tools_pkg.models import get_required_mcp_servers
servers = get_required_mcp_servers("planner", linear_enabled=False)
assert "linear" not in servers
def test_linear_included_when_enabled(self):
"""Linear should be included when linear_enabled=True for agents with optional Linear."""
from agents.tools_pkg.models import get_required_mcp_servers
servers = get_required_mcp_servers("planner", linear_enabled=True)
assert "linear" in servers
def test_browser_resolved_to_electron_for_electron_project(self):
"""Browser should resolve to 'electron' for Electron projects."""
from agents.tools_pkg.models import get_required_mcp_servers
# Mock ELECTRON_MCP_ENABLED
os.environ["ELECTRON_MCP_ENABLED"] = "true"
try:
servers = get_required_mcp_servers(
"qa_reviewer", project_capabilities={"is_electron": True}
)
assert "electron" in servers
assert "browser" not in servers
assert "puppeteer" not in servers
finally:
os.environ.pop("ELECTRON_MCP_ENABLED", None)
def test_browser_resolved_to_puppeteer_for_web_frontend(self):
"""Browser should resolve to 'puppeteer' for web frontend projects when enabled."""
from agents.tools_pkg.models import get_required_mcp_servers
# Puppeteer requires explicit opt-in via project config
servers = get_required_mcp_servers(
"qa_reviewer",
project_capabilities={"is_web_frontend": True, "is_electron": False},
mcp_config={"PUPPETEER_MCP_ENABLED": "true"},
)
assert "puppeteer" in servers
assert "browser" not in servers
assert "electron" not in servers
def test_puppeteer_not_included_when_disabled(self):
"""Puppeteer should NOT be included when not explicitly enabled (default)."""
from agents.tools_pkg.models import get_required_mcp_servers
# Default behavior: puppeteer is NOT auto-enabled for web frontends
servers = get_required_mcp_servers(
"qa_reviewer",
project_capabilities={"is_web_frontend": True, "is_electron": False},
)
assert "puppeteer" not in servers
assert "browser" not in servers
class TestGetDefaultThinkingLevel:
"""Tests for get_default_thinking_level() function."""
def test_returns_low_for_coder(self):
"""Coder should return 'low' thinking level."""
from agents.tools_pkg.models import get_default_thinking_level
result = get_default_thinking_level("coder")
assert result == "low"
def test_returns_high_for_qa_reviewer(self):
"""QA reviewer should return 'high' thinking level."""
from agents.tools_pkg.models import get_default_thinking_level
result = get_default_thinking_level("qa_reviewer")
assert result == "high"
def test_returns_high_for_spec_critic(self):
"""Spec critic should return 'high' thinking level."""
from agents.tools_pkg.models import get_default_thinking_level
result = get_default_thinking_level("spec_critic")
assert result == "high"
def test_can_convert_to_budget_via_phase_config(self):
"""Verify thinking level can be converted to budget using phase_config."""
from agents.tools_pkg.models import get_default_thinking_level
from phase_config import THINKING_BUDGET_MAP
level = get_default_thinking_level("qa_reviewer")
budget = THINKING_BUDGET_MAP.get(level)
assert budget == THINKING_BUDGET_MAP["high"]
class TestGetAllowedTools:
"""Tests for get_allowed_tools() function."""
def test_coder_includes_write_tools(self):
"""Coder should have Write, Edit, Bash tools."""
from agents.tools_pkg.permissions import get_allowed_tools
tools = get_allowed_tools("coder")
assert "Write" in tools
assert "Edit" in tools
assert "Bash" in tools
def test_qa_reviewer_has_write_for_reports(self):
"""QA reviewer needs Write/Edit to create qa_report.md and update implementation_plan.json."""
from agents.tools_pkg.permissions import get_allowed_tools
tools = get_allowed_tools("qa_reviewer")
assert "Read" in tools
assert "Bash" in tools # Can run tests
assert "Write" in tools # Needs to write qa_report.md
assert "Edit" in tools # Needs to edit implementation_plan.json
def test_pr_reviewer_is_read_only(self):
"""PR reviewer should only have Read tools."""
from agents.tools_pkg.permissions import get_allowed_tools
tools = get_allowed_tools("pr_reviewer")
assert "Read" in tools
assert "Write" not in tools
assert "Edit" not in tools
assert "Bash" not in tools
def test_merge_resolver_has_no_tools(self):
"""Merge resolver is text-only, no tools."""
from agents.tools_pkg.permissions import get_allowed_tools
tools = get_allowed_tools("merge_resolver")
# Should have no file operation tools
assert "Read" not in tools
assert "Write" not in tools
assert "Bash" not in tools
def test_raises_for_unknown_type(self):
"""Should raise ValueError for unknown agent types."""
from agents.tools_pkg.permissions import get_allowed_tools
with pytest.raises(ValueError):
get_allowed_tools("definitely_not_a_real_agent")
class TestGetAllAgentTypes:
"""Tests for get_all_agent_types() function."""
def test_returns_sorted_list(self):
"""Should return a sorted list of all agent types."""
from agents.tools_pkg.permissions import get_all_agent_types
types = get_all_agent_types()
assert isinstance(types, list)
assert types == sorted(types)
assert len(types) > 10 # Should have many agent types