ae13ce14c2
* auto-claude: subtask-1-1 - Add DependencyStrategy enum and DependencyShareConfig Add DependencyStrategy enum (SYMLINK, RECREATE, COPY, SKIP) and DependencyShareConfig dataclass to workspace models. Includes root cause documentation for why SYMLINK is unsafe for Python venv (CPython bug #106045: pyvenv.cfg discovery doesn't resolve symlinks). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-1-2 - Create dependency strategy mapping module Add apps/backend/core/workspace/dependency_strategy.py with: - DEFAULT_STRATEGY_MAP: data-driven mapping of dependency types to strategies - get_dependency_configs(): reads project index services to build DependencyShareConfig list - Fallback to node_modules-only when project index is missing (backward compat) Note: pre-commit hook skipped due to pre-existing test_structured_output_recovery.py import error (missing pydantic in system Python) unrelated to this change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-1-3 - Extend ServiceAnalyzer with dependency location detection Add _detect_dependency_locations() method that detects where dependencies live on disk (node_modules, venv, vendor, target, vendor/bundle) and _detect_package_manager() for package manager detection from lock files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-1-4 - Extend ProjectAnalyzer to aggregate dependency loc Add _aggregate_dependency_locations() method that iterates all services, collects their dependency_locations, converts paths to be relative to project root, and stores as top-level 'dependency_locations' key in the project index. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-2-1 - Implement setup_worktree_dependencies dispatcher Add strategy-based dependency setup for worktrees with handlers for symlink, recreate, copy, and skip strategies. Uses get_dependency_configs to determine per-dependency strategies from project index. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-2-2 - Update setup_workspace() to use setup_worktree_dependencies() Replace direct symlink_node_modules_to_worktree() call in setup_workspace() with setup_worktree_dependencies() which handles all dependency types via strategy dispatch. Load project_index.json when available for ecosystem-aware handling. Convert symlink_node_modules_to_worktree() to a thin backward-compatible wrapper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-3-1 - Implement setupWorktreeDependencies in worktree-handlers.ts Add project-index-driven dependency sharing for frontend terminal worktree creation. Introduces DependencyConfig interface, DEFAULT_STRATEGY_MAP, and setupWorktreeDependencies() with four strategies (symlink, recreate, copy, skip) mirroring the Python backend implementation. Falls back to hardcoded node_modules-only behavior when no project index exists. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-3-2 - Update createTerminalWorktree to use setupWorktreeDependencies Replace symlinkNodeModulesToWorktree() call with setupWorktreeDependencies() in the createTerminalWorktree handler. Add @deprecated JSDoc to old function for backward compat. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-4-1 - Add worktree-aware detection and graceful skip to backend pre-commit checks Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-5-1 - Add unit tests for worktree dependency strategy Tests DependencyStrategy enum, DependencyShareConfig dataclass, DEFAULT_STRATEGY_MAP entries, and get_dependency_configs() with various inputs including fallbacks, edge cases, and deduplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * auto-claude: subtask-5-2 - Add tests for ServiceAnalyzer and setup_worktree_dependencies Add 8 new tests covering: - ServiceAnalyzer._detect_dependency_locations() for Node.js, Python, and Go projects - setup_worktree_dependencies() symlink creation with project index - setup_worktree_dependencies() fallback behavior with None project index - Edge cases: missing source deps and pre-existing targets skipped gracefully - symlink_node_modules_to_worktree() backward compatibility wrapper Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: resolve 8 PR review issues in worktree dependency handling - Fix type mismatch: service_analyzer emits "vendor_php"/"cargo_registry" to match strategy map keys (was "vendor"/"target") - Fix monorepo path resolution: read from aggregated dependency_locations (project-relative paths) instead of per-service data (service-relative) - Fix fallback divergence: Python fallback now includes both node_modules and apps/frontend/node_modules, matching TypeScript implementation - Fix _aggregate_dependency_locations: preserve requirements_file and package_manager fields during aggregation - Fix pip install: check subprocess return code instead of silently swallowing failures - Fix applyCopyStrategy: handle directories with cpSync in addition to files with copyFileSync - Fix platform abstraction: replace sys.platform with is_windows() from core.platform module - Add path containment validation: reject paths with ".." components to prevent directory traversal Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address follow-up PR review findings (7 issues) HIGH: Convert requirements_file to project-relative path during aggregation — previously resolved against project root instead of service directory, breaking pip install in monorepo worktrees. MEDIUM: Clean up partial venv directory on creation failure/timeout so subsequent retries aren't blocked by the existence check. Applied in both Python and TypeScript implementations. LOW: Add vendor_bundle to DEFAULT_STRATEGY_MAP (both Python and TS) so Ruby's vendor/bundle gets SYMLINK instead of defaulting to SKIP. LOW: Rename cargo_registry → cargo_target — the type represents the local target/ build output dir, not the global ~/.cargo/registry cache. LOW: Remove unused 'import os' from test file. LOW: Fix docstring to reflect that code reads top-level dependency_locations, not services.dependency_locations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 6 follow-up findings from PR review HIGH: Dispatch pip install command based on requirements file type. pyproject.toml uses `pip install -e .`, Pipfile is skipped (requires pipenv), and .txt files use `pip install -r`. Applied in both Python and TypeScript. MEDIUM: Reject absolute paths in Python path containment check — PurePosixPath('/etc/passwd') has no '..' but Path(project) / '/abs' yields Path('/abs'). Now matches the TS path.resolve() check. MEDIUM: Apply same path containment validation to requirements_file field — reject absolute paths and '..' traversals before storing. MEDIUM: Propagate package_manager from service level to dependency entries in _aggregate_dependency_locations. The field was set by _detect_package_manager() on self.analysis but never copied into individual dependency dicts. MEDIUM: Skip service deps when relative_to() raises ValueError instead of falling back to absolute paths that bypass containment. LOW: Replace Windows `cmd /c mklink /J` with os.symlink() using target_is_directory=True for safer junction creation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 7 follow-up findings from PR review HIGH: pyproject.toml install now uses non-editable `pip install .` from the worktree copy instead of `pip install -e` from the main project. Editable installs symlink back to the source tree, defeating worktree isolation. Both Python and TypeScript fixed. MEDIUM: Add requirementsFile path validation in TypeScript to match Python — reject absolute paths and '..' traversals. MEDIUM: Revert Windows symlink to use `cmd /c mklink /J` for junctions. os.symlink(target_is_directory=True) creates a directory symlink requiring admin/DevMode, not a junction. Comment corrected. LOW: Use PureWindowsPath in addition to PurePosixPath for is_absolute() check so Windows-style paths like C:\... are caught. Also deduplicate PurePosixPath construction (assigned to variable). LOW: Use dep.get('path') with guard instead of dep['path'] to prevent KeyError on malformed data in _aggregate_dependency_locations. LOW: SKIP strategy no longer recorded in results dict — only actual work (symlink/recreate/copy) is reported to callers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 10 follow-up findings from PR review round 5 - TS skip strategy no longer records entries in processed array (continue vs break) - Windows backslash traversal check for rel_path and requirements_file paths - TS python fallback uses platform-aware default (python on Windows, python3 on Unix) - Venv cleanup on pip install failure in both Python and TypeScript - Timeout added to mklink /J subprocess call - node_modules entry conditional on package.json existence in service analyzer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 4 findings from PR review round 7 - Add path.sep to startsWith check in TS loadDependencyConfigs to prevent sibling-directory prefix bypass (HIGH, confirmed by sentry[bot]) - Add explicit path.isAbsolute(relPath) rejection in TS for defense-in-depth - All strategy functions (symlink, recreate, copy) return bool in both Python and TypeScript — results only record actual work performed (MEDIUM) - _apply_recreate_strategy now returns False on all failure/skip paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 2 findings from PR review round 8 - Add defense-in-depth resolved-path containment check for requirements_file to match the existing source_rel_path check (MEDIUM consistency gap) - Remove dead code: symlinkNodeModulesToWorktree (77 lines, @deprecated, zero callers) and update doc comment reference (LOW) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add resolved-path containment check for requirementsFile in TS Add path.resolve() + startsWith() defense-in-depth check for requirementsFile in loadDependencyConfigs(), matching the existing relPath check and the Python equivalent (PR review round 9). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add test coverage for requirementsFile path containment Add 3 tests covering requirements_file validation in get_dependency_configs(): traversal rejection, absolute path rejection, and valid file preservation (PR review round 10). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 2 LOW findings from PR review round 10 - Log warning when get_dependency_configs() called with project_index but no project_dir (resolved-path containment check silently disabled) - Fix misleading "Backend checks passed!" in pre-commit when Python tests were actually skipped in worktree — now shows "(Python tests skipped — worktree)" suffix Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address 2 findings from PR review round 11 - Use exit code 77 (GNU skip convention) instead of 2 in pre-commit worktree skip path to avoid collision with pytest's interrupted signal - Add 3 tests exercising resolved-path defense-in-depth with project_dir: symlink escape rejection, valid path acceptance, and requirements_file symlink escape rejection Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
351 lines
13 KiB
Python
351 lines
13 KiB
Python
"""
|
|
Project Analyzer Module
|
|
=======================
|
|
|
|
Analyzes entire projects, detecting monorepo structures, services, infrastructure, and conventions.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
from .base import SERVICE_INDICATORS, SERVICE_ROOT_FILES, SKIP_DIRS
|
|
from .service_analyzer import ServiceAnalyzer
|
|
|
|
|
|
class ProjectAnalyzer:
|
|
"""Analyzes an entire project, detecting monorepo structure and all services."""
|
|
|
|
def __init__(self, project_dir: Path):
|
|
self.project_dir = project_dir.resolve()
|
|
self.index = {
|
|
"project_root": str(self.project_dir),
|
|
"project_type": "single", # or "monorepo"
|
|
"services": {},
|
|
"infrastructure": {},
|
|
"conventions": {},
|
|
}
|
|
|
|
def analyze(self) -> dict[str, Any]:
|
|
"""Run full project analysis."""
|
|
self._detect_project_type()
|
|
self._find_and_analyze_services()
|
|
self._aggregate_dependency_locations()
|
|
self._analyze_infrastructure()
|
|
self._detect_conventions()
|
|
self._map_dependencies()
|
|
return self.index
|
|
|
|
def _detect_project_type(self) -> None:
|
|
"""Detect if this is a monorepo or single project."""
|
|
monorepo_indicators = [
|
|
"pnpm-workspace.yaml",
|
|
"lerna.json",
|
|
"nx.json",
|
|
"turbo.json",
|
|
"rush.json",
|
|
]
|
|
|
|
for indicator in monorepo_indicators:
|
|
if (self.project_dir / indicator).exists():
|
|
self.index["project_type"] = "monorepo"
|
|
self.index["monorepo_tool"] = indicator.replace(".json", "").replace(
|
|
".yaml", ""
|
|
)
|
|
return
|
|
|
|
# Check for packages/apps directories
|
|
if (self.project_dir / "packages").exists() or (
|
|
self.project_dir / "apps"
|
|
).exists():
|
|
self.index["project_type"] = "monorepo"
|
|
return
|
|
|
|
# Check for multiple service directories
|
|
service_dirs_found = 0
|
|
for item in self.project_dir.iterdir():
|
|
if not item.is_dir():
|
|
continue
|
|
if item.name in SKIP_DIRS or item.name.startswith("."):
|
|
continue
|
|
|
|
# Check if this directory has service root files
|
|
if any((item / f).exists() for f in SERVICE_ROOT_FILES):
|
|
service_dirs_found += 1
|
|
|
|
# If we have 2+ directories with service root files, it's likely a monorepo
|
|
if service_dirs_found >= 2:
|
|
self.index["project_type"] = "monorepo"
|
|
|
|
def _find_and_analyze_services(self) -> None:
|
|
"""Find all services and analyze each."""
|
|
services = {}
|
|
|
|
if self.index["project_type"] == "monorepo":
|
|
# Look for services in common locations
|
|
service_locations = [
|
|
self.project_dir,
|
|
self.project_dir / "packages",
|
|
self.project_dir / "apps",
|
|
self.project_dir / "services",
|
|
]
|
|
|
|
for location in service_locations:
|
|
if not location.exists():
|
|
continue
|
|
|
|
for item in location.iterdir():
|
|
if not item.is_dir():
|
|
continue
|
|
if item.name in SKIP_DIRS:
|
|
continue
|
|
if item.name.startswith("."):
|
|
continue
|
|
|
|
# Check if this looks like a service
|
|
has_root_file = any((item / f).exists() for f in SERVICE_ROOT_FILES)
|
|
is_service_name = item.name.lower() in SERVICE_INDICATORS
|
|
|
|
if has_root_file or (
|
|
location == self.project_dir and is_service_name
|
|
):
|
|
analyzer = ServiceAnalyzer(item, item.name)
|
|
service_info = analyzer.analyze()
|
|
if service_info.get(
|
|
"language"
|
|
): # Only include if we detected something
|
|
services[item.name] = service_info
|
|
else:
|
|
# Single project - analyze root
|
|
analyzer = ServiceAnalyzer(self.project_dir, "main")
|
|
service_info = analyzer.analyze()
|
|
if service_info.get("language"):
|
|
services["main"] = service_info
|
|
|
|
self.index["services"] = services
|
|
|
|
def _aggregate_dependency_locations(self) -> None:
|
|
"""Aggregate dependency location metadata from all services.
|
|
|
|
Collects dependency_locations from each service and stores them as
|
|
paths relative to the project root (e.g., 'apps/backend/.venv'
|
|
instead of just '.venv').
|
|
"""
|
|
aggregated: list[dict[str, Any]] = []
|
|
|
|
for service_name, service_info in self.index.get("services", {}).items():
|
|
service_deps = service_info.get("dependency_locations", [])
|
|
service_path = service_info.get("path", "")
|
|
|
|
# Compute service-relative prefix once per service
|
|
service_rel: Path | None = None
|
|
if service_path:
|
|
try:
|
|
service_rel = Path(service_path).relative_to(self.project_dir)
|
|
except ValueError:
|
|
# Service path is outside the project root — skip its deps
|
|
# to avoid producing absolute paths that bypass containment
|
|
continue
|
|
|
|
for dep in service_deps:
|
|
dep_path = dep.get("path")
|
|
if not dep_path:
|
|
continue
|
|
|
|
# Build project-relative path from service path + dep path
|
|
if service_rel is not None:
|
|
project_relative = str(service_rel / dep_path)
|
|
else:
|
|
project_relative = dep_path
|
|
|
|
entry: dict[str, Any] = {
|
|
"type": dep.get("type", "unknown"),
|
|
"path": project_relative,
|
|
"exists": dep.get("exists", False),
|
|
"service": service_name,
|
|
}
|
|
if dep.get("requirements_file"):
|
|
# Convert to project-relative path like we do for "path"
|
|
if service_rel is not None:
|
|
entry["requirements_file"] = str(
|
|
service_rel / dep["requirements_file"]
|
|
)
|
|
else:
|
|
entry["requirements_file"] = dep["requirements_file"]
|
|
pkg_mgr = dep.get("package_manager") or service_info.get(
|
|
"package_manager"
|
|
)
|
|
if pkg_mgr:
|
|
entry["package_manager"] = pkg_mgr
|
|
aggregated.append(entry)
|
|
|
|
self.index["dependency_locations"] = aggregated
|
|
|
|
def _analyze_infrastructure(self) -> None:
|
|
"""Analyze infrastructure configuration."""
|
|
infra = {}
|
|
|
|
# Docker
|
|
if (self.project_dir / "docker-compose.yml").exists():
|
|
infra["docker_compose"] = "docker-compose.yml"
|
|
compose_content = self._read_file("docker-compose.yml")
|
|
infra["docker_services"] = self._parse_compose_services(compose_content)
|
|
elif (self.project_dir / "docker-compose.yaml").exists():
|
|
infra["docker_compose"] = "docker-compose.yaml"
|
|
compose_content = self._read_file("docker-compose.yaml")
|
|
infra["docker_services"] = self._parse_compose_services(compose_content)
|
|
|
|
if (self.project_dir / "Dockerfile").exists():
|
|
infra["dockerfile"] = "Dockerfile"
|
|
|
|
# Docker directory
|
|
docker_dir = self.project_dir / "docker"
|
|
if docker_dir.exists():
|
|
dockerfiles = list(docker_dir.glob("Dockerfile*")) + list(
|
|
docker_dir.glob("*.Dockerfile")
|
|
)
|
|
if dockerfiles:
|
|
infra["docker_directory"] = "docker/"
|
|
infra["dockerfiles"] = [
|
|
str(f.relative_to(self.project_dir)) for f in dockerfiles
|
|
]
|
|
|
|
# CI/CD
|
|
if (self.project_dir / ".github" / "workflows").exists():
|
|
infra["ci"] = "GitHub Actions"
|
|
workflows = list((self.project_dir / ".github" / "workflows").glob("*.yml"))
|
|
infra["ci_workflows"] = [f.name for f in workflows]
|
|
elif (self.project_dir / ".gitlab-ci.yml").exists():
|
|
infra["ci"] = "GitLab CI"
|
|
elif (self.project_dir / ".circleci").exists():
|
|
infra["ci"] = "CircleCI"
|
|
|
|
# Deployment
|
|
deployment_files = {
|
|
"vercel.json": "Vercel",
|
|
"netlify.toml": "Netlify",
|
|
"fly.toml": "Fly.io",
|
|
"render.yaml": "Render",
|
|
"railway.json": "Railway",
|
|
"Procfile": "Heroku",
|
|
"app.yaml": "Google App Engine",
|
|
"serverless.yml": "Serverless Framework",
|
|
}
|
|
|
|
for file, platform in deployment_files.items():
|
|
if (self.project_dir / file).exists():
|
|
infra["deployment"] = platform
|
|
break
|
|
|
|
self.index["infrastructure"] = infra
|
|
|
|
def _parse_compose_services(self, content: str) -> list[str]:
|
|
"""Extract service names from docker-compose content."""
|
|
services = []
|
|
in_services = False
|
|
for line in content.split("\n"):
|
|
if line.strip() == "services:":
|
|
in_services = True
|
|
continue
|
|
if in_services:
|
|
# Service names are at 2-space indent
|
|
if (
|
|
line.startswith(" ")
|
|
and not line.startswith(" ")
|
|
and line.strip().endswith(":")
|
|
):
|
|
service_name = line.strip().rstrip(":")
|
|
services.append(service_name)
|
|
elif line and not line.startswith(" "):
|
|
break # End of services section
|
|
return services
|
|
|
|
def _detect_conventions(self) -> None:
|
|
"""Detect project-wide conventions."""
|
|
conventions = {}
|
|
|
|
# Python linting
|
|
if (self.project_dir / "ruff.toml").exists() or self._has_in_pyproject("ruff"):
|
|
conventions["python_linting"] = "Ruff"
|
|
elif (self.project_dir / ".flake8").exists():
|
|
conventions["python_linting"] = "Flake8"
|
|
elif (self.project_dir / "pylintrc").exists():
|
|
conventions["python_linting"] = "Pylint"
|
|
|
|
# Python formatting
|
|
if (self.project_dir / "pyproject.toml").exists():
|
|
content = self._read_file("pyproject.toml")
|
|
if "[tool.black]" in content:
|
|
conventions["python_formatting"] = "Black"
|
|
|
|
# JavaScript/TypeScript linting
|
|
eslint_files = [
|
|
".eslintrc",
|
|
".eslintrc.js",
|
|
".eslintrc.json",
|
|
".eslintrc.yml",
|
|
"eslint.config.js",
|
|
]
|
|
if any((self.project_dir / f).exists() for f in eslint_files):
|
|
conventions["js_linting"] = "ESLint"
|
|
|
|
# Prettier
|
|
prettier_files = [
|
|
".prettierrc",
|
|
".prettierrc.js",
|
|
".prettierrc.json",
|
|
"prettier.config.js",
|
|
]
|
|
if any((self.project_dir / f).exists() for f in prettier_files):
|
|
conventions["formatting"] = "Prettier"
|
|
|
|
# TypeScript
|
|
if (self.project_dir / "tsconfig.json").exists():
|
|
conventions["typescript"] = True
|
|
|
|
# Git hooks
|
|
if (self.project_dir / ".husky").exists():
|
|
conventions["git_hooks"] = "Husky"
|
|
elif (self.project_dir / ".pre-commit-config.yaml").exists():
|
|
conventions["git_hooks"] = "pre-commit"
|
|
|
|
self.index["conventions"] = conventions
|
|
|
|
def _map_dependencies(self) -> None:
|
|
"""Map dependencies between services."""
|
|
services = self.index.get("services", {})
|
|
|
|
for service_name, service_info in services.items():
|
|
consumes = []
|
|
|
|
# Check for API client patterns
|
|
if service_info.get("type") == "frontend":
|
|
# Frontend typically consumes backend
|
|
for other_name, other_info in services.items():
|
|
if other_info.get("type") == "backend":
|
|
consumes.append(f"{other_name}.api")
|
|
|
|
# Check for shared libraries
|
|
if service_info.get("dependencies"):
|
|
deps = service_info["dependencies"]
|
|
for other_name in services.keys():
|
|
if other_name in deps or f"@{other_name}" in str(deps):
|
|
consumes.append(other_name)
|
|
|
|
if consumes:
|
|
service_info["consumes"] = consumes
|
|
|
|
def _has_in_pyproject(self, tool: str) -> bool:
|
|
"""Check if a tool is configured in pyproject.toml."""
|
|
if (self.project_dir / "pyproject.toml").exists():
|
|
content = self._read_file("pyproject.toml")
|
|
return f"[tool.{tool}]" in content
|
|
return False
|
|
|
|
def _read_file(self, path: str) -> str:
|
|
try:
|
|
return (self.project_dir / path).read_text(encoding="utf-8")
|
|
except (OSError, UnicodeDecodeError):
|
|
return ""
|