6a6247bbf2
* Fix UTF-8 encoding for Priorities 1-2 (Core & Agents - 18 instances) Add encoding="utf-8" to file operations in: - Priority 1: Core Infrastructure (8 instances) - core/progress.py (6 read operations) - core/debug.py (1 append operation) - core/workspace/setup.py (1 read operation) - Priority 2: Agent System (10 instances) - agents/utils.py (1 read) - agents/tools_pkg/tools/subtask.py (1 read, 1 write) - agents/tools_pkg/tools/memory.py (2 read, 1 write, 1 append) - agents/tools_pkg/tools/qa.py (1 read, 1 write) - agents/tools_pkg/tools/progress.py (1 read) All changes use double quotes for ruff format compliance. * Fix UTF-8 encoding for Priorities 3-4 (Spec & Project - 26 instances) Add encoding="utf-8" to file operations in: - Priority 3: Spec Pipeline (21 instances) - spec/context.py (4: 2 read, 2 write) - spec/complexity.py (3: 2 read, 1 write) - spec/requirements.py (3: 2 read, 1 write) - spec/validator.py (3 write operations) - spec/writer.py (2: 1 read, 1 write) - spec/discovery.py (1 read) - spec/pipeline/orchestrator.py (2 read) - spec/phases/requirements_phases.py (1 write) - spec/validate_pkg/auto_fix.py (2: 1 read, 1 write) - Priority 4: Project Analyzer (5 instances) - project/analyzer.py (2: 1 read, 1 write) - project/config_parser.py (2 read operations) - project/stack_detector.py (1 read) All changes use double quotes for ruff format compliance. * Fix UTF-8 encoding for Priorities 5-7 (Services, Analysis, Ideation - 43 instances) Add encoding="utf-8" to file operations in: - Priority 5: Services (12 instances) - services/recovery.py (8: 4 read, 4 write) - services/context.py (4 read operations) - Priority 6: Analysis & QA (6 instances) - analysis/analyzers/__init__.py (2 write) - analysis/insight_extractor.py (1 read) - qa/criteria.py (2: 1 read, 1 write) - qa/report.py (1 read) - Priority 7: Ideation & Roadmap (25 instances) - ideation/analyzer.py (3 read) - ideation/formatter.py (4 read, 1 write) - ideation/phase_executor.py (5: 3 read, 2 write) - ideation/runner.py (1 read) - runners/roadmap/competitor_analyzer.py (3: 1 read, 2 write) - runners/roadmap/graph_integration.py (3 write) - runners/roadmap/orchestrator.py (1 read) - runners/roadmap/phases.py (2 read) - runners/insights_runner.py (3 read) All changes use double quotes for ruff format compliance. * Fix UTF-8 encoding for Priorities 8-14 (All remaining - 85+ instances) Add encoding="utf-8" to file operations across all remaining modules: Priorities 8-10 (Merge, Memory, Integrations - 26 instances): - merge/ (4 files) - memory/ (3 files) - context/ (3 files) - integrations/ (4 files) Priorities 11-14 (GitHub, GitLab, AI, Other - 59 instances): - runners/github/ (19 files) - runners/gitlab/ (3 files) - runners/ai_analyzer/ (1 file) All changes use double quotes for ruff format compliance. Applied using Python regex script for efficiency. * Fix UTF-8 encoding for missed instances (23 instances) Fix remaining instances missed by batch script: - cli/batch_commands.py (3 instances) - cli/followup_commands.py (1 instance) - core/client.py (1 instance) - phase_config.py (1 instance) - planner_lib/context.py (4 instances) - prediction/main.py (1 instance) - prediction/memory_loader.py (1 instance) - prompts_pkg/prompts.py (2 instances) - review/formatters.py (1 instance) - review/state.py (2 instances) - spec/phases/spec_phases.py (1 instance) - spec/pipeline/models.py (1 instance) - spec/validate_pkg/validators/context_validator.py (1 instance) - spec/validate_pkg/validators/implementation_plan_validator.py (1 instance) - ui/status.py (2 instances) All encoding parameters use double quotes for ruff format compliance. Verified: 0 instances without encoding remain in source code. * Fix missed os.fdopen() calls and duplicate encoding bug Thorough verification found 3 additional issues: - runners/github/file_lock.py:462 - os.fdopen missing encoding - runners/github/trust.py:442 - os.fdopen missing encoding - runners/insights_runner.py:372 - duplicate encoding parameter All fixed. Final count: 251 instances with encoding="utf-8" * Fix missed Path.read_text() and Path.write_text() encoding (99 instances) Gemini Code Assist review found instances we missed: - Path.read_text() without encoding: 77 instances → fixed - Path.write_text() without encoding: 22 instances → fixed Total UTF-8 encoding fixes: 350 instances across codebase - open() operations: 251 instances - Path.read_text(): 98 instances - Path.write_text(): 30 instances All text file operations now explicitly use encoding="utf-8". Addresses feedback from PR #782 review. * Fix critical syntax errors from CodeRabbit review - Fix os.getpid() syntax error in core/workspace/models.py (2 instances) Changed: os.getpid(, encoding="utf-8") -> str(os.getpid()) - Fix json.dumps invalid encoding parameter (3 instances) json.dumps() doesn't accept encoding parameter Changed: json.dumps(data, encoding="utf-8") -> json.dumps(data) Files: runners/ai_analyzer/cache_manager.py, runners/github/test_file_lock.py - Fix tempfile.NamedTemporaryFile missing encoding Added encoding="utf-8" to spec/requirements.py:22 - Fix subprocess.run text=True to encoding Changed: text=True -> encoding="utf-8" in core/workspace/setup.py:375 All critical syntax errors from CodeRabbit review resolved. * Fix critical syntax errors in test_context_gatherer.py - Line 78: Move encoding="utf-8" outside of JS string content Changed: write_text("...encoding="utf-8"...") To: write_text("...", encoding="utf-8") - Line 102: Move encoding="utf-8" outside of JS string content Changed: write_text("...encoding="utf-8"...") To: write_text("...", encoding="utf-8") Fixes syntax errors where encoding parameter was incorrectly placed inside the JavaScript code string instead of as write_text() parameter. * Fix CodeRabbit issues: UnicodeDecodeError handling and trailing newlines - Add UnicodeDecodeError to exception handling in agents/utils.py and spec/validate_pkg/auto_fix.py - Fix trailing newline preservation in merge/file_merger.py (2 locations) - Add encoding parameter to atomic_write() in runners/github/file_lock.py These fixes ensure robust error handling for malformed UTF-8 files and preserve file formatting during merge operations. * Fix test fixture to use UTF-8 encoding consistently Update spec_file fixture in tests/conftest.py to write spec file with encoding="utf-8" to match how it's read in validators. This ensures consistency between test fixtures and production code. * Fix linting errors and security vulnerabilities from merge - Remove unused tree-sitter methods in semantic_analyzer.py that caused F821 undefined name errors - Fix regex injection vulnerability in bump-version.js by properly escaping all regex special characters - Add escapeRegex() function to prevent security issues when version string is used in RegExp constructor Resolves ruff linting failures and CodeQL security alerts. * Fix code formatting for ruff compliance Apply formatting fixes to meet line length requirements: - context/builder.py: Split long line with array slicing - planner_lib/context.py: Split long ternary expression - spec/requirements.py: Split long tempfile.NamedTemporaryFile call Resolves ruff format check failures. * Fix missing UTF-8 encoding in init.py gitignore operations Found by pre-commit hook testing in PR #795: - Line 96: Path.read_text() without encoding - Line 122: Path.write_text() without encoding These handle .gitignore file operations and could fail on Windows with special characters in gitignore comments or entries. Total fixes in PR #782: 253 instances (was 251, +2 from init.py) * Add pre-commit hook for UTF-8 encoding enforcement 1. Encoding Check Script (scripts/check_encoding.py): - Validates all file operations have encoding="utf-8" - Checks open(), Path.read_text(), Path.write_text() - Checks json.load/dump with open() - Allows binary mode without encoding - Windows-compatible emoji output with UTF-8 reconfiguration 2. Pre-commit Config (.pre-commit-config.yaml): - Added check-file-encoding hook for apps/backend/ - Runs automatically before commits - Scoped to backend Python files only 3. Tests (tests/test_check_encoding.py): - Comprehensive test coverage (10 tests, all passing) - Tests detection of missing encoding - Tests allowlist for binary files - Tests multiple issues in single file - Tests file type filtering Purpose: - Prevent regression of 251 UTF-8 encoding fixes from PR #782 - Catch missing encoding in new code during development - Fast feedback loop for developers Implementation Notes: - Hook scoped to apps/backend/ to avoid false positives in test code - Uses simple regex matching for speed - Compatible with existing pre-commit infrastructure - Already caught 6 real issues in apps/backend/core/progress.py Related: PR #782 - Fix Windows UTF-8 encoding errors * Address CodeRabbit and Gemini review feedback Fixes based on automated review comments: 1. Binary Mode Detection (Critical Fix): - Replaced brittle regex with robust pattern: r'["'][rwax+]*b[rwax+]*["']' - Now correctly detects all binary modes: rb, wb, ab, r+b, w+b, etc. - Prevents false positives on text mode 'w' without 'b' - Added comprehensive tests for wb, ab, and text w modes 2. Encoding Detection Robustness (Critical Fix): - Changed from 'encoding=' string match to word boundary regex: r'\bencoding\s*=' - Now handles encoding with spaces: encoding = "utf-8" - Prevents false matches of substrings containing 'encoding=' - Applied across all checks (open, read_text, write_text, json.load, json.dump) - Added test for spaces around equals sign 3. Test Coverage Improvements: - Added json.dump() with encoding test (passing case) - Added json.dump() without encoding test (failing case) - Fixed test assertions to match actual behavior (== 1 not == 2) - Added 6 new tests for improved binary/text mode coverage - Total tests increased from 10 to 16, all passing ✅ 4. Code Cleanup: - Removed unused pytest import (CodeQL warning) - Simplified check_files() to remove unused variable tracking All changes validated with comprehensive test suite (16/16 passing). Related: PR #795 review feedback from CodeRabbit and Gemini Code Assist * docs: Add UTF-8 encoding guidelines and Windows development guide 1. CONTRIBUTING.md: - Added concise file encoding section after Code Style - DO/DON'T examples for common file operations - Covers open(), Path methods, json operations - References PR #782 and windows-development.md 2. guides/windows-development.md (NEW): - Comprehensive Windows development guide - File encoding (cp1252 vs UTF-8 issue) - Line endings, path separators, shell commands - Development environment recommendations - Common pitfalls and solutions - Testing guidelines 3. .github/PULL_REQUEST_TEMPLATE.md: - Added encoding checklist item for Python PRs - Helps catch missing encoding during review 4. guides/README.md: - Added windows-development.md to guide index - Organized with CLI-USAGE and linux guides Purpose: Educate developers about UTF-8 encoding requirements to prevent regressions of the 251 encoding issues fixed in PR #782. Automated checking via pre-commit hooks (PR #795) + developer education ensures long-term Windows compatibility. Related: - PR #782: Fix Windows UTF-8 encoding errors (251 instances) - PR #795: Add pre-commit hooks for encoding enforcement * Address review comments from CodeRabbit and Gemini 1. Fix CONTRIBUTING.md markdown linting issues - Add blank lines around code blocks (MD031) - Add JSON write example with ensure_ascii=False (Gemini suggestion) 2. Fix guides/windows-development.md markdown linting (39 violations) - Rename duplicate headings: "The Problem"/"The Solution" → "Problem"/"Solution" (MD024) - Add blank lines around all code blocks (MD031) - Add language specifiers to code blocks (MD040) - Add blank lines before/after headings (MD022) - Wrap long lines to <=80 characters (MD013) - Add blank line before list (MD032) - Use Gemini's idiomatic line ending normalization pattern 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix additional UTF-8 encoding issues and improve encoding check script - Add encoding="utf-8" to 5 files that were missing it: - cli/workspace_commands.py: read_text for worktree config - context/pattern_discovery.py: read_text with errors param - context/search.py: read_text with errors param - core/sentry.py: open for package.json version detection - core/workspace/setup.py: open for security profile JSON - Improve check_encoding.py script to reduce false positives: - Use negative lookbehind to exclude os.open(), urlopen(), etc. - Handle nested parentheses correctly when checking args - Skip self.method.read_text() calls (custom methods, not Path) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix missing UTF-8 encoding in locked_write() function Add encoding parameter to locked_write() async context manager and use it in os.fdopen() call. This fixes HIGH priority issue from PR review where locked_write() was missing UTF-8 encoding support, which could cause encoding errors on Windows when writing files with non-ASCII content. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add UnicodeDecodeError handling for file loading resilience Address CodeRabbit review feedback: - runner.py: Add UnicodeDecodeError to exception handling when loading batch files - trust.py: Add exception handling in get_state() and get_all_states() to gracefully handle corrupted state files instead of failing completely Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix atomic_write to handle binary mode correctly The atomic_write function was unconditionally passing encoding to os.fdopen, which would crash with ValueError if called with binary mode (e.g., 'wb'). Apply the same fix used in locked_write: only pass encoding for text modes. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix run_git() call with invalid parameters in setup.py Remove capture_output and encoding kwargs from run_git() call - these parameters are already handled internally by run_git() and passing them causes TypeError since the function doesn't accept them. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix CodeQL warnings and potential double-newline bug - Remove unused is_path_call variables in check_encoding.py - Remove unused failed_count variable in check_encoding.py - Remove unused escapeRegex function in bump-version.js - Fix potential double-newline when adding imports in file_merger.py (strip trailing newlines from content_after before inserting) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix Ruff formatting: wrap long line in file_merger.py Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add UnicodeDecodeError handling to all JSON file loading Comprehensively add UnicodeDecodeError to exception handlers across the codebase to handle legacy-encoded or corrupted files gracefully: - 32+ locations now catch UnicodeDecodeError alongside OSError and json.JSONDecodeError - context/builder.py: Regenerate index on decode failure - planner_lib/context.py: Use empty dicts on decode failure - check_encoding.py: Handle OSError for unreadable files - cleanup.py: Handle decode errors in index pruning This ensures the codebase is robust against non-UTF-8 files that may exist from previous Windows runs with cp1252 encoding. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add explanatory comments to empty except clauses Address CodeQL notices about empty except clauses with just 'pass' by adding explanatory comments describing the intent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix review issues from Andy's Auto Claude PR Review 1. [HIGH] Fix double-close bug in trust.py:449 - Remove try/except around os.fdopen since it takes ownership of fd - The with statement handles closing, no need for explicit os.close() 2. [LOW] Fix dead code in file_merger.py:87,159 - Simplify endswith check to just '\n' since content is already normalized to LF at that point 3. [LOW] Fix escaped backslash-n in test_context_gatherer.py:150 - Change "\n" (literal backslash-n) to "\n" (actual newline) 4. [LOW] Fix coder.md examples missing encoding parameter - Add encoding="utf-8" to read_text() and open() calls in examples Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: TamerineSky <TamerineSky@users.noreply.github.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
334 lines
11 KiB
Python
334 lines
11 KiB
Python
"""
|
|
Test File Locking for Concurrent Operations
|
|
===========================================
|
|
|
|
Demonstrates file locking preventing data corruption in concurrent scenarios.
|
|
"""
|
|
|
|
import asyncio
|
|
import json
|
|
import tempfile
|
|
import time
|
|
from pathlib import Path
|
|
|
|
from file_lock import (
|
|
FileLock,
|
|
FileLockTimeout,
|
|
locked_json_read,
|
|
locked_json_update,
|
|
locked_json_write,
|
|
locked_read,
|
|
locked_write,
|
|
)
|
|
|
|
|
|
async def test_basic_file_lock():
|
|
"""Test basic file locking mechanism."""
|
|
print("\n=== Test 1: Basic File Lock ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "test.txt"
|
|
test_file.write_text("initial content", encoding="utf-8")
|
|
|
|
# Acquire lock and hold it
|
|
async with FileLock(test_file, timeout=5.0):
|
|
print("✓ Lock acquired successfully")
|
|
# Do work while holding lock
|
|
await asyncio.sleep(0.1)
|
|
print("✓ Lock held during work")
|
|
|
|
print("✓ Lock released automatically")
|
|
|
|
|
|
async def test_locked_write():
|
|
"""Test atomic locked write operations."""
|
|
print("\n=== Test 2: Locked Write ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "data.json"
|
|
|
|
# Write data with locking
|
|
data = {"count": 0, "items": ["a", "b", "c"]}
|
|
async with locked_write(test_file, timeout=5.0) as f:
|
|
json.dump(data, f, indent=2)
|
|
|
|
print(f"✓ Written to {test_file.name}")
|
|
|
|
# Verify data was written correctly
|
|
with open(test_file, encoding="utf-8") as f:
|
|
loaded = json.load(f)
|
|
assert loaded == data
|
|
print(f"✓ Data verified: {loaded}")
|
|
|
|
|
|
async def test_locked_json_helpers():
|
|
"""Test JSON helper functions."""
|
|
print("\n=== Test 3: JSON Helpers ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "data.json"
|
|
|
|
# Write JSON
|
|
data = {"users": [], "total": 0}
|
|
await locked_json_write(test_file, data, timeout=5.0)
|
|
print(f"✓ JSON written: {data}")
|
|
|
|
# Read JSON
|
|
loaded = await locked_json_read(test_file, timeout=5.0)
|
|
assert loaded == data
|
|
print(f"✓ JSON read: {loaded}")
|
|
|
|
|
|
async def test_locked_json_update():
|
|
"""Test atomic read-modify-write updates."""
|
|
print("\n=== Test 4: Atomic Updates ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "counter.json"
|
|
|
|
# Initialize counter
|
|
await locked_json_write(test_file, {"count": 0}, timeout=5.0)
|
|
print("✓ Counter initialized to 0")
|
|
|
|
# Define update function
|
|
def increment_counter(data):
|
|
data["count"] += 1
|
|
return data
|
|
|
|
# Perform 5 atomic updates
|
|
for i in range(5):
|
|
await locked_json_update(test_file, increment_counter, timeout=5.0)
|
|
|
|
# Verify final count
|
|
final = await locked_json_read(test_file, timeout=5.0)
|
|
assert final["count"] == 5
|
|
print(f"✓ Counter incremented 5 times: {final}")
|
|
|
|
|
|
async def test_concurrent_updates_without_lock():
|
|
"""Demonstrate data corruption WITHOUT file locking."""
|
|
print("\n=== Test 5: Concurrent Updates WITHOUT Locking (UNSAFE) ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "unsafe.json"
|
|
|
|
# Initialize counter
|
|
test_file.write_text(json.dumps({"count": 0}), encoding="utf-8")
|
|
|
|
async def unsafe_increment():
|
|
"""Increment without locking - RACE CONDITION!"""
|
|
# Read
|
|
with open(test_file, encoding="utf-8") as f:
|
|
data = json.load(f)
|
|
|
|
# Simulate some processing
|
|
await asyncio.sleep(0.01)
|
|
|
|
# Write
|
|
data["count"] += 1
|
|
with open(test_file, "w", encoding="utf-8") as f:
|
|
json.dump(data, f)
|
|
|
|
# Run 10 concurrent increments
|
|
await asyncio.gather(*[unsafe_increment() for _ in range(10)])
|
|
|
|
# Check final count
|
|
with open(test_file, encoding="utf-8") as f:
|
|
final = json.load(f)
|
|
|
|
print("✗ Expected count: 10")
|
|
print(f"✗ Actual count: {final['count']} (CORRUPTED due to race condition)")
|
|
print(
|
|
f"✗ Lost updates: {10 - final['count']} (multiple processes overwrote each other)"
|
|
)
|
|
|
|
|
|
async def test_concurrent_updates_with_lock():
|
|
"""Demonstrate data integrity WITH file locking."""
|
|
print("\n=== Test 6: Concurrent Updates WITH Locking (SAFE) ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "safe.json"
|
|
|
|
# Initialize counter
|
|
await locked_json_write(test_file, {"count": 0}, timeout=5.0)
|
|
|
|
async def safe_increment():
|
|
"""Increment with locking - NO RACE CONDITION!"""
|
|
|
|
def increment(data):
|
|
# Simulate some processing
|
|
time.sleep(0.01)
|
|
data["count"] += 1
|
|
return data
|
|
|
|
await locked_json_update(test_file, increment, timeout=5.0)
|
|
|
|
# Run 10 concurrent increments
|
|
await asyncio.gather(*[safe_increment() for _ in range(10)])
|
|
|
|
# Check final count
|
|
final = await locked_json_read(test_file, timeout=5.0)
|
|
|
|
assert final["count"] == 10
|
|
print("✓ Expected count: 10")
|
|
print(f"✓ Actual count: {final['count']} (CORRECT with file locking)")
|
|
print("✓ No data corruption - all updates applied successfully")
|
|
|
|
|
|
async def test_lock_timeout():
|
|
"""Test lock timeout behavior."""
|
|
print("\n=== Test 7: Lock Timeout ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "timeout.json"
|
|
test_file.write_text(json.dumps({"data": "test"}), encoding="utf-8")
|
|
|
|
# Acquire lock and hold it
|
|
lock1 = FileLock(test_file, timeout=1.0)
|
|
await lock1.__aenter__()
|
|
print("✓ First lock acquired")
|
|
|
|
try:
|
|
# Try to acquire second lock with short timeout
|
|
lock2 = FileLock(test_file, timeout=0.5)
|
|
await lock2.__aenter__()
|
|
print("✗ Second lock acquired (should have timed out!)")
|
|
except FileLockTimeout as e:
|
|
print(f"✓ Second lock timed out as expected: {e}")
|
|
finally:
|
|
await lock1.__aexit__(None, None, None)
|
|
print("✓ First lock released")
|
|
|
|
|
|
async def test_index_update_pattern():
|
|
"""Test the index update pattern used in models.py."""
|
|
print("\n=== Test 8: Index Update Pattern (Production Pattern) ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
index_file = Path(tmpdir) / "index.json"
|
|
|
|
# Simulate multiple PR reviews updating the index concurrently
|
|
async def add_review(pr_number: int, status: str):
|
|
"""Add or update a PR review in the index."""
|
|
|
|
def update_index(current_data):
|
|
if current_data is None:
|
|
current_data = {"reviews": [], "last_updated": None}
|
|
|
|
reviews = current_data.get("reviews", [])
|
|
existing = next(
|
|
(r for r in reviews if r["pr_number"] == pr_number), None
|
|
)
|
|
|
|
entry = {
|
|
"pr_number": pr_number,
|
|
"status": status,
|
|
"timestamp": time.time(),
|
|
}
|
|
|
|
if existing:
|
|
reviews = [
|
|
entry if r["pr_number"] == pr_number else r for r in reviews
|
|
]
|
|
else:
|
|
reviews.append(entry)
|
|
|
|
current_data["reviews"] = reviews
|
|
current_data["last_updated"] = time.time()
|
|
|
|
return current_data
|
|
|
|
await locked_json_update(index_file, update_index, timeout=5.0)
|
|
|
|
# Simulate 5 concurrent review updates
|
|
print("Simulating 5 concurrent PR review updates...")
|
|
await asyncio.gather(
|
|
add_review(101, "approved"),
|
|
add_review(102, "changes_requested"),
|
|
add_review(103, "commented"),
|
|
add_review(104, "approved"),
|
|
add_review(105, "approved"),
|
|
)
|
|
|
|
# Verify all reviews were recorded
|
|
final_index = await locked_json_read(index_file, timeout=5.0)
|
|
assert len(final_index["reviews"]) == 5
|
|
print("✓ All 5 reviews recorded correctly")
|
|
print(f"✓ Index state: {len(final_index['reviews'])} reviews")
|
|
|
|
# Update an existing review
|
|
await add_review(102, "approved") # Change status
|
|
updated_index = await locked_json_read(index_file, timeout=5.0)
|
|
assert len(updated_index["reviews"]) == 5 # Still 5, not 6
|
|
review_102 = next(r for r in updated_index["reviews"] if r["pr_number"] == 102)
|
|
assert review_102["status"] == "approved"
|
|
print("✓ Review #102 updated from 'changes_requested' to 'approved'")
|
|
print("✓ No duplicate entries created")
|
|
|
|
|
|
async def test_atomic_write_failure():
|
|
"""Test that failed writes don't corrupt existing files."""
|
|
print("\n=== Test 9: Atomic Write Failure Handling ===")
|
|
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
test_file = Path(tmpdir) / "important.json"
|
|
|
|
# Write initial data
|
|
initial_data = {"important": "data", "version": 1}
|
|
await locked_json_write(test_file, initial_data, timeout=5.0)
|
|
print(f"✓ Initial data written: {initial_data}")
|
|
|
|
# Try to write invalid data that will fail
|
|
try:
|
|
async with locked_write(test_file, timeout=5.0) as f:
|
|
f.write("{invalid json")
|
|
# Simulate an error during write
|
|
raise Exception("Simulated write failure")
|
|
except Exception as e:
|
|
print(f"✓ Write failed as expected: {e}")
|
|
|
|
# Verify original data is intact (atomic write rolled back)
|
|
current_data = await locked_json_read(test_file, timeout=5.0)
|
|
assert current_data == initial_data
|
|
print(f"✓ Original data intact after failed write: {current_data}")
|
|
print(
|
|
"✓ Atomic write prevented corruption (temp file discarded, original preserved)"
|
|
)
|
|
|
|
|
|
async def main():
|
|
"""Run all tests."""
|
|
print("=" * 70)
|
|
print("File Locking Tests - Preventing Concurrent Operation Corruption")
|
|
print("=" * 70)
|
|
|
|
tests = [
|
|
test_basic_file_lock,
|
|
test_locked_write,
|
|
test_locked_json_helpers,
|
|
test_locked_json_update,
|
|
test_concurrent_updates_without_lock,
|
|
test_concurrent_updates_with_lock,
|
|
test_lock_timeout,
|
|
test_index_update_pattern,
|
|
test_atomic_write_failure,
|
|
]
|
|
|
|
for test in tests:
|
|
try:
|
|
await test()
|
|
except Exception as e:
|
|
print(f"✗ Test failed: {e}")
|
|
import traceback
|
|
|
|
traceback.print_exc()
|
|
|
|
print("\n" + "=" * 70)
|
|
print("All Tests Completed!")
|
|
print("=" * 70)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
asyncio.run(main())
|