Files
Aperant/apps/frontend/scripts/verify-linux-packages.cjs
T
StillKnotKnown bad1a9b2c4 fix: align Linux package builds (AppImage/deb/Flatpak) with target-specific extraResources (#1623)
* fix: align Linux package builds (AppImage/deb/Flatpak) with target-specific extraResources

The .deb, AppImage, and Flatpak builds had inconsistent package inclusion
due to ${os}-${arch} template variables in global extraResources not
expanding consistently for all Linux targets.

Changes:
- Move Python runtime from global extraResources to target-specific config
- Add platform-specific extraResources for mac, win, appImage, deb, flatpak
- Create verify-linux-packages.cjs script to inspect package contents
- Add verify:linux and test:verify-linux npm scripts
- Update release.yml and beta-release.yml CI/CD workflows with verification

This ensures electron-builder processes resources correctly for each package
type, avoiding template variable resolution issues and enabling early detection
of missing critical files (secretstorage, pydantic_core, claude_agent_sdk).

* style: apply Biome formatting to verify-linux-packages.cjs

* fix: address PR review feedback

- Fix Windows Python path: use python directory, not python.exe directly
- Move Linux extraResources to linux block (not appImage/deb targets)
- Remove redundant __dirname shadowing in verification script
- Tighten file pattern matching to reduce false positives
- Export CRITICAL_PACKAGES for use in tests
- Only run main() when script is executed directly

The appImage and deb blocks are target options in electron-builder,
not platform options. They don't support extraResources. Use
build.linux.extraResources instead for all Linux targets.

The Windows tarball extracts with a 'python' directory containing
python.exe, so copy the entire directory, not the exe file directly.

* fix: use ${os} template variable for platform paths

Use ${os} instead of hardcoded platform names (darwin, win32) to match
the directory structure created by download-python.cjs.

The download script uses electron-builder platform names (mac, win, linux)
via toElectronBuilderPlatform(), so the extraResources paths must use
${os} which resolves to these same names:
- macOS: ${os} -> mac (not darwin)
- Windows: ${os} -> win (not win32)
- Linux: ${os} -> linux

* fix: address PR review feedback for verification script

- Export verification functions (findPackages, verifyFileList, verifyAppImage, verifyDeb, verifyFlatpak)
- Extract common verification logic to reduce duplication (DRY)
- Fix pattern matching to avoid false positives:
  - Python binary check explicitly excludes python-site-packages
  - Backend check requires resources/ prefix
  - Package check requires python-site-packages/ prefix
- Update tests to use actual exported functions instead of re-implementing logic
- Fix test data to match real package file formats (AppImage uses './' prefix)
- All 12 unit tests now pass

This addresses all issues raised by CodeRabbit AI and Auto Claude PR reviews.

* refactor: extract Flatpak size threshold to named constant

Add FLATPAK_MIN_SIZE_MB constant (50 MB) alongside CRITICAL_PACKAGES.
This makes the threshold self-documenting and centralized for easier maintenance.

Also improves error message to include expected size for clarity.

* fix: add maxBuffer to spawnSync and remove Flatpak early return

- Add 50MB maxBuffer to bsdtar spawnSync call for large AppImages
- Add 50MB maxBuffer to dpkg-deb spawnSync call for large deb packages
- Remove early return in verifyFlatpak when flatpak CLI is missing
- File existence/size checks now run even without flatpak CLI

Fixes CodeRabbit review feedback.

* test: fix test to actually call findPackages and address low severity issues

- Fix test to properly call findPackages() with mocked fs.existsSync/readdirSync
- Add test for missing dist directory handling
- Add test for duplicate package warnings
- Change commandExists to use POSIX-compliant 'command -v' instead of 'which'
- Add warnings for duplicate packages in findPackages function
- Remove unused imports and variables from tests

Fixes AI review findings:
- NEW-001 [MEDIUM]: Test now calls findPackages function
- NEW-003 [LOW]: Duplicate packages now trigger warnings
- NEW-005 [LOW]: commandExists now uses POSIX-compliant 'command -v'

* security: fix shell injection vulnerability in commandExists

- Change from shell interpolation (sh -c 'command -v \$cmd') to direct which call
- Use spawnSync with argument array to avoid shell interpretation
- This prevents potential command injection if function is called with untrusted input

Fixes CodeRabbit security review finding.

* fix: normalize paths to handle trailing slashes from archive tools

- Add normalizePath helper to remove trailing slashes
- Update Python binary and backend directory checks to use normalized paths
- Add test case for paths with trailing slashes (bsdtar/dpkg-deb output)

Fixes CodeRabbit review finding about archive tools emitting directories
with trailing slashes like './resources/python/' or 'resources/python/'.

Now handles:
- './resources/python'
- './resources/python/'
- 'resources/python'
- 'resources/python/'

* fix: fail CI when critical verification tools are missing

- Add critical flag when bsdtar or dpkg-deb is missing
- Update main function to fail when critical verifications are skipped
- Provide helpful installation instructions when tools are missing

Fixes Sentry review finding about CI false positives. The script now
exits with error code 1 when AppImage or deb verification is skipped
due to missing required tools (bsdtar, dpkg-deb).

Note: flatpak CLI remains non-critical since basic file/size checks
still work without it.

Regarding CodeRabbit's suggestion to use platform helpers:
- This is a standalone .cjs script that runs in Node.js (not Electron)
- Platform helpers (findExecutable) are TypeScript modules for the app
- The 'which' command is universally available on Linux CI systems

* fix: add spawnSync error checks and improve Flatpak tests

MEDIUM severity fixes:
- Add result.error checks in verifyAppImage for spawnSync failures
- Add result.error checks in verifyDeb for spawnSync failures
- These catch OS-level spawn failures (permission denied, memory issues)

LOW severity fix:
- Refactor Flatpak tests to call actual verifyFlatpak function
- Mock fs.existsSync and fs.statSync similar to findPackages tests
- Add test case for non-existent Flatpak files

Increases test coverage confidence by testing actual function implementation
instead of manually reimplementing logic in tests.

Fixes AI review findings NEW-CODE-001, NEW-CODE-002, NEW-CODE-003.

* test: add spawnSync mock coverage for verifyAppImage and verifyDeb

- Add test coverage for verifyAppImage and verifyDeb functions
- Mock spawnSync at module level by clearing require cache
- Test cases cover:
  - Successful extraction
  - result.error set (OS-level spawn failures)
  - result.status !== 0 (tool returns error)
  - Missing required tools (bsdtar, dpkg-deb)

Fixes AI review finding NEW-001 [MEDIUM] about missing test coverage
for the newly-added error handling code.

Increases test count from 16 to 24 tests.

---------

Co-authored-by: StillKnotKnown <stillknotknown@users.noreply.github.com>
Co-authored-by: Andy <119136210+AndyMik90@users.noreply.github.com>
2026-01-30 13:09:23 +01:00

407 lines
12 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env node
/**
* Verify Linux package contents to ensure alignment between AppImage, deb, and Flatpak.
*
* This script extracts and inspects each Linux package format to verify that critical
* files (Python binary, backend code, Python packages) are present and correctly bundled.
*
* Usage: node scripts/verify-linux-packages.cjs [dist-dir]
*/
const fs = require('fs');
const path = require('path');
const { spawnSync } = require('child_process');
// Critical Python packages that must be present
const CRITICAL_PACKAGES = [
'secretstorage', // Linux OAuth token storage
'pydantic_core',
'claude_agent_sdk',
'dotenv',
];
// Minimum expected Flatpak file size (50 MB)
// Flatpak files are large OCI archives; anything smaller is suspicious
// Based on observed minimum sizes of valid builds
const FLATPAK_MIN_SIZE_MB = 50;
// Colors for terminal output
const colors = {
reset: '\x1b[0m',
red: '\x1b[31m',
green: '\x1b[32m',
yellow: '\x1b[33m',
blue: '\x1b[34m',
cyan: '\x1b[36m',
};
function log(message, color = colors.reset) {
console.log(`${color}${message}${colors.reset}`);
}
function logSuccess(message) {
log(`${message}`, colors.green);
}
function logError(message) {
log(`${message}`, colors.red);
}
function logWarning(message) {
log(`${message}`, colors.yellow);
}
function logInfo(message) {
log(` ${message}`, colors.cyan);
}
/**
* Check if a command exists
* Uses 'which' directly without shell interpolation to prevent command injection
*/
function commandExists(cmd) {
const result = spawnSync('which', [cmd], { stdio: 'ignore' });
return result.status === 0;
}
/**
* Find all Linux packages in the dist directory
*/
function findPackages(distDir) {
const packages = {
appImage: null,
deb: null,
flatpak: null,
};
if (!fs.existsSync(distDir)) {
logError(`Distribution directory not found: ${distDir}`);
return packages;
}
const files = fs.readdirSync(distDir);
for (const file of files) {
const fullPath = path.join(distDir, file);
if (file.endsWith('.AppImage')) {
if (!packages.appImage) {
packages.appImage = fullPath;
} else {
logWarning(`Multiple AppImage files found, using first: ${path.basename(packages.appImage)}`);
}
} else if (file.endsWith('.deb')) {
if (!packages.deb) {
packages.deb = fullPath;
} else {
logWarning(`Multiple deb files found, using first: ${path.basename(packages.deb)}`);
}
} else if (file.endsWith('.flatpak')) {
if (!packages.flatpak) {
packages.flatpak = fullPath;
} else {
logWarning(`Multiple Flatpak files found, using first: ${path.basename(packages.flatpak)}`);
}
}
}
return packages;
}
/**
* Common file list verification logic
* @param {string[]} files - List of files from package
* @param {string} packageType - Type of package (for error messages)
* @returns {Object} Verification result with verified flag and issues array
*
* File formats:
* - AppImage (bsdtar): './resources/python', './resources/backend/file.py'
* - deb (dpkg-deb -c): 'resources/python', 'resources/backend/file.py' (in last column)
*/
function verifyFileList(files, packageType) {
const issues = [];
// Normalize paths by removing trailing slashes (archive tools commonly add these)
const normalizePath = (p) => p.replace(/\/+$/, '');
// Check for Python binary directory
// AppImage: './resources/python' or './resources/python/' (with trailing slash)
// deb: 'resources/python' or 'resources/python/' (with trailing slash)
// Must NOT match 'resources/python-site-packages'
const pythonBinFound = files.some((f) => {
const normalized = normalizePath(f);
return (
(normalized === './resources/python' ||
normalized === 'resources/python' ||
normalized.endsWith('/resources/python')) &&
!f.includes('python-site-packages')
);
});
if (!pythonBinFound) {
issues.push(`Python binary directory not found in ${packageType}`);
}
// Check for backend directory (must be under resources/)
const backendFound = files.some((f) => {
const normalized = normalizePath(f);
return (
f.includes('./resources/backend/') ||
f.includes('resources/backend/') ||
normalized === './resources/backend' ||
normalized === 'resources/backend'
);
});
if (!backendFound) {
issues.push(`Backend directory not found in ${packageType}`);
}
// Check for critical Python packages (must be under python-site-packages/)
for (const pkg of CRITICAL_PACKAGES) {
// Match: './resources/python-site-packages/secretstorage/__init__.py'
// Match: 'resources/python-site-packages/secretstorage/__init__.py'
// Don't match: '/some/other/path/secretstorage/'
const found = files.some(
(f) => f.includes(`python-site-packages/${pkg}/`) || f.includes(`python-site-packages/${pkg}.`),
);
if (!found) {
issues.push(`Python package not found: ${pkg}`);
}
}
return {
verified: issues.length === 0,
issues,
fileCount: files.filter((f) => f.trim()).length,
};
}
/**
* Verify AppImage contents using bsdtar (libarchive)
*/
function verifyAppImage(appImagePath) {
logInfo(`Verifying AppImage: ${path.basename(appImagePath)}`);
// Check if bsdtar is available
if (!commandExists('bsdtar')) {
logWarning('bsdtar not found. Install with: sudo apt-get install libarchive-tools');
logWarning('Skipping AppImage verification');
return { verified: false, reason: 'bsdtar not available', critical: true };
}
// Extract file list from AppImage using bsdtar
const result = spawnSync('bsdtar', ['-t', '-f', appImagePath], {
stdio: 'pipe',
encoding: 'utf-8',
maxBuffer: 50 * 1024 * 1024, // 50MB buffer for large file listings
});
// Check for spawn errors (e.g., permission denied, memory issues)
if (result.error) {
logError(`Failed to execute bsdtar: ${result.error.message}`);
return { verified: false, reason: `Command execution failed: ${result.error.message}` };
}
if (result.status !== 0) {
logError(`Failed to read AppImage: ${result.stderr}`);
return { verified: false, reason: 'Failed to extract file list' };
}
const files = result.stdout.split('\n');
return verifyFileList(files, 'AppImage');
}
/**
* Verify deb package contents
*/
function verifyDeb(debPath) {
logInfo(`Verifying deb package: ${path.basename(debPath)}`);
// Check if dpkg is available
if (!commandExists('dpkg-deb')) {
logWarning('dpkg-deb not found. Skipping deb verification');
return { verified: false, reason: 'dpkg-deb not available', critical: true };
}
// List contents of deb package
const result = spawnSync('dpkg-deb', ['-c', debPath], {
stdio: 'pipe',
encoding: 'utf-8',
maxBuffer: 50 * 1024 * 1024, // 50MB buffer for large file listings
});
// Check for spawn errors (e.g., permission denied, memory issues)
if (result.error) {
logError(`Failed to execute dpkg-deb: ${result.error.message}`);
return { verified: false, reason: `Command execution failed: ${result.error.message}` };
}
if (result.status !== 0) {
logError(`Failed to read deb package: ${result.stderr}`);
return { verified: false, reason: 'Failed to extract file list' };
}
const files = result.stdout.split('\n');
return verifyFileList(files, 'deb package');
}
/**
* Verify Flatpak package contents
* Note: Flatpak is more complex to inspect, so we do basic validation
*/
function verifyFlatpak(flatpakPath) {
logInfo(`Verifying Flatpak package: ${path.basename(flatpakPath)}`);
const issues = [];
// Check if flatpak command is available for detailed validation
const hasFlatpakCli = commandExists('flatpak');
if (!hasFlatpakCli) {
logWarning('flatpak command not found. Skipping detailed Flatpak verification');
// Continue with basic file existence/size checks
}
// Check if file exists and is not empty
if (!fs.existsSync(flatpakPath)) {
return { verified: false, issues: ['Flatpak file does not exist'] };
}
const stats = fs.statSync(flatpakPath);
if (stats.size === 0) {
return { verified: false, issues: ['Flatpak file is empty'] };
}
// Flatpak files are large OCI archives, so we just verify file size and basic structure
// Detailed content inspection would require mounting or extracting the flatpak
if (stats.size < FLATPAK_MIN_SIZE_MB * 1024 * 1024) {
// Less than minimum size is suspicious
issues.push(
`Flatpak file seems too small (${(stats.size / 1024 / 1024).toFixed(2)} MB, expected at least ${FLATPAK_MIN_SIZE_MB} MB)`,
);
}
return {
verified: issues.length === 0,
issues,
size: stats.size,
};
}
/**
* Main verification function
*/
function main() {
const distDir = process.argv[2] || path.join(__dirname, '..', 'dist');
log('\n=== Linux Package Verification ===\n', colors.blue);
logInfo(`Distribution directory: ${distDir}\n`);
const packages = findPackages(distDir);
// Report found packages
if (packages.appImage) {
logSuccess(`Found AppImage: ${path.basename(packages.appImage)}`);
} else {
logWarning('No AppImage found');
}
if (packages.deb) {
logSuccess(`Found deb: ${path.basename(packages.deb)}`);
} else {
logWarning('No deb package found');
}
if (packages.flatpak) {
logSuccess(`Found Flatpak: ${path.basename(packages.flatpak)}`);
} else {
logWarning('No Flatpak package found');
}
if (!packages.appImage && !packages.deb && !packages.flatpak) {
logError('\nNo Linux packages found to verify!');
process.exit(1);
}
log('');
// Verify each package
const results = {};
if (packages.appImage) {
results.appImage = verifyAppImage(packages.appImage);
}
if (packages.deb) {
results.deb = verifyDeb(packages.deb);
}
if (packages.flatpak) {
results.flatpak = verifyFlatpak(packages.flatpak);
}
// Print results
log('\n=== Verification Results ===\n', colors.blue);
let hasFailures = false;
let hasCriticalSkips = false;
for (const [type, result] of Object.entries(results)) {
if (result.reason) {
if (result.critical) {
logError(`${type}: CRITICAL - SKIPPED (${result.reason})`);
hasCriticalSkips = true;
} else {
logWarning(`${type}: SKIPPED (${result.reason})`);
}
} else if (result.verified) {
logSuccess(`${type}: VERIFIED`);
if (result.fileCount) {
logInfo(` Files: ${result.fileCount}`);
}
if (result.size) {
logInfo(` Size: ${(result.size / 1024 / 1024).toFixed(2)} MB`);
}
} else {
logError(`${type}: FAILED`);
hasFailures = true;
for (const issue of result.issues || []) {
logError(` - ${issue}`);
}
}
}
log('');
if (hasFailures || hasCriticalSkips) {
logError('\n=== VERIFICATION FAILED ===\n');
if (hasFailures) {
log('Some packages are missing critical files. This will cause runtime errors.\n', colors.red);
}
if (hasCriticalSkips) {
log('Some packages could not be verified due to missing required tools.\n', colors.red);
log('Install required tools:\n', colors.red);
log(' - bsdtar: sudo apt-get install libarchive-tools\n', colors.red);
log(' - dpkg-deb: sudo apt-get install dpkg\n', colors.red);
}
process.exit(1);
} else {
logSuccess('\n=== ALL PACKAGES VERIFIED ===\n');
log('All Linux packages contain the required files.\n', colors.green);
process.exit(0);
}
}
// Only run main if this file is executed directly (not imported)
if (require.main === module) {
main();
}
// Export for testing
module.exports = {
CRITICAL_PACKAGES,
findPackages,
verifyFileList,
verifyAppImage,
verifyDeb,
verifyFlatpak,
};