diff --git a/.clauderules b/.clauderules new file mode 100644 index 00000000..70101d61 --- /dev/null +++ b/.clauderules @@ -0,0 +1,63 @@ +# Claude Code Rules - Follow Every Rule Exactly + +You must prioritize straightforward code semantics, well-named types, clear function signatures, and robust, carefully-chosen abstractions. Think about how your decisions might impact these aspects of code quality before proposing any changes. + +You have access to all modern Python features from Python 3.13, 3.12, 3.11... + +**When you're done making changes, remove any redundant comments; remaining comments should only apply to complex code segments, adding relevant context.** + +## 1. Code Discipline + +* Eliminate superfluous `try`/`catch` and `if` branches through strict typing and static analysis. +* Use pure functions unless you must mutate fixed state—then wrap that state in a class. +* Every function is **referentially transparent**: same inputs ⇒ same outputs, no hidden state, no unintended I/O. +* Put side-effects in injectable "effect handlers"; keep core logic pure. + +## 2. Naming + +* Choose descriptive, non-abbreviated names—no 3-letter acronyms or non-standard contractions. +* Anyone reading a function's type signature alone should grasp its purpose without extra context. + +## 3. Typing + +* Maintain **strict, exhaustive** typing; never bypass the type-checker. +* Default to `Literal[...]` when an enum-like set is needed. +* Prefer built-in types; when two values share structure but differ in meaning, enforce separation: + * Use `typing.NewType` for primitives (zero runtime cost). + * For serializable objects, add a `type: str` field that states the object's identity. + +## 4. Pydantic + +* Read, respect, and rely on Pydantic documentation. +* Centralize a common `ConfigDict` with `frozen=True` and `strict=True` (or stricter) and reuse it everywhere. +* For hierarchies of `BaseModel` variants, declare a discriminated union with `typing.Annotated[Base, Field(discriminator='variant')]`; publish a single `TypeAdapter[Base]` so all variants share one strict validator. + +## 5. IDs & UUIDs + +* Subclass Pydantic's `UUID4` for custom ID types. +* Generate fresh IDs with `uuid.uuid4()`. +* Create idempotency keys by hashing *persisted* state plus a **function-specific salt** to avoid collisions after crashes. + +## 6. Error Handling + +* Catch an exception **only** where you can handle or transform it meaningfully. +* State in the docstring **where** each exception is expected to be handled and **why**. + +## 7. Dependencies + +* Introduce new external dependencies only after approval. +* Request only libraries common in production environments. + +## 8. Use of `@final` & Freezing + +* Mark classes, methods, and variables as `@final` or otherwise immutable wherever applicable. + +## 9. Repository Workflow + +If you spot a rule violation within code that you've not been asked to work on directly, inform the user rather than patching it ad-hoc. + +--- + +### One-Sentence Summary + +Write strictly-typed, pure, self-describing Python that uses Pydantic, well-scoped side-effects, immutable state, approved dependencies, and explicit error handling. \ No newline at end of file diff --git a/.cursorrules b/.cursorrules new file mode 100644 index 00000000..2f64c4b9 --- /dev/null +++ b/.cursorrules @@ -0,0 +1,64 @@ +# follow **every** rule exactly; report any violation instead of silently fixing it. + +You must prioritize straightforward code semantics, well-named types, clear function signatures, and robust, carefully-chosen abstractions. Think about how your decisions might impact these aspects of code quality before proposing any changes. + +You can use the advanced features of `typing`. You have access to all of the new features from Python 3.13, 3.12, 3.11... + +**When you're done making your changes, remove any redundant comments that you may have left; the comments that remain should only apply to complex segments of code, adding relevant context.** + +## 1. Code Discipline + +* Eliminate superfluous `try` / `catch` and `if` branches through strict typing and static analysis. +* Use pure functions unless you must mutate fixed state—then wrap that state in a class. +* Every function is **referentially transparent**: same inputs ⇒ same outputs, no hidden state, no unintended I/O. +* Put side-effects in injectable “effect handlers”; keep core logic pure. + +## 2. Naming + +* Choose descriptive, non-abbreviated names—no 3-letter acronyms or non-standard contractions. +* Anyone reading a function’s type signature alone should grasp its purpose without extra context. + +## 3. Typing + +* Maintain **strict, exhaustive** typing; never bypass the type-checker. +* Default to `Literal[...]` when an enum-like set is needed. +* Prefer built-in types; when two values share structure but differ in meaning, enforce separation: + * Use `typing.NewType` for primitives (zero runtime cost). + * For serialisable objects, add a `type: str` field that states the object’s identity. + +## 4. Pydantic + +* Read, respect, and rely on Pydantic docs. +* Centralise a common `ConfigDict` with `frozen=True` and `strict=True` (or stricter) and reuse it everywhere. +* For hierarchies of `BaseModel` variants, declare a discriminated union with `typing.Annotated[Base, Field(discriminator='variant')]`; publish a single `TypeAdapter[Base]` so all variants share one strict validator. + +## 5. IDs & UUIDs + +* Subclass Pydantic’s `UUID4` for custom ID types. +* Generate fresh IDs with `uuid.uuid4()`. +* Create idempotency keys by hashing *persisted* state plus a **function-specific salt** to avoid collisions after crashes. + +## 6. Error Handling + +* Catch an exception **only** where you can handle or transform it meaningfully. +* State in the docstring **where** each exception is expected to be handled and **why**. + +## 7. Dependencies + +* Introduce new external dependencies only after approval. +* Request only libraries common in production environments. + +## 8. Use of `@final` & Freezing + +* Mark classes, methods, and variables as `@final` or otherwise immutable wherever applicable. + +## 9. Repository Workflow + +If you spot a rule violation within code that you've not been asked to work on directly, inform the user rather than patching it ad-hoc. + + +--- + +### One-Sentence Summary + +Write strictly-typed, pure, self-describing Python that uses Pydantic, well-scoped side-effects, immutable state, approved dependencies, and explicit error handling diff --git a/.envrc b/.envrc new file mode 100644 index 00000000..3550a30f --- /dev/null +++ b/.envrc @@ -0,0 +1 @@ +use flake diff --git a/.githooks/post-checkout b/.githooks/post-checkout new file mode 100755 index 00000000..5abf8ed9 --- /dev/null +++ b/.githooks/post-checkout @@ -0,0 +1,3 @@ +#!/bin/sh +command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-checkout' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; } +git lfs post-checkout "$@" diff --git a/.githooks/post-commit b/.githooks/post-commit new file mode 100755 index 00000000..b8b76c2c --- /dev/null +++ b/.githooks/post-commit @@ -0,0 +1,3 @@ +#!/bin/sh +command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-commit' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; } +git lfs post-commit "$@" diff --git a/.githooks/post-merge b/.githooks/post-merge new file mode 100755 index 00000000..726f9098 --- /dev/null +++ b/.githooks/post-merge @@ -0,0 +1,3 @@ +#!/bin/sh +command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-merge' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; } +git lfs post-merge "$@" diff --git a/.githooks/pre-push b/.githooks/pre-push new file mode 100755 index 00000000..5f26dc45 --- /dev/null +++ b/.githooks/pre-push @@ -0,0 +1,3 @@ +#!/bin/sh +command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'pre-push' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; } +git lfs pre-push "$@" diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 00000000..16b1988c --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,3 @@ +* @ToxicPine +* @AlexCheema +* @GeluVrabie diff --git a/.github/actions/conditional-commit/action.yml b/.github/actions/conditional-commit/action.yml new file mode 100644 index 00000000..5d18fbf6 --- /dev/null +++ b/.github/actions/conditional-commit/action.yml @@ -0,0 +1,16 @@ +name: Commit if changed +description: "Create a commit when the working tree is dirty" + +inputs: + message: + description: "Commit message" + required: true + +runs: + using: composite + steps: + - name: Commit changed files + shell: bash + run: | + git diff --quiet && exit 0 + git commit -am "${{ inputs.message }}" diff --git a/.github/actions/format/action.yml b/.github/actions/format/action.yml new file mode 100644 index 00000000..5df1b5f4 --- /dev/null +++ b/.github/actions/format/action.yml @@ -0,0 +1,10 @@ +name: Format Code + +description: "Run code formatter" + +runs: + using: "composite" + steps: + - name: Format code + run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just fmt + shell: bash diff --git a/.github/actions/lint-check/action.yml b/.github/actions/lint-check/action.yml new file mode 100644 index 00000000..7d69c90d --- /dev/null +++ b/.github/actions/lint-check/action.yml @@ -0,0 +1,10 @@ +name: Lint Check + +description: "Check for lint errors" + +runs: + using: "composite" + steps: + - name: Lint check + run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just lint-check + shell: bash diff --git a/.github/actions/lint/action.yml b/.github/actions/lint/action.yml new file mode 100644 index 00000000..05f7939c --- /dev/null +++ b/.github/actions/lint/action.yml @@ -0,0 +1,10 @@ +name: Lint Code + +description: "Run code linter" + +runs: + using: "composite" + steps: + - name: Lint code + run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just lint + shell: bash diff --git a/.github/actions/regenerate-protobufs/action.yml b/.github/actions/regenerate-protobufs/action.yml new file mode 100644 index 00000000..6da2a7a4 --- /dev/null +++ b/.github/actions/regenerate-protobufs/action.yml @@ -0,0 +1,10 @@ +name: Regenerate Protobufs + +description: "Regenerate protobuf files" + +runs: + using: "composite" + steps: + - name: Regenerate protobufs + run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just regenerate-protobufs + shell: bash diff --git a/.github/actions/setup-python-uv/action.yml b/.github/actions/setup-python-uv/action.yml new file mode 100644 index 00000000..b3eb2c03 --- /dev/null +++ b/.github/actions/setup-python-uv/action.yml @@ -0,0 +1,20 @@ +name: Setup Python & uv + +description: "Regenerate Python environment from uv.lock" + +runs: + using: "composite" + steps: + - name: Install uv + uses: astral-sh/setup-uv@v6 + with: + enable-cache: true + cache-dependency-glob: uv.lock + + - name: Install Python + run: uv python install + shell: bash + + - name: Sync + run: uv sync --locked --all-extras --dev + shell: bash diff --git a/.github/actions/typecheck/action.yml b/.github/actions/typecheck/action.yml new file mode 100644 index 00000000..cd52d6e3 --- /dev/null +++ b/.github/actions/typecheck/action.yml @@ -0,0 +1,12 @@ +name: Type Check + +description: "Run type checker" + +runs: + using: "composite" + steps: + - name: Run type checker + run: | + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just sync + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just check + shell: bash diff --git a/.github/actions/unit-test/action.yml b/.github/actions/unit-test/action.yml new file mode 100644 index 00000000..65f5e07b --- /dev/null +++ b/.github/actions/unit-test/action.yml @@ -0,0 +1,12 @@ +name: Unit Test + +description: "Run unit tests" + +runs: + using: "composite" + steps: + - name: Run unit tests + run: | + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just sync-clean + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just test-fast + shell: bash diff --git a/.github/actions/verify-clean/action.yml b/.github/actions/verify-clean/action.yml new file mode 100644 index 00000000..976e6a7d --- /dev/null +++ b/.github/actions/verify-clean/action.yml @@ -0,0 +1,20 @@ +name: Verify Clean Working Tree + +description: "Fail the job if the previous step left the working tree dirty" + +inputs: + step: + description: "The name of the step that just executed" + required: true + +runs: + using: composite + steps: + - name: Check git diff + shell: bash + run: | + if ! git diff --quiet; then + echo "Error: ${{ inputs.step }} left working tree dirty." >&2 + git --no-pager diff >&2 + exit 1 + fi \ No newline at end of file diff --git a/.github/benchmark-dashboard/README.md b/.github/benchmark-dashboard/README.md new file mode 100644 index 00000000..1db78344 --- /dev/null +++ b/.github/benchmark-dashboard/README.md @@ -0,0 +1,159 @@ +# EXO Benchmark Dashboard + +A fully self-contained, browser-based dashboard for tracking EXO benchmark performance over time. + +## Features + +- 📊 **Success Rate Tracking**: Monitor cluster reliability across commits +- ⚡ **Response Time Analysis**: Track average request completion times +- 🎯 **Throughput Metrics**: Tokens per second visualization +- 📈 **Request Distribution**: Success/failure breakdown over time +- 🔄 **Auto-Refresh**: Updates every 60 seconds +- 📺 **TV-Ready**: Large, clear visualizations perfect for display +- 🔐 **Secure**: Credentials stored in browser localStorage only +- 🌐 **No Backend**: Directly accesses S3 from the browser + +## Quick Start + +### Option 1: Direct File Access (Simplest) + +Just open the HTML file directly in your browser: + +```bash +open .github/benchmark-dashboard/index.html +``` + +Then click "Configure AWS Credentials" and enter your keys. + +### Option 2: URL Parameters (For Quick Setup) + +```bash +# Serve with credentials in URL (they'll be moved to localStorage) +open ".github/benchmark-dashboard/index.html?accessKey=YOUR_KEY&secretKey=YOUR_SECRET®ion=us-east-1" +``` + +The credentials will be saved to localStorage and removed from the URL immediately. + +### Option 3: Simple HTTP Server + +```bash +# From repo root +python3 -m http.server 8080 + +# Then open: http://localhost:8080/.github/benchmark-dashboard/ +``` + +## AWS Credentials + +The dashboard needs read-only access to the `exo-benchmark-results` S3 bucket. + +### Required IAM Permissions + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:GetObject", + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::exo-benchmark-results", + "arn:aws:s3:::exo-benchmark-results/*" + ] + } + ] +} +``` + +### Security Notes + +- ✅ Credentials stored in browser `localStorage` only +- ✅ Never sent to any server (except AWS) +- ✅ All S3 access happens client-side +- ✅ Use read-only IAM credentials +- ⚠️ Don't commit credentials to git +- ⚠️ Use a dedicated read-only IAM user + +## TV/Kiosk Mode + +For permanent display on a TV: + +### macOS +```bash +open -a "Google Chrome" --args --kiosk ".github/benchmark-dashboard/index.html" +``` + +### Linux +```bash +chromium-browser --kiosk --app="file://$(pwd)/.github/benchmark-dashboard/index.html" +``` + +### Auto-start on Boot + +Create a simple startup script: + +```bash +#!/bin/bash +# /usr/local/bin/start-benchmark-dashboard.sh + +cd /path/to/exo +python3 -m http.server 8080 & +sleep 2 +chromium-browser --kiosk http://localhost:8080/.github/benchmark-dashboard/ +``` + +## Data Displayed + +### Summary Cards +- **Latest Success Rate**: Most recent benchmark success percentage with trend +- **Avg Response Time**: Latest average response time in ms with trend +- **Total Benchmarks**: Count of all benchmarks run +- **Active Configurations**: Number of unique benchmark configs + +### Charts +1. **Success Rate Over Time**: Line chart showing reliability trends +2. **Average Response Time**: Performance over time (lower is better) +3. **Throughput**: Tokens/second metric (higher is better) +4. **Request Distribution**: Stacked bar chart of successes/failures + +## How It Works + +1. **Loads AWS SDK**: Uses AWS SDK for JavaScript (browser version) +2. **Lists S3 Objects**: Fetches all files from `s3://exo-benchmark-results/bench/` +3. **Downloads Results**: Fetches each JSON result file +4. **Parses & Visualizes**: Uses Chart.js to create interactive charts +5. **Auto-Refreshes**: Polls S3 every 60 seconds for new results + +## Customization + +To modify the dashboard: + +1. Edit `index.html` +2. Adjust `REFRESH_INTERVAL` for different polling frequency +3. Modify chart colors/styles in the Chart.js configuration +4. Add new metrics by extending the results parsing + +## Troubleshooting + +**"AWS credentials not configured"** +- Click "Configure AWS Credentials" and enter your keys + +**"Error loading benchmark data"** +- Check AWS credentials are correct +- Verify S3 bucket name is `exo-benchmark-results` +- Ensure IAM user has read permissions +- Check browser console for detailed errors + +**"No benchmark results found"** +- Wait for benchmark workflows to run +- Verify results are being uploaded to S3 +- Check S3 bucket has files in `bench/` prefix + +**Charts not updating** +- Check browser console for errors +- Verify network connectivity to S3 +- Try refreshing the page manually + diff --git a/.github/benchmark-dashboard/index.html b/.github/benchmark-dashboard/index.html new file mode 100644 index 00000000..5f72a831 --- /dev/null +++ b/.github/benchmark-dashboard/index.html @@ -0,0 +1,1641 @@ + + + + + + EXO Benchmark Dashboard + + + + + + + + + + + + +
+

🚀 EXO Benchmark Dashboard

+

Real-time performance tracking across commits

+
Loading...
+
+ +
+
+
Latest Success Rate
+
--%
+
+
+
+
Avg Response Time
+
-- ms
+
+
+
+
Time to First Token
+
-- ms
+
+
+
+
Decode Speed
+
-- t/s
+
+
+
+
Total Benchmarks
+
--
+
+
+
Active Configurations
+
--
+
+
+ +
+

📋 All Tests Summary

+ + + + + + + + + + + + + + + +
NameStrategySuccess RatePrefill Timems per token
Loading...
+
+ +
+ +
+ +
+
Loading benchmark data...
+
+ + + + diff --git a/.github/configs/README.md b/.github/configs/README.md new file mode 100644 index 00000000..4a399c88 --- /dev/null +++ b/.github/configs/README.md @@ -0,0 +1,186 @@ +# EXO Benchmark Configurations + +This directory contains configuration files for the EXO staged benchmark system. + +## Overview + +The staged benchmark system allows you to run complex, multi-stage load tests against EXO clusters. Each stage can have different characteristics: + +- **Prompt Length**: Number of tokens in the input prompt +- **Generation Length**: Maximum tokens to generate in the response +- **Time Between Requests**: Delay (in seconds) between firing consecutive requests +- **Iterations**: Number of requests to send in this stage + +Requests are **fire-and-forget** - they don't wait for the previous request to complete. This allows you to test overlapping request handling and measure success rates under load. + +## Configuration Files + +### `bench_simple.yaml` +A minimal configuration that replicates the behavior of the original `bench.py` script: +- Single stage with 1 iteration +- Short prompt (~20 tokens) +- Generates up to 100 tokens + +This is useful for quick smoke tests. + +### `bench_config.yaml` +A comprehensive multi-stage benchmark with: +1. **Warmup** (10 requests): Light load with short prompts +2. **Medium Load** (20 requests): Moderate load with medium prompts +3. **Stress Test** (30 requests): Heavy overlapping requests with long prompts +4. **Cooldown** (5 requests): Light load to wind down + +This tests the cluster's behavior under varying load patterns. + +## Configuration Schema + +```yaml +# Hardware configuration - maps runner labels to instance counts +hardware_plan: + M3ULTRA_GPU80_512GB: 4 + +# Environment variables to set on each node (optional) +environment: + OVERRIDE_MEMORY_MB: 512 + +# Timeout for instance and runner readiness (seconds) +timeout_seconds: 600 + +# Model instances to run concurrently +model_ids: + - "mlx-community/Llama-3.2-1B-Instruct-4bit" + +# Benchmark stages +stages: + - name: "stage_name" # Human-readable name for this stage + prompt_length: 100 # Target prompt length in tokens + generation_length: 200 # Max tokens to generate + time_between_requests: 2.0 # Seconds between firing requests + iterations: 10 # Number of requests in this stage +``` + +## Running Benchmarks + +### Via GitHub Actions + +**Automatic (every commit):** +- The **`bench`** workflow runs automatically on every push +- Uses `bench_simple.yaml` as the default configuration +- All settings (hardware plan, timeout, environment variables, models, stages) are defined in the config file + +**Manual (on-demand):** +1. Go to **Actions** → **bench** workflow +2. Click **Run workflow** +3. Configure: + - **Config File**: Path to your YAML config (default: `.github/configs/bench_simple.yaml`) + - `.github/configs/bench_simple.yaml` for quick tests + - `.github/configs/bench_config.yaml` for complex multi-stage tests + +All other settings (hardware plan, timeout, environment variables, models, stages) are read from the specified config file. + +### Via Command Line + +```bash +# Start EXO on localhost:8000 +uv run exo --api-port 8000 + +# Run simple benchmark (1 stage, 1 iteration) +python3 .github/scripts/bench.py \ + --api-port 8000 \ + --config .github/configs/bench_simple.yaml \ + --expected-nodes 1 \ + --is-primary true \ + --timeout-seconds 600 + +# Run complex staged benchmark (4 stages, multiple iterations) +python3 .github/scripts/bench.py \ + --api-port 8000 \ + --config .github/configs/bench_config.yaml \ + --expected-nodes 1 \ + --is-primary true \ + --timeout-seconds 600 +``` + +## Output Metrics + +For each stage, the benchmark reports: + +- **Total Requests**: Number of requests fired +- **Successful Requests**: Requests that completed successfully +- **Failed Requests**: Requests that encountered errors +- **Success Rate**: Percentage of successful requests +- **Total Tokens**: Sum of all tokens generated across successful requests +- **Avg Tokens/Request**: Average tokens per successful request +- **Avg Time/Request**: Average completion time per successful request + +A JSON summary is also printed for easy parsing and storage. + +## Creating Custom Benchmarks + +To create a custom benchmark: + +1. Copy an existing config file (e.g., `bench_config.yaml`) +2. Modify the stages to match your test scenario +3. Save it in this directory with a descriptive name +4. Run it using the workflow or command line + +### Example: Sustained Load Test + +```yaml +hardware_plan: + M3ULTRA_GPU80_512GB: 2 + +environment: + OVERRIDE_MEMORY_MB: 1024 + +timeout_seconds: 600 + +model_ids: + - "mlx-community/Llama-3.2-1B-Instruct-4bit" + +stages: + - name: "sustained_load" + prompt_length: 200 + generation_length: 150 + time_between_requests: 0.5 # Very fast - 2 requests/second + iterations: 100 # Run for ~50 seconds +``` + +### Example: Varying Prompt Sizes + +```yaml +hardware_plan: + M4PRO_GPU16_24GB: 3 + +timeout_seconds: 900 + +model_ids: + - "mlx-community/Llama-3.2-1B-Instruct-4bit" + +stages: + - name: "tiny_prompts" + prompt_length: 10 + generation_length: 100 + time_between_requests: 1.0 + iterations: 10 + + - name: "medium_prompts" + prompt_length: 200 + generation_length: 100 + time_between_requests: 1.0 + iterations: 10 + + - name: "large_prompts" + prompt_length: 1000 + generation_length: 100 + time_between_requests: 1.0 + iterations: 10 +``` + +## Tips + +- **Overlapping Requests**: Set `time_between_requests` < expected completion time to test concurrent request handling +- **Sequential Requests**: Set `time_between_requests` > expected completion time to ensure requests don't overlap +- **Realistic Load**: Model real usage patterns by varying prompt/generation lengths across stages +- **Success Rate**: A 100% success rate indicates the cluster handled the load well; lower rates suggest capacity limits + diff --git a/.github/configs/bench_config.yaml b/.github/configs/bench_config.yaml new file mode 100644 index 00000000..2477a4ff --- /dev/null +++ b/.github/configs/bench_config.yaml @@ -0,0 +1,49 @@ +# EXO Staged Benchmark Configuration +# This configuration defines a multi-stage load test for EXO clusters + +# Hardware configuration - maps runner labels to instance counts +hardware_plan: + M3ULTRA_GPU80_512GB: 4 + +# Environment variables to set on each node (optional) +environment: + OVERRIDE_MEMORY_MB: 512 + +# Timeout for instance and runner readiness (seconds) +timeout_seconds: 600 + +# Multiple instances run concurrently on the cluster +model_ids: + - "mlx-community/Qwen3-0.6B-4bit" + - "mlx-community/Qwen3-0.6B-4bit" + +# Stages run sequentially, each with its own characteristics +stages: + # Stage 1: Light load with short prompts + - name: "warmup" + prompt_length: 50 # Number of tokens in prompt + generation_length: 100 # Max tokens to generate + time_between_requests: 5.0 # Seconds between firing requests + iterations: 10 # Number of requests to send in this stage + + # Stage 2: Medium load with medium prompts + - name: "medium_load" + prompt_length: 200 + generation_length: 150 + time_between_requests: 3.0 + iterations: 20 + + # Stage 3: Heavy load with long prompts - requests will overlap + - name: "stress_test" + prompt_length: 500 + generation_length: 200 + time_between_requests: 1.0 # Fast firing - will definitely overlap + iterations: 30 + + # Stage 4: Cool down with simple prompts + - name: "cooldown" + prompt_length: 50 + generation_length: 50 + time_between_requests: 10.0 + iterations: 5 + diff --git a/.github/configs/bench_simple.yaml b/.github/configs/bench_simple.yaml new file mode 100644 index 00000000..9a76b6db --- /dev/null +++ b/.github/configs/bench_simple.yaml @@ -0,0 +1,125 @@ +# Simple single-shot benchmark +# Tests 2 instances concurrently on 2 nodes + +# Hardware configuration - maps runner labels to instance counts +hardware_plan: + puffin4: 1 + puffin8: 1 + +# Environment variables to set on each node +environment: + PLACEHOLDER: "placeholder" + # OVERRIDE_MEMORY_MB: 50000 + MLX_METAL_FAST_SYNCH: 1 + +# Timeout for instance and runner readiness (seconds) +timeout_seconds: 1800 + +# Model instances to run concurrently +model_ids: + # - "mlx-community/DeepSeek-V3.1-8bit" + # - "mlx-community/Kimi-K2-Instruct-4bit" + - "mlx-community/Kimi-K2-Thinking" + # - "mlx-community/Qwen3-235B-A22B-4bit" + # - "mlx-community/Llama-3.3-70B-Instruct-4bit" + # - "mlx-community/Llama-3.3-70B-Instruct-8bit" + # - "mlx-community/Llama-3.2-1B-Instruct-4bit" + +# Sharding strategy: "Pipeline" or "Tensor" +sharding: "Tensor" + +# Instance type: "MlxRing" or "MlxIbv" +instance_meta: "MlxIbv" + +# If true, run requests sequentially (no overlap); if false, fire-and-forget (default: false) +no_overlap: true + +# Benchmark stages +# pp: 64, 256, 1024, 2048, 4096, 8192, 16384 +# g: 64, 512 +stages: + # - name: "simple" + # prompt_length: 512 + # generation_length: 10 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp64_g64" + # prompt_length: 64 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp64_g64" + # prompt_length: 64 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp64_g512" + # prompt_length: 64 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 10 + # - name: "pp256_g64" + # prompt_length: 256 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 5 + - name: "pp256_g64" + prompt_length: 256 + generation_length: 64 + time_between_requests: 2.0 + iterations: 5 + # - name: "pp256_g512" + # prompt_length: 256 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 10 + # - name: "pp1024_g64" + # prompt_length: 1024 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp1024_g512" + # prompt_length: 1024 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 10 + # - name: "pp2048_g64" + # prompt_length: 2048 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp2048_g512" + # prompt_length: 2048 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 10 + # - name: "pp4096_g64" + # prompt_length: 4096 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 4 + # - name: "pp4096_g512" + # prompt_length: 4096 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 10 + # - name: "pp8192_g64" + # prompt_length: 8192 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp8192_g512" + # prompt_length: 8192 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 5 + # - name: "pp16384_g64" + # prompt_length: 16384 + # generation_length: 64 + # time_between_requests: 2.0 + # iterations: 10 + # - name: "pp16384_g512" + # prompt_length: 16384 + # generation_length: 512 + # time_between_requests: 2.0 + # iterations: 10 diff --git a/.github/scripts/bench.py b/.github/scripts/bench.py new file mode 100644 index 00000000..0ba73d58 --- /dev/null +++ b/.github/scripts/bench.py @@ -0,0 +1,1399 @@ +#!/usr/bin/env python3 + +# type: ignore +""" +Unified benchmark script for EXO. +Runs single or multi-stage benchmarks with configurable load patterns. +Requests are fire-and-forget, allowing overlapping execution. + +Simple benchmark (1 iteration): --config .github/configs/bench_simple.yaml +Complex benchmark (multiple stages): --config .github/configs/bench_config.yaml +""" + +# pyright: reportAny=false, reportUnknownArgumentType=false, reportUnknownVariableType=false +from __future__ import annotations + +import argparse +import asyncio +import contextlib +import json +import sys +import time +import urllib.error +import urllib.request +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Mapping + +import yaml + + +def _format_http_error(error: Exception) -> str: + """Format HTTP error with full response details for debugging.""" + if isinstance(error, urllib.error.HTTPError): + try: + body = error.read().decode("utf-8", errors="replace") + except Exception: + body = "" + + headers_str = ( + "\n".join(f" {k}: {v}" for k, v in error.headers.items()) + if error.headers + else "" + ) + + return ( + f"HTTP {error.code} {error.reason}\n" + f"URL: {error.url}\n" + f"Headers:\n{headers_str}\n" + f"Body: {body}" + ) + elif isinstance(error, urllib.error.URLError): + return f"URLError: {error.reason}" + else: + return str(error) + + +def _http_request( + url: str, *, method: str = "GET", data: Mapping[str, Any] | None = None +) -> dict[str, Any]: + headers = {"Content-Type": "application/json"} + payload: bytes | None = None + if data is not None: + payload = json.dumps(data).encode("utf-8") + req = urllib.request.Request(url, data=payload, headers=headers, method=method) + try: + with urllib.request.urlopen(req, timeout=300) as resp: # nosec - runner-local API + body = resp.read().decode("utf-8") + try: + return json.loads(body) + except json.JSONDecodeError: + return {"raw": body} + except Exception as e: + error_details = _format_http_error(e) + print(f"HTTP request failed:\n{error_details}") + raise + + +async def _http_request_async( + url: str, *, method: str = "GET", data: Mapping[str, Any] | None = None +) -> dict[str, Any]: + """Async version that runs in executor to not block event loop.""" + loop = asyncio.get_event_loop() + return await loop.run_in_executor( + None, lambda: _http_request(url, method=method, data=data) + ) + + +async def _http_stream_async( + url: str, *, method: str = "POST", data: Mapping[str, Any], timeout: int = 300 +) -> list[tuple[str, float]]: + """Async streaming request. Returns list of (line, timestamp) tuples.""" + + def _stream() -> list[tuple[str, float]]: + headers = {"Content-Type": "application/json"} + payload = json.dumps(data).encode("utf-8") + req = urllib.request.Request(url, data=payload, headers=headers, method=method) + lines: list[tuple[str, float]] = [] + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: # nosec - runner-local API + for raw_line in resp: + timestamp = time.monotonic() + line = raw_line.decode("utf-8", errors="replace").rstrip("\n\r") + if line: + lines.append((line, timestamp)) + return lines + except Exception as e: + error_details = _format_http_error(e) + print(f"HTTP request failed:\n{error_details}") + raise + + loop = asyncio.get_event_loop() + return await loop.run_in_executor(None, _stream) + + +def fetch_state(api_base: str) -> dict[str, Any]: + return _http_request(f"{api_base}/state") + + +def unwrap_tagged_union(obj: Any) -> tuple[str | None, Any]: + """Extract tag and payload from tagged union format {Tag: {fields...}}. + + Returns (tag_name, payload) if the object is a tagged union, otherwise (None, obj). + """ + if not isinstance(obj, dict): + return None, obj + + keys = list(obj.keys()) + if len(keys) == 1 and isinstance(keys[0], str): + tag = keys[0] + payload = obj[tag] + return tag, payload + + return None, obj + + +def collect_metrics_snapshot(state: Mapping[str, Any]) -> MetricsSnapshot: + """Collect current metrics snapshot from state.""" + timestamp = time.time() + + # Collect memory for each node + node_memory: dict[str, MemorySnapshot] = {} + node_profiles: Mapping[str, Any] = state.get("nodeProfiles", {}) + + for node_id, profile in node_profiles.items(): + if not isinstance(profile, dict): + continue + + memory = profile.get("memory", {}) + if not isinstance(memory, dict): + continue + + # Parse memory values - they're objects with 'inBytes' field + def get_bytes(mem_obj: Any) -> int: + if isinstance(mem_obj, dict): + return int(mem_obj.get("inBytes", 0)) + return 0 + + ram_total = get_bytes(memory.get("ramTotal")) + ram_available = get_bytes(memory.get("ramAvailable")) + swap_total = get_bytes(memory.get("swapTotal")) + swap_available = get_bytes(memory.get("swapAvailable")) + + node_memory[node_id] = MemorySnapshot( + ram_total_bytes=ram_total, + ram_available_bytes=ram_available, + ram_used_bytes=max(ram_total - ram_available, 0), + swap_total_bytes=swap_total, + swap_available_bytes=swap_available, + swap_used_bytes=max(swap_total - swap_available, 0), + ) + + # Collect task counts per instance and per node + instance_tasks: list[InstanceTaskSnapshot] = [] + instances: Mapping[str, Any] = state.get("instances", {}) + tasks: Mapping[str, Any] = state.get("tasks", {}) + print(f"[DEBUG] Num tasks: {len(tasks)}. Num instances: {len(instances)}.") + + # Map instance_id -> node_ids (instances can span multiple nodes) + instance_to_nodes: dict[str, set[str]] = {} + for instance_id, instance_wrapped in instances.items(): + # Unwrap tagged Instance union (MlxRingInstance or MlxIbvInstance) + _instance_tag, instance_data = unwrap_tagged_union(instance_wrapped) + if not isinstance(instance_data, dict): + continue + + shard_assignments = instance_data.get("shardAssignments", {}) + if not isinstance(shard_assignments, dict): + continue + + # Get all nodes that this instance uses + node_to_runner = shard_assignments.get("nodeToRunner", {}) + if isinstance(node_to_runner, dict): + instance_to_nodes[instance_id] = set(node_to_runner.keys()) + + # Count tasks per instance (only Pending and Running exist in state; completed tasks are deleted) + instance_task_counts: dict[str, dict[str, int]] = {} + for instance_id in instances: + instance_task_counts[instance_id] = { + "Pending": 0, + "Running": 0, + } + + # Iterate through tasks and count by instance and status + tasks_matched = 0 + tasks_skipped = 0 + + for _task_id, task_wrapper in tasks.items(): + if not isinstance(task_wrapper, dict): + print(f"[DEBUG] Task wrapper is not a dict: {task_wrapper}") + tasks_skipped += 1 + continue + + # Extract actual task from wrapper (e.g., {"ChatCompletion": {...}}) + if len(task_wrapper) != 1: + print( + f"[DEBUG] Task wrapper has unexpected number of keys: {len(task_wrapper)}" + ) + tasks_skipped += 1 + continue + + _task_type, task_data = next(iter(task_wrapper.items())) + + if not isinstance(task_data, dict): + print(f"[DEBUG] Task data is not a dict: {task_data}") + tasks_skipped += 1 + continue + + instance_id = task_data.get("instanceId") + task_status = task_data.get("taskStatus") + + if not instance_id or instance_id not in instance_task_counts: + tasks_skipped += 1 + continue + + if task_status not in ["Pending", "Running"]: + tasks_skipped += 1 + continue + + # Count this task + instance_task_counts[instance_id][task_status] += 1 + tasks_matched += 1 + + if tasks_skipped > 0: + print( + f"[DEBUG] Task matching: {tasks_matched} matched, {tasks_skipped} skipped (from {len(tasks)} total)" + ) + + # Build snapshots for each instance (assign to primary node - first in sorted order) + for instance_id, counts in instance_task_counts.items(): + pending = counts["Pending"] + running = counts["Running"] + total_active = pending + running + + node_ids = instance_to_nodes.get(instance_id, set()) + primary_node = sorted(node_ids)[0] if node_ids else "unknown" + + instance_tasks.append( + InstanceTaskSnapshot( + instance_id=instance_id, + node_id=primary_node, + pending_tasks=pending, + running_tasks=running, + total_active_tasks=total_active, + ) + ) + + # Aggregate tasks per node + node_task_counts: dict[str, dict[str, int]] = {} + node_instance_counts: dict[str, int] = {} + + for instance_snapshot in instance_tasks: + node_id = instance_snapshot.node_id + + if node_id not in node_task_counts: + node_task_counts[node_id] = { + "Pending": 0, + "Running": 0, + } + node_instance_counts[node_id] = 0 + + node_task_counts[node_id]["Pending"] += instance_snapshot.pending_tasks + node_task_counts[node_id]["Running"] += instance_snapshot.running_tasks + node_instance_counts[node_id] += 1 + + # Build node snapshots + node_tasks: list[NodeTaskSnapshot] = [] + for node_id, counts in node_task_counts.items(): + pending = counts["Pending"] + running = counts["Running"] + total_active = pending + running + + node_tasks.append( + NodeTaskSnapshot( + node_id=node_id, + pending_tasks=pending, + running_tasks=running, + total_active_tasks=total_active, + instance_count=node_instance_counts.get(node_id, 0), + ) + ) + + return MetricsSnapshot( + timestamp=timestamp, + node_memory=node_memory, + instance_tasks=instance_tasks, + node_tasks=node_tasks, + ) + + +def get_topology_node_count(state: Mapping[str, Any]) -> int: + """Get the number of nodes in the topology.""" + topology = state.get("topology", {}) + nodes = topology.get("nodes", []) + return len(nodes) + + +def count_instances_by_model(state: Mapping[str, Any], model_id: str) -> int: + """Count how many instances exist for a given model_id.""" + instances: Mapping[str, Any] = state.get("instances", {}) + count = 0 + for instance_wrapped in instances.values(): + # Unwrap tagged Instance union + _instance_tag, instance_data = unwrap_tagged_union(instance_wrapped) + if not isinstance(instance_data, dict): + continue + + shard = instance_data.get("shardAssignments", {}) + if isinstance(shard, dict) and shard.get("modelId") == model_id: + count += 1 + return count + + +def get_all_instance_ids_for_model( + state: Mapping[str, Any], model_id: str +) -> list[str]: + """Get all instance IDs for a given model_id.""" + instances: Mapping[str, Any] = state.get("instances", {}) + instance_ids = [] + for instance_id, instance_wrapped in instances.items(): + # Unwrap tagged Instance union + _instance_tag, instance_data = unwrap_tagged_union(instance_wrapped) + if not isinstance(instance_data, dict): + continue + + shard = instance_data.get("shardAssignments", {}) + if isinstance(shard, dict) and shard.get("modelId") == model_id: + instance_ids.append(instance_id) + return instance_ids + + +def count_ready_instances_by_model(state: Mapping[str, Any], model_id: str) -> int: + """Count how many instances for a model have all their runners ready.""" + instances: Mapping[str, Any] = state.get("instances", {}) + ready_count = 0 + + for instance_id, instance_wrapped in instances.items(): + # Unwrap tagged Instance union + _instance_tag, instance_data = unwrap_tagged_union(instance_wrapped) + if not isinstance(instance_data, dict): + continue + + shard = instance_data.get("shardAssignments", {}) + if not isinstance(shard, dict) or shard.get("modelId") != model_id: + continue + + # Check if all runners for this instance are ready + runner_ids = get_runner_ids_for_instance(state, instance_id) + if len(runner_ids) == 0: + continue + + # Fixed runner status names: RunnerReady and RunnerRunning (not LoadedRunnerStatus/RunningRunnerStatus) + all_ready = all( + get_runner_status_kind(state, rid) in {"RunnerReady", "RunnerRunning"} + for rid in runner_ids + ) + + if all_ready: + ready_count += 1 + + return ready_count + + +def get_runner_ids_for_instance( + state: Mapping[str, Any], instance_id: str +) -> list[str]: + instances: Mapping[str, Any] = state.get("instances", {}) + instance_wrapped = instances.get(instance_id, {}) + + # Unwrap tagged Instance union + _instance_tag, instance_data = unwrap_tagged_union(instance_wrapped) + if not isinstance(instance_data, dict): + return [] + + shard_assignments = instance_data.get("shardAssignments", {}) + if not isinstance(shard_assignments, dict): + return [] + + r2s = shard_assignments.get("runnerToShard", {}) + if isinstance(r2s, dict): + return list(r2s.keys()) + return [] + + +def get_runner_status_kind(state: Mapping[str, Any], runner_id: str) -> str | None: + runners: Mapping[str, Any] = state.get("runners", {}) + status_obj = runners.get(runner_id) + if not isinstance(status_obj, dict): + return None + if len(status_obj) == 1: + return next(iter(status_obj.keys())) + return None + + +async def wait_for_topology_ready( + api_base: str, expected_nodes: int, timeout_s: int +) -> None: + """Wait for all expected nodes to appear in the topology.""" + print( + f"Waiting for {expected_nodes} node(s) to appear in topology (timeout: {timeout_s}s)..." + ) + start = time.monotonic() + while True: + state = fetch_state(api_base) + node_count = get_topology_node_count(state) + elapsed = time.monotonic() - start + print( + f" Topology has {node_count}/{expected_nodes} nodes (elapsed: {elapsed:.1f}s)" + ) + + if node_count >= expected_nodes: + print(f"All {expected_nodes} node(s) are in topology!") + return + + if elapsed > timeout_s: + raise TimeoutError( + f"Timed out waiting for topology. Expected {expected_nodes} nodes, got {node_count}" + ) + await asyncio.sleep(2) + + +async def wait_for_instances_ready( + api_base: str, model_id: str, expected_count: int, timeout_s: int +) -> list[str]: + """Wait for a specific count of instances for a model to be fully ready.""" + print( + f"Waiting for {expected_count} instance(s) of {model_id} to be ready (timeout: {timeout_s}s)..." + ) + start = time.monotonic() + while True: + state = fetch_state(api_base) + + total_count = count_instances_by_model(state, model_id) + ready_count = count_ready_instances_by_model(state, model_id) + elapsed = time.monotonic() - start + + print( + f" Model {model_id}: {ready_count}/{expected_count} ready ({total_count} total) (elapsed: {elapsed:.1f}s)" + ) + + if ready_count >= expected_count: + instance_ids = get_all_instance_ids_for_model(state, model_id) + print( + f"All {expected_count} instance(s) ready! Instance IDs: {instance_ids}" + ) + return instance_ids + + if elapsed > timeout_s: + raise TimeoutError( + f"Timed out waiting for instances. Expected {expected_count} ready instances of {model_id}, " + f"got {ready_count} ready out of {total_count} total" + ) + await asyncio.sleep(2) + + +async def wait_for_all_instances_deleted(api_base: str, model_id: str) -> None: + """Wait for all instances of a model to be deleted.""" + print(f"Waiting for all instances of {model_id} to be deleted...") + start = time.monotonic() + while True: + state = fetch_state(api_base) + count = count_instances_by_model(state, model_id) + if count == 0: + elapsed = time.monotonic() - start + print(f"All instances of {model_id} deleted after {elapsed:.1f}s") + return + await asyncio.sleep(2) + + +async def wait_for_tasks_drained(api_base: str, timeout_s: int = 600) -> None: + """Wait for all tasks in the cluster to be drained (completed or failed). + + Tasks are deleted from state when complete, so we wait until there are no + pending or running tasks remaining. + """ + print(f"\n{'=' * 80}") + print("⏳ WAITING FOR ALL TASKS TO DRAIN") + print(f"{'=' * 80}") + start = time.monotonic() + + while True: + state = fetch_state(api_base) + snapshot = collect_metrics_snapshot(state) + + # Count total active tasks across all nodes + total_pending = sum(node.pending_tasks for node in snapshot.node_tasks) + total_running = sum(node.running_tasks for node in snapshot.node_tasks) + total_active = total_pending + total_running + + elapsed = time.monotonic() - start + + if total_active == 0: + print(f"✅ All tasks drained after {elapsed:.1f}s") + return + + print( + f" [{elapsed:.1f}s] Still draining: {total_active} active tasks ({total_pending} pending, {total_running} running)" + ) + + # Print per-node breakdown if there are active tasks + if snapshot.node_tasks: + for node_snapshot in snapshot.node_tasks: + if node_snapshot.total_active_tasks > 0: + node_short = node_snapshot.node_id[-4:] + print( + f" Node ...{node_short}: {node_snapshot.running_tasks} running, {node_snapshot.pending_tasks} pending" + ) + + if elapsed > timeout_s: + print( + f"⚠️ WARNING: Timed out waiting for tasks to drain after {timeout_s}s" + ) + print( + f" Remaining: {total_active} tasks ({total_pending} pending, {total_running} running)" + ) + return + + await asyncio.sleep(2) + + +def generate_prompt(length: int) -> str: + """Generate a prompt of approximately the specified token length.""" + # Rough approximation: 1 token ≈ 4 characters + # Use a repeating pattern that's easy to generate + base_text = "The quick brown fox jumps over the lazy dog. " + target_chars = length * 4 + repetitions = (target_chars // len(base_text)) + 1 + return (base_text * repetitions)[:target_chars] + + +@dataclass(frozen=True) +class StageConfig: + name: str + prompt_length: int + generation_length: int + time_between_requests: float + iterations: int + + +@dataclass +class RequestResult: + request_id: int + success: bool + tokens: int + elapsed_s: float + started_at: float + completed_at: float + time_to_first_token_s: float | None = None + decode_tps: float | None = None + error: str | None = None + + +@dataclass +class StageResult: + name: str + total_requests: int + successful_requests: int + failed_requests: int + success_rate: float + total_tokens: int + total_time: float + avg_tokens_per_request: float + avg_time_per_request: float + avg_time_to_first_token: float | None + std_time_to_first_token: float | None + avg_decode_tps: float | None + avg_ms_per_token: float | None + std_ms_per_token: float | None + request_results: list[RequestResult] + stage_started_at: float + stage_completed_at: float + + +@dataclass(frozen=True) +class MemorySnapshot: + """Memory snapshot for a node at a point in time.""" + + ram_total_bytes: int + ram_available_bytes: int + ram_used_bytes: int + swap_total_bytes: int + swap_available_bytes: int + swap_used_bytes: int + + +@dataclass(frozen=True) +class InstanceTaskSnapshot: + """Task counts for an instance at a point in time. + + Note: Tasks are deleted from state when complete, so we only track active tasks. + total_active_tasks = pending + running. + """ + + instance_id: str + node_id: str + pending_tasks: int + running_tasks: int + total_active_tasks: int + + +@dataclass(frozen=True) +class NodeTaskSnapshot: + """Task counts for a node at a point in time. + + Note: Tasks are deleted from state when complete, so we only track active tasks. + total_active_tasks = pending + running across all instances on this node. + """ + + node_id: str + pending_tasks: int + running_tasks: int + total_active_tasks: int + instance_count: int + + +@dataclass(frozen=True) +class MetricsSnapshot: + """System metrics snapshot at a point in time.""" + + timestamp: float + node_memory: dict[str, MemorySnapshot] + instance_tasks: list[InstanceTaskSnapshot] + node_tasks: list[NodeTaskSnapshot] + + +async def run_single_request( + api_base: str, + model_id: str, + prompt: str, + max_tokens: int, + request_id: int, + timeout: int = 300, +) -> RequestResult: + """Run a single chat completion request and return its result.""" + started_at = time.time() + start = time.monotonic() + try: + lines = await _http_stream_async( + f"{api_base}/v1/chat/completions", + method="POST", + data={ + "model": model_id, + "messages": [{"role": "user", "content": prompt}], + "max_tokens": max_tokens, + "temperature": 0.7, + }, + timeout=timeout, + ) + + tokens = 0 + got_done = False + first_token_time: float | None = None + last_token_time: float | None = None + + for line, timestamp in lines: + if not line.startswith("data:"): + continue + payload = line[len("data:") :].strip() + if payload == "[DONE]": + got_done = True + break + try: + obj = json.loads(payload) + content = obj.get("choices", [{}])[0].get("delta", {}).get("content") + if content: + if first_token_time is None: + first_token_time = timestamp + last_token_time = timestamp + tokens += 1 + except json.JSONDecodeError: + continue + + elapsed = time.monotonic() - start + completed_at = time.time() + + # Calculate TTFT and decode TPS + time_to_first_token: float | None = None + decode_tps: float | None = None + + if first_token_time is not None: + time_to_first_token = first_token_time - start + + # Decode TPS: tokens per second after first token + if last_token_time is not None and tokens > 1: + decode_time = last_token_time - first_token_time + if decode_time > 0: + decode_tps = (tokens - 1) / decode_time + + # Request is only successful if we got at least one token AND a [DONE] marker + if tokens == 0: + print( + f" Request #{request_id}: FAILED - no tokens generated in {elapsed:.2f}s" + ) + return RequestResult( + request_id=request_id, + success=False, + tokens=0, + elapsed_s=elapsed, + started_at=started_at, + completed_at=completed_at, + time_to_first_token_s=time_to_first_token, + decode_tps=decode_tps, + error="No tokens generated", + ) + + if not got_done: + print( + f" Request #{request_id}: FAILED - incomplete response (no [DONE]) after {elapsed:.2f}s" + ) + return RequestResult( + request_id=request_id, + success=False, + tokens=tokens, + elapsed_s=elapsed, + started_at=started_at, + completed_at=completed_at, + time_to_first_token_s=time_to_first_token, + decode_tps=decode_tps, + error="Incomplete response (no [DONE] marker)", + ) + + ttft_str = ( + f"{time_to_first_token:.3f}s" if time_to_first_token is not None else "N/A" + ) + tps_str = f"{decode_tps:.1f} t/s" if decode_tps is not None else "N/A" + print( + f" Request #{request_id}: SUCCESS - {tokens} tokens in {elapsed:.2f}s (TTFT: {ttft_str}, Decode: {tps_str})" + ) + return RequestResult( + request_id=request_id, + success=True, + tokens=tokens, + elapsed_s=elapsed, + started_at=started_at, + completed_at=completed_at, + time_to_first_token_s=time_to_first_token, + decode_tps=decode_tps, + ) + + except Exception as e: + elapsed = time.monotonic() - start + completed_at = time.time() + error_details = _format_http_error(e) + print(f" Request #{request_id}: FAILED - {error_details}") + return RequestResult( + request_id=request_id, + success=False, + tokens=0, + elapsed_s=elapsed, + started_at=started_at, + completed_at=completed_at, + time_to_first_token_s=None, + decode_tps=None, + error=error_details, + ) + + +async def monitor_metrics( + api_base: str, + metrics_snapshots: list[MetricsSnapshot], + stop_event: asyncio.Event, + interval_seconds: float = 5.0, +) -> None: + """Background task that collects metrics snapshots every interval_seconds.""" + print(f"\n{'=' * 80}") + print(f"🔍 METRICS MONITORING STARTED (polling every {interval_seconds}s)") + print(f"{'=' * 80}\n") + + snapshot_count = 0 + while not stop_event.is_set(): + try: + snapshot_count += 1 + state = fetch_state(api_base) + snapshot = collect_metrics_snapshot(state) + metrics_snapshots.append(snapshot) + + # Print detailed summary + node_count = len(snapshot.node_memory) + instance_count = len(snapshot.instance_tasks) + + # Aggregate task counts from node level (only active tasks in state) + total_pending = sum(node.pending_tasks for node in snapshot.node_tasks) + total_running = sum(node.running_tasks for node in snapshot.node_tasks) + total_active = sum(node.total_active_tasks for node in snapshot.node_tasks) + + # Print detailed breakdown + print( + f"\n[METRICS #{snapshot_count}] {node_count} nodes, {instance_count} instances | Active Tasks: {total_active} ({total_pending} pending, {total_running} running)" + ) + + # Print per-node breakdown (only if there are nodes) + if snapshot.node_tasks: + for node_snapshot in snapshot.node_tasks: + node_short = node_snapshot.node_id[-4:] + print( + f" Node ...{node_short}: {node_snapshot.total_active_tasks} active ({node_snapshot.pending_tasks} pending, {node_snapshot.running_tasks} running) across {node_snapshot.instance_count} instances" + ) + + except Exception as e: + print(f"[METRICS] Error collecting snapshot: {e}") + import traceback + + traceback.print_exc() + + # Wait for interval or until stopped + with contextlib.suppress(asyncio.TimeoutError): + await asyncio.wait_for(stop_event.wait(), timeout=interval_seconds) + + +async def run_stage( + api_base: str, + model_id: str, + stage: StageConfig, + no_overlap: bool = False, +) -> StageResult: + """Run a single benchmark stage with fire-and-forget requests (or sequential if no_overlap=True).""" + print("=" * 80) + print(f"STAGE: {stage.name}") + print("=" * 80) + print(f" Prompt Length: {stage.prompt_length} tokens") + print(f" Generation Length: {stage.generation_length} tokens") + print(f" Time Between Reqs: {stage.time_between_requests}s") + print(f" Iterations: {stage.iterations}") + print(f" No Overlap: {no_overlap}") + print("=" * 80) + + stage_started_at = time.time() + prompt = generate_prompt(stage.prompt_length) + results: list[RequestResult] = [] + + if no_overlap: + # Sequential execution: wait for each request to complete before starting next + print("\nRunning requests sequentially (no overlap)...") + for i in range(stage.iterations): + result = await run_single_request( + api_base, model_id, prompt, stage.generation_length, i + 1 + ) + results.append(result) + + # Wait before starting next request (except after last one) + if i < stage.iterations - 1: + await asyncio.sleep(stage.time_between_requests) + else: + # Concurrent execution: fire-and-forget with delays between starts + print("\nRunning requests concurrently (with overlap)...") + tasks: list[asyncio.Task[RequestResult]] = [] + + # Fire off requests with delays between them + for i in range(stage.iterations): + task = asyncio.create_task( + run_single_request( + api_base, model_id, prompt, stage.generation_length, i + 1 + ) + ) + tasks.append(task) + + # Wait before firing next request (except after last one) + if i < stage.iterations - 1: + await asyncio.sleep(stage.time_between_requests) + + # Wait for all requests to complete + print(f"\nWaiting for all {len(tasks)} HTTP requests to complete...") + results = list(await asyncio.gather(*tasks)) + + # Wait for all tasks in the cluster to be drained + print("\nHTTP requests completed. Now waiting for cluster tasks to drain...") + await wait_for_tasks_drained(api_base, timeout_s=600) + + stage_completed_at = time.time() + + # Compute statistics + successful = sum(1 for r in results if r.success) + failed = len(results) - successful + success_rate = successful / len(results) if results else 0.0 + total_tokens = sum(r.tokens for r in results) + total_time = sum(r.elapsed_s for r in results) + avg_tokens = total_tokens / successful if successful > 0 else 0.0 + avg_time = total_time / successful if successful > 0 else 0.0 + + # Calculate average TTFT and decode TPS for successful requests only + successful_results = [r for r in results if r.success] + + # Skip first iteration if there are more than 1 iterations (warmup) + results_for_stats = ( + successful_results[1:] if len(successful_results) > 1 else successful_results + ) + + # TTFT statistics + ttft_values = [ + r.time_to_first_token_s + for r in results_for_stats + if r.time_to_first_token_s is not None + ] + avg_ttft = sum(ttft_values) / len(ttft_values) if ttft_values else None + + if avg_ttft is not None and len(ttft_values) > 1: + variance_ttft = sum((x - avg_ttft) ** 2 for x in ttft_values) / len(ttft_values) + std_ttft = variance_ttft**0.5 + else: + std_ttft = None + + # Decode TPS and ms per token statistics + decode_tps_values = [ + r.decode_tps for r in results_for_stats if r.decode_tps is not None + ] + avg_decode_tps = ( + sum(decode_tps_values) / len(decode_tps_values) if decode_tps_values else None + ) + + # Convert to ms per token + ms_per_token_values = ( + [1000.0 / tps for tps in decode_tps_values] if decode_tps_values else [] + ) + avg_ms_per_token = ( + sum(ms_per_token_values) / len(ms_per_token_values) + if ms_per_token_values + else None + ) + + if avg_ms_per_token is not None and len(ms_per_token_values) > 1: + variance_ms_per_token = sum( + (x - avg_ms_per_token) ** 2 for x in ms_per_token_values + ) / len(ms_per_token_values) + std_ms_per_token = variance_ms_per_token**0.5 + else: + std_ms_per_token = None + + return StageResult( + name=stage.name, + total_requests=len(results), + successful_requests=successful, + failed_requests=failed, + success_rate=success_rate, + total_tokens=total_tokens, + total_time=total_time, + avg_tokens_per_request=avg_tokens, + avg_time_per_request=avg_time, + avg_time_to_first_token=avg_ttft, + std_time_to_first_token=std_ttft, + avg_decode_tps=avg_decode_tps, + avg_ms_per_token=avg_ms_per_token, + std_ms_per_token=std_ms_per_token, + request_results=list(results), + stage_started_at=stage_started_at, + stage_completed_at=stage_completed_at, + ) + + +async def run_benchmark( + api_base: str, + config_path: Path, + expected_nodes: int, + is_primary: bool, + timeout_seconds: int, + results_output_path: Path | None = None, + git_commit: str | None = None, + hardware_labels: list[str] | None = None, +) -> int: + """Run the full staged benchmark.""" + benchmark_started_at = time.time() + + # Load configuration + with open(config_path) as f: + config = yaml.safe_load(f) + + # Support both model_id (legacy) and model_ids (new) + if "model_ids" in config: + model_ids = config["model_ids"] + elif "model_id" in config: + model_ids = [config["model_id"]] + else: + raise ValueError("Config must contain either 'model_id' or 'model_ids'") + + # Get sharding and instance_meta (optional, defaults to None if not specified) + sharding: str | None = config.get("sharding") + instance_meta: str | None = config.get("instance_meta") + + # Get no_overlap flag (optional, defaults to False) + no_overlap: bool = config.get("no_overlap", False) + + stages = [StageConfig(**s) for s in config["stages"]] + + print("=" * 80) + print("EXO BENCHMARK") + print("=" * 80) + print(f"Configuration File: {config_path}") + print(f"Model IDs: {model_ids}") + print(f"Instance Count: {len(model_ids)}") + print( + f"Sharding: {sharding if sharding else 'not specified (defaults to Pipeline)'}" + ) + print( + f"Instance Type: {instance_meta if instance_meta else 'not specified (defaults to MlxRing)'}" + ) + print(f"No Overlap: {no_overlap}") + print(f"Stages: {len(stages)}") + print(f"Expected Nodes: {expected_nodes}") + print(f"Is Primary: {is_primary}") + print("=" * 80) + + try: + # Wait for all nodes to join the topology first + await wait_for_topology_ready( + api_base, expected_nodes, timeout_s=timeout_seconds + ) + + # Add 30 second delay to allow topology to stabilize before creating instances + print( + "\nWaiting 30 seconds for topology to stabilize before creating instances..." + ) + await asyncio.sleep(30) + print("Proceeding with instance creation\n") + + # Count how many instances we need for each unique model_id + from collections import Counter + + model_counts = Counter(model_ids) + + print("\nTarget instance counts by model:") + for model_id, count in model_counts.items(): + print(f" {model_id}: {count} instance(s)") + print() + + # Track all instance IDs (collected at the end) + all_instance_ids: list[str] = [] + + if is_primary: + # Primary: create instances one at a time, waiting for count to increase + for idx, model_id in enumerate(model_ids): + # Determine current and target counts for this model + current_state = fetch_state(api_base) + current_ready = count_ready_instances_by_model(current_state, model_id) + target_count = current_ready + 1 + + print("=" * 80) + print( + f"[PRIMARY] Creating instance {idx + 1}/{len(model_ids)} for model: {model_id}" + ) + print( + f"[PRIMARY] Current ready count for {model_id}: {current_ready}, target: {target_count}" + ) + + # Build instance creation request data + instance_data: dict[str, Any] = {"model_id": model_id} + if sharding is not None: + instance_data["sharding"] = sharding + if instance_meta is not None: + instance_data["instance_meta"] = instance_meta + + response = await _http_request_async( + f"{api_base}/instance", method="POST", data=instance_data + ) + print(f"[PRIMARY] Instance creation response: {response}") + + # Wait for one more instance of this model to be ready + await wait_for_instances_ready( + api_base, model_id, target_count, timeout_s=timeout_seconds + ) + print(f"[PRIMARY] Instance {idx + 1}/{len(model_ids)} is ready") + print("=" * 80) + else: + # Secondary: wait for expected counts of each model to be ready + print("[SECONDARY] Waiting for all instances to be created and ready...") + for model_id, expected_count in model_counts.items(): + await wait_for_instances_ready( + api_base, model_id, expected_count, timeout_s=timeout_seconds + ) + + # Collect all instance IDs for all models + state = fetch_state(api_base) + for model_id in model_counts: + ids = get_all_instance_ids_for_model(state, model_id) + all_instance_ids.extend(ids) + + # Count total runners + total_runners = 0 + for instance_id in all_instance_ids: + runner_ids = get_runner_ids_for_instance(state, instance_id) + total_runners += len(runner_ids) + + print( + f"\nAll {len(all_instance_ids)} instance(s) with {total_runners} total runner(s) are ready!" + ) + print(f"Instance IDs: {all_instance_ids}") + + if is_primary: + # Run all stages once (requests will use available instances) + # We use the first model_id for the benchmark requests + benchmark_model_id = model_ids[0] + print(f"\n{'=' * 80}") + print(f"RUNNING BENCHMARK (using model: {benchmark_model_id})") + print(f"Instances available: {len(all_instance_ids)}") + print(f"{'=' * 80}") + + # Start metrics monitoring with 500ms interval to catch fast-completing tasks + metrics_snapshots: list[MetricsSnapshot] = [] + stop_monitoring = asyncio.Event() + monitoring_task = asyncio.create_task( + monitor_metrics( + api_base, metrics_snapshots, stop_monitoring, interval_seconds=0.5 + ) + ) + + stage_results: list[StageResult] = [] + for stage in stages: + result = await run_stage( + api_base, benchmark_model_id, stage, no_overlap=no_overlap + ) + stage_results.append(result) + + # Stop metrics monitoring + print("\nStopping metrics monitoring...") + stop_monitoring.set() + await monitoring_task + print(f"Collected {len(metrics_snapshots)} metrics snapshots") + + # Print final results + print("\n" + "=" * 80) + print("BENCHMARK COMPLETE - RESULTS SUMMARY") + print("=" * 80) + print(f"Instances tested: {len(all_instance_ids)}") + print(f"Model IDs: {model_ids}") + print(f"Instance IDs: {all_instance_ids}") + + for result in stage_results: + print(f"\nStage: {result.name}") + print(f" Total Requests: {result.total_requests}") + print(f" Successful: {result.successful_requests}") + print(f" Failed: {result.failed_requests}") + print(f" Success Rate: {result.success_rate * 100:.1f}%") + print(f" Total Tokens: {result.total_tokens}") + print(f" Avg Tokens/Request: {result.avg_tokens_per_request:.1f}") + print(f" Avg Time/Request: {result.avg_time_per_request:.2f}s") + if result.avg_time_to_first_token is not None: + if result.std_time_to_first_token is not None: + print( + f" Avg TTFT: {result.avg_time_to_first_token:.3f}s ± {result.std_time_to_first_token:.3f}s" + ) + else: + print( + f" Avg TTFT: {result.avg_time_to_first_token:.3f}s" + ) + if result.avg_ms_per_token is not None: + if result.std_ms_per_token is not None: + print( + f" Avg ms/token: {result.avg_ms_per_token:.2f}ms ± {result.std_ms_per_token:.2f}ms" + ) + else: + print(f" Avg ms/token: {result.avg_ms_per_token:.2f}ms") + if result.avg_decode_tps is not None: + print(f" Avg Decode TPS: {result.avg_decode_tps:.2f} tokens/s") + + benchmark_completed_at = time.time() + + # Build comprehensive results document + results_doc = { + "metadata": { + "benchmark_started_at": benchmark_started_at, + "benchmark_completed_at": benchmark_completed_at, + "total_duration_s": benchmark_completed_at - benchmark_started_at, + "git_commit": git_commit, + "config_file": str(config_path), + "hardware_labels": hardware_labels or [], + "expected_nodes": expected_nodes, + "timeout_seconds": timeout_seconds, + }, + "cluster": { + "model_ids": model_ids, + "instance_ids": all_instance_ids, + "instance_count": len(all_instance_ids), + "runner_count": total_runners, + "sharding": sharding, + "instance_meta": instance_meta, + }, + "configuration": { + "stages": [ + { + "name": stage.name, + "prompt_length": stage.prompt_length, + "generation_length": stage.generation_length, + "time_between_requests": stage.time_between_requests, + "iterations": stage.iterations, + } + for stage in stages + ] + }, + "results": { + "stages": [ + { + "name": r.name, + "total_requests": r.total_requests, + "successful_requests": r.successful_requests, + "failed_requests": r.failed_requests, + "success_rate": round(r.success_rate, 4), + "total_tokens": r.total_tokens, + "avg_tokens_per_request": round( + r.avg_tokens_per_request, 2 + ), + "avg_time_per_request": round(r.avg_time_per_request, 3), + "avg_time_to_first_token": round( + r.avg_time_to_first_token, 3 + ) + if r.avg_time_to_first_token is not None + else None, + "std_time_to_first_token": round( + r.std_time_to_first_token, 3 + ) + if r.std_time_to_first_token is not None + else None, + "avg_decode_tps": round(r.avg_decode_tps, 2) + if r.avg_decode_tps is not None + else None, + "avg_ms_per_token": round(r.avg_ms_per_token, 2) + if r.avg_ms_per_token is not None + else None, + "std_ms_per_token": round(r.std_ms_per_token, 2) + if r.std_ms_per_token is not None + else None, + "stage_started_at": r.stage_started_at, + "stage_completed_at": r.stage_completed_at, + "stage_duration_s": r.stage_completed_at + - r.stage_started_at, + "requests": [ + { + "request_id": req.request_id, + "success": req.success, + "tokens": req.tokens, + "elapsed_s": round(req.elapsed_s, 3), + "started_at": req.started_at, + "completed_at": req.completed_at, + "time_to_first_token_s": round( + req.time_to_first_token_s, 3 + ) + if req.time_to_first_token_s is not None + else None, + "decode_tps": round(req.decode_tps, 2) + if req.decode_tps is not None + else None, + "error": req.error, + } + for req in r.request_results + ], + } + for r in stage_results + ] + }, + "metrics": { + "snapshots": [ + { + "timestamp": snapshot.timestamp, + "node_memory": { + node_id: { + "ram_total_bytes": mem.ram_total_bytes, + "ram_available_bytes": mem.ram_available_bytes, + "ram_used_bytes": mem.ram_used_bytes, + "swap_total_bytes": mem.swap_total_bytes, + "swap_available_bytes": mem.swap_available_bytes, + "swap_used_bytes": mem.swap_used_bytes, + } + for node_id, mem in snapshot.node_memory.items() + }, + "instance_tasks": [ + { + "instance_id": inst.instance_id, + "node_id": inst.node_id, + "pending_tasks": inst.pending_tasks, + "running_tasks": inst.running_tasks, + "total_active_tasks": inst.total_active_tasks, + } + for inst in snapshot.instance_tasks + ], + "node_tasks": [ + { + "node_id": node.node_id, + "pending_tasks": node.pending_tasks, + "running_tasks": node.running_tasks, + "total_active_tasks": node.total_active_tasks, + "instance_count": node.instance_count, + } + for node in snapshot.node_tasks + ], + } + for snapshot in metrics_snapshots + ] + }, + } + + # Output JSON summary + print("\n" + "=" * 80) + print("JSON RESULTS") + print("=" * 80) + print(json.dumps(results_doc, indent=2)) + print("=" * 80) + + # Save to file if path provided + if results_output_path: + print(f"Saving results to: {results_output_path}") + with open(results_output_path, "w") as f: + json.dump(results_doc, f, indent=2) + print("Results saved successfully") + + # Cleanup all instances + for instance_id in all_instance_ids: + print(f"[PRIMARY] Cleaning up instance: {instance_id}") + await _http_request_async( + f"{api_base}/instance/{instance_id}", method="DELETE" + ) + print(f"[PRIMARY] Instance {instance_id} deleted successfully") + else: + print( + "[SECONDARY] Waiting with cluster (primary handles benchmark execution)" + ) + # Secondary nodes wait until all instances of all models are deleted + for model_id in model_counts: + await wait_for_all_instances_deleted(api_base, model_id) + + return 0 + + except TimeoutError as e: + print("=" * 80) + print(f"TIMEOUT ERROR: {e}") + print("=" * 80) + return 1 + except Exception as e: + print("=" * 80) + print(f"ERROR: {e}") + import traceback + + traceback.print_exc() + print("=" * 80) + return 1 + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Run unified benchmark for EXO (single or multi-stage)" + ) + parser.add_argument("--api-port", type=int, required=True) + parser.add_argument( + "--config", type=Path, required=True, help="Path to YAML config file" + ) + parser.add_argument( + "--expected-nodes", + type=int, + required=True, + help="Total number of nodes expected in the cluster", + ) + parser.add_argument( + "--is-primary", type=str, choices=["true", "false"], required=True + ) + parser.add_argument("--timeout-seconds", type=int, default=1800) + parser.add_argument( + "--output", type=Path, help="Path to save detailed results JSON" + ) + parser.add_argument("--git-commit", type=str, help="Git commit hash for metadata") + parser.add_argument( + "--hardware-labels", type=str, help="Comma-separated hardware labels" + ) + args = parser.parse_args() + + api_base = f"http://localhost:{args.api_port}" + is_primary = args.is_primary.lower() == "true" + hardware_labels = args.hardware_labels.split(",") if args.hardware_labels else None + + return asyncio.run( + run_benchmark( + api_base, + args.config, + args.expected_nodes, + is_primary, + args.timeout_seconds, + results_output_path=args.output, + git_commit=args.git_commit, + hardware_labels=hardware_labels, + ) + ) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/scripts/build_matrix.py b/.github/scripts/build_matrix.py new file mode 100644 index 00000000..2f139350 --- /dev/null +++ b/.github/scripts/build_matrix.py @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 +import json +import os +from typing import NotRequired, TypedDict, cast + +import yaml + + +class MatrixEntry(TypedDict): + label: str + index: int + + +class MatrixInclude(TypedDict): + label: str + index: int + is_primary: bool + expected_nodes: int + + +class Config(TypedDict): + hardware_plan: dict[str, int] + timeout_seconds: NotRequired[int] + environment: NotRequired[dict[str, str]] + + +# Read the config file +config_file: str = os.environ["CONFIG_FILE"] +with open(config_file, "r") as f: + config: Config = cast(Config, yaml.safe_load(f)) + +# Extract hardware plan from config +plan: dict[str, int] = config["hardware_plan"] +if not plan: + raise ValueError(f"No hardware_plan found in {config_file}") + +# Build matrix entries +entries: list[MatrixEntry] = [] +for label, count in plan.items(): + for idx in range(count): + entries.append({"label": label, "index": idx}) + +total_nodes: int = len(entries) +matrix: dict[str, list[MatrixInclude]] = { + "include": [ + { + "label": e["label"], + "index": e["index"], + "is_primary": (i == 0), + "expected_nodes": total_nodes, + } + for i, e in enumerate(entries) + ] +} + +# Extract other config values +timeout_seconds: int = config.get("timeout_seconds", 600) +environment: dict[str, str] = config.get("environment", {}) + +# Output to GitHub Actions +with open(os.environ["GITHUB_OUTPUT"], "a") as f: + f.write(f"matrix={json.dumps(matrix)}\n") + f.write(f"config_file={config_file}\n") + f.write(f"timeout_seconds={timeout_seconds}\n") + f.write(f"environment={json.dumps(environment)}\n") + +print(f"Matrix: {json.dumps(matrix)}") +print(f"Config file: {config_file}") +print(f"Timeout: {timeout_seconds}") +print(f"Environment: {json.dumps(environment)}") diff --git a/.github/workflows/BENCH_USAGE.md b/.github/workflows/BENCH_USAGE.md new file mode 100644 index 00000000..b61d31da --- /dev/null +++ b/.github/workflows/BENCH_USAGE.md @@ -0,0 +1,156 @@ +# Benchmark Workflow Usage + +## Overview + +The `bench_matrix.yml` workflow enables distributed benchmarking of models across multiple self-hosted macOS runners with different hardware configurations. + +## Workflow Inputs + +| Input | Description | Default | Required | +|-------|-------------|---------|----------| +| `model_id` | Model ID to benchmark | `mlx-community/Llama-3.2-1B-Instruct-4bit` | Yes | +| `hardware_plan` | JSON mapping of runner labels to counts | `{"M4PRO_GPU16_24GB": 1}` | Yes | +| `prompt` | Benchmark prompt text | `What is the capital of France?` | No | +| `timeout_seconds` | Timeout for instance/runner readiness | `600` | No | + +## Hardware Plan Format + +The `hardware_plan` input is a JSON object mapping runner labels to the number of machines: + +```json +{ + "M4PRO_GPU16_24GB": 2, + "M3ULTRA_GPU80_512GB": 1 +} +``` + +This example would: +- Start 2 runners with the `M4PRO_GPU16_24GB` label +- Start 1 runner with the `M3ULTRA_GPU80_512GB` label +- Total of 3 runners coordinating on a single distributed inference instance + +## How It Works + +1. **Planning Job** (`plan`) + - Runs on `ubuntu-latest` + - Parses the `hardware_plan` JSON + - Generates a dynamic matrix with one entry per runner + - Only the first runner (index 0) is marked as `is_primary` + +2. **Benchmark Worker Jobs** (`bench_worker`) + - Each job runs on a self-hosted macOS runner with the specified label + - All runners start EXO in parallel + - The primary runner creates the model instance + - All runners wait for their assigned runner to be ready (Loaded/Running status) + - The primary runner executes the benchmark and prints results + - The primary runner deletes the instance + +## Example Usage + +### Single Machine Benchmark + +```yaml +model_id: mlx-community/Llama-3.2-1B-Instruct-4bit +hardware_plan: '{"M4PRO_GPU16_24GB": 1}' +prompt: What is the capital of France? +timeout_seconds: 600 +``` + +### Multi-Machine Distributed Benchmark + +```yaml +model_id: mlx-community/Llama-3.2-3B-Instruct-4bit +hardware_plan: '{"M4PRO_GPU16_24GB": 2, "M3ULTRA_GPU80_512GB": 1}' +prompt: Explain quantum computing in simple terms. +timeout_seconds: 900 +``` + +## Benchmark Output + +The primary runner outputs a JSON object with benchmark results: + +```json +{ + "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit", + "instance_id": "abc-123-def", + "tokens": 42, + "elapsed_s": 2.451, + "tps": 17.136 +} +``` + +Where: +- `tokens`: Number of chunks/tokens generated +- `elapsed_s`: Total elapsed time in seconds +- `tps`: Tokens per second (tokens / elapsed_s) + +## Runner Requirements + +Each self-hosted runner must: +- Be labeled with appropriate hardware tags (e.g., `M4PRO_GPU16_24GB`) +- Have the `self-hosted` and `macOS` labels +- Have Nix installed with flakes enabled +- Have network connectivity to other runners in the same job + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ GitHub Actions Workflow (bench_matrix.yml) │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌────────────────┐ │ +│ │ Plan Job │ │ +│ │ (ubuntu) │──┬─► Matrix: [{label, index, primary}] │ +│ └────────────────┘ │ │ +│ │ │ +│ ┌───────────────────▼──────────────────────────────────┐ │ +│ │ Bench Worker Jobs (Matrix) │ │ +│ ├──────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ Runner 0 (Primary) Runner 1 Runner 2 │ │ +│ │ ┌─────────────┐ ┌─────────────┐ ┌──────────┐ │ │ +│ │ │ Start EXO │ │ Start EXO │ │ Start EXO│ │ │ +│ │ │ Create Inst │ │ Wait... │ │ Wait... │ │ │ +│ │ │ Wait Ready │ │ Wait Ready │ │ Wait... │ │ │ +│ │ │ Run Bench │ │ (idle) │ │ (idle) │ │ │ +│ │ │ Print TPS │ │ │ │ │ │ │ +│ │ │ Delete Inst │ │ │ │ │ │ │ +│ │ └─────────────┘ └─────────────┘ └──────────┘ │ │ +│ └───────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Implementation Details + +### `scripts/bench.py` + +A standalone Python script that: +- Creates instance (primary only) +- Polls `/state` endpoint until instance and all runners are ready +- Executes chat completion with timing (primary only) +- Parses SSE stream and counts tokens +- Computes TPS metrics +- Cleans up instance (primary only) + +### Key Functions + +- `wait_for_instance()`: Polls until instance with model_id appears +- `wait_for_runners_ready()`: Polls until expected number of runners reach Loaded/Running status +- `run_benchmark()`: Executes chat completion, measures time, counts tokens + +## Troubleshooting + +### Instance never becomes ready +- Check EXO logs in the workflow output +- Verify model_id is valid and accessible +- Increase `timeout_seconds` + +### Runner mismatch +- Ensure hardware_plan counts match available labeled runners +- Check runner labels match exactly (case-sensitive) + +### Network issues +- Verify runners can communicate on the network +- Check firewall rules between runner hosts + diff --git a/.github/workflows/bench.yml b/.github/workflows/bench.yml new file mode 100644 index 00000000..dda16435 --- /dev/null +++ b/.github/workflows/bench.yml @@ -0,0 +1,305 @@ +name: bench + +on: [push] + +jobs: + plan: + if: contains(github.event.head_commit.message, '/bench') + runs-on: ubuntu-latest + outputs: + matrix: ${{ steps.build.outputs.matrix }} + config_file: ${{ steps.build.outputs.config_file }} + timeout_seconds: ${{ steps.build.outputs.timeout_seconds }} + environment: ${{ steps.build.outputs.environment }} + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Build matrix from config file + id: build + shell: bash + run: | + set -euo pipefail + CONFIG_FILE='.github/configs/bench_simple.yaml' + export CONFIG_FILE + echo "Config file: $CONFIG_FILE" + python3 .github/scripts/build_matrix.py + + bench_worker: + needs: plan + strategy: + fail-fast: false + matrix: ${{ fromJSON(needs.plan.outputs.matrix) }} + name: "bench on ${{ matrix.label }} [${{ matrix.index }}]" + runs-on: [self-hosted, macOS, "${{ matrix.label }}"] + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + lfs: false + + - name: Configure git user + run: | + git config --local user.email "github-actions@users.noreply.github.com" + git config --local user.name "github-actions bot" + shell: bash + + # TODO: this is mega hacky and I'd like a simpler solution. + - name: Setup Nix Environment + run: | + echo "Checking for nix installation..." + + # Check if nix is already available + if command -v nix >/dev/null 2>&1; then + echo "Nix already in PATH" + # Try sourcing profile scripts to set up environment properly + elif [ -f /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh ]; then + echo "Sourcing multi-user nix-daemon profile script" + source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh + elif [ -f "$HOME/.nix-profile/etc/profile.d/nix.sh" ]; then + echo "Sourcing single-user nix profile script" + source "$HOME/.nix-profile/etc/profile.d/nix.sh" + elif [ -f /nix/var/nix/profiles/per-user/$USER/profile/etc/profile.d/nix.sh ]; then + echo "Sourcing per-user nix profile script" + source /nix/var/nix/profiles/per-user/$USER/profile/etc/profile.d/nix.sh + elif [ -f /etc/profile.d/nix.sh ]; then + echo "Sourcing system-wide nix profile script" + source /etc/profile.d/nix.sh + # Fallback: manually add nix to PATH if binary exists + elif [ -f /nix/var/nix/profiles/default/bin/nix ]; then + echo "Found nix binary, manually adding to PATH" + export PATH="/nix/var/nix/profiles/default/bin:$PATH" + elif [ -f "$HOME/.nix-profile/bin/nix" ]; then + echo "Found nix binary in user profile, manually adding to PATH" + export PATH="$HOME/.nix-profile/bin:$PATH" + else + echo "Nix not found. Debugging info:" + echo "USER: $USER" + echo "HOME: $HOME" + echo "Current PATH: $PATH" + echo "" + echo "Checking common Nix locations:" + echo " /nix/var/nix/profiles/default/bin/nix:" + ls -la /nix/var/nix/profiles/default/bin/nix 2>/dev/null || echo " Not found" + echo " /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh:" + ls -la /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh 2>/dev/null || echo " Not found" + echo " ~/.nix-profile/etc/profile.d/nix.sh:" + ls -la "$HOME/.nix-profile/etc/profile.d/nix.sh" 2>/dev/null || echo " Not found" + echo " /nix/var/nix/profiles/per-user/$USER/profile/etc/profile.d/nix.sh:" + ls -la "/nix/var/nix/profiles/per-user/$USER/profile/etc/profile.d/nix.sh" 2>/dev/null || echo " Not found" + echo "" + echo "/nix directory structure:" + ls -la /nix 2>/dev/null || echo " /nix directory not found" + echo "" + echo "/nix/var:" + ls -la /nix/var 2>/dev/null || echo " /nix/var not found" + echo "" + echo "/nix/store:" + ls -la /nix/store 2>/dev/null | head -20 || echo " /nix/store not found" + echo "" + echo "GitHub Actions runner is running as user '$USER'." + echo "If Nix is installed for a different user, either:" + echo " 1. Install Nix for user '$USER' (multi-user install recommended)" + echo " 2. Configure the runner service to run as the user with Nix installed" + echo " 3. Ensure Nix is installed system-wide with proper daemon setup" + exit 1 + fi + + # Verify nix is available and persist to GITHUB_ENV + if command -v nix >/dev/null 2>&1; then + echo "✓ Nix is available" + nix --version + echo "PATH=$PATH" >> $GITHUB_ENV + if [ -n "$NIX_PATH" ]; then + echo "NIX_PATH=$NIX_PATH" >> $GITHUB_ENV + fi + else + echo "ERROR: Failed to set up Nix" + echo "PATH after setup attempt: $PATH" + exit 1 + fi + shell: bash + + - name: Setup EXO_HOME and API_PORT + run: | + EXO_HOME=$(mktemp -d -t exo-e2e-XXXXXXXX) + API_PORT=$((49152 + RANDOM % (65535 - 49152 + 1))) + EXO_MODELS_DIR="$HOME/.exo/models" + EXO_LIBP2P_NAMESPACE="bench-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" + echo "EXO_HOME=$EXO_HOME" >> "$GITHUB_ENV" + echo "API_PORT=$API_PORT" >> "$GITHUB_ENV" + echo "EXO_MODELS_DIR=$EXO_MODELS_DIR" >> "$GITHUB_ENV" + echo "EXO_LIBP2P_NAMESPACE=$EXO_LIBP2P_NAMESPACE" >> "$GITHUB_ENV" + echo "Created EXO_HOME: $EXO_HOME" + echo "Generated API_PORT: $API_PORT" + echo "Using models from: $EXO_MODELS_DIR" + echo "Using libp2p namespace: $EXO_LIBP2P_NAMESPACE" + shell: bash + + - name: Configure local MLX if available + run: | + echo "=== DEBUG: Checking for local MLX configuration ===" + MODIFIED=false + + echo "Checking for /Users/Shared/mlx directory..." + if [ -d "/Users/Shared/mlx" ]; then + echo "✓ Found /Users/Shared/mlx" + ls -la /Users/Shared/mlx | head -5 + echo "Enabling local mlx path in pyproject.toml" + sed -i.bak 's|^# mlx = { path = "/Users/Shared/mlx", editable=true }$|mlx = { path = "/Users/Shared/mlx", editable=true }|' pyproject.toml + MODIFIED=true + else + echo "✗ /Users/Shared/mlx not found, will use PyPI version" + fi + + echo "Checking for /Users/Shared/mlx-lm directory..." + if [ -d "/Users/Shared/mlx-lm" ]; then + echo "✓ Found /Users/Shared/mlx-lm" + ls -la /Users/Shared/mlx-lm | head -5 + echo "Enabling local mlx-lm path in pyproject.toml" + sed -i.bak 's|^# mlx-lm = { path = "/Users/Shared/mlx-lm", editable=true }$|mlx-lm = { path = "/Users/Shared/mlx-lm", editable=true }|' pyproject.toml + MODIFIED=true + else + echo "✗ /Users/Shared/mlx-lm not found, will use PyPI version" + fi + + if [ "$MODIFIED" = true ]; then + echo "=== Modified pyproject.toml [tool.uv.sources] section: ===" + sed -n '/\[tool\.uv\.sources\]/,/^\[/{/^\[tool\.uv\.sources\]/p; /^\[/!p;}' pyproject.toml + echo "=== Regenerating uv.lock with local MLX paths... ===" + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop --command uv lock --upgrade-package mlx --upgrade-package mlx-lm + echo "✓ Lock file regenerated" + else + echo "⚠ No local MLX directories found, using PyPI packages" + fi + echo "=== DEBUG: Local MLX configuration complete ===" + shell: bash + + - name: Sync dependencies + run: | + if [ -d "/Users/Shared/test" ]; then + pushd /Users/Shared/test + uv sync --reinstall + popd + fi + echo "Running just sync to ensure clean dependencies..." + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop --command just sync + shell: bash + + - name: Start EXO and run bench script + shell: bash + env: + IS_PRIMARY: ${{ matrix.is_primary }} + EXPECTED_NODES: ${{ matrix.expected_nodes }} + HARDWARE_LABEL: ${{ matrix.label }} + CONFIG_FILE: ${{ needs.plan.outputs.config_file }} + TIMEOUT_SECONDS: ${{ needs.plan.outputs.timeout_seconds }} + ENVIRONMENT_JSON: ${{ needs.plan.outputs.environment }} + run: | + set -euo pipefail + + # Parse environment variables from config + ENV_VARS="" + if [ -n "$ENVIRONMENT_JSON" ] && [ "$ENVIRONMENT_JSON" != "{}" ]; then + ENV_VARS=$(echo "$ENVIRONMENT_JSON" | python3 -c "import sys, json; env = json.load(sys.stdin); print(' '.join([f'{k}={v}' for k, v in env.items()]))") + fi + + echo "Starting EXO with API_PORT=${API_PORT} EXO_HOME=${EXO_HOME} EXO_LIBP2P_NAMESPACE=${EXO_LIBP2P_NAMESPACE}" + echo "Environment variables from config: $ENV_VARS" + LOG_FILE=/tmp/exo.log + : > "$LOG_FILE" + + MASTER_FLAG="" + if [ "$IS_PRIMARY" = "true" ]; then + MASTER_FLAG="-m" + fi + + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop --command bash -c \ + "EXO_HOME=$EXO_HOME EXO_MODELS_DIR=$EXO_MODELS_DIR EXO_LIBP2P_NAMESPACE=$EXO_LIBP2P_NAMESPACE $ENV_VARS PYTHONUNBUFFERED=1 PYTHONDEBUG=1 PYTHONPATH=. uv run exo $MASTER_FLAG --api-port $API_PORT" \ + >> "$LOG_FILE" 2>&1 & + + EXO_PID=$! + echo "Started EXO in background with PID: $EXO_PID" + echo "Log file: $LOG_FILE" + + cleanup() { + echo '=== EXO log (tail) ===' + tail -n 300 "$LOG_FILE" || true + if ps -p "$EXO_PID" >/dev/null 2>&1; then + echo "Killing EXO (PID $EXO_PID)" + kill "$EXO_PID" || true + fi + } + trap cleanup EXIT + + for i in $(seq 1 60); do + if curl -s "http://localhost:${API_PORT}/state" >/dev/null 2>&1; then + echo "EXO API ready" + break + fi + if ! ps -p "$EXO_PID" >/dev/null 2>&1; then + echo "EXO terminated early"; sed -n '1,200p' "$LOG_FILE" || true; exit 1 + fi + sleep 1 + done + + RESULTS_FILE="/tmp/bench_results_${GITHUB_RUN_ID}_${GITHUB_RUN_ATTEMPT}_$(date +%s).json" + echo "Results will be saved to: $RESULTS_FILE" + echo "RESULTS_FILE=$RESULTS_FILE" >> "$GITHUB_ENV" + + echo "Running bench script with config: $CONFIG_FILE, timeout: $TIMEOUT_SECONDS" + nix --extra-experimental-features nix-command --extra-experimental-features flakes develop --command bash -c \ + "PYTHONUNBUFFERED=1 uv run --no-project --with pyyaml --with pydantic python .github/scripts/bench.py \ + --api-port $API_PORT \ + --config $CONFIG_FILE \ + --expected-nodes ${EXPECTED_NODES} \ + --is-primary ${IS_PRIMARY} \ + --timeout-seconds ${TIMEOUT_SECONDS} \ + --output $RESULTS_FILE \ + --git-commit ${GITHUB_SHA} \ + --hardware-labels ${HARDWARE_LABEL}" + + - name: Install AWS CLI + if: always() && env.RESULTS_FILE && matrix.is_primary + run: | + if ! command -v aws &> /dev/null; then + echo "AWS CLI not found, installing..." + brew install awscli + else + echo "AWS CLI already installed" + fi + shell: bash + + - name: Upload results to S3 + if: always() && env.RESULTS_FILE && matrix.is_primary + env: + AWS_ACCESS_KEY_ID: ${{ secrets.S3_BENCHMARKS_AWS_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_BENCHMARKS_AWS_SECRET_ACCESS_KEY }} + AWS_DEFAULT_REGION: us-east-1 + run: | + echo "Checking for results file: $RESULTS_FILE" + echo "Is primary: ${{ matrix.is_primary }}" + + if [ -f "$RESULTS_FILE" ]; then + TIMESTAMP=$(date -u +%Y/%m/%d/%H%M%S) + S3_KEY="bench/${TIMESTAMP}_${GITHUB_SHA:0:8}_${GITHUB_RUN_ID}.json" + echo "Uploading results to s3://exo-benchmark-results/$S3_KEY" + + aws s3 cp "$RESULTS_FILE" "s3://exo-benchmark-results/$S3_KEY" \ + --content-type application/json \ + --metadata "commit=${GITHUB_SHA},run_id=${GITHUB_RUN_ID},branch=${GITHUB_REF_NAME}" + + echo "Results uploaded successfully" + echo "View at: https://exo-benchmark-results.s3.amazonaws.com/$S3_KEY" + else + echo "Results file not found at: $RESULTS_FILE" + echo "Skipping upload" + fi + shell: bash + + - name: Cleanup EXO_HOME + run: | + echo "Cleaning up EXO_HOME: $EXO_HOME" + rm -rf "$EXO_HOME" + shell: bash + if: always() diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml new file mode 100644 index 00000000..e78c3198 --- /dev/null +++ b/.github/workflows/pipeline.yml @@ -0,0 +1,183 @@ +name: ci-pipeline + +on: + push: + pull_request: + branches: + - staging + - main + +jobs: + typecheck: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + lfs: false + + - uses: cachix/install-nix-action@v31 + with: + nix_path: nixpkgs=channel:nixos-unstable + + - name: Configure git user + run: | + git config --local user.email "github-actions@users.noreply.github.com" + git config --local user.name "github-actions bot" + shell: bash + + - name: Pull LFS files + run: | + echo "Pulling Git LFS files..." + git lfs pull + shell: bash + + - name: Setup Nix Environment + run: | + echo "Checking for nix installation..." + + # Check if nix binary exists directly + if [ -f /nix/var/nix/profiles/default/bin/nix ]; then + echo "Found nix binary at /nix/var/nix/profiles/default/bin/nix" + export PATH="/nix/var/nix/profiles/default/bin:$PATH" + echo "PATH=$PATH" >> $GITHUB_ENV + nix --version + elif [ -f /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh ]; then + echo "Found nix profile script, sourcing..." + source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh + nix --version + elif command -v nix >/dev/null 2>&1; then + echo "Nix already in PATH" + nix --version + else + echo "Nix not found. Debugging info:" + echo "Contents of /nix/var/nix/profiles/default/:" + ls -la /nix/var/nix/profiles/default/ 2>/dev/null || echo "Directory not found" + echo "Contents of /nix/var/nix/profiles/default/bin/:" + ls -la /nix/var/nix/profiles/default/bin/ 2>/dev/null || echo "Directory not found" + exit 1 + fi + shell: bash + + - name: Configure basedpyright include for local MLX + run: | + RUNNER_LABELS='${{ toJSON(runner.labels) }}' + if echo "$RUNNER_LABELS" | grep -q "local_mlx"; then + if [ -d "/Users/Shared/mlx" ]; then + echo "Updating [tool.basedpyright].include to use /Users/Shared/mlx" + awk ' + BEGIN { in=0 } + /^\[tool\.basedpyright\]/ { in=1; print; next } + in && /^\[/ { in=0 } # next section + in && /^[ \t]*include[ \t]*=/ { + print "include = [\"/Users/Shared/mlx\"]" + next + } + { print } + ' pyproject.toml > pyproject.toml.tmp && mv pyproject.toml.tmp pyproject.toml + + echo "New [tool.basedpyright] section:" + sed -n '/^\[tool\.basedpyright\]/,/^\[/p' pyproject.toml | sed '$d' || true + else + echo "local_mlx tag present but /Users/Shared/mlx not found; leaving pyproject unchanged." + fi + else + echo "Runner does not have 'local_mlx' tag; leaving pyproject unchanged." + fi + shell: bash + + - uses: ./.github/actions/typecheck + + nix-flake-check: + name: Check Nix flake + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + lfs: false + + - uses: cachix/install-nix-action@v31 + with: + nix_path: nixpkgs=channel:nixos-unstable + + - name: Run nix flake check + run: | + nix flake check + shell: bash + +# ci: +# needs: typecheck +# runs-on: ubuntu-latest +# permissions: +# contents: read +# env: +# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} +# steps: +# - name: Checkout repository +# uses: actions/checkout@v4 +# with: +# fetch-depth: 0 +# token: ${{ secrets.GITHUB_TOKEN }} +# lfs: true +# +# - name: Configure git user +# run: | +# git config --local user.email "github-actions@users.noreply.github.com" +# git config --local user.name "github-actions bot" +# shell: bash +# +# - name: Pull LFS files +# run: | +# echo "Pulling Git LFS files..." +# git lfs pull +# shell: bash +# +# - name: Setup EXO_HOME and API_PORT +# run: | +# EXO_HOME=$(mktemp -d -t exo-ci-XXXXXXXX) +# # Generate random port (macOS compatible method) +# API_PORT=$((49152 + RANDOM % (65535 - 49152 + 1))) +# echo "EXO_HOME=$EXO_HOME" >> $GITHUB_ENV +# echo "API_PORT=$API_PORT" >> $GITHUB_ENV +# echo "Created EXO_HOME: $EXO_HOME" +# echo "Generated API_PORT: $API_PORT" +# shell: bash +# +# - name: Setup Nix Environment +# run: | +# echo "Checking for nix installation..." +# +# # Check if nix binary exists directly +# if [ -f /nix/var/nix/profiles/default/bin/nix ]; then +# echo "Found nix binary at /nix/var/nix/profiles/default/bin/nix" +# export PATH="/nix/var/nix/profiles/default/bin:$PATH" +# echo "PATH=$PATH" >> $GITHUB_ENV +# nix --version +# elif [ -f /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh ]; then +# echo "Found nix profile script, sourcing..." +# source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh +# nix --version +# elif command -v nix >/dev/null 2>&1; then +# echo "Nix already in PATH" +# nix --version +# else +# echo "Nix not found. Debugging info:" +# echo "Contents of /nix/var/nix/profiles/default/:" +# ls -la /nix/var/nix/profiles/default/ 2>/dev/null || echo "Directory not found" +# echo "Contents of /nix/var/nix/profiles/default/bin/:" +# ls -la /nix/var/nix/profiles/default/bin/ 2>/dev/null || echo "Directory not found" +# exit 1 +# fi +# shell: bash +# +# - uses: ./.github/actions/lint-check +# +# - uses: ./.github/actions/unit-test +# +# - name: Cleanup EXO_HOME +# run: | +# echo "Cleaning up EXO_HOME: $EXO_HOME" +# rm -rf "$EXO_HOME" +# shell: bash +# if: always() diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..befc8b3b --- /dev/null +++ b/.gitignore @@ -0,0 +1,24 @@ +# gitingest +digest.txt + +# python +**/__pycache__ + +# nix +.direnv/ + + +# xcode / macos +*.xcuserstate +**/.DS_Store + + +# rust +target/ +**/*.rs.bk +*.pdb + +# svelte +dashboard/build/ +dashboard/node_modules/ +dashboard/.svelte-kit/ diff --git a/.idea/.gitignore b/.idea/.gitignore new file mode 100644 index 00000000..5ddb3d79 --- /dev/null +++ b/.idea/.gitignore @@ -0,0 +1,9 @@ +# Default ignored files +/shelf/ +/workspace.xml +# Editor-based HTTP Client requests +/httpRequests/ +# Datasource local storage ignored files +/dataSources/ +/dataSources.local.xml +workspace.xml \ No newline at end of file diff --git a/.idea/LanguageServersSettings.xml b/.idea/LanguageServersSettings.xml new file mode 100644 index 00000000..7d92ce2f --- /dev/null +++ b/.idea/LanguageServersSettings.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/.idea/exo-v2.iml b/.idea/exo-v2.iml new file mode 100644 index 00000000..aa638174 --- /dev/null +++ b/.idea/exo-v2.iml @@ -0,0 +1,31 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/.idea/externalDependencies.xml b/.idea/externalDependencies.xml new file mode 100644 index 00000000..60785b21 --- /dev/null +++ b/.idea/externalDependencies.xml @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/.idea/inspectionProfiles/Project_Default.xml b/.idea/inspectionProfiles/Project_Default.xml new file mode 100644 index 00000000..12df2a84 --- /dev/null +++ b/.idea/inspectionProfiles/Project_Default.xml @@ -0,0 +1,14 @@ + + + + \ No newline at end of file diff --git a/.idea/misc.xml b/.idea/misc.xml new file mode 100644 index 00000000..4c4cf56c --- /dev/null +++ b/.idea/misc.xml @@ -0,0 +1,10 @@ + + + + + + + + \ No newline at end of file diff --git a/.idea/modules.xml b/.idea/modules.xml new file mode 100644 index 00000000..0ccec085 --- /dev/null +++ b/.idea/modules.xml @@ -0,0 +1,8 @@ + + + + + + + + \ No newline at end of file diff --git a/.idea/pyright-overrides.xml b/.idea/pyright-overrides.xml new file mode 100644 index 00000000..9216c0c4 --- /dev/null +++ b/.idea/pyright-overrides.xml @@ -0,0 +1,18 @@ + + + + + + \ No newline at end of file diff --git a/.idea/pyright.xml b/.idea/pyright.xml new file mode 100644 index 00000000..9f3391a8 --- /dev/null +++ b/.idea/pyright.xml @@ -0,0 +1,9 @@ + + + + + \ No newline at end of file diff --git a/.idea/vcs.xml b/.idea/vcs.xml new file mode 100644 index 00000000..35eb1ddf --- /dev/null +++ b/.idea/vcs.xml @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/.mlx_typings/.gitkeep b/.mlx_typings/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/.mlx_typings/mlx/core/__init__.pyi b/.mlx_typings/mlx/core/__init__.pyi new file mode 100644 index 00000000..48680a80 --- /dev/null +++ b/.mlx_typings/mlx/core/__init__.pyi @@ -0,0 +1,5420 @@ +import enum +import pathlib +import types +from typing import ( + Annotated, + Callable, + Literal, + Mapping, + Sequence, + TypeAlias, + overload, +) + +import numpy +from mlx.nn.layers import Module +from numpy.typing import ArrayLike as _ArrayLike + +from . import cuda as cuda +from . import distributed as distributed +from . import metal as metal +from . import random as random + +class ArrayAt: + """A helper object to apply updates at specific indices.""" + def __getitem__(self, indices: object | None) -> ArrayAt: ... + def add( + self, + value: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def subtract( + self, + value: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def multiply( + self, + value: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def divide( + self, + value: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def maximum( + self, + value: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def minimum( + self, + value: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + +class ArrayIterator: + """A helper object to iterate over the 1st dimension of an array.""" + def __next__(self) -> array: ... + def __iter__(self) -> ArrayIterator: ... + +class ArrayLike: + """ + Any Python object which has an ``__mlx__array__`` method that + returns an :obj:`array`. + """ + def __init__(self, arg: object, /) -> None: ... + +class Device: + """A device to run operations on.""" + def __init__(self, type: DeviceType, index: int = ...) -> None: ... + @property + def type(self) -> DeviceType: ... + def __repr__(self) -> str: ... + def __eq__(self, arg: object, /) -> bool: ... + +class DeviceType(enum.Enum): + cpu = ... # type: ignore + gpu = ... #  type: ignore + def __eq__(self, arg: object, /) -> bool: ... + +class Dtype: + """ + An object to hold the type of a :class:`array`. + + See the :ref:`list of types ` for more details + on available data types. + """ + @property + def size(self) -> int: + """Size of the type in bytes.""" + + def __repr__(self) -> str: ... + def __eq__(self, arg: object, /) -> bool: ... + def __hash__(self) -> int: ... + +class DtypeCategory(enum.Enum): + """ + Type to hold categories of :class:`dtypes `. + + * :attr:`~mlx.core.generic` + + * :ref:`bool_ ` + * :attr:`~mlx.core.number` + + * :attr:`~mlx.core.integer` + + * :attr:`~mlx.core.unsignedinteger` + + * :ref:`uint8 ` + * :ref:`uint16 ` + * :ref:`uint32 ` + * :ref:`uint64 ` + + * :attr:`~mlx.core.signedinteger` + + * :ref:`int8 ` + * :ref:`int32 ` + * :ref:`int64 ` + + * :attr:`~mlx.core.inexact` + + * :attr:`~mlx.core.floating` + + * :ref:`float16 ` + * :ref:`bfloat16 ` + * :ref:`float32 ` + * :ref:`float64 ` + + * :attr:`~mlx.core.complexfloating` + + * :ref:`complex64 ` + + See also :func:`~mlx.core.issubdtype`. + """ + + complexfloating = ... + floating = ... + inexact = ... + signedinteger = ... + unsignedinteger = ... + integer = ... + number = ... + generic = ... + +class FunctionExporter: + """ + A context managing class for exporting multiple traces of the same + function to a file. + + Make an instance of this class by calling fun:`mx.exporter`. + """ + def close(self) -> None: ... + def __enter__(self) -> FunctionExporter: ... + def __exit__( + self, + exc_type: object | None = ..., + exc_value: object | None = ..., + traceback: object | None = ..., + ) -> None: ... + def __call__(self, *args, **kwargs) -> None: ... + +class Stream: + """A stream for running operations on a given device.""" + @property + def device(self) -> Device: ... + def __repr__(self) -> str: ... + def __eq__(self, arg: object, /) -> bool: ... + +class StreamContext: + """ + A context manager for setting the current device and stream. + + See :func:`stream` for usage. + + Args: + s: The stream or device to set as the default. + """ + def __init__(self, s: Stream | Device) -> None: ... + def __enter__(self) -> None: ... + def __exit__( + self, + exc_type: type | None = ..., + exc_value: object | None = ..., + traceback: object | None = ..., + ) -> None: ... + +def abs(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise absolute value. + + Args: + a (array): Input array. + + Returns: + array: The absolute value of ``a``. + """ + +def add( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise addition. + + Add two arrays with numpy-style broadcasting semantics. Either or both input arrays + can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The sum of ``a`` and ``b``. + """ + +def addmm( + c: array, + a: array, + b: array, + /, + alpha: float = ..., + beta: float = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Matrix multiplication with addition and optional scaling. + + Perform the (possibly batched) matrix multiplication of two arrays and add to the result + with optional scaling factors. + + Args: + c (array): Input array or scalar. + a (array): Input array or scalar. + b (array): Input array or scalar. + alpha (float, optional): Scaling factor for the + matrix product of ``a`` and ``b`` (default: ``1``) + beta (float, optional): Scaling factor for ``c`` (default: ``1``) + + Returns: + array: ``alpha * (a @ b) + beta * c`` + """ + +def all( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + An `and` reduction over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +def allclose( + a: array, + b: array, + /, + rtol: float = ..., + atol: float = ..., + *, + equal_nan: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Approximate comparison of two arrays. + + Infinite values are considered equal if they have the same sign, NaN values are not equal unless ``equal_nan`` is ``True``. + + The arrays are considered equal if: + + .. code-block:: + + all(abs(a - b) <= (atol + rtol * abs(b))) + + Note unlike :func:`array_equal`, this function supports numpy-style + broadcasting. + + Args: + a (array): Input array. + b (array): Input array. + rtol (float): Relative tolerance. + atol (float): Absolute tolerance. + equal_nan (bool): If ``True``, NaNs are considered equal. + Defaults to ``False``. + + Returns: + array: The boolean output scalar indicating if the arrays are close. + """ + +def any( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + An `or` reduction over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +@overload +def arange( + start: int | float, + stop: int | float, + step: int | float | None, + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Generates ranges of numbers. + + Generate numbers in the half-open interval ``[start, stop)`` in + increments of ``step``. + + Args: + start (float or int, optional): Starting value which defaults to ``0``. + stop (float or int): Stopping value. + step (float or int, optional): Increment which defaults to ``1``. + dtype (Dtype, optional): Specifies the data type of the output. If unspecified will default to ``float32`` if any of ``start``, ``stop``, or ``step`` are ``float``. Otherwise will default to ``int32``. + + Returns: + array: The range of values. + + Note: + Following the Numpy convention the actual increment used to + generate numbers is ``dtype(start + step) - dtype(start)``. + This can lead to unexpected results for example if `start + step` + is a fractional value and the `dtype` is integral. + """ + +@overload +def arange( + stop: int | float, + step: int | float | None = ..., + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: ... +def arccos(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse cosine. + + Args: + a (array): Input array. + + Returns: + array: The inverse cosine of ``a``. + """ + +def arccosh(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse hyperbolic cosine. + + Args: + a (array): Input array. + + Returns: + array: The inverse hyperbolic cosine of ``a``. + """ + +def arcsin(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse sine. + + Args: + a (array): Input array. + + Returns: + array: The inverse sine of ``a``. + """ + +def arcsinh(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse hyperbolic sine. + + Args: + a (array): Input array. + + Returns: + array: The inverse hyperbolic sine of ``a``. + """ + +def arctan(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse tangent. + + Args: + a (array): Input array. + + Returns: + array: The inverse tangent of ``a``. + """ + +def arctan2(a: array, b: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse tangent of the ratio of two arrays. + + Args: + a (array): Input array. + b (array): Input array. + + Returns: + array: The inverse tangent of the ratio of ``a`` and ``b``. + """ + +def arctanh(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse hyperbolic tangent. + + Args: + a (array): Input array. + + Returns: + array: The inverse hyperbolic tangent of ``a``. + """ + +def argmax( + a: array, + /, + axis: int | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Indices of the maximum values along the axis. + + Args: + a (array): Input array. + axis (int, optional): Optional axis to reduce over. If unspecified + this defaults to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The ``uint32`` array with the indices of the maximum values. + """ + +def argmin( + a: array, + /, + axis: int | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Indices of the minimum values along the axis. + + Args: + a (array): Input array. + axis (int, optional): Optional axis to reduce over. If unspecified + this defaults to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The ``uint32`` array with the indices of the minimum values. + """ + +def argpartition( + a: array, + /, + kth: int, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Returns the indices that partition the array. + + The ordering of the elements within a partition in given by the indices + is undefined. + + Args: + a (array): Input array. + kth (int): Element index at the ``kth`` position in the output will + give the sorted position. All indices before the ``kth`` position + will be of elements less or equal to the element at the ``kth`` + index and all indices after will be of elements greater or equal + to the element at the ``kth`` index. + axis (int or None, optional): Optional axis to partition over. + If ``None``, this partitions over the flattened array. + If unspecified, it defaults to ``-1``. + + Returns: + array: The ``uint32`` array containing indices that partition the input. + """ + +def argsort( + a: array, + /, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Returns the indices that sort the array. + + Args: + a (array): Input array. + axis (int or None, optional): Optional axis to sort over. + If ``None``, this sorts over the flattened array. + If unspecified, it defaults to -1 (sorting over the last axis). + + Returns: + array: The ``uint32`` array containing indices that sort the input. + """ + +class array: + """An N-dimensional array object.""" + def __init__( + self: array, + val: scalar | list | tuple | numpy.ndarray | array, + dtype: Dtype | None = ..., + ) -> None: ... + def __buffer__(self, flags, /): + """ + Return a buffer object that exposes the underlying memory of the object. + """ + + def __release_buffer__(self, buffer, /): + """ + Release the buffer object that exposes the underlying memory of the object. + """ + + @property + def size(self) -> int: + """Number of elements in the array.""" + + @property + def ndim(self) -> int: + """The array's dimension.""" + + @property + def itemsize(self) -> int: + """The size of the array's datatype in bytes.""" + + @property + def nbytes(self) -> int: + """The number of bytes in the array.""" + + @property + def shape(self) -> tuple[int, ...]: + """ + The shape of the array as a Python tuple. + + Returns: + tuple(int): A tuple containing the sizes of each dimension. + """ + + @property + def dtype(self) -> Dtype: + """The array's :class:`Dtype`.""" + + @property + def real(self) -> array: + """The real part of a complex array.""" + + @property + def imag(self) -> array: + """The imaginary part of a complex array.""" + + def item(self) -> scalar: + """ + Access the value of a scalar array. + + Returns: + Standard Python scalar. + """ + + def tolist(self) -> list_or_scalar: + """ + Convert the array to a Python :class:`list`. + + Returns: + list: The Python list. + + If the array is a scalar then a standard Python scalar is returned. + + If the array has more than one dimension then the result is a nested + list of lists. + + The value type of the list corresponding to the last dimension is either + ``bool``, ``int`` or ``float`` depending on the ``dtype`` of the array. + """ + + def astype(self, dtype: Dtype, stream: Stream | Device | None = ...) -> array: + """ + Cast the array to a specified type. + + Args: + dtype (Dtype): Type to which the array is cast. + stream (Stream): Stream (or device) for the operation. + + Returns: + array: The array with type ``dtype``. + """ + + def __array_namespace__(self, api_version: str | None = ...) -> types.ModuleType: + """ + Returns an object that has all the array API functions on it. + + See the `Python array API `_ + for more information. + + Args: + api_version (str, optional): String representing the version + of the array API spec to return. Default: ``None``. + + Returns: + out (Any): An object representing the array API namespace. + """ + + def __getitem__(self, arg: object | None) -> array: ... + def __setitem__( + self, + arg0: object | None, + arg1: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> None: ... + @property + def at(self) -> ArrayAt: + """ + Used to apply updates at the given indices. + + .. note:: + + Regular in-place updates map to assignment. For instance ``x[idx] += y`` + maps to ``x[idx] = x[idx] + y``. As a result, assigning to the + same index ignores all but one update. Using ``x.at[idx].add(y)`` + will correctly apply all updates to all indices. + + .. list-table:: + :header-rows: 1 + + * - array.at syntax + - In-place syntax + * - ``x = x.at[idx].add(y)`` + - ``x[idx] += y`` + * - ``x = x.at[idx].subtract(y)`` + - ``x[idx] -= y`` + * - ``x = x.at[idx].multiply(y)`` + - ``x[idx] *= y`` + * - ``x = x.at[idx].divide(y)`` + - ``x[idx] /= y`` + * - ``x = x.at[idx].maximum(y)`` + - ``x[idx] = mx.maximum(x[idx], y)`` + * - ``x = x.at[idx].minimum(y)`` + - ``x[idx] = mx.minimum(x[idx], y)`` + + Example: + >>> a = mx.array([0, 0]) + >>> idx = mx.array([0, 1, 0, 1]) + >>> a[idx] += 1 + >>> a + array([1, 1], dtype=int32) + >>> + >>> a = mx.array([0, 0]) + >>> a.at[idx].add(1) + array([2, 2], dtype=int32) + """ + + def __len__(self) -> int: ... + def __iter__(self) -> ArrayIterator: ... + def __getstate__(self) -> tuple: ... + def __setstate__(self, arg: tuple, /) -> None: ... + def __dlpack__(self) -> _ArrayLike: ... + def __dlpack_device__(self) -> tuple: ... + def __copy__(self) -> array: ... + def __deepcopy__(self, memo: dict) -> array: ... + def __add__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __iadd__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __radd__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __sub__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __isub__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rsub__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __mul__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __imul__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rmul__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __truediv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __itruediv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rtruediv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __div__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rdiv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __floordiv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ifloordiv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rfloordiv__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __mod__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __imod__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rmod__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __eq__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array | bool: ... + def __lt__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __le__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __gt__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ge__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ne__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array | bool: ... + def __neg__(self) -> array: ... + def __bool__(self) -> bool: ... + def __repr__(self) -> str: ... + def __matmul__(self, other: array) -> array: ... + def __imatmul__(self, other: array) -> array: ... + def __pow__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rpow__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ipow__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __invert__(self) -> array: ... + def __and__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __iand__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __or__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ior__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __lshift__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ilshift__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __rshift__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __irshift__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __xor__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __ixor__( + self, + other: bool + | int + | float + | array + | Annotated[_ArrayLike, dict(order="C", device="cpu", writable=False)] + | complex + | ArrayLike, + ) -> array: ... + def __int__(self) -> int: ... + def __float__(self) -> float: ... + def flatten( + self, + start_axis: int = ..., + end_axis: int = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`flatten`.""" + + def reshape(self, *shape, stream: Stream | Device | None = ...) -> array: + """ + Equivalent to :func:`reshape` but the shape can be passed either as a + :obj:`tuple` or as separate arguments. + + See :func:`reshape` for full documentation. + """ + + def squeeze( + self, + axis: int | Sequence[int] | None = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`squeeze`.""" + + def abs(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`abs`.""" + + def __abs__(self) -> array: + """See :func:`abs`.""" + + def square(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`square`.""" + + def sqrt(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`sqrt`.""" + + def rsqrt(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`rsqrt`.""" + + def reciprocal(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`reciprocal`.""" + + def exp(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`exp`.""" + + def log(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`log`.""" + + def log2(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`log2`.""" + + def log10(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`log10`.""" + + def sin(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`sin`.""" + + def cos(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`cos`.""" + + def log1p(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`log1p`.""" + + def all( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`all`.""" + + def any( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`any`.""" + + def moveaxis( + self, source: int, destination: int, *, stream: Stream | Device | None = ... + ) -> array: + """See :func:`moveaxis`.""" + + def swapaxes( + self, axis1: int, axis2: int, *, stream: Stream | Device | None = ... + ) -> array: + """See :func:`swapaxes`.""" + + def transpose(self, *axes, stream: Stream | Device | None = ...) -> array: + """ + Equivalent to :func:`transpose` but the axes can be passed either as + a tuple or as separate arguments. + + See :func:`transpose` for full documentation. + """ + + @property + def T(self) -> array: + """Equivalent to calling ``self.transpose()`` with no arguments.""" + + def sum( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`sum`.""" + + def prod( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`prod`.""" + + def min( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`min`.""" + + def max( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`max`.""" + + def logcumsumexp( + self, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`logcumsumexp`.""" + + def logsumexp( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`logsumexp`.""" + + def mean( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`mean`.""" + + def std( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + ddof: int = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`std`.""" + + def var( + self, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + ddof: int = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`var`.""" + + def split( + self, + indices_or_sections: int | tuple[int, ...], + axis: int = ..., + *, + stream: Stream | Device | None = ..., + ) -> list[array]: + """See :func:`split`.""" + + def argmin( + self, + axis: int | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`argmin`.""" + + def argmax( + self, + axis: int | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`argmax`.""" + + def cumsum( + self, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`cumsum`.""" + + def cumprod( + self, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`cumprod`.""" + + def cummax( + self, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`cummax`.""" + + def cummin( + self, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`cummin`.""" + + def round( + self, decimals: int = ..., *, stream: Stream | Device | None = ... + ) -> array: + """See :func:`round`.""" + + def diagonal( + self, + offset: int = ..., + axis1: int = ..., + axis2: int = ..., + stream: Stream | Device | None = ..., + ) -> array: + """See :func:`diagonal`.""" + + def diag(self, k: int = ..., *, stream: Stream | Device | None = ...) -> array: + """Extract a diagonal or construct a diagonal matrix.""" + + def conj(self, *, stream: Stream | Device | None = ...) -> array: + """See :func:`conj`.""" + + def view(self, dtype: Dtype, *, stream: Stream | Device | None = ...) -> array: + """See :func:`view`.""" + +def array_equal( + a: scalar | array, + b: scalar | array, + equal_nan: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Array equality check. + + Compare two arrays for equality. Returns ``True`` if and only if the arrays + have the same shape and their values are equal. The arrays need not have + the same type to be considered equal. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + equal_nan (bool): If ``True``, NaNs are considered equal. + Defaults to ``False``. + + Returns: + array: A scalar boolean array. + """ + +def as_strided( + a: array, + /, + shape: Sequence[int] | None = ..., + strides: Sequence[int] | None = ..., + offset: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Create a view into the array with the given shape and strides. + + The resulting array will always be as if the provided array was row + contiguous regardless of the provided arrays storage order and current + strides. + + .. note:: + Note that this function should be used with caution as it changes + the shape and strides of the array directly. This can lead to the + resulting array pointing to invalid memory locations which can + result into crashes. + + Args: + a (array): Input array + shape (list(int), optional): The shape of the resulting array. If + None it defaults to ``a.shape()``. + strides (list(int), optional): The strides of the resulting array. If + None it defaults to the reverse exclusive cumulative product of + ``a.shape()``. + offset (int): Skip that many elements from the beginning of the input + array. + + Returns: + array: The output array which is the strided view of the input. + """ + +def async_eval(*args: MX_ARRAY_TREE) -> None: + """ + Asynchronously evaluate an :class:`array` or tree of :class:`array`. + + .. note:: + + This is an experimental API and may change in future versions. + + Args: + *args (arrays or trees of arrays): Each argument can be a single array + or a tree of arrays. If a tree is given the nodes can be a Python + :class:`list`, :class:`tuple` or :class:`dict`. Leaves which are not + arrays are ignored. + + Example: + >>> x = mx.array(1.0) + >>> y = mx.exp(x) + >>> mx.async_eval(y) + >>> print(y) + >>> + >>> y = mx.exp(x) + >>> mx.async_eval(y) + >>> z = y + 3 + >>> mx.async_eval(z) + >>> print(z) + """ + +def atleast_1d( + *arys: array, stream: Stream | Device | None = ... +) -> array | list[array]: + """ + Convert all arrays to have at least one dimension. + + Args: + *arys: Input arrays. + stream (Stream | Device | None, optional): The stream to execute the operation on. + + Returns: + array or list(array): An array or list of arrays with at least one dimension. + """ + +def atleast_2d( + *arys: array, stream: Stream | Device | None = ... +) -> array | list[array]: + """ + Convert all arrays to have at least two dimensions. + + Args: + *arys: Input arrays. + stream (Stream | Device | None, optional): The stream to execute the operation on. + + Returns: + array or list(array): An array or list of arrays with at least two dimensions. + """ + +def atleast_3d( + *arys: array, stream: Stream | Device | None = ... +) -> array | list[array]: + """ + Convert all arrays to have at least three dimensions. + + Args: + *arys: Input arrays. + stream (Stream | Device | None, optional): The stream to execute the operation on. + + Returns: + array or list(array): An array or list of arrays with at least three dimensions. + """ + +bfloat16: Dtype = ... + +def bitwise_and( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise bitwise and. + + Take the bitwise and of two arrays with numpy-style broadcasting + semantics. Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The bitwise and ``a & b``. + """ + +def bitwise_invert(a: scalar | array, stream: Stream | Device | None = ...) -> array: + """ + Element-wise bitwise inverse. + + Take the bitwise complement of the input. + + Args: + a (array): Input array or scalar. + + Returns: + array: The bitwise inverse ``~a``. + """ + +def bitwise_or( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise bitwise or. + + Take the bitwise or of two arrays with numpy-style broadcasting + semantics. Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The bitwise or``a | b``. + """ + +def bitwise_xor( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise bitwise xor. + + Take the bitwise exclusive or of two arrays with numpy-style + broadcasting semantics. Either or both input arrays can also be + scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The bitwise xor ``a ^ b``. + """ + +def block_masked_mm( + a: array, + b: array, + /, + block_size: int = ..., + mask_out: array | None = ..., + mask_lhs: array | None = ..., + mask_rhs: array | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + r""" + Matrix multiplication with block masking. + + Perform the (possibly batched) matrix multiplication of two arrays and with blocks + of size ``block_size x block_size`` optionally masked out. + + Assuming ``a`` with shape (..., `M`, `K`) and b with shape (..., `K`, `N`) + + * ``lhs_mask`` must have shape (..., :math:`\lceil` `M` / ``block_size`` :math:`\rceil`, :math:`\lceil` `K` / ``block_size`` :math:`\rceil`) + + * ``rhs_mask`` must have shape (..., :math:`\lceil` `K` / ``block_size`` :math:`\rceil`, :math:`\lceil` `N` / ``block_size`` :math:`\rceil`) + + * ``out_mask`` must have shape (..., :math:`\lceil` `M` / ``block_size`` :math:`\rceil`, :math:`\lceil` `N` / ``block_size`` :math:`\rceil`) + + Note: Only ``block_size=64`` and ``block_size=32`` are currently supported + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + block_size (int): Size of blocks to be masked. Must be ``32`` or ``64``. Default: ``64``. + mask_out (array, optional): Mask for output. Default: ``None``. + mask_lhs (array, optional): Mask for ``a``. Default: ``None``. + mask_rhs (array, optional): Mask for ``b``. Default: ``None``. + + Returns: + array: The output array. + """ + +def broadcast_arrays( + *arrays: array, stream: Stream | Device | None = ... +) -> tuple[array, ...]: + """ + Broadcast arrays against one another. + + The broadcasting semantics are the same as Numpy. + + Args: + *arrays (array): The input arrays. + + Returns: + tuple(array): The output arrays with the broadcasted shape. + """ + +def broadcast_shapes(*shapes: Sequence[int]) -> tuple[int]: + """ + Broadcast shapes. + + Returns the shape that results from broadcasting the supplied array shapes + against each other. + + Args: + *shapes (Sequence[int]): The shapes to broadcast. + + Returns: + tuple: The broadcasted shape. + + Raises: + ValueError: If the shapes cannot be broadcast. + + Example: + >>> mx.broadcast_shapes((1,), (3, 1)) + (3, 1) + >>> mx.broadcast_shapes((6, 7), (5, 6, 1), (7,)) + (5, 6, 7) + >>> mx.broadcast_shapes((5, 1, 4), (1, 3, 1)) + (5, 3, 4) + """ + +def broadcast_to( + a: scalar | array, + /, + shape: Sequence[int], + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Broadcast an array to the given shape. + + The broadcasting semantics are the same as Numpy. + + Args: + a (array): Input array. + shape (list(int)): The shape to broadcast to. + + Returns: + array: The output array with the new shape. + """ + +def ceil(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise ceil. + + Args: + a (array): Input array. + + Returns: + array: The ceil of ``a``. + """ + +def checkpoint(fun: Callable) -> Callable: ... +def clear_cache() -> None: + """ + Clear the memory cache. + + After calling this, :func:`get_cache_memory` should return ``0``. + """ + +def clip( + a: array, + /, + a_min: scalar | array | None, + a_max: scalar | array | None, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Clip the values of the array between the given minimum and maximum. + + If either ``a_min`` or ``a_max`` are ``None``, then corresponding edge + is ignored. At least one of ``a_min`` and ``a_max`` cannot be ``None``. + The input ``a`` and the limits must broadcast with one another. + + Args: + a (array): Input array. + a_min (scalar or array or None): Minimum value to clip to. + a_max (scalar or array or None): Maximum value to clip to. + + Returns: + array: The clipped array. + """ + +def compile( + fun: Callable, + inputs: object | None = ..., + outputs: object | None = ..., + shapeless: bool = ..., +) -> Callable: + """ + Returns a compiled function which produces the same output as ``fun``. + + Args: + fun (Callable): A function which takes a variable number of + :class:`array` or trees of :class:`array` and returns + a variable number of :class:`array` or trees of :class:`array`. + inputs (list or dict, optional): These inputs will be captured during + the function compilation along with the inputs to ``fun``. The ``inputs`` + can be a :obj:`list` or a :obj:`dict` containing arbitrarily nested + lists, dictionaries, or arrays. Leaf nodes that are not + :obj:`array` are ignored. Default: ``None`` + outputs (list or dict, optional): These outputs will be captured and + updated in a compiled function. The ``outputs`` can be a + :obj:`list` or a :obj:`dict` containing arbitrarily nested lists, + dictionaries, or arrays. Leaf nodes that are not :obj:`array` are ignored. + Default: ``None`` + shapeless (bool, optional): A function compiled with the ``shapeless`` + option enabled will not be recompiled when the input shape changes. Not all + functions can be compiled with ``shapeless`` enabled. Attempting to compile + such functions with shapeless enabled will throw. Note, changing the number + of dimensions or type of any input will result in a recompilation even with + ``shapeless`` set to ``True``. Default: ``False`` + + Returns: + Callable: A compiled function which has the same input arguments + as ``fun`` and returns the the same output(s). + """ + +complex64: Dtype = ... +complexfloating: DtypeCategory = ... + +def concat( + arrays: list[array], + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """See :func:`concatenate`.""" + +def concatenate( + arrays: list[array], + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Concatenate the arrays along the given axis. + + Args: + arrays (list(array)): Input :obj:`list` or :obj:`tuple` of arrays. + axis (int, optional): Optional axis to concatenate along. If + unspecified defaults to ``0``. + + Returns: + array: The concatenated array. + """ + +def conj(a: array, *, stream: Stream | Device | None = ...) -> array: + """ + Return the elementwise complex conjugate of the input. + Alias for `mx.conjugate`. + + Args: + a (array): Input array + + Returns: + array: The output array. + """ + +def conjugate(a: array, *, stream: Stream | Device | None = ...) -> array: + """ + Return the elementwise complex conjugate of the input. + Alias for `mx.conj`. + + Args: + a (array): Input array + + Returns: + array: The output array. + """ + +def contiguous( + a: array, + /, + allow_col_major: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Force an array to be row contiguous. Copy if necessary. + + Args: + a (array): The input to make contiguous + allow_col_major (bool): Consider column major as contiguous and don't copy + + Returns: + array: The row or col contiguous output. + """ + +def conv1d( + input: array, + weight: array, + /, + stride: int = ..., + padding: int = ..., + dilation: int = ..., + groups: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + 1D convolution over an input with several channels + + Args: + input (array): Input array of shape ``(N, L, C_in)``. + weight (array): Weight array of shape ``(C_out, K, C_in)``. + stride (int, optional): Kernel stride. Default: ``1``. + padding (int, optional): Input padding. Default: ``0``. + dilation (int, optional): Kernel dilation. Default: ``1``. + groups (int, optional): Input feature groups. Default: ``1``. + + Returns: + array: The convolved array. + """ + +def conv2d( + input: array, + weight: array, + /, + stride: int | tuple[int, int] = ..., + padding: int | tuple[int, int] = ..., + dilation: int | tuple[int, int] = ..., + groups: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + 2D convolution over an input with several channels + + Args: + input (array): Input array of shape ``(N, H, W, C_in)``. + weight (array): Weight array of shape ``(C_out, KH, KW, C_in)``. + stride (int or tuple(int), optional): :obj:`tuple` of size 2 with + kernel strides. All spatial dimensions get the same stride if + only one number is specified. Default: ``1``. + padding (int or tuple(int), optional): :obj:`tuple` of size 2 with + symmetric input padding. All spatial dimensions get the same + padding if only one number is specified. Default: ``0``. + dilation (int or tuple(int), optional): :obj:`tuple` of size 2 with + kernel dilation. All spatial dimensions get the same dilation + if only one number is specified. Default: ``1`` + groups (int, optional): input feature groups. Default: ``1``. + + Returns: + array: The convolved array. + """ + +def conv3d( + input: array, + weight: array, + /, + stride: int | tuple[int, int, int] = ..., + padding: int | tuple[int, int, int] = ..., + dilation: int | tuple[int, int, int] = ..., + groups: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + 3D convolution over an input with several channels + + Note: Only the default ``groups=1`` is currently supported. + + Args: + input (array): Input array of shape ``(N, D, H, W, C_in)``. + weight (array): Weight array of shape ``(C_out, KD, KH, KW, C_in)``. + stride (int or tuple(int), optional): :obj:`tuple` of size 3 with + kernel strides. All spatial dimensions get the same stride if + only one number is specified. Default: ``1``. + padding (int or tuple(int), optional): :obj:`tuple` of size 3 with + symmetric input padding. All spatial dimensions get the same + padding if only one number is specified. Default: ``0``. + dilation (int or tuple(int), optional): :obj:`tuple` of size 3 with + kernel dilation. All spatial dimensions get the same dilation + if only one number is specified. Default: ``1`` + groups (int, optional): input feature groups. Default: ``1``. + + Returns: + array: The convolved array. + """ + +def conv_general( + input: array, + weight: array, + /, + stride: int | Sequence[int] = ..., + padding: int | Sequence[int] | tuple[Sequence[int] | Sequence[int]] = ..., + kernel_dilation: int | Sequence[int] = ..., + input_dilation: int | Sequence[int] = ..., + groups: int = ..., + flip: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + General convolution over an input with several channels + + Args: + input (array): Input array of shape ``(N, ..., C_in)``. + weight (array): Weight array of shape ``(C_out, ..., C_in)``. + stride (int or list(int), optional): :obj:`list` with kernel strides. + All spatial dimensions get the same stride if + only one number is specified. Default: ``1``. + padding (int, list(int), or tuple(list(int), list(int)), optional): + :obj:`list` with input padding. All spatial dimensions get the same + padding if only one number is specified. Default: ``0``. + kernel_dilation (int or list(int), optional): :obj:`list` with + kernel dilation. All spatial dimensions get the same dilation + if only one number is specified. Default: ``1`` + input_dilation (int or list(int), optional): :obj:`list` with + input dilation. All spatial dimensions get the same dilation + if only one number is specified. Default: ``1`` + groups (int, optional): Input feature groups. Default: ``1``. + flip (bool, optional): Flip the order in which the spatial dimensions of + the weights are processed. Performs the cross-correlation operator when + ``flip`` is ``False`` and the convolution operator otherwise. + Default: ``False``. + + Returns: + array: The convolved array. + """ + +def conv_transpose1d( + input: array, + weight: array, + /, + stride: int = ..., + padding: int = ..., + dilation: int = ..., + output_padding: int = ..., + groups: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + 1D transposed convolution over an input with several channels + + Args: + input (array): Input array of shape ``(N, L, C_in)``. + weight (array): Weight array of shape ``(C_out, K, C_in)``. + stride (int, optional): Kernel stride. Default: ``1``. + padding (int, optional): Input padding. Default: ``0``. + dilation (int, optional): Kernel dilation. Default: ``1``. + output_padding (int, optional): Output padding. Default: ``0``. + groups (int, optional): Input feature groups. Default: ``1``. + + Returns: + array: The convolved array. + """ + +def conv_transpose2d( + input: array, + weight: array, + /, + stride: int | tuple[int, int] = ..., + padding: int | tuple[int, int] = ..., + dilation: int | tuple[int, int] = ..., + output_padding: int | tuple[int, int] = ..., + groups: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + 2D transposed convolution over an input with several channels + + Note: Only the default ``groups=1`` is currently supported. + + Args: + input (array): Input array of shape ``(N, H, W, C_in)``. + weight (array): Weight array of shape ``(C_out, KH, KW, C_in)``. + stride (int or tuple(int), optional): :obj:`tuple` of size 2 with + kernel strides. All spatial dimensions get the same stride if + only one number is specified. Default: ``1``. + padding (int or tuple(int), optional): :obj:`tuple` of size 2 with + symmetric input padding. All spatial dimensions get the same + padding if only one number is specified. Default: ``0``. + dilation (int or tuple(int), optional): :obj:`tuple` of size 2 with + kernel dilation. All spatial dimensions get the same dilation + if only one number is specified. Default: ``1`` + output_padding (int or tuple(int), optional): :obj:`tuple` of size 2 with + output padding. All spatial dimensions get the same output + padding if only one number is specified. Default: ``0``. + groups (int, optional): input feature groups. Default: ``1``. + + Returns: + array: The convolved array. + """ + +def conv_transpose3d( + input: array, + weight: array, + /, + stride: int | tuple[int, int, int] = ..., + padding: int | tuple[int, int, int] = ..., + dilation: int | tuple[int, int, int] = ..., + output_padding: int | tuple[int, int, int] = ..., + groups: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + 3D transposed convolution over an input with several channels + + Note: Only the default ``groups=1`` is currently supported. + + Args: + input (array): Input array of shape ``(N, D, H, W, C_in)``. + weight (array): Weight array of shape ``(C_out, KD, KH, KW, C_in)``. + stride (int or tuple(int), optional): :obj:`tuple` of size 3 with + kernel strides. All spatial dimensions get the same stride if + only one number is specified. Default: ``1``. + padding (int or tuple(int), optional): :obj:`tuple` of size 3 with + symmetric input padding. All spatial dimensions get the same + padding if only one number is specified. Default: ``0``. + dilation (int or tuple(int), optional): :obj:`tuple` of size 3 with + kernel dilation. All spatial dimensions get the same dilation + if only one number is specified. Default: ``1`` + output_padding (int or tuple(int), optional): :obj:`tuple` of size 3 with + output padding. All spatial dimensions get the same output + padding if only one number is specified. Default: ``0``. + groups (int, optional): input feature groups. Default: ``1``. + + Returns: + array: The convolved array. + """ + +def convolve( + a: array, v: array, /, mode: str = ..., *, stream: Stream | Device | None = ... +) -> array: + """ + The discrete convolution of 1D arrays. + + If ``v`` is longer than ``a``, then they are swapped. + The conv filter is flipped following signal processing convention. + + Args: + a (array): 1D Input array. + v (array): 1D Input array. + mode (str, optional): {'full', 'valid', 'same'} + + Returns: + array: The convolved array. + """ + +def cos(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise cosine. + + Args: + a (array): Input array. + + Returns: + array: The cosine of ``a``. + """ + +def cosh(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise hyperbolic cosine. + + Args: + a (array): Input array. + + Returns: + array: The hyperbolic cosine of ``a``. + """ + +cpu: DeviceType = ... + +def cummax( + a: array, + /, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Return the cumulative maximum of the elements along the given axis. + + Args: + a (array): Input array + axis (int, optional): Optional axis to compute the cumulative maximum + over. If unspecified the cumulative maximum of the flattened array is + returned. + reverse (bool): Perform the cumulative maximum in reverse. + inclusive (bool): The i-th element of the output includes the i-th + element of the input. + + Returns: + array: The output array. + """ + +def cummin( + a: array, + /, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Return the cumulative minimum of the elements along the given axis. + + Args: + a (array): Input array + axis (int, optional): Optional axis to compute the cumulative minimum + over. If unspecified the cumulative minimum of the flattened array is + returned. + reverse (bool): Perform the cumulative minimum in reverse. + inclusive (bool): The i-th element of the output includes the i-th + element of the input. + + Returns: + array: The output array. + """ + +def cumprod( + a: array, + /, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Return the cumulative product of the elements along the given axis. + + Args: + a (array): Input array + axis (int, optional): Optional axis to compute the cumulative product + over. If unspecified the cumulative product of the flattened array is + returned. + reverse (bool): Perform the cumulative product in reverse. + inclusive (bool): The i-th element of the output includes the i-th + element of the input. + + Returns: + array: The output array. + """ + +def cumsum( + a: array, + /, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Return the cumulative sum of the elements along the given axis. + + Args: + a (array): Input array + axis (int, optional): Optional axis to compute the cumulative sum + over. If unspecified the cumulative sum of the flattened array is + returned. + reverse (bool): Perform the cumulative sum in reverse. + inclusive (bool): The i-th element of the output includes the i-th + element of the input. + + Returns: + array: The output array. + """ + +class custom_function: + """ + Set up a function for custom gradient and vmap definitions. + + This class is meant to be used as a function decorator. Instances are + callables that behave identically to the wrapped function. However, when + a function transformation is used (e.g. computing gradients using + :func:`value_and_grad`) then the functions defined via + :meth:`custom_function.vjp`, :meth:`custom_function.jvp` and + :meth:`custom_function.vmap` are used instead of the default transformation. + + Note, all custom transformations are optional. Undefined transformations + fall back to the default behaviour. + + Example: + + .. code-block:: python + + import mlx.core as mx + + @mx.custom_function + def f(x, y): + return mx.sin(x) * y + + @f.vjp + def f_vjp(primals, cotangent, output): + x, y = primals + return cotan * mx.cos(x) * y, cotan * mx.sin(x) + + @f.jvp + def f_jvp(primals, tangents): + x, y = primals + dx, dy = tangents + return dx * mx.cos(x) * y + dy * mx.sin(x) + + @f.vmap + def f_vmap(inputs, axes): + x, y = inputs + ax, ay = axes + if ay != ax and ax is not None: + y = y.swapaxes(ay, ax) + return mx.sin(x) * y, (ax or ay) + + All ``custom_function`` instances behave as pure functions. Namely, any + variables captured will be treated as constants and no gradients will be + computed with respect to the captured arrays. For instance: + + .. code-block:: python + + import mlx.core as mx + + def g(x, y): + @mx.custom_function + def f(x): + return x * y + + @f.vjp + def f_vjp(x, dx, fx): + # Note that we have only x, dx and fx and nothing with respect to y + raise ValueError("Abort!") + + return f(x) + + x = mx.array(2.0) + y = mx.array(3.0) + print(g(x, y)) # prints 6.0 + print(mx.grad(g)(x, y)) # Raises exception + print(mx.grad(g, argnums=1)(x, y)) # prints 0.0 + """ + def __init__(self, f: Callable) -> None: ... + def __call__(self, *args, **kwargs) -> object: ... + def vjp(self, f: Callable): + """ + Define a custom vjp for the wrapped function. + + The vjp function takes three arguments: + + - *primals*: A pytree that contains all the positional arguments to + the function. It could be a single array, a tuple of arrays or a + full blown tuple of dicts of arrays etc. + - *cotangents*: A pytree that matches the structure of the output + but contains the cotangents (usually the gradients of the loss + function with respect to the outputs). + - *outputs*: The outputs of the function to be used to avoid + recomputing them for the gradient computation. + + The vjp function should return the same pytree structure as the + primals but containing the corresponding computed cotangents. + """ + + def jvp(self, f: Callable): + """ + Define a custom jvp for the wrapped function. + + The jvp function takes two arguments: + + - *primals*: A pytree that contains all the positional arguments to + the function. It could be a single array, a tuple of arrays or a + full blown tuple of dicts of arrays etc. + - *tangents*: A pytree that matches the structure of the inputs but + instead contains the gradients wrt to each input. Tangents could + be ``None`` if some inputs don't have an associated gradient. + + The jvp function should return the same pytree structure as the + outputs of the function but containing the tangents. + """ + + def vmap(self, f: Callable): + """ + Define a custom vectorization transformation for the wrapped function. + + The vmap function takes two arguments: + + - *inputs*: A pytree that contains all the positional arguments to + the function. It could be a single array, a tuple of arrays or a + full blown tuple of dicts of arrays etc. + - *axes*: A pytree that matches the structure of the inputs but + instead contains the vectorization axis for each input or + ``None`` if an input is not vectorized. + + The vmap function should return the outputs of the original + function but vectorized over the provided axes. It should also + return a pytree with the vectorization axes of each output. If some + outputs are no longer vectorized, then their vectorization axis + should be ``None``. + """ + +def default_device() -> Device: + """Get the default device.""" + +def default_stream(device: Device) -> Stream: + """Get the device's default stream.""" + +def degrees(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Convert angles from radians to degrees. + + Args: + a (array): Input array. + + Returns: + array: The angles in degrees. + """ + +def depends(inputs: array | Sequence[array], dependencies: array | Sequence[array]): + """ + Insert dependencies between arrays in the graph. The outputs are + identical to ``inputs`` but with dependencies on ``dependencies``. + + Args: + inputs (array or Sequence[array]): The input array or arrays. + dependencies (array or Sequence[array]): The array or arrays + to insert dependencies on. + + Returns: + array or Sequence[array]: The outputs which depend on dependencies. + """ + +def dequantize( + w: array, + /, + scales: array, + biases: array | None = ..., + group_size: int = ..., + bits: int = ..., + mode: str = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + r""" + Dequantize the matrix ``w`` using quantization parameters. + + Args: + w (array): Matrix to be dequantized + scales (array): The scales to use per ``group_size`` elements of ``w``. + biases (array, optional): The biases to use per ``group_size`` + elements of ``w``. Default: ``None``. + group_size (int, optional): The size of the group in ``w`` that shares a + scale and bias. Default: ``64``. + bits (int, optional): The number of bits occupied by each element in + ``w``. Default: ``4``. + mode (str, optional): The quantization mode. Default: ``"affine"``. + + Returns: + array: The dequantized version of ``w`` + + Notes: + The currently supported quantization modes are ``"affine"`` and ``mxfp4``. + + For ``affine`` quantization, given the notation in :func:`quantize`, + we compute :math:`w_i` from :math:`\hat{w_i}` and corresponding :math:`s` + and :math:`\beta` as follows + + .. math:: + + w_i = s \hat{w_i} + \beta + """ + +def diag(a: array, /, k: int = ..., *, stream: Stream | Device | None = ...) -> array: + """ + Extract a diagonal or construct a diagonal matrix. + If ``a`` is 1-D then a diagonal matrix is constructed with ``a`` on the + :math:`k`-th diagonal. If ``a`` is 2-D then the :math:`k`-th diagonal is + returned. + + Args: + a (array): 1-D or 2-D input array. + k (int, optional): The diagonal to extract or construct. + Default: ``0``. + + Returns: + array: The extracted diagonal or the constructed diagonal matrix. + """ + +def diagonal( + a: array, + offset: int = ..., + axis1: int = ..., + axis2: int = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Return specified diagonals. + + If ``a`` is 2-D, then a 1-D array containing the diagonal at the given + ``offset`` is returned. + + If ``a`` has more than two dimensions, then ``axis1`` and ``axis2`` + determine the 2D subarrays from which diagonals are extracted. The new + shape is the original shape with ``axis1`` and ``axis2`` removed and a + new dimension inserted at the end corresponding to the diagonal. + + Args: + a (array): Input array + offset (int, optional): Offset of the diagonal from the main diagonal. + Can be positive or negative. Default: ``0``. + axis1 (int, optional): The first axis of the 2-D sub-arrays from which + the diagonals should be taken. Default: ``0``. + axis2 (int, optional): The second axis of the 2-D sub-arrays from which + the diagonals should be taken. Default: ``1``. + + Returns: + array: The diagonals of the array. + """ + +def disable_compile() -> None: + """ + Globally disable compilation. Setting the environment variable + ``MLX_DISABLE_COMPILE`` can also be used to disable compilation. + """ + +def divide( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise division. + + Divide two arrays with numpy-style broadcasting semantics. Either or both + input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The quotient ``a / b``. + """ + +def divmod( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise quotient and remainder. + + The fuction ``divmod(a, b)`` is equivalent to but faster than + ``(a // b, a % b)``. The function uses numpy-style broadcasting + semantics. Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + tuple(array, array): The quotient ``a // b`` and remainder ``a % b``. + """ + +e: float = ... + +def einsum(subscripts: str, *operands, stream: Stream | Device | None = ...) -> array: + """ + Perform the Einstein summation convention on the operands. + + Args: + subscripts (str): The Einstein summation convention equation. + *operands (array): The input arrays. + + Returns: + array: The output array. + """ + +def einsum_path(subscripts: str, *operands): + """ + Compute the contraction order for the given Einstein summation. + + Args: + subscripts (str): The Einstein summation convention equation. + *operands (array): The input arrays. + + Returns: + tuple(list(tuple(int, int)), str): + The einsum path and a string containing information about the + chosen path. + """ + +def enable_compile() -> None: + """ + Globally enable compilation. This will override the environment + variable ``MLX_DISABLE_COMPILE`` if set. + """ + +def equal( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise equality. + + Equality comparison on two arrays with numpy-style broadcasting semantics. + Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The element-wise comparison ``a == b``. + """ + +def erf(a: array, /, *, stream: Stream | Device | None = ...) -> array: + r""" + Element-wise error function. + + .. math:: + \mathrm{erf}(x) = \frac{2}{\sqrt{\pi}} \int_0^x e^{-t^2} \, dt + + Args: + a (array): Input array. + + Returns: + array: The error function of ``a``. + """ + +def erfinv(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise inverse of :func:`erf`. + + Args: + a (array): Input array. + + Returns: + array: The inverse error function of ``a``. + """ + +euler_gamma: float = ... + +type MX_ARRAY_TREE = ( + array + | Module + | list[MX_ARRAY_TREE] + | tuple[MX_ARRAY_TREE, ...] + | Mapping[str, MX_ARRAY_TREE] +) + +def eval(*args: MX_ARRAY_TREE | None) -> None: + """ + Evaluate an :class:`array` or tree of :class:`array`. + + Args: + *args (arrays or trees of arrays): Each argument can be a single array + or a tree of arrays. If a tree is given the nodes can be a Python + :class:`list`, :class:`tuple` or :class:`dict`. Leaves which are not + arrays are ignored. + """ + +def exp(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise exponential. + + Args: + a (array): Input array. + + Returns: + array: The exponential of ``a``. + """ + +def expand_dims( + a: array, + /, + axis: int | Sequence[int], + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Add a size one dimension at the given axis. + + Args: + a (array): Input array. + axes (int or tuple(int)): The index of the inserted dimensions. + + Returns: + array: The array with inserted dimensions. + """ + +def expm1(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise exponential minus 1. + + Computes ``exp(x) - 1`` with greater precision for small ``x``. + + Args: + a (array): Input array. + + Returns: + array: The expm1 of ``a``. + """ + +def export_function( + arg0: object, fun: Callable, *args, shapeless: bool = ..., **kwargs +) -> None: + """ + Export an MLX function. + + Example input arrays must be provided to export a function. The example + inputs can be variable ``*args`` and ``**kwargs`` or a tuple of arrays + and/or dictionary of string keys with array values. + + .. warning:: + + This is part of an experimental API which is likely to + change in future versions of MLX. Functions exported with older + versions of MLX may not be compatible with future versions. + + Args: + file (str or Callable): Either a file path to export the function + to or a callback. + fun (Callable): A function which takes as input zero or more + :class:`array` and returns one or more :class:`array`. + *args (array): Example array inputs to the function. + shapeless (bool, optional): Whether or not the function allows + inputs with variable shapes. Default: ``False``. + **kwargs (array): Additional example keyword array inputs to the + function. + + Example: + + .. code-block:: python + + def fun(x, y): + return x + y + + x = mx.array(1) + y = mx.array([1, 2, 3]) + mx.export_function("fun.mlxfn", fun, x, y=y) + """ + +def export_to_dot(file: object, *args, **kwargs) -> None: + """ + Export a graph to DOT format for visualization. + + A variable number of output arrays can be provided for exporting + The graph exported will recursively include all unevaluated inputs of + the provided outputs. + + Args: + file (str): The file path to export to. + *args (array): The output arrays. + **kwargs (dict[str, array]): Provide some names for arrays in the + graph to make the result easier to parse. + + Example: + >>> a = mx.array(1) + mx.array(2) + >>> mx.export_to_dot("graph.dot", a) + >>> x = mx.array(1) + >>> y = mx.array(2) + >>> mx.export_to_dot("graph.dot", x + y, x=x, y=y) + """ + +def exporter(file: str, fun: Callable, *, shapeless: bool = ...) -> FunctionExporter: + """ + Make a callable object to export multiple traces of a function to a file. + + .. warning:: + + This is part of an experimental API which is likely to + change in future versions of MLX. Functions exported with older + versions of MLX may not be compatible with future versions. + + Args: + file (str): File path to export the function to. + shapeless (bool, optional): Whether or not the function allows + inputs with variable shapes. Default: ``False``. + + Example: + + .. code-block:: python + + def fun(*args): + return sum(args) + + with mx.exporter("fun.mlxfn", fun) as exporter: + exporter(mx.array(1)) + exporter(mx.array(1), mx.array(2)) + exporter(mx.array(1), mx.array(2), mx.array(3)) + """ + +def eye( + n: int, + m: int | None = ..., + k: int = ..., + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Create an identity matrix or a general diagonal matrix. + + Args: + n (int): The number of rows in the output. + m (int, optional): The number of columns in the output. Defaults to n. + k (int, optional): Index of the diagonal. Defaults to 0 (main diagonal). + dtype (Dtype, optional): Data type of the output array. Defaults to float32. + stream (Stream, optional): Stream or device. Defaults to None. + + Returns: + array: An array where all elements are equal to zero, except for the k-th diagonal, whose values are equal to one. + """ + +class finfo: + """Get information on floating-point types.""" + def __init__(self, arg: Dtype, /) -> None: ... + @property + def min(self) -> float: + """The smallest representable number.""" + + @property + def max(self) -> float: + """The largest representable number.""" + + @property + def eps(self) -> float: + """ + The difference between 1.0 and the next smallest + representable number larger than 1.0. + """ + + @property + def dtype(self) -> Dtype: + """The :obj:`Dtype`.""" + + def __repr__(self) -> str: ... + +def flatten( + a: array, + /, + start_axis: int = ..., + end_axis: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Flatten an array. + + The axes flattened will be between ``start_axis`` and ``end_axis``, + inclusive. Negative axes are supported. After converting negative axis to + positive, axes outside the valid range will be clamped to a valid value, + ``start_axis`` to ``0`` and ``end_axis`` to ``ndim - 1``. + + Args: + a (array): Input array. + start_axis (int, optional): The first dimension to flatten. Defaults to ``0``. + end_axis (int, optional): The last dimension to flatten. Defaults to ``-1``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The flattened array. + + Example: + >>> a = mx.array([[1, 2], [3, 4]]) + >>> mx.flatten(a) + array([1, 2, 3, 4], dtype=int32) + >>> + >>> mx.flatten(a, start_axis=0, end_axis=-1) + array([1, 2, 3, 4], dtype=int32) + """ + +float16: Dtype = ... +float32: Dtype = ... +float64: Dtype = ... +floating: DtypeCategory = ... + +def floor(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise floor. + + Args: + a (array): Input array. + + Returns: + array: The floor of ``a``. + """ + +def floor_divide( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise integer division. + + If either array is a floating point type then it is equivalent to + calling :func:`floor` after :func:`divide`. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The quotient ``a // b``. + """ + +def full( + shape: int | Sequence[int], + vals: scalar | array, + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Construct an array with the given value. + + Constructs an array of size ``shape`` filled with ``vals``. If ``vals`` + is an :obj:`array` it must be broadcastable to the given ``shape``. + + Args: + shape (int or list(int)): The shape of the output array. + vals (float or int or array): Values to fill the array with. + dtype (Dtype, optional): Data type of the output array. If + unspecified the output type is inferred from ``vals``. + + Returns: + array: The output array with the specified shape and values. + """ + +def gather_mm( + a: array, + b: array, + /, + lhs_indices: array, + rhs_indices: array, + *, + sorted_indices: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Matrix multiplication with matrix-level gather. + + Performs a gather of the operands with the given indices followed by a + (possibly batched) matrix multiplication of two arrays. This operation + is more efficient than explicitly applying a :func:`take` followed by a + :func:`matmul`. + + The indices ``lhs_indices`` and ``rhs_indices`` contain flat indices + along the batch dimensions (i.e. all but the last two dimensions) of + ``a`` and ``b`` respectively. + + For ``a`` with shape ``(A1, A2, ..., AS, M, K)``, ``lhs_indices`` + contains indices from the range ``[0, A1 * A2 * ... * AS)`` + + For ``b`` with shape ``(B1, B2, ..., BS, M, K)``, ``rhs_indices`` + contains indices from the range ``[0, B1 * B2 * ... * BS)`` + + If only one index is passed and it is sorted, the ``sorted_indices`` + flag can be passed for a possible faster implementation. + + Args: + a (array): Input array. + b (array): Input array. + lhs_indices (array, optional): Integer indices for ``a``. Default: ``None`` + rhs_indices (array, optional): Integer indices for ``b``. Default: ``None`` + sorted_indices (bool, optional): May allow a faster implementation + if the passed indices are sorted. Default: ``False``. + + Returns: + array: The output array. + """ + +def gather_qmm( + x: array, + w: array, + /, + scales: array, + biases: array | None = ..., + lhs_indices: array | None = ..., + rhs_indices: array | None = ..., + transpose: bool = ..., + group_size: int = ..., + bits: int = ..., + mode: str = ..., + *, + sorted_indices: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Perform quantized matrix multiplication with matrix-level gather. + + This operation is the quantized equivalent to :func:`gather_mm`. + Similar to :func:`gather_mm`, the indices ``lhs_indices`` and + ``rhs_indices`` contain flat indices along the batch dimensions (i.e. + all but the last two dimensions) of ``x`` and ``w`` respectively. + + Note that ``scales`` and ``biases`` must have the same batch dimensions + as ``w`` since they represent the same quantized matrix. + + Args: + x (array): Input array + w (array): Quantized matrix packed in unsigned integers + scales (array): The scales to use per ``group_size`` elements of ``w`` + biases (array, optional): The biases to use per ``group_size`` + elements of ``w``. Default: ``None``. + lhs_indices (array, optional): Integer indices for ``x``. Default: ``None``. + rhs_indices (array, optional): Integer indices for ``w``. Default: ``None``. + transpose (bool, optional): Defines whether to multiply with the + transposed ``w`` or not, namely whether we are performing + ``x @ w.T`` or ``x @ w``. Default: ``True``. + group_size (int, optional): The size of the group in ``w`` that + shares a scale and bias. Default: ``64``. + bits (int, optional): The number of bits occupied by each element in + ``w``. Default: ``4``. + mode (str, optional): The quantization mode. Default: ``"affine"``. + sorted_indices (bool, optional): May allow a faster implementation + if the passed indices are sorted. Default: ``False``. + + Returns: + array: The result of the multiplication of ``x`` with ``w`` + after gathering using ``lhs_indices`` and ``rhs_indices``. + """ + +generic: DtypeCategory = ... + +def get_active_memory() -> int: + """ + Get the actively used memory in bytes. + + Note, this will not always match memory use reported by the system because + it does not include cached memory buffers. + """ + +def get_cache_memory() -> int: + """ + Get the cache size in bytes. + + The cache includes memory not currently used that has not been returned + to the system allocator. + """ + +def get_peak_memory() -> int: + """ + Get the peak amount of used memory in bytes. + + The maximum memory used recorded from the beginning of the program + execution or since the last call to :func:`reset_peak_memory`. + """ + +gpu: DeviceType = ... + +def grad( + fun: Callable, + argnums: int | Sequence[int] | None = ..., + argnames: str | Sequence[str] = ..., +) -> Callable: + """ + Returns a function which computes the gradient of ``fun``. + + Args: + fun (Callable): A function which takes a variable number of + :class:`array` or trees of :class:`array` and returns + a scalar output :class:`array`. + argnums (int or list(int), optional): Specify the index (or indices) + of the positional arguments of ``fun`` to compute the gradient + with respect to. If neither ``argnums`` nor ``argnames`` are + provided ``argnums`` defaults to ``0`` indicating ``fun``'s first + argument. + argnames (str or list(str), optional): Specify keyword arguments of + ``fun`` to compute gradients with respect to. It defaults to [] so + no gradients for keyword arguments by default. + + Returns: + Callable: A function which has the same input arguments as ``fun`` and + returns the gradient(s). + """ + +def greater( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise greater than. + + Strict greater than on two arrays with numpy-style broadcasting semantics. + Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The element-wise comparison ``a > b``. + """ + +def greater_equal( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise greater or equal. + + Greater than or equal on two arrays with numpy-style broadcasting semantics. + Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The element-wise comparison ``a >= b``. + """ + +def hadamard_transform( + a: array, scale: float | None = ..., stream: Stream | Device | None = ... +) -> array: + """ + Perform the Walsh-Hadamard transform along the final axis. + + Equivalent to: + + .. code-block:: python + + from scipy.linalg import hadamard + + y = (hadamard(len(x)) @ x) * scale + + Supports sizes ``n = m*2^k`` for ``m`` in ``(1, 12, 20, 28)`` and ``2^k + <= 8192`` for float32 and ``2^k <= 16384`` for float16/bfloat16. + + Args: + a (array): Input array or scalar. + scale (float): Scale the output by this factor. + Defaults to ``1/sqrt(a.shape[-1])`` so that the Hadamard matrix is orthonormal. + + Returns: + array: The transformed array. + """ + +def identity( + n: int, dtype: Dtype | None = ..., *, stream: Stream | Device | None = ... +) -> array: + """ + Create a square identity matrix. + + Args: + n (int): The number of rows and columns in the output. + dtype (Dtype, optional): Data type of the output array. Defaults to float32. + stream (Stream, optional): Stream or device. Defaults to None. + + Returns: + array: An identity matrix of size n x n. + """ + +class iinfo: + """Get information on integer types.""" + def __init__(self, arg: Dtype, /) -> None: ... + @property + def min(self) -> int: + """The smallest representable number.""" + + @property + def max(self) -> int: + """The largest representable number.""" + + @property + def dtype(self) -> Dtype: + """The :obj:`Dtype`.""" + + def __repr__(self) -> str: ... + +def imag(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Returns the imaginary part of a complex array. + + Args: + a (array): Input array. + + Returns: + array: The imaginary part of ``a``. + """ + +def import_function(file: str) -> Callable: + """ + Import a function from a file. + + The imported function can be called either with ``*args`` and + ``**kwargs`` or with a tuple of arrays and/or dictionary of string + keys with array values. Imported functions always return a tuple of + arrays. + + .. warning:: + + This is part of an experimental API which is likely to + change in future versions of MLX. Functions exported with older + versions of MLX may not be compatible with future versions. + + Args: + file (str): The file path to import the function from. + + Returns: + Callable: The imported function. + + Example: + >>> fn = mx.import_function("function.mlxfn") + >>> out = fn(a, b, x=x, y=y)[0] + >>> + >>> out = fn((a, b), {"x": x, "y": y}[0] + """ + +inexact: DtypeCategory = ... +inf: float = ... + +def inner(a: array, b: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Ordinary inner product of vectors for 1-D arrays, in higher dimensions a sum product over the last axes. + + Args: + a (array): Input array + b (array): Input array + + Returns: + array: The inner product. + """ + +int16: Dtype = ... +int32: Dtype = ... +int64: Dtype = ... +int8: Dtype = ... +integer: DtypeCategory = ... + +def is_available(device: Device) -> bool: + """Check if a back-end is available for the given device.""" + +def isclose( + a: array, + b: array, + /, + rtol: float = ..., + atol: float = ..., + *, + equal_nan: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Returns a boolean array where two arrays are element-wise equal within a tolerance. + + Infinite values are considered equal if they have the same sign, NaN values are + not equal unless ``equal_nan`` is ``True``. + + Two values are considered equal if: + + .. code-block:: + + abs(a - b) <= (atol + rtol * abs(b)) + + Note unlike :func:`array_equal`, this function supports numpy-style + broadcasting. + + Args: + a (array): Input array. + b (array): Input array. + rtol (float): Relative tolerance. + atol (float): Absolute tolerance. + equal_nan (bool): If ``True``, NaNs are considered equal. + Defaults to ``False``. + + Returns: + array: The boolean output scalar indicating if the arrays are close. + """ + +def isfinite(a: array, stream: Stream | Device | None = ...) -> array: + """ + Return a boolean array indicating which elements are finite. + + An element is finite if it is not infinite or NaN. + + Args: + a (array): Input array. + + Returns: + array: The boolean array indicating which elements are finite. + """ + +def isinf(a: array, stream: Stream | Device | None = ...) -> array: + """ + Return a boolean array indicating which elements are +/- inifnity. + + Args: + a (array): Input array. + + Returns: + array: The boolean array indicating which elements are +/- infinity. + """ + +def isnan(a: array, stream: Stream | Device | None = ...) -> array: + """ + Return a boolean array indicating which elements are NaN. + + Args: + a (array): Input array. + + Returns: + array: The boolean array indicating which elements are NaN. + """ + +def isneginf(a: array, stream: Stream | Device | None = ...) -> array: + """ + Return a boolean array indicating which elements are negative infinity. + + Args: + a (array): Input array. + stream (Stream | Device | None): Optional stream or device. + + Returns: + array: The boolean array indicating which elements are negative infinity. + """ + +def isposinf(a: array, stream: Stream | Device | None = ...) -> array: + """ + Return a boolean array indicating which elements are positive infinity. + + Args: + a (array): Input array. + stream (Stream | Device | None): Optional stream or device. + + Returns: + array: The boolean array indicating which elements are positive infinity. + """ + +def issubdtype(arg1: Dtype | DtypeCategory, arg2: Dtype | DtypeCategory) -> bool: + """ + Check if a :obj:`Dtype` or :obj:`DtypeCategory` is a subtype + of another. + + Args: + arg1 (Dtype | DtypeCategory: First dtype or category. + arg2 (Dtype | DtypeCategory: Second dtype or category. + + Returns: + bool: + A boolean indicating if the first input is a subtype of the + second input. + + Example: + + >>> ints = mx.array([1, 2, 3], dtype=mx.int32) + >>> mx.issubdtype(ints.dtype, mx.integer) + True + >>> mx.issubdtype(ints.dtype, mx.floating) + False + + >>> floats = mx.array([1, 2, 3], dtype=mx.float32) + >>> mx.issubdtype(floats.dtype, mx.integer) + False + >>> mx.issubdtype(floats.dtype, mx.floating) + True + + Similar types of different sizes are not subdtypes of each other: + + >>> mx.issubdtype(mx.float64, mx.float32) + False + >>> mx.issubdtype(mx.float32, mx.float64) + False + + but both are subtypes of `floating`: + + >>> mx.issubdtype(mx.float64, mx.floating) + True + >>> mx.issubdtype(mx.float32, mx.floating) + True + + For convenience, dtype-like objects are allowed too: + + >>> mx.issubdtype(mx.float32, mx.inexact) + True + >>> mx.issubdtype(mx.signedinteger, mx.floating) + False + """ + +def jvp( + fun: Callable, primals: list[array], tangents: list[array] +) -> tuple[list[array], list[array]]: + """ + Compute the Jacobian-vector product. + + This computes the product of the Jacobian of a function ``fun`` evaluated + at ``primals`` with the ``tangents``. + + Args: + fun (Callable): A function which takes a variable number of :class:`array` + and returns a single :class:`array` or list of :class:`array`. + primals (list(array)): A list of :class:`array` at which to + evaluate the Jacobian. + tangents (list(array)): A list of :class:`array` which are the + "vector" in the Jacobian-vector product. The ``tangents`` should be the + same in number, shape, and type as the inputs of ``fun`` (i.e. the ``primals``). + + Returns: + list(array): A list of the Jacobian-vector products which + is the same in number, shape, and type of the inputs to ``fun``. + """ + +def kron(a: array, b: array, *, stream: Stream | Device | None = ...) -> array: + """ + Compute the Kronecker product of two arrays ``a`` and ``b``. + + Args: + a (array): The first input array. + b (array): The second input array. + stream (Stream | Device | None, optional): Optional stream or + device for execution. Default: ``None``. + + Returns: + array: The Kronecker product of ``a`` and ``b``. + + Examples: + >>> a = mx.array([[1, 2], [3, 4]]) + >>> b = mx.array([[0, 5], [6, 7]]) + >>> result = mx.kron(a, b) + >>> print(result) + array([[0, 5, 0, 10], + [6, 7, 12, 14], + [0, 15, 0, 20], + [18, 21, 24, 28]], dtype=int32) + """ + +def left_shift( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise left shift. + + Shift the bits of the first input to the left by the second using + numpy-style broadcasting semantics. Either or both input arrays can + also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The bitwise left shift ``a << b``. + """ + +def less( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise less than. + + Strict less than on two arrays with numpy-style broadcasting semantics. + Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The element-wise comparison ``a < b``. + """ + +def less_equal( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise less than or equal. + + Less than or equal on two arrays with numpy-style broadcasting semantics. + Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The element-wise comparison ``a <= b``. + """ + +def linspace( + start, + stop, + num: int | None = ..., + dtype: Dtype | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate ``num`` evenly spaced numbers over interval ``[start, stop]``. + + Args: + start (scalar): Starting value. + stop (scalar): Stopping value. + num (int, optional): Number of samples, defaults to ``50``. + dtype (Dtype, optional): Specifies the data type of the output, + default to ``float32``. + + Returns: + array: The range of values. + """ + +def load( + file: str | pathlib.Path, + /, + format: str | None = ..., + return_metadata: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array | dict[str, array]: + """ + Load array(s) from a binary file. + + The supported formats are ``.npy``, ``.npz``, ``.safetensors``, and + ``.gguf``. + + Args: + file (str, pathlib.Path): File in which the array is saved. + format (str, optional): Format of the file. If ``None``, the + format is inferred from the file extension. Supported formats: + ``npy``, ``npz``, and ``safetensors``. Default: ``None``. + return_metadata (bool, optional): Load the metadata for formats + which support matadata. The metadata will be returned as an + additional dictionary. Default: ``False``. + Returns: + array or dict: + A single array if loading from a ``.npy`` file or a dict + mapping names to arrays if loading from a ``.npz`` or + ``.safetensors`` file. If ``return_metadata`` is ``True`` an + additional dictionary of metadata will be returned. + + Warning: + + When loading unsupported quantization formats from GGUF, tensors + will automatically cast to ``mx.float16`` + """ + +def log(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise natural logarithm. + + Args: + a (array): Input array. + + Returns: + array: The natural logarithm of ``a``. + """ + +def log10(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise base-10 logarithm. + + Args: + a (array): Input array. + + Returns: + array: The base-10 logarithm of ``a``. + """ + +def log1p(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise natural log of one plus the array. + + Args: + a (array): Input array. + + Returns: + array: The natural logarithm of one plus ``a``. + """ + +def log2(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise base-2 logarithm. + + Args: + a (array): Input array. + + Returns: + array: The base-2 logarithm of ``a``. + """ + +def logaddexp( + a: scalar | array, + b: scalar | array, + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise log-add-exp. + + This is a numerically stable log-add-exp of two arrays with numpy-style + broadcasting semantics. Either or both input arrays can also be scalars. + + The computation is is a numerically stable version of ``log(exp(a) + exp(b))``. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The log-add-exp of ``a`` and ``b``. + """ + +def logcumsumexp( + a: array, + /, + axis: int | None = ..., + *, + reverse: bool = ..., + inclusive: bool = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Return the cumulative logsumexp of the elements along the given axis. + + Args: + a (array): Input array + axis (int, optional): Optional axis to compute the cumulative logsumexp + over. If unspecified the cumulative logsumexp of the flattened array is + returned. + reverse (bool): Perform the cumulative logsumexp in reverse. + inclusive (bool): The i-th element of the output includes the i-th + element of the input. + + Returns: + array: The output array. + """ + +def logical_and( + a: array, b: array, /, *, stream: Stream | Device | None = ... +) -> array: + """ + Element-wise logical and. + + Args: + a (array): First input array or scalar. + b (array): Second input array or scalar. + + Returns: + array: The boolean array containing the logical and of ``a`` and ``b``. + """ + +def logical_not(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise logical not. + + Args: + a (array): Input array or scalar. + + Returns: + array: The boolean array containing the logical not of ``a``. + """ + +def logical_or(a: array, b: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise logical or. + + Args: + a (array): First input array or scalar. + b (array): Second input array or scalar. + + Returns: + array: The boolean array containing the logical or of ``a`` and ``b``. + """ + +def logsumexp( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + A `log-sum-exp` reduction over the given axes. + + The log-sum-exp reduction is a numerically stable version of: + + .. code-block:: + + log(sum(exp(a), axis)) + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +def matmul(a: array, b: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Matrix multiplication. + + Perform the (possibly batched) matrix multiplication of two arrays. This function supports + broadcasting for arrays with more than two dimensions. + + - If the first array is 1-D then a 1 is prepended to its shape to make it + a matrix. Similarly if the second array is 1-D then a 1 is appended to its + shape to make it a matrix. In either case the singleton dimension is removed + from the result. + - A batched matrix multiplication is performed if the arrays have more than + 2 dimensions. The matrix dimensions for the matrix product are the last + two dimensions of each input. + - All but the last two dimensions of each input are broadcast with one another using + standard numpy-style broadcasting semantics. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The matrix product of ``a`` and ``b``. + """ + +def max( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + A `max` reduction over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +def maximum( + a: scalar | array, + b: scalar | array, + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise maximum. + + Take the element-wise max of two arrays with numpy-style broadcasting + semantics. Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The max of ``a`` and ``b``. + """ + +def mean( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Compute the mean(s) over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array of means. + """ + +def meshgrid( + *arrays: array, + sparse: bool | None = ..., + indexing: str | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate multidimensional coordinate grids from 1-D coordinate arrays + + Args: + *arrays (array): Input arrays. + sparse (bool, optional): If ``True``, a sparse grid is returned in which each output + array has a single non-zero element. If ``False``, a dense grid is returned. + Defaults to ``False``. + indexing (str, optional): Cartesian ('xy') or matrix ('ij') indexing of the output arrays. + Defaults to ``'xy'``. + + Returns: + list(array): The output arrays. + """ + +def min( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + A `min` reduction over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +def minimum( + a: scalar | array, + b: scalar | array, + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise minimum. + + Take the element-wise min of two arrays with numpy-style broadcasting + semantics. Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The min of ``a`` and ``b``. + """ + +def moveaxis( + a: array, + /, + source: int, + destination: int, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Move an axis to a new position. + + Args: + a (array): Input array. + source (int): Specifies the source axis. + destination (int): Specifies the destination axis. + + Returns: + array: The array with the axis moved. + """ + +def multiply( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise multiplication. + + Multiply two arrays with numpy-style broadcasting semantics. Either or both + input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The multiplication ``a * b``. + """ + +nan: float = ... + +def nan_to_num( + a: scalar | array, + nan: float = ..., + posinf: float | None = ..., + neginf: float | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Replace NaN and Inf values with finite numbers. + + Args: + a (array): Input array + nan (float, optional): Value to replace NaN with. Default: ``0``. + posinf (float, optional): Value to replace positive infinities + with. If ``None``, defaults to largest finite value for the + given data type. Default: ``None``. + neginf (float, optional): Value to replace negative infinities + with. If ``None``, defaults to the negative of the largest + finite value for the given data type. Default: ``None``. + + Returns: + array: Output array with NaN and Inf replaced. + """ + +def negative(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise negation. + + Args: + a (array): Input array. + + Returns: + array: The negative of ``a``. + """ + +def new_stream(device: Device) -> Stream: + """Make a new stream on the given device.""" + +newaxis: None = ... + +def not_equal( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise not equal. + + Not equal comparison on two arrays with numpy-style broadcasting semantics. + Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The element-wise comparison ``a != b``. + """ + +number: DtypeCategory = ... + +def ones( + shape: int | Sequence[int], + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Construct an array of ones. + + Args: + shape (int or list(int)): The shape of the output array. + dtype (Dtype, optional): Data type of the output array. If + unspecified the output type defaults to ``float32``. + + Returns: + array: The array of ones with the specified shape. + """ + +def ones_like(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + An array of ones like the input. + + Args: + a (array): The input to take the shape and type from. + + Returns: + array: The output array filled with ones. + """ + +def outer(a: array, b: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Compute the outer product of two 1-D arrays, if the array's passed are not 1-D a flatten op will be run beforehand. + + Args: + a (array): Input array + b (array): Input array + + Returns: + array: The outer product. + """ + +def pad( + a: array, + pad_width: int | tuple[int] | tuple[int, int] | list[tuple[int, int]], + mode: Literal["constant", "edge"] = ..., + constant_values: scalar | array = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Pad an array with a constant value + + Args: + a (array): Input array. + pad_width (int, tuple(int), tuple(int, int) or list(tuple(int, int))): Number of padded + values to add to the edges of each axis:``((before_1, after_1), + (before_2, after_2), ..., (before_N, after_N))``. If a single pair + of integers is passed then ``(before_i, after_i)`` are all the same. + If a single integer or tuple with a single integer is passed then + all axes are extended by the same number on each side. + mode: Padding mode. One of the following strings: + "constant" (default): Pads with a constant value. + "edge": Pads with the edge values of array. + constant_value (array or scalar, optional): Optional constant value + to pad the edges of the array with. + + Returns: + array: The padded array. + """ + +def partition( + a: array, + /, + kth: int, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Returns a partitioned copy of the array such that the smaller ``kth`` + elements are first. + + The ordering of the elements in partitions is undefined. + + Args: + a (array): Input array. + kth (int): Element at the ``kth`` index will be in its sorted + position in the output. All elements before the kth index will + be less or equal to the ``kth`` element and all elements after + will be greater or equal to the ``kth`` element in the output. + axis (int or None, optional): Optional axis to partition over. + If ``None``, this partitions over the flattened array. + If unspecified, it defaults to ``-1``. + + Returns: + array: The partitioned array. + """ + +def permute_dims( + a: array, + /, + axes: Sequence[int] | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """See :func:`transpose`.""" + +pi: float = ... + +def power( + a: scalar | array, + b: scalar | array, + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise power operation. + + Raise the elements of a to the powers in elements of b with numpy-style + broadcasting semantics. Either or both input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: Bases of ``a`` raised to powers in ``b``. + """ + +def prod( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + An product reduction over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +def put_along_axis( + a: array, + /, + indices: array, + values: array, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Put values along an axis at the specified indices. + + Args: + a (array): Destination array. + indices (array): Indices array. These should be broadcastable with + the input array excluding the `axis` dimension. + values (array): Values array. These should be broadcastable with + the indices. + + axis (int or None): Axis in the destination to put the values to. If + ``axis == None`` the destination is flattened prior to the put + operation. + + Returns: + array: The output array. + """ + +def quantize( + w: array, + /, + group_size: int = ..., + bits: int = ..., + mode: str = ..., + *, + stream: Stream | Device | None = ..., +) -> tuple[array, array, array]: + r""" + Quantize the matrix ``w`` using ``bits`` bits per element. + + Note, every ``group_size`` elements in a row of ``w`` are quantized + together. Hence, number of columns of ``w`` should be divisible by + ``group_size``. In particular, the rows of ``w`` are divided into groups of + size ``group_size`` which are quantized together. + + .. warning:: + + ``quantize`` currently only supports 2D inputs with the second + dimension divisible by ``group_size`` + + The supported quantization modes are ``"affine"`` and ``"mxfp4"``. They + are described in more detail below. + + Args: + w (array): Matrix to be quantized + group_size (int, optional): The size of the group in ``w`` that shares a + scale and bias. Default: ``64``. + bits (int, optional): The number of bits occupied by each element of + ``w`` in the returned quantized matrix. Default: ``4``. + mode (str, optional): The quantization mode. Default: ``"affine"``. + + Returns: + tuple: A tuple with either two or three elements containing: + + * w_q (array): The quantized version of ``w`` + * scales (array): The quantization scales + * biases (array): The quantization biases (returned for ``mode=="affine"``). + + Notes: + The ``affine`` mode quantizes groups of :math:`g` consecutive + elements in a row of ``w``. For each group the quantized + representation of each element :math:`\hat{w_i}` is computed as follows: + + .. math:: + + \begin{aligned} + \alpha &= \max_i w_i \\ + \beta &= \min_i w_i \\ + s &= \frac{\alpha - \beta}{2^b - 1} \\ + \hat{w_i} &= \textrm{round}\left( \frac{w_i - \beta}{s}\right). + \end{aligned} + + After the above computation, :math:`\hat{w_i}` fits in :math:`b` bits + and is packed in an unsigned 32-bit integer from the lower to upper + bits. For instance, for 4-bit quantization we fit 8 elements in an + unsigned 32 bit integer where the 1st element occupies the 4 least + significant bits, the 2nd bits 4-7 etc. + + To dequantize the elements of ``w``, we also save :math:`s` and + :math:`\beta` which are the returned ``scales`` and + ``biases`` respectively. + + The ``mxfp4`` mode similarly quantizes groups of :math:`g` elements + of ``w``. For ``mxfp4`` the group size must be ``32``. The elements + are quantized to 4-bit precision floating-point values (E2M1) with a + shared 8-bit scale per group. Unlike ``affine`` quantization, + ``mxfp4`` does not have a bias value. More details on the format can + be found in the `specification `_. + """ + +def quantized_matmul( + x: array, + w: array, + /, + scales: array, + biases: array | None = ..., + transpose: bool = ..., + group_size: int = ..., + bits: int = ..., + mode: str = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Perform the matrix multiplication with the quantized matrix ``w``. The + quantization uses one floating point scale and bias per ``group_size`` of + elements. Each element in ``w`` takes ``bits`` bits and is packed in an + unsigned 32 bit integer. + + Args: + x (array): Input array + w (array): Quantized matrix packed in unsigned integers + scales (array): The scales to use per ``group_size`` elements of ``w`` + biases (array, optional): The biases to use per ``group_size`` + elements of ``w``. Default: ``None``. + transpose (bool, optional): Defines whether to multiply with the + transposed ``w`` or not, namely whether we are performing + ``x @ w.T`` or ``x @ w``. Default: ``True``. + group_size (int, optional): The size of the group in ``w`` that + shares a scale and bias. Default: ``64``. + bits (int, optional): The number of bits occupied by each element in + ``w``. Default: ``4``. + mode (str, optional): The quantization mode. Default: ``"affine"``. + + Returns: + array: The result of the multiplication of ``x`` with ``w``. + """ + +def radians(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Convert angles from degrees to radians. + + Args: + a (array): Input array. + + Returns: + array: The angles in radians. + """ + +def real(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Returns the real part of a complex array. + + Args: + a (array): Input array. + + Returns: + array: The real part of ``a``. + """ + +def reciprocal(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise reciprocal. + + Args: + a (array): Input array. + + Returns: + array: The reciprocal of ``a``. + """ + +def remainder( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise remainder of division. + + Computes the remainder of dividing a with b with numpy-style + broadcasting semantics. Either or both input arrays can also be + scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The remainder of ``a // b``. + """ + +def repeat( + array: array, + repeats: int, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Repeat an array along a specified axis. + + Args: + array (array): Input array. + repeats (int): The number of repetitions for each element. + axis (int, optional): The axis in which to repeat the array along. If + unspecified it uses the flattened array of the input and repeats + along axis 0. + stream (Stream, optional): Stream or device. Defaults to ``None``. + + Returns: + array: The resulting repeated array. + """ + +def reset_peak_memory() -> None: + """Reset the peak memory to zero.""" + +def reshape( + a: array, /, shape: Sequence[int], *, stream: Stream | Device | None = ... +) -> array: + """ + Reshape an array while preserving the size. + + Args: + a (array): Input array. + shape (tuple(int)): New shape. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The reshaped array. + """ + +def right_shift( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise right shift. + + Shift the bits of the first input to the right by the second using + numpy-style broadcasting semantics. Either or both input arrays can + also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The bitwise right shift ``a >> b``. + """ + +def roll( + a: array, + shift: int | tuple[int], + axis: int | tuple[int] | None = ..., + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Roll array elements along a given axis. + + Elements that are rolled beyond the end of the array are introduced at + the beggining and vice-versa. + + If the axis is not provided the array is flattened, rolled and then the + shape is restored. + + Args: + a (array): Input array + shift (int or tuple(int)): The number of places by which elements + are shifted. If positive the array is rolled to the right, if + negative it is rolled to the left. If an int is provided but the + axis is a tuple then the same value is used for all axes. + axis (int or tuple(int), optional): The axis or axes along which to + roll the elements. + """ + +def round( + a: array, /, decimals: int = ..., stream: Stream | Device | None = ... +) -> array: + """ + Round to the given number of decimals. + + Basically performs: + + .. code-block:: python + + s = 10**decimals + x = round(x * s) / s + + Args: + a (array): Input array + decimals (int): Number of decimal places to round to. (default: 0) + + Returns: + array: An array of the same type as ``a`` rounded to the + given number of decimals. + """ + +def rsqrt(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise reciprocal and square root. + + Args: + a (array): Input array. + + Returns: + array: One over the square root of ``a``. + """ + +def save(file: str | pathlib.Path, arr: array) -> None: + """ + Save the array to a binary file in ``.npy`` format. + + Args: + file (str, pathlib.Path): File to which the array is saved + arr (array): Array to be saved. + """ + +def save_gguf( + file: str | pathlib.Path, + arrays: dict[str, array], + metadata: dict[str, array | str | list[str]], +): + """ + Save array(s) to a binary file in ``.gguf`` format. + + See the `GGUF documentation + `_ for + more information on the format. + + Args: + file (file, str, pathlib.Path): File in which the array is saved. + arrays (dict(str, array)): The dictionary of names to arrays to + be saved. + metadata (dict(str, array | str | list(str))): The dictionary + of metadata to be saved. The values can be a scalar or 1D + obj:`array`, a :obj:`str`, or a :obj:`list` of :obj:`str`. + """ + +def save_safetensors( + file: str | pathlib.Path, + arrays: dict[str, array], + metadata: dict[str, str] | None = ..., +): + """ + Save array(s) to a binary file in ``.safetensors`` format. + + See the `Safetensors documentation + `_ for more + information on the format. + + Args: + file (file, str, pathlib.Path): File in which the array is saved. + arrays (dict(str, array)): The dictionary of names to arrays to + be saved. + metadata (dict(str, str), optional): The dictionary of + metadata to be saved. + """ + +def savez(file: str | pathlib.Path, *args, **kwargs): + """ + Save several arrays to a binary file in uncompressed ``.npz`` + format. + + .. code-block:: python + + import mlx.core as mx + + x = mx.ones((10, 10)) + mx.savez("my_path.npz", x=x) + + import mlx.nn as nn + from mlx.utils import tree_flatten + + model = nn.TransformerEncoder(6, 128, 4) + flat_params = tree_flatten(model.parameters()) + mx.savez("model.npz", **dict(flat_params)) + + Args: + file (file, str, pathlib.Path): Path to file to which the arrays are saved. + *args (arrays): Arrays to be saved. + **kwargs (arrays): Arrays to be saved. Each array will be saved + with the associated keyword as the output file name. + """ + +def savez_compressed(file: str | pathlib.Path, *args, **kwargs): + """ + Save several arrays to a binary file in compressed ``.npz`` format. + + Args: + file (file, str, pathlib.Path): Path to file to which the arrays are saved. + *args (arrays): Arrays to be saved. + **kwargs (arrays): Arrays to be saved. Each array will be saved + with the associated keyword as the output file name. + """ + +def segmented_mm( + a: array, b: array, /, segments: array, *, stream: Stream | Device | None = ... +) -> array: + """ + Perform a matrix multiplication but segment the inner dimension and + save the result for each segment separately. + + Args: + a (array): Input array of shape ``MxK``. + b (array): Input array of shape ``KxN``. + segments (array): The offsets into the inner dimension for each segment. + + Returns: + array: The result per segment of shape ``MxN``. + """ + +def set_cache_limit(limit: int) -> int: + """ + Set the free cache limit. + + If using more than the given limit, free memory will be reclaimed + from the cache on the next allocation. To disable the cache, set + the limit to ``0``. + + The cache limit defaults to the memory limit. See + :func:`set_memory_limit` for more details. + + Args: + limit (int): The cache limit in bytes. + + Returns: + int: The previous cache limit in bytes. + """ + +def set_default_device(device: Device | DeviceType) -> None: + """Set the default device.""" + +def set_default_stream(stream: Stream) -> None: + """ + Set the default stream. + + This will make the given stream the default for the + streams device. It will not change the default device. + + Args: + stream (stream): Stream to make the default. + """ + +def set_memory_limit(limit: int) -> int: + """ + Set the memory limit. + + The memory limit is a guideline for the maximum amount of memory to use + during graph evaluation. If the memory limit is exceeded and there is no + more RAM (including swap when available) allocations will result in an + exception. + + When metal is available the memory limit defaults to 1.5 times the + maximum recommended working set size reported by the device. + + Args: + limit (int): Memory limit in bytes. + + Returns: + int: The previous memory limit in bytes. + """ + +def set_wired_limit(limit: int) -> int: + """ + Set the wired size limit. + + .. note:: + * This function is only useful on macOS 15.0 or higher. + * The wired limit should remain strictly less than the total + memory size. + + The wired limit is the total size in bytes of memory that will be kept + resident. The default value is ``0``. + + Setting a wired limit larger than system wired limit is an error. You can + increase the system wired limit with: + + .. code-block:: + + sudo sysctl iogpu.wired_limit_mb= + + Use :func:`device_info` to query the system wired limit + (``"max_recommended_working_set_size"``) and the total memory size + (``"memory_size"``). + + Args: + limit (int): The wired limit in bytes. + + Returns: + int: The previous wired limit in bytes. + """ + +def sigmoid(a: array, /, *, stream: Stream | Device | None = ...) -> array: + r""" + Element-wise logistic sigmoid. + + The logistic sigmoid function is: + + .. math:: + \mathrm{sigmoid}(x) = \frac{1}{1 + e^{-x}} + + Args: + a (array): Input array. + + Returns: + array: The logistic sigmoid of ``a``. + """ + +def sign(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise sign. + + Args: + a (array): Input array. + + Returns: + array: The sign of ``a``. + """ + +signedinteger: DtypeCategory = ... + +def sin(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise sine. + + Args: + a (array): Input array. + + Returns: + array: The sine of ``a``. + """ + +def sinh(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise hyperbolic sine. + + Args: + a (array): Input array. + + Returns: + array: The hyperbolic sine of ``a``. + """ + +def slice( + a: array, + start_indices: array, + axes: Sequence[int], + slice_size: Sequence[int], + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Extract a sub-array from the input array. + + Args: + a (array): Input array + start_indices (array): The index location to start the slice at. + axes (tuple(int)): The axes corresponding to the indices in ``start_indices``. + slice_size (tuple(int)): The size of the slice. + + Returns: + array: The sliced output array. + + Example: + + >>> a = mx.array([[1, 2, 3], [4, 5, 6]]) + >>> mx.slice(a, start_indices=mx.array(1), axes=(0,), slice_size=(1, 2)) + array([[4, 5]], dtype=int32) + >>> + >>> mx.slice(a, start_indices=mx.array(1), axes=(1,), slice_size=(2, 1)) + array([[2], + [5]], dtype=int32) + """ + +def slice_update( + a: array, + update: array, + start_indices: array, + axes: Sequence[int], + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Update a sub-array of the input array. + + Args: + a (array): The input array to update + update (array): The update array. + start_indices (array): The index location to start the slice at. + axes (tuple(int)): The axes corresponding to the indices in ``start_indices``. + + Returns: + array: The output array with the same shape and type as the input. + + Example: + + >>> a = mx.zeros((3, 3)) + >>> mx.slice_update(a, mx.ones((1, 2)), start_indices=mx.array(1, 1), axes=(0, 1)) + array([[0, 0, 0], + [0, 1, 0], + [0, 1, 0]], dtype=float32) + """ + +def softmax( + a: array, + /, + axis: int | Sequence[int] | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Perform the softmax along the given axis. + + This operation is a numerically stable version of: + + .. code-block:: + + exp(a) / sum(exp(a), axis, keepdims=True) + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or axes to compute + the softmax over. If unspecified this performs the softmax over + the full array. + + Returns: + array: The output of the softmax. + """ + +def sort( + a: array, + /, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Returns a sorted copy of the array. + + Args: + a (array): Input array. + axis (int or None, optional): Optional axis to sort over. + If ``None``, this sorts over the flattened array. + If unspecified, it defaults to -1 (sorting over the last axis). + + Returns: + array: The sorted array. + """ + +def split( + a: array, + /, + indices_or_sections: int | Sequence[int], + axis: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Split an array along a given axis. + + Args: + a (array): Input array. + indices_or_sections (int or list(int)): If ``indices_or_sections`` + is an integer the array is split into that many sections of equal + size. An error is raised if this is not possible. If ``indices_or_sections`` + is a list, the list contains the indices of the start of each subarray + along the given axis. + axis (int, optional): Axis to split along, defaults to `0`. + + Returns: + list(array): A list of split arrays. + """ + +def sqrt(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise square root. + + Args: + a (array): Input array. + + Returns: + array: The square root of ``a``. + """ + +def square(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise square. + + Args: + a (array): Input array. + + Returns: + array: The square of ``a``. + """ + +def squeeze( + a: array, + /, + axis: int | Sequence[int] | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Remove length one axes from an array. + + Args: + a (array): Input array. + axis (int or tuple(int), optional): Axes to remove. Defaults + to ``None`` in which case all size one axes are removed. + + Returns: + array: The output array with size one axes removed. + """ + +def stack( + arrays: list[array], + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Stacks the arrays along a new axis. + + Args: + arrays (list(array)): A list of arrays to stack. + axis (int, optional): The axis in the result array along which the + input arrays are stacked. Defaults to ``0``. + stream (Stream, optional): Stream or device. Defaults to ``None``. + + Returns: + array: The resulting stacked array. + """ + +def std( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + ddof: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Compute the standard deviation(s) over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + ddof (int, optional): The divisor to compute the variance + is ``N - ddof``, defaults to 0. + + Returns: + array: The output array of standard deviations. + """ + +def stop_gradient(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Stop gradients from being computed. + + The operation is the identity but it prevents gradients from flowing + through the array. + + Args: + a (array): Input array. + + Returns: + array: + The unchanged input ``a`` but without gradient flowing + through it. + """ + +def stream(s: Stream | Device) -> StreamContext: + """ + Create a context manager to set the default device and stream. + + Args: + s: The :obj:`Stream` or :obj:`Device` to set as the default. + + Returns: + A context manager that sets the default device and stream. + + Example: + + .. code-block::python + + import mlx.core as mx + + # Create a context manager for the default device and stream. + with mx.stream(mx.cpu): + # Operations here will use mx.cpu by default. + pass + """ + +def subtract( + a: scalar | array, + b: scalar | array, + stream: Stream | Device | None = ..., +) -> array: + """ + Element-wise subtraction. + + Subtract one array from another with numpy-style broadcasting semantics. Either or both + input arrays can also be scalars. + + Args: + a (array): Input array or scalar. + b (array): Input array or scalar. + + Returns: + array: The difference ``a - b``. + """ + +def sum( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Sum reduce the array over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + + Returns: + array: The output array with the corresponding axes reduced. + """ + +def swapaxes( + a: array, /, axis1: int, axis2: int, *, stream: Stream | Device | None = ... +) -> array: + """ + Swap two axes of an array. + + Args: + a (array): Input array. + axis1 (int): Specifies the first axis. + axis2 (int): Specifies the second axis. + + Returns: + array: The array with swapped axes. + """ + +def synchronize(stream: Stream | None = ...) -> None: + """ + Synchronize with the given stream. + + Args: + stream (Stream, optional): The stream to synchronize with. If ``None`` + then the default stream of the default device is used. + Default: ``None``. + """ + +def take( + a: array, + /, + indices: int | array, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Take elements along an axis. + + The elements are taken from ``indices`` along the specified axis. + If the axis is not specified the array is treated as a flattened + 1-D array prior to performing the take. + + As an example, if the ``axis=1`` this is equivalent to ``a[:, indices, ...]``. + + Args: + a (array): Input array. + indices (int or array): Integer index or input array with integral type. + axis (int, optional): Axis along which to perform the take. If unspecified + the array is treated as a flattened 1-D vector. + + Returns: + array: The indexed values of ``a``. + """ + +def take_along_axis( + a: array, + /, + indices: array, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Take values along an axis at the specified indices. + + Args: + a (array): Input array. + indices (array): Indices array. These should be broadcastable with + the input array excluding the `axis` dimension. + axis (int or None): Axis in the input to take the values from. If + ``axis == None`` the array is flattened to 1D prior to the indexing + operation. + + Returns: + array: The output array. + """ + +def tan(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise tangent. + + Args: + a (array): Input array. + + Returns: + array: The tangent of ``a``. + """ + +def tanh(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + Element-wise hyperbolic tangent. + + Args: + a (array): Input array. + + Returns: + array: The hyperbolic tangent of ``a``. + """ + +def tensordot( + a: array, + b: array, + /, + axes: int | list[Sequence[int]] = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Compute the tensor dot product along the specified axes. + + Args: + a (array): Input array + b (array): Input array + axes (int or list(list(int)), optional): The number of dimensions to + sum over. If an integer is provided, then sum over the last + ``axes`` dimensions of ``a`` and the first ``axes`` dimensions of + ``b``. If a list of lists is provided, then sum over the + corresponding dimensions of ``a`` and ``b``. Default: 2. + + Returns: + array: The tensor dot product. + """ + +def tile( + a: array, + reps: int | Sequence[int], + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Construct an array by repeating ``a`` the number of times given by ``reps``. + + Args: + a (array): Input array + reps (int or list(int)): The number of times to repeat ``a`` along each axis. + + Returns: + array: The tiled array. + """ + +def topk( + a: array, + /, + k: int, + axis: int | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Returns the ``k`` largest elements from the input along a given axis. + + The elements will not necessarily be in sorted order. + + Args: + a (array): Input array. + k (int): ``k`` top elements to be returned + axis (int or None, optional): Optional axis to select over. + If ``None``, this selects the top ``k`` elements over the + flattened array. If unspecified, it defaults to ``-1``. + + Returns: + array: The top ``k`` elements from the input. + """ + +def trace( + a: array, + /, + offset: int = ..., + axis1: int = ..., + axis2: int = ..., + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Return the sum along a specified diagonal in the given array. + + Args: + a (array): Input array + offset (int, optional): Offset of the diagonal from the main diagonal. + Can be positive or negative. Default: ``0``. + axis1 (int, optional): The first axis of the 2-D sub-arrays from which + the diagonals should be taken. Default: ``0``. + axis2 (int, optional): The second axis of the 2-D sub-arrays from which + the diagonals should be taken. Default: ``1``. + dtype (Dtype, optional): Data type of the output array. If + unspecified the output type is inferred from the input array. + + Returns: + array: Sum of specified diagonal. + """ + +def transpose( + a: array, + /, + axes: Sequence[int] | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Transpose the dimensions of the array. + + Args: + a (array): Input array. + axes (list(int), optional): Specifies the source axis for each axis + in the new array. The default is to reverse the axes. + + Returns: + array: The transposed array. + """ + +def tri( + n: int, + m: int, + k: int, + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + An array with ones at and below the given diagonal and zeros elsewhere. + + Args: + n (int): The number of rows in the output. + m (int, optional): The number of cols in the output. Defaults to ``None``. + k (int, optional): The diagonal of the 2-D array. Defaults to ``0``. + dtype (Dtype, optional): Data type of the output array. Defaults to ``float32``. + stream (Stream, optional): Stream or device. Defaults to ``None``. + + Returns: + array: Array with its lower triangle filled with ones and zeros elsewhere + """ + +def tril(x: array, k: int, *, stream: Stream | Device | None = ...) -> array: + """ + Zeros the array above the given diagonal. + + Args: + x (array): input array. + k (int, optional): The diagonal of the 2-D array. Defaults to ``0``. + stream (Stream, optional): Stream or device. Defaults to ``None``. + + Returns: + array: Array zeroed above the given diagonal + """ + +def triu(x: array, k: int, *, stream: Stream | Device | None = ...) -> array: + """ + Zeros the array below the given diagonal. + + Args: + x (array): input array. + k (int, optional): The diagonal of the 2-D array. Defaults to ``0``. + stream (Stream, optional): Stream or device. Defaults to ``None``. + + Returns: + array: Array zeroed below the given diagonal + """ + +uint16: Dtype = ... +uint32: Dtype = ... +uint64: Dtype = ... +uint8: Dtype = ... + +def unflatten( + a: array, + /, + axis: int, + shape: Sequence[int], + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Unflatten an axis of an array to a shape. + + Args: + a (array): Input array. + axis (int): The axis to unflatten. + shape (tuple(int)): The shape to unflatten to. At most one + entry can be ``-1`` in which case the corresponding size will be + inferred. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The unflattened array. + + Example: + >>> a = mx.array([1, 2, 3, 4]) + >>> mx.unflatten(a, 0, (2, -1)) + array([[1, 2], [3, 4]], dtype=int32) + """ + +unsignedinteger: DtypeCategory = ... + +def value_and_grad( + fun: Callable, + argnums: int | Sequence[int] | None = ..., + argnames: str | Sequence[str] = ..., +) -> Callable: + """ + Returns a function which computes the value and gradient of ``fun``. + + The function passed to :func:`value_and_grad` should return either + a scalar loss or a tuple in which the first element is a scalar + loss and the remaining elements can be anything. + + .. code-block:: python + + import mlx.core as mx + + def mse(params, inputs, targets): + outputs = forward(params, inputs) + lvalue = (outputs - targets).square().mean() + return lvalue + + # Returns lvalue, dlvalue/dparams + lvalue, grads = mx.value_and_grad(mse)(params, inputs, targets) + + def lasso(params, inputs, targets, a=1.0, b=1.0): + outputs = forward(params, inputs) + mse = (outputs - targets).square().mean() + l1 = mx.abs(outputs - targets).mean() + + loss = a*mse + b*l1 + + return loss, mse, l1 + + (loss, mse, l1), grads = mx.value_and_grad(lasso)(params, inputs, targets) + + Args: + fun (Callable): A function which takes a variable number of + :class:`array` or trees of :class:`array` and returns + a scalar output :class:`array` or a tuple the first element + of which should be a scalar :class:`array`. + argnums (int or list(int), optional): Specify the index (or indices) + of the positional arguments of ``fun`` to compute the gradient + with respect to. If neither ``argnums`` nor ``argnames`` are + provided ``argnums`` defaults to ``0`` indicating ``fun``'s first + argument. + argnames (str or list(str), optional): Specify keyword arguments of + ``fun`` to compute gradients with respect to. It defaults to [] so + no gradients for keyword arguments by default. + + Returns: + Callable: A function which returns a tuple where the first element + is the output of `fun` and the second element is the gradients w.r.t. + the loss. + """ + +def var( + a: array, + /, + axis: int | Sequence[int] | None = ..., + keepdims: bool = ..., + ddof: int = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Compute the variance(s) over the given axes. + + Args: + a (array): Input array. + axis (int or list(int), optional): Optional axis or + axes to reduce over. If unspecified this defaults + to reducing over the entire array. + keepdims (bool, optional): Keep reduced axes as + singleton dimensions, defaults to `False`. + ddof (int, optional): The divisor to compute the variance + is ``N - ddof``, defaults to 0. + + Returns: + array: The output array of variances. + """ + +def view( + a: scalar | array, dtype: Dtype, stream: Stream | Device | None = ... +) -> array: + """ + View the array as a different type. + + The output shape changes along the last axis if the input array's + type and the input ``dtype`` do not have the same size. + + Note: the view op does not imply that the input and output arrays share + their underlying data. The view only gaurantees that the binary + representation of each element (or group of elements) is the same. + + Args: + a (array): Input array or scalar. + dtype (Dtype): The data type to change to. + + Returns: + array: The array with the new type. + """ + +def vjp( + fun: Callable, primals: list[array], cotangents: list[array] +) -> tuple[list[array], list[array]]: + """ + Compute the vector-Jacobian product. + + Computes the product of the ``cotangents`` with the Jacobian of a + function ``fun`` evaluated at ``primals``. + + Args: + fun (Callable): A function which takes a variable number of :class:`array` + and returns a single :class:`array` or list of :class:`array`. + primals (list(array)): A list of :class:`array` at which to + evaluate the Jacobian. + cotangents (list(array)): A list of :class:`array` which are the + "vector" in the vector-Jacobian product. The ``cotangents`` should be the + same in number, shape, and type as the outputs of ``fun``. + + Returns: + list(array): A list of the vector-Jacobian products which + is the same in number, shape, and type of the outputs of ``fun``. + """ + +def vmap(fun: Callable, in_axes: object = ..., out_axes: object = ...) -> Callable: + """ + Returns a vectorized version of ``fun``. + + Args: + fun (Callable): A function which takes a variable number of + :class:`array` or a tree of :class:`array` and returns + a variable number of :class:`array` or a tree of :class:`array`. + in_axes (int, optional): An integer or a valid prefix tree of the + inputs to ``fun`` where each node specifies the vmapped axis. If + the value is ``None`` then the corresponding input(s) are not vmapped. + Defaults to ``0``. + out_axes (int, optional): An integer or a valid prefix tree of the + outputs of ``fun`` where each node specifies the vmapped axis. If + the value is ``None`` then the corresponding outputs(s) are not vmapped. + Defaults to ``0``. + + Returns: + Callable: The vectorized function. + """ + +def where( + condition: scalar | array, + x: scalar | array, + y: scalar | array, + /, + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Select from ``x`` or ``y`` according to ``condition``. + + The condition and input arrays must be the same shape or + broadcastable with each another. + + Args: + condition (array): The condition array. + x (array): The input selected from where condition is ``True``. + y (array): The input selected from where condition is ``False``. + + Returns: + array: The output containing elements selected from + ``x`` and ``y``. + """ + +def zeros( + shape: int | Sequence[int], + dtype: Dtype | None = ..., + *, + stream: Stream | Device | None = ..., +) -> array: + """ + Construct an array of zeros. + + Args: + shape (int or list(int)): The shape of the output array. + dtype (Dtype, optional): Data type of the output array. If + unspecified the output type defaults to ``float32``. + + Returns: + array: The array of zeros with the specified shape. + """ + +def zeros_like(a: array, /, *, stream: Stream | Device | None = ...) -> array: + """ + An array of zeros like the input. + + Args: + a (array): The input to take the shape and type from. + + Returns: + array: The output array filled with zeros. + """ + +scalar: TypeAlias = int | float | bool +list_or_scalar: TypeAlias = scalar | list["list_or_scalar"] +bool_: Dtype = ... diff --git a/.mlx_typings/mlx/core/cuda/__init__.pyi b/.mlx_typings/mlx/core/cuda/__init__.pyi new file mode 100644 index 00000000..cb7e23ba --- /dev/null +++ b/.mlx_typings/mlx/core/cuda/__init__.pyi @@ -0,0 +1,2 @@ +def is_available() -> bool: + """Check if the CUDA back-end is available.""" diff --git a/.mlx_typings/mlx/core/distributed/__init__.pyi b/.mlx_typings/mlx/core/distributed/__init__.pyi new file mode 100644 index 00000000..15a952c4 --- /dev/null +++ b/.mlx_typings/mlx/core/distributed/__init__.pyi @@ -0,0 +1,216 @@ +from typing import Sequence + +from mlx.core import Device, Dtype, Stream, array + +class Group: + """ + An :class:`mlx.core.distributed.Group` represents a group of independent mlx + processes that can communicate. + """ + def rank(self) -> int: + """Get the rank of this process""" + + def size(self) -> int: + """Get the size of the group""" + + def split(self, color: int, key: int = ...) -> Group: + """ + Split the group to subgroups based on the provided color. + + Processes that use the same color go to the same group. The ``key`` + argument defines the rank in the new group. The smaller the key the + smaller the rank. If the key is negative then the rank in the + current group is used. + + Args: + color (int): A value to group processes into subgroups. + key (int, optional): A key to optionally change the rank ordering + of the processes. + """ + +def all_gather( + x: array, *, group: Group | None = ..., stream: Stream | Device | None = ... +) -> array: + """ + Gather arrays from all processes. + + Gather the ``x`` arrays from all processes in the group and concatenate + them along the first axis. The arrays should all have the same shape. + + Args: + x (array): Input array. + group (Group): The group of processes that will participate in the + gather. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The concatenation of all ``x`` arrays. + """ + +def all_max( + x: array, *, group: Group | None = ..., stream: Stream | Device | None = ... +) -> array: + """ + All reduce max. + + Find the maximum of the ``x`` arrays from all processes in the group. + + Args: + x (array): Input array. + group (Group): The group of processes that will participate in the + reduction. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The maximum of all ``x`` arrays. + """ + +def all_min( + x: array, *, group: Group | None = ..., stream: Stream | Device | None = ... +) -> array: + """ + All reduce min. + + Find the minimum of the ``x`` arrays from all processes in the group. + + Args: + x (array): Input array. + group (Group): The group of processes that will participate in the + reduction. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The minimum of all ``x`` arrays. + """ + +def all_sum( + x: array, *, group: Group | None = ..., stream: Stream | Device | None = ... +) -> array: + """ + All reduce sum. + + Sum the ``x`` arrays from all processes in the group. + + Args: + x (array): Input array. + group (Group): The group of processes that will participate in the + reduction. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The sum of all ``x`` arrays. + """ + +def init(strict: bool = ..., backend: str = ...) -> Group: + """ + Initialize the communication backend and create the global communication group. + + Example: + + .. code:: python + + import mlx.core as mx + + group = mx.distributed.init(backend="ring") + + Args: + strict (bool, optional): If set to False it returns a singleton group + in case ``mx.distributed.is_available()`` returns False otherwise + it throws a runtime error. Default: ``False`` + backend (str, optional): Which distributed backend to initialize. + Possible values ``mpi``, ``ring``, ``nccl``, ``any``. If set to ``any`` all + available backends are tried and the first one that succeeds + becomes the global group which will be returned in subsequent + calls. Default: ``any`` + + Returns: + Group: The group representing all the launched processes. + """ + +def is_available() -> bool: + """Check if a communication backend is available.""" + +def recv( + shape: Sequence[int], + dtype: Dtype, + src: int, + *, + group: Group | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Recv an array with shape ``shape`` and dtype ``dtype`` from process + with rank ``src``. + + Args: + shape (tuple[int]): The shape of the array we are receiving. + dtype (Dtype): The data type of the array we are receiving. + src (int): Rank of the source process in the group. + group (Group): The group of processes that will participate in the + recv. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The array that was received from ``src``. + """ + +def recv_like( + x: array, + src: int, + *, + group: Group | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Recv an array with shape and type like ``x`` from process with rank + ``src``. + + It is equivalent to calling ``mx.distributed.recv(x.shape, x.dtype, src)``. + + Args: + x (array): An array defining the shape and dtype of the array we are + receiving. + src (int): Rank of the source process in the group. + group (Group): The group of processes that will participate in the + recv. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: The array that was received from ``src``. + """ + +def send( + x: array, + dst: int, + *, + group: Group | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Send an array from the current process to the process that has rank + ``dst`` in the group. + + Args: + x (array): Input array. + dst (int): Rank of the destination process in the group. + group (Group): The group of processes that will participate in the + sned. If set to ``None`` the global group is used. Default: + ``None``. + stream (Stream, optional): Stream or device. Defaults to ``None`` + in which case the default stream of the default device is used. + + Returns: + array: An array identical to ``x`` which when evaluated the send is performed. + """ diff --git a/.mlx_typings/mlx/core/metal/__init__.pyi b/.mlx_typings/mlx/core/metal/__init__.pyi new file mode 100644 index 00000000..983f0067 --- /dev/null +++ b/.mlx_typings/mlx/core/metal/__init__.pyi @@ -0,0 +1,38 @@ +def clear_cache() -> None: ... +def device_info() -> dict[str, str | int]: + """ + Get information about the GPU device and system settings. + + Currently returns: + + * ``architecture`` + * ``max_buffer_size`` + * ``max_recommended_working_set_size`` + * ``memory_size`` + * ``resource_limit`` + + Returns: + dict: A dictionary with string keys and string or integer values. + """ + +def get_active_memory() -> int: ... +def get_cache_memory() -> int: ... +def get_peak_memory() -> int: ... +def is_available() -> bool: + """Check if the Metal back-end is available.""" + +def reset_peak_memory() -> None: ... +def set_cache_limit(limit: int) -> int: ... +def set_memory_limit(limit: int) -> int: ... +def set_wired_limit(limit: int) -> int: ... +def start_capture(path: str) -> None: + """ + Start a Metal capture. + + Args: + path (str): The path to save the capture which should have + the extension ``.gputrace``. + """ + +def stop_capture() -> None: + """Stop a Metal capture.""" diff --git a/.mlx_typings/mlx/core/random/__init__.pyi b/.mlx_typings/mlx/core/random/__init__.pyi new file mode 100644 index 00000000..4116e0ec --- /dev/null +++ b/.mlx_typings/mlx/core/random/__init__.pyi @@ -0,0 +1,301 @@ +from typing import Sequence + +from mlx.core import Device, Dtype, Stream, array, scalar +from mlx.core.distributed import state as state + +def bernoulli( + p: scalar | array = ..., + shape: Sequence[int] | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate Bernoulli random values. + + The values are sampled from the bernoulli distribution with parameter + ``p``. The parameter ``p`` can be a :obj:`float` or :obj:`array` and + must be broadcastable to ``shape``. + + Args: + p (float or array, optional): Parameter of the Bernoulli + distribution. Default: ``0.5``. + shape (list(int), optional): Shape of the output. + Default: ``p.shape``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The array of random integers. + """ + +def categorical( + logits: array, + axis: int = ..., + shape: Sequence[int] | None = ..., + num_samples: int | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Sample from a categorical distribution. + + The values are sampled from the categorical distribution specified by + the unnormalized values in ``logits``. Note, at most one of ``shape`` + or ``num_samples`` can be specified. If both are ``None``, the output + has the same shape as ``logits`` with the ``axis`` dimension removed. + + Args: + logits (array): The *unnormalized* categorical distribution(s). + axis (int, optional): The axis which specifies the distribution. + Default: ``-1``. + shape (list(int), optional): The shape of the output. This must + be broadcast compatible with ``logits.shape`` with the ``axis`` + dimension removed. Default: ``None`` + num_samples (int, optional): The number of samples to draw from each + of the categorical distributions in ``logits``. The output will have + ``num_samples`` in the last dimension. Default: ``None``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The ``shape``-sized output array with type ``uint32``. + """ + +def gumbel( + shape: Sequence[int] = ..., + dtype: Dtype | None = ..., + key: Stream | Device | None = ..., + stream: array | None = ..., +) -> array: + """ + Sample from the standard Gumbel distribution. + + The values are sampled from a standard Gumbel distribution + which CDF ``exp(-exp(-x))``. + + Args: + shape (list(int)): The shape of the output. + dtype (Dtype, optional): The data type of the output. + Default: ``float32``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: + The :class:`array` with shape ``shape`` and distributed according + to the Gumbel distribution. + """ + +def key(seed: int) -> array: + """ + Get a PRNG key from a seed. + + Args: + seed (int): Seed for the PRNG. + + Returns: + array: The PRNG key array. + """ + +def laplace( + shape: Sequence[int] = ..., + dtype: Dtype | None = ..., + loc: float = ..., + scale: float = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Sample numbers from a Laplace distribution. + + Args: + shape (list(int), optional): Shape of the output. Default: ``()``. + dtype (Dtype, optional): Type of the output. Default: ``float32``. + loc (float, optional): Mean of the distribution. Default: ``0.0``. + scale (float, optional): The scale "b" of the Laplace distribution. + Default:``1.0``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The output array of random values. + """ + +def multivariate_normal( + mean: array, + cov: array, + shape: Sequence[int] = ..., + dtype: Dtype | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate jointly-normal random samples given a mean and covariance. + + The matrix ``cov`` must be positive semi-definite. The behavior is + undefined if it is not. The only supported ``dtype`` is ``float32``. + + Args: + mean (array): array of shape ``(..., n)``, the mean of the + distribution. + cov (array): array of shape ``(..., n, n)``, the covariance + matrix of the distribution. The batch shape ``...`` must be + broadcast-compatible with that of ``mean``. + shape (list(int), optional): The output shape must be + broadcast-compatible with ``mean.shape[:-1]`` and ``cov.shape[:-2]``. + If empty, the result shape is determined by broadcasting the batch + shapes of ``mean`` and ``cov``. Default: ``[]``. + dtype (Dtype, optional): The output type. Default: ``float32``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The output array of random values. + """ + +def normal( + shape: Sequence[int] = ..., + dtype: Dtype | None = ..., + loc: scalar | array | None = ..., + scale: scalar | array | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + r""" + Generate normally distributed random numbers. + + If ``loc`` and ``scale`` are not provided the "standard" normal + distribution is used. That means $x \sim \mathcal{N}(0, 1)$ for + real numbers and $\text{Re}(x),\text{Im}(x) \sim \mathcal{N}(0, + \frac{1}{2})$ for complex numbers. + + Args: + shape (list(int), optional): Shape of the output. Default: ``()``. + dtype (Dtype, optional): Type of the output. Default: ``float32``. + loc (scalar or array, optional): Mean of the distribution. + Default: ``None``. + scale (scalar or array, optional): Standard deviation of the + distribution. Default: ``None``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The output array of random values. + """ + +def permutation( + x: int | array, + axis: int = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate a random permutation or permute the entries of an array. + + Args: + x (int or array, optional): If an integer is provided a random + permtuation of ``mx.arange(x)`` is returned. Otherwise the entries + of ``x`` along the given axis are randomly permuted. + axis (int, optional): The axis to permute along. Default: ``0``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: + The generated random permutation or randomly permuted input array. + """ + +def randint( + low: scalar | array, + high: scalar | array, + shape: Sequence[int] = ..., + dtype: Dtype | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate random integers from the given interval. + + The values are sampled with equal probability from the integers in + half-open interval ``[low, high)``. The lower and upper bound can be + scalars or arrays and must be broadcastable to ``shape``. + + Args: + low (scalar or array): Lower bound of the interval. + high (scalar or array): Upper bound of the interval. + shape (list(int), optional): Shape of the output. Default: ``()``. + dtype (Dtype, optional): Type of the output. Default: ``int32``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The array of random integers. + """ + +def seed(seed: int) -> None: + """ + Seed the global PRNG. + + Args: + seed (int): Seed for the global PRNG. + """ + +def split(key: array, num: int = ..., stream: Stream | Device | None = ...) -> array: + """ + Split a PRNG key into sub keys. + + Args: + key (array): Input key to split. + num (int, optional): Number of sub keys. Default: ``2``. + + Returns: + array: The array of sub keys with ``num`` as its first dimension. + """ + +def truncated_normal( + lower: scalar | array, + upper: scalar | array, + shape: Sequence[int] | None = ..., + dtype: Dtype | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate values from a truncated normal distribution. + + The values are sampled from the truncated normal distribution + on the domain ``(lower, upper)``. The bounds ``lower`` and ``upper`` + can be scalars or arrays and must be broadcastable to ``shape``. + + Args: + lower (scalar or array): Lower bound of the domain. + upper (scalar or array): Upper bound of the domain. + shape (list(int), optional): The shape of the output. + Default:``()``. + dtype (Dtype, optional): The data type of the output. + Default: ``float32``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The output array of random values. + """ + +def uniform( + low: scalar | array = ..., + high: scalar | array = ..., + shape: Sequence[int] = ..., + dtype: Dtype | None = ..., + key: array | None = ..., + stream: Stream | Device | None = ..., +) -> array: + """ + Generate uniformly distributed random numbers. + + The values are sampled uniformly in the half-open interval ``[low, high)``. + The lower and upper bound can be scalars or arrays and must be + broadcastable to ``shape``. + + Args: + low (scalar or array, optional): Lower bound of the distribution. + Default: ``0``. + high (scalar or array, optional): Upper bound of the distribution. + Default: ``1``. + shape (list(int), optional): Shape of the output. Default:``()``. + dtype (Dtype, optional): Type of the output. Default: ``float32``. + key (array, optional): A PRNG key. Default: ``None``. + + Returns: + array: The output array random values. + """ diff --git a/.mlx_typings/mlx/nn/__init__.pyi b/.mlx_typings/mlx/nn/__init__.pyi new file mode 100644 index 00000000..4c999379 --- /dev/null +++ b/.mlx_typings/mlx/nn/__init__.pyi @@ -0,0 +1,9 @@ +""" +This type stub file was generated by pyright. +""" + +from layers import * +from utils import * + +from . import init as init +from . import losses as losses diff --git a/.mlx_typings/mlx/nn/init.pyi b/.mlx_typings/mlx/nn/init.pyi new file mode 100644 index 00000000..efa453e6 --- /dev/null +++ b/.mlx_typings/mlx/nn/init.pyi @@ -0,0 +1,284 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Callable, Literal + +import mlx.core as mx + +def constant(value: float, dtype: mx.Dtype = ...) -> Callable[[mx.array], mx.array]: + r"""An initializer that returns an array filled with ``value``. + + Args: + value (float): The value to fill the array with. + dtype (Dtype, optional): The data type of the array. Default: + ``float32``. + + Returns: + Callable[[array], array]: An initializer that returns an array with the + same shape as the input, filled with ``value``. + + Example: + + >>> init_fn = nn.init.constant(0.5) + >>> init_fn(mx.zeros((2, 2))) + array([[0.5, 0.5], + [0.5, 0.5]], dtype=float32) + """ + +def normal( + mean: float = ..., std: float = ..., dtype: mx.Dtype = ... +) -> Callable[[mx.array], mx.array]: + r"""An initializer that returns samples from a normal distribution. + + Args: + mean (float, optional): Mean of the normal distribution. Default: + ``0.0``. + std (float, optional): Standard deviation of the normal distribution. + Default: ``1.0``. + dtype (Dtype, optional): The data type of the array. Default: + ``float32``. + + Returns: + Callable[[array], array]: An initializer that returns an array with the + same shape as the input, filled with samples from a normal distribution. + + Example: + + >>> init_fn = nn.init.normal() + >>> init_fn(mx.zeros((2, 2))) + array([[-0.982273, -0.534422], + [0.380709, 0.0645099]], dtype=float32) + """ + +def uniform( + low: float = ..., high: float = ..., dtype: mx.Dtype = ... +) -> Callable[[mx.array], mx.array]: + r"""An initializer that returns samples from a uniform distribution. + + Args: + low (float, optional): The lower bound of the uniform distribution. + Default: ``0.0``. + high (float, optional): The upper bound of the uniform distribution. + Default: ``1.0`` + dtype (Dtype, optional): The data type of the array. Default: ``float32``. + + Returns: + Callable[[array], array]: An initializer that returns an array + with the same shape as the input, filled with samples from a uniform + distribution + + Example: + + >>> init_fn = nn.init.uniform(low=0, high=1) + >>> init_fn(mx.zeros((2, 2))) + array([[0.883935, 0.863726], + [0.617261, 0.417497]], dtype=float32) + """ + +def identity(dtype: mx.Dtype = ...) -> Callable[[mx.array], mx.array]: + r"""An initializer that returns an identity matrix. + + Args: + dtype (Dtype, optional): The data type of the array. Defaults: + ``float32``. + + Returns: + Callable[[array], array]: An initializer that returns an identity + matrix with the same shape as the input. + + Example: + + >>> init_fn = nn.init.identity() + >>> init_fn(mx.zeros((2, 2))) + array([[1, 0], + [0, 1]], dtype=float32) + """ + +def glorot_normal(dtype: mx.Dtype = ...) -> Callable[[mx.array, float], mx.array]: + r"""A Glorot normal initializer. + + This initializer samples from a normal distribution with a standard + deviation computed from the number of input (``fan_in``) and output + (``fan_out``) units according to: + + .. math:: + \sigma = \gamma \sqrt{\frac{2.0}{\text{fan\_in} + \text{fan\_out}}} + + For more details see the original reference: `Understanding the difficulty + of training deep feedforward neural networks + `_ + + Args: + dtype (Dtype, optional): The data type of the array. Default: ``float32``. + + Returns: + Callable[[array, float], array]: An initializer that returns an array + with the same shape as the input, filled with samples from the Glorot + normal distribution. + + Example: + + >>> init_fn = nn.init.glorot_normal() + >>> init_fn(mx.zeros((2, 2))) + array([[0.191107, 1.61278], + [-0.150594, -0.363207]], dtype=float32) + >>> init_fn(mx.zeros((2, 2)), gain=4.0) + array([[1.89613, -4.53947], + [4.48095, 0.995016]], dtype=float32) + """ + +def glorot_uniform(dtype: mx.Dtype = ...) -> Callable[[mx.array, float], mx.array]: + r"""A Glorot uniform initializer. + + This initializer samples from a uniform distribution with a range + computed from the number of input (``fan_in``) and output (``fan_out``) + units according to: + + .. math:: + \sigma = \gamma \sqrt{\frac{6.0}{\text{fan\_in} + \text{fan\_out}}} + + For more details see the original reference: `Understanding the difficulty + of training deep feedforward neural networks + `_ + + Args: + dtype (Dtype, optional): The data type of the array. Default: ``float32``. + + Returns: + Callable[[array, float], array]: An initializer that returns an array + with the same shape as the input, filled with samples from the Glorot + uniform distribution. + + Example: + + >>> init_fn = nn.init.glorot_uniform() + >>> init_fn(mx.zeros((2, 2))) + array([[0.223404, -0.890597], + [-0.379159, -0.776856]], dtype=float32) + >>> init_fn(mx.zeros((2, 2)), gain=4.0) + array([[-1.90041, 3.02264], + [-0.912766, 4.12451]], dtype=float32) + """ + +def he_normal( + dtype: mx.Dtype = ..., +) -> Callable[[mx.array, Literal["fan_in", "fan_out"], float], mx.array]: + r"""Build a He normal initializer. + + This initializer samples from a normal distribution with a standard + deviation computed from the number of input (``fan_in``) or output + (``fan_out``) units according to: + + .. math:: + \sigma = \gamma \frac{1}{\sqrt{\text{fan}}} + + where :math:`\text{fan}` is either the number of input units when the + ``mode`` is ``"fan_in"`` or output units when the ``mode`` is + ``"fan_out"``. + + For more details see the original reference: `Delving Deep into Rectifiers: + Surpassing Human-Level Performance on ImageNet Classification + `_ + + Args: + dtype (Dtype, optional): The data type of the array. Defaults to mx.float32. + + Returns: + Callable[[array, str, float], array]: An initializer that returns an + array with the same shape as the input, filled with samples from the He + normal distribution. + + Example: + + >>> init_fn = nn.init.he_normal() + >>> init_fn(mx.zeros((2, 2))) # uses fan_in + array([[-1.25211, 0.458835], + [-0.177208, -0.0137595]], dtype=float32) + >>> init_fn(mx.zeros((2, 2)), mode="fan_out", gain=5) + array([[5.6967, 4.02765], + [-4.15268, -2.75787]], dtype=float32) + """ + +def he_uniform( + dtype: mx.Dtype = ..., +) -> Callable[[mx.array, Literal["fan_in", "fan_out"], float], mx.array]: + r"""A He uniform (Kaiming uniform) initializer. + + This initializer samples from a uniform distribution with a range + computed from the number of input (``fan_in``) or output (``fan_out``) + units according to: + + .. math:: + + \sigma = \gamma \sqrt{\frac{3.0}{\text{fan}}} + + where :math:`\text{fan}` is either the number of input units when the + ``mode`` is ``"fan_in"`` or output units when the ``mode`` is + ``"fan_out"``. + + For more details see the original reference: `Delving Deep into Rectifiers: + Surpassing Human-Level Performance on ImageNet Classification + `_ + + + Args: + dtype (Dtype, optional): The data type of the array. Default: ``float32``. + + Returns: + Callable[[array, str, float], array]: An initializer that returns an + array with the same shape as the input, filled with samples from the + He uniform distribution. + + Example: + + >>> init_fn = nn.init.he_uniform() + >>> init_fn(mx.zeros((2, 2))) # uses fan_in + array([[0.0300242, -0.0184009], + [0.793615, 0.666329]], dtype=float32) + >>> init_fn(mx.zeros((2, 2)), mode="fan_out", gain=5) + array([[-1.64331, -2.16506], + [1.08619, 5.79854]], dtype=float32) + """ + +def sparse( + sparsity: float, mean: float = ..., std: float = ..., dtype: mx.Dtype = ... +) -> Callable[[mx.array], mx.array]: + r"""An initializer that returns a sparse matrix. + + Args: + sparsity (float): The fraction of elements in each column to be set to + zero. + mean (float, optional): Mean of the normal distribution. Default: + ``0.0``. + std (float, optional): Standard deviation of the normal distribution. + Default: ``1.0``. + dtype (Dtype, optional): The data type of the array. Default: + ``float32``. + + Returns: + Callable[[array], array]: An initializer that returns an array with the + same shape as the input, filled with samples from a normal distribution. + + Example: + + >>> init_fn = nn.init.sparse(sparsity=0.5) + >>> init_fn(mx.zeros((2, 2))) + array([[-1.91187, -0.117483], + [0, 0]], dtype=float32) + """ + +def orthogonal( + gain: float = ..., dtype: mx.Dtype = ... +) -> Callable[[mx.array], mx.array]: + r"""An initializer that returns an orthogonal matrix. + + Args: + gain (float, optional): Scaling factor for the orthogonal matrix. + Default: ``1.0``. + dtype (Dtype, optional): Data type of the array. Default: ``float32``. + + Returns: + Callable[[array], array]: An initializer that returns + an orthogonal matrix with the same shape as the input. + """ diff --git a/.mlx_typings/mlx/nn/layers/__init__.pyi b/.mlx_typings/mlx/nn/layers/__init__.pyi new file mode 100644 index 00000000..f22856cd --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/__init__.pyi @@ -0,0 +1,20 @@ +""" +This type stub file was generated by pyright. +""" + +from activations import * +from base import * +from containers import * +from convolution import * +from convolution_transpose import * +from distributed import * +from dropout import * +from embedding import * +from linear import * +from normalization import * +from pooling import * +from positional_encoding import * +from quantized import * +from recurrent import * +from transformer import * +from upsample import * diff --git a/.mlx_typings/mlx/nn/layers/activations.pyi b/.mlx_typings/mlx/nn/layers/activations.pyi new file mode 100644 index 00000000..adacb4da --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/activations.pyi @@ -0,0 +1,523 @@ +""" +This type stub file was generated by pyright. +""" + +from functools import partial +from typing import Any + +import mlx.core as mx +from base import Module + +@partial(mx.compile, shapeless=True) +def sigmoid(x: mx.array) -> mx.array: + r"""Applies the sigmoid function. + + .. math:: + \text{Sigmoid}(x) = \sigma(x) = \frac{1}{1 + \exp(-x)} + """ + +@partial(mx.compile, shapeless=True) +def relu(x: mx.array) -> mx.array: + r"""Applies the Rectified Linear Unit. + + Simply ``mx.maximum(x, 0)``. + """ + +@partial(mx.compile, shapeless=True) +def relu2(x: mx.array) -> mx.array: + r"""Applies the ReLU² activation function. + + Applies :math:`\max(0, x)^2` element wise. + """ + +@partial(mx.compile, shapeless=True) +def relu6(x: mx.array) -> mx.array: + r"""Applies the Rectified Linear Unit 6. + + Applies :math:`\min(\max(x, 0), 6)` element wise. + """ + +@partial(mx.compile, shapeless=True) +def leaky_relu(x: mx.array, negative_slope=...) -> mx.array: + r"""Applies the Leaky Rectified Linear Unit. + + Simply ``mx.maximum(negative_slope * x, x)``. + """ + +@partial(mx.compile, shapeless=True) +def log_softmax(x: mx.array, axis=...): + r"""Applies the Log Softmax function. + + Applies :math:`x + \log \sum_i e^{x_i}` element wise. + """ + +@partial(mx.compile, shapeless=True) +def elu(x: mx.array, alpha=...) -> mx.array: + r"""Applies the Exponential Linear Unit. + + Simply ``mx.where(x > 0, x, alpha * (mx.exp(x) - 1))``. + """ + +@partial(mx.compile, shapeless=True) +def softmax(x: mx.array, axis=...) -> mx.array: + r"""Applies the Softmax function. + + Applies :math:`\frac{e^{x_i}}{\sum_j e^{x_j}}` element wise. + """ + +@partial(mx.compile, shapeless=True) +def softplus(x: mx.array) -> mx.array: + r"""Applies the Softplus function. + + Applies :math:`\log(1 + \exp(x))` element wise. + """ + +@partial(mx.compile, shapeless=True) +def softsign(x: mx.array) -> mx.array: + r"""Applies the Softsign function. + + Applies :math:`\frac{x}{1 + |x|}` element wise. + """ + +@partial(mx.compile, shapeless=True) +def softshrink(x: mx.array, lambd: float = ...) -> mx.array: + r"""Applies the Softshrink activation function. + + .. math:: + \text{softshrink}(x) = \begin{cases} + x - \lambda & \text{if } x > \lambda \\ + x + \lambda & \text{if } x < -\lambda \\ + 0 & \text{otherwise} + \end{cases} + """ + +@partial(mx.compile, shapeless=True) +def celu(x: mx.array, alpha=...) -> mx.array: + r"""Applies the Continuously Differentiable Exponential Linear Unit. + + Applies :math:`\max(0, x) + \min(0, \alpha * (\exp(x / \alpha) - 1))` + element wise. + """ + +@partial(mx.compile, shapeless=True) +def silu(x: mx.array) -> mx.array: + r"""Applies the Sigmoid Linear Unit. Also known as Swish. + + Applies :math:`x \sigma(x)` element wise, where :math:`\sigma(\cdot)` is + the logistic sigmoid. + """ + +@partial(mx.compile, shapeless=True) +def log_sigmoid(x: mx.array) -> mx.array: + r"""Applies the Log Sigmoid function. + + Applies :math:`\log(\sigma(x)) = -\log(1 + e^{-x})` element wise. + """ + +@partial(mx.compile, shapeless=True) +def gelu(x: mx.array) -> mx.array: + r"""Applies the Gaussian Error Linear Units function. + + .. math:: + \textrm{GELU}(x) = x * \Phi(x) + + where :math:`\Phi(x)` is the Gaussian CDF. + + See also :func:`gelu_approx` and :func:`gelu_fast_approx` for faster + approximations. + """ + +@partial(mx.compile, shapeless=True) +def gelu_approx(x: mx.array) -> mx.array: + r"""An approximation to Gaussian Error Linear Unit. + + See :func:`gelu` for the exact computation. + + This function approximates ``gelu`` with a maximum absolute error :math:`< + 0.0005` in the range :math:`[-6, 6]` using the following + + .. math:: + + x = 0.5 * x * \left(1 + \text{Tanh}\left((\sqrt{2 / \pi} * \left(x + 0.044715 * x^3\right)\right)\right) + + """ + +@partial(mx.compile, shapeless=True) +def gelu_fast_approx(x: mx.array) -> mx.array: + r"""A fast approximation to Gaussian Error Linear Unit. + + See :func:`gelu` for the exact computation. + + This function approximates ``gelu`` with a maximum absolute error :math:`< + 0.015` in the range :math:`[-6, 6]` using the following + + .. math:: + + x = x \sigma\left(1.702 x\right) + + where :math:`\sigma(\cdot)` is the logistic sigmoid. + + References: + - https://github.com/hendrycks/GELUs + - https://arxiv.org/abs/1606.08415 + """ + +def glu(x: mx.array, axis: int = ...) -> mx.array: + r"""Applies the gated linear unit function. + + This function splits the ``axis`` dimension of the input into two halves + (:math:`a` and :math:`b`) and applies :math:`a * \sigma(b)`. + + .. math:: + \textrm{GLU}(x) = a * \sigma(b) + + Args: + axis (int): The dimension to split along. Default: ``-1`` + """ + +@partial(mx.compile, shapeless=True) +def step(x: mx.array, threshold: float = ...) -> mx.array: + r"""Applies the Step Activation Function. + + This function implements a binary step activation, where the output is set + to 1 if the input is greater than a specified threshold, and 0 otherwise. + + .. math:: + \text{step}(x) = \begin{cases} + 0 & \text{if } x < \text{threshold} \\ + 1 & \text{if } x \geq \text{threshold} + \end{cases} + + Args: + threshold: The value to threshold at. + """ + +@partial(mx.compile, shapeless=True) +def selu(x: mx.array) -> mx.array: + r"""Applies the Scaled Exponential Linear Unit. + + .. math:: + \text{selu}(x) = \begin{cases} + \lambda x & \text{if } x > 0 \\ + \lambda \alpha (\exp(x) - 1) & \text{if } x \leq 0 + \end{cases} + + where :math:`\lambda = 1.0507` and :math:`\alpha = 1.67326`. + + See also :func:`elu`. + """ + +@partial(mx.compile, shapeless=True) +def prelu(x: mx.array, alpha: mx.array) -> mx.array: + r"""Applies the element-wise parametric ReLU. + + .. math:: + \text{PReLU}(x) = \max(0,x) + a * \min(0,x) + + where :math:`a` is an array. + """ + +@partial(mx.compile, shapeless=True) +def mish(x: mx.array) -> mx.array: + r"""Applies the Mish function, element-wise. + + Mish: A Self Regularized Non-Monotonic Neural Activation Function. + + Reference: https://arxiv.org/abs/1908.08681 + + .. math:: + \text{Mish}(x) = x * \text{Tanh}(\text{Softplus}(x)) + + """ + +@partial(mx.compile, shapeless=True) +def hardswish(x: mx.array) -> mx.array: + r"""Applies the hardswish function, element-wise. + + .. math:: + \text{Hardswish}(x) = x * \min(\max(x + 3, 0), 6) / 6 + """ + +@partial(mx.compile, shapeless=True) +def hard_tanh(x: mx.array, min_val=..., max_val=...) -> mx.array: + r"""Applies the HardTanh function. + + Applies :math:`\max(\min(x, \text{max\_val}), \text{min\_val})` element-wise. + """ + +@partial(mx.compile, shapeless=True) +def hard_shrink(x: mx.array, lambd=...) -> mx.array: + r"""Applies the HardShrink activation function. + + .. math:: + \text{hardshrink}(x) = \begin{cases} + x & \text{if } x > \lambda \\ + x & \text{if } x < -\lambda \\ + 0 & \text{otherwise} + \end{cases} + """ + +@partial(mx.compile, shapeless=True) +def softmin(x: mx.array, axis=...) -> mx.array: + r"""Applies the Softmin function. + + Applies :math:`\frac{e^{-x_i}}{\sum_j e^{-x_j}}` element-wise. + """ + +def tanh(x: mx.array) -> mx.array: + """Applies the hyperbolic tangent function. + + Simply ``mx.tanh(x)``. + """ + +class GLU(Module): + r"""Applies the gated linear unit function. + + This function splits the ``axis`` dimension of the input into two halves + (:math:`a` and :math:`b`) and applies :math:`a * \sigma(b)`. + + .. math:: + \textrm{GLU}(x) = a * \sigma(b) + + Args: + axis (int): The dimension to split along. Default: ``-1`` + """ + def __init__(self, axis: int = ...) -> None: ... + def __call__(self, x) -> Any: ... + +@_make_activation_module(sigmoid) +class Sigmoid(Module): + r"""Applies the sigmoid function, element-wise. + + .. math:: + \text{Sigmoid}(x) = \sigma(x) = \frac{1}{1 + \exp(-x)} + """ + +@_make_activation_module(mish) +class Mish(Module): + r"""Applies the Mish function, element-wise. + + Reference: https://arxiv.org/abs/1908.08681 + + .. math:: + \text{Mish}(x) = x * \text{Tanh}(\text{Softplus}(x)) + + """ + +@_make_activation_module(relu) +class ReLU(Module): + r"""Applies the Rectified Linear Unit. + Simply ``mx.maximum(x, 0)``. + + See :func:`relu` for the functional equivalent. + """ + +@_make_activation_module(relu2) +class ReLU2(Module): + r"""Applies the ReLU² activation function. + + See :func:`relu2` for the functional equivalent. + """ + +@_make_activation_module(relu6) +class ReLU6(Module): + r"""Applies the Rectified Linear Unit 6. + + See :func:`relu6` for the functional equivalent. + """ + +class LeakyReLU(Module): + r"""Applies the Leaky Rectified Linear Unit. + + Simply ``mx.maximum(negative_slope * x, x)``. + + Args: + negative_slope: Controls the angle of the negative slope. Default: ``1e-2`` + """ + def __init__(self, negative_slope=...) -> None: ... + def __call__(self, x): ... + +class ELU(Module): + r"""Applies the Exponential Linear Unit. + Simply ``mx.where(x > 0, x, alpha * (mx.exp(x) - 1))``. + + See :func:`elu` for the functional equivalent. + + Args: + alpha: the :math:`\alpha` value for the ELU formulation. Default: ``1.0`` + """ + def __init__(self, alpha=...) -> None: ... + def __call__(self, x): ... + +@_make_activation_module(softmax) +class Softmax(Module): + r"""Applies the Softmax function. + + See :func:`softmax` for the functional equivalent. + """ + +@_make_activation_module(softplus) +class Softplus(Module): + r"""Applies the Softplus function. + + See :func:`softplus` for the functional equivalent. + """ + +@_make_activation_module(softsign) +class Softsign(Module): + r"""Applies the Softsign function. + + See :func:`softsign` for the functional equivalent. + """ + +class Softshrink(Module): + r"""Applies the Softshrink function. + + See :func:`softshrink` for the functional equivalent. + + Args: + lambd: the :math:`\lambda` value for Softshrink. Default: ``0.5`` + """ + def __init__(self, lambd=...) -> None: ... + def __call__(self, x): ... + +class CELU(Module): + r"""Applies the Continuously Differentiable Exponential Linear Unit. + Applies :math:`\max(0, x) + \min(0, \alpha * (\exp(x / \alpha) - 1))` + element wise. + + See :func:`celu` for the functional equivalent. + + Args: + alpha: the :math:`\alpha` value for the CELU formulation. Default: ``1.0`` + """ + def __init__(self, alpha=...) -> None: ... + def __call__(self, x): ... + +@_make_activation_module(silu) +class SiLU(Module): + r"""Applies the Sigmoid Linear Unit. Also known as Swish. + + See :func:`silu` for the functional equivalent. + """ + +@_make_activation_module(log_softmax) +class LogSoftmax(Module): + r"""Applies the Log Softmax function. + + See :func:`log_softmax` for the functional equivalent. + """ + +@_make_activation_module(log_sigmoid) +class LogSigmoid(Module): + r"""Applies the Log Sigmoid function. + + See :func:`log_sigmoid` for the functional equivalent. + """ + +class PReLU(Module): + r"""Applies the element-wise parametric ReLU. + Applies :math:`\max(0, x) + a * \min(0, x)` element wise, where :math:`a` + is an array. + + See :func:`prelu` for the functional equivalent. + + Args: + num_parameters: number of :math:`a` to learn. Default: ``1`` + init: the initial value of :math:`a`. Default: ``0.25`` + """ + def __init__(self, num_parameters=..., init=...) -> None: ... + def __call__(self, x: mx.array): ... + +class GELU(Module): + r"""Applies the Gaussian Error Linear Units. + + .. math:: + \textrm{GELU}(x) = x * \Phi(x) + + where :math:`\Phi(x)` is the Gaussian CDF. + + However, if ``approx`` is set to 'precise' or 'fast' it applies + + .. math:: + \textrm{GELUApprox}(x) &= 0.5 * x * \left(1 + \text{Tanh}\left((\sqrt{2 / \pi} * \left(x + 0.044715 * x^3\right)\right)\right) \\ + \textrm{GELUFast}(x) &= x * \sigma\left(1.702 * x\right) + + respectively. + + .. note:: + For compatibility with the PyTorch API, 'tanh' can be used as an alias + for 'precise'. + + See :func:`gelu`, :func:`gelu_approx` and :func:`gelu_fast_approx` for the + functional equivalents and information regarding error bounds. + + + Args: + approx ('none' | 'precise' | 'fast'): Which approximation to gelu to use if any. + """ + def __init__(self, approx=...) -> None: ... + def __call__(self, x): ... + +@_make_activation_module(tanh) +class Tanh(Module): + r"""Applies the hyperbolic tangent function. + + See :func:`tanh` for the functional equivalent. + """ + +@_make_activation_module(hardswish) +class Hardswish(Module): + r"""Applies the hardswish function, element-wise. + + See :func:`hardswish` for the functional equivalent. + """ + +class Step(Module): + r"""Applies the Step Activation Function. + + This function implements a binary step activation, where the output is set + to 1 if the input is greater than a specified threshold, and 0 otherwise. + + .. math:: + \text{step}(x) = \begin{cases} + 0 & \text{if } x < \text{threshold} \\ + 1 & \text{if } x \geq \text{threshold} + \end{cases} + + Args: + threshold: The value to threshold at. + """ + def __init__(self, threshold: float = ...) -> None: ... + def __call__(self, x: mx.array): ... + +@_make_activation_module(selu) +class SELU(Module): + r"""Applies the Scaled Exponential Linear Unit. + + See :func:`selu` for the functional equivalent. + """ + +@_make_activation_module(hard_tanh) +class HardTanh(Module): + r"""Applies the HardTanh function. + + See :func:`hard_tanh` for the functional equivalent. + """ + +@_make_activation_module(hard_shrink) +class HardShrink(Module): + r"""Applies the HardShrink function. + + See :func:`hard_shrink` for the functional equivalent. + + Args: + lambd: the :math:`\lambda` value for Hardshrink. Default: ``0.5`` + """ + +@_make_activation_module(softmin) +class Softmin(Module): + r"""Applies the Softmin function. + + See :func:`softmin` for the functional equivalent. + """ diff --git a/.mlx_typings/mlx/nn/layers/base.pyi b/.mlx_typings/mlx/nn/layers/base.pyi new file mode 100644 index 00000000..a4abf36b --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/base.pyi @@ -0,0 +1,393 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Any, Callable, List, Optional, Tuple, Union + +import mlx.core as mx + +class Module(dict): + """Base class for building neural networks with MLX. + + All the layers provided in :mod:`layers` subclass this class and + your models should do the same. + + A ``Module`` can contain other ``Module`` instances or :class:`mlx.core.array` + instances in arbitrary nesting of python lists or dicts. The ``Module`` + then allows recursively extracting all the :class:`mlx.core.array` instances + using :meth:`Module.parameters`. + + In addition, the ``Module`` has the concept of trainable and non trainable + parameters (called "frozen"). When using :func:`value_and_grad` + the gradients are returned only with respect to the trainable parameters. + All arrays in a module are trainable unless they are added in the "frozen" + set by calling :meth:`freeze`. + + .. code-block:: python + + import mlx.core as mx + import mlx.nn as nn + + class MyMLP(nn.Module): + def __init__(self, in_dims: int, out_dims: int, hidden_dims: int = 16): + super().__init__() + + self.in_proj = nn.Linear(in_dims, hidden_dims) + self.out_proj = nn.Linear(hidden_dims, out_dims) + + def __call__(self, x): + x = self.in_proj(x) + x = mx.maximum(x, 0) + return self.out_proj(x) + + model = MyMLP(2, 1) + + # All the model parameters are created but since MLX is lazy by + # default, they are not evaluated yet. Calling `mx.eval` actually + # allocates memory and initializes the parameters. + mx.eval(model.parameters()) + + # Setting a parameter to a new value is as simply as accessing that + # parameter and assigning a new array to it. + model.in_proj.weight = model.in_proj.weight * 2 + mx.eval(model.parameters()) + """ + + __call__: Callable + def __init__(self) -> None: + """Should be called by the subclasses of ``Module``.""" + + @property + def training(self): # -> bool: + """Boolean indicating if the model is in training mode.""" + + @property + def state(self): # -> Self: + """The module's state dictionary + + The module's state dictionary contains any attribute set on the + module including parameters in :meth:`Module.parameters` + + Unlike :meth:`Module.parameters`, the :attr:`Module.state` property is + a reference to the module's state. Updates to it will be reflected in + the original module. + """ + + def __repr__(self): # -> str: + ... + def __getattr__(self, key: str): # -> None: + ... + def __setattr__(self, key: str, val: Any): # -> None: + ... + def __delattr__(self, name): # -> None: + ... + def load_weights( + self, + file_or_weights: Union[str, List[Tuple[str, mx.array]]], + strict: bool = ..., + ) -> Module: + """ + Update the model's weights from a ``.npz``, a ``.safetensors`` file, or a list. + + Args: + file_or_weights (str or list(tuple(str, mx.array))): The path to + the weights ``.npz`` file (``.npz`` or ``.safetensors``) or a list + of pairs of parameter names and arrays. + strict (bool, optional): If ``True`` then checks that the provided + weights exactly match the parameters of the model. Otherwise, + only the weights actually contained in the model are loaded and + shapes are not checked. Default: ``True``. + + Returns: + The module instance after updating the weights. + + Example: + + .. code-block:: python + + import mlx.core as mx + import mlx.nn as nn + model = nn.Linear(10, 10) + + # Load from file + model.load_weights("weights.npz") + + # Load from .safetensors file + model.load_weights("weights.safetensors") + + # Load from list + weights = [ + ("weight", mx.random.uniform(shape=(10, 10))), + ("bias", mx.zeros((10,))), + ] + model.load_weights(weights) + + # Missing weight + weights = [ + ("weight", mx.random.uniform(shape=(10, 10))), + ] + + # Raises a ValueError exception + model.load_weights(weights) + + # Ok, only updates the weight but not the bias + model.load_weights(weights, strict=False) + """ + + def save_weights(self, file: str): # -> None: + """ + Save the model's weights to a file. The saving method is determined by the file extension: + - ``.npz`` will use :func:`mx.savez` + - ``.safetensors`` will use :func:`mx.save_safetensors` + """ + + @staticmethod + def is_module(value): # -> bool: + ... + @staticmethod + def valid_child_filter(module, key, value): # -> bool: + ... + @staticmethod + def valid_parameter_filter(module, key, value): # -> bool: + ... + @staticmethod + def trainable_parameter_filter(module, key, value): # -> bool: + ... + def filter_and_map( + self, + filter_fn: Callable[[Module, str, Any], bool], + map_fn: Optional[Callable] = ..., + is_leaf_fn: Optional[Callable[[Module, str, Any], bool]] = ..., + ): # -> dict[Any, Any | dict[Any, Any | dict[Any, Any] | list[Any]] | dict[Any, Any] | list[Any]]: + """Recursively filter the contents of the module using ``filter_fn``, + namely only select keys and values where ``filter_fn`` returns true. + + This is used to implement :meth:`parameters` and :meth:`trainable_parameters` + but it can also be used to extract any subset of the module's parameters. + + Args: + filter_fn (Callable): Given a value, the key in which it is found + and the containing module, decide whether to keep the value or + drop it. + map_fn (Callable, optional): Optionally transform the value before + returning it. + is_leaf_fn (Callable, optional): Given a value, the key in which it + is found and the containing module decide if it is a leaf. + + Returns: + A dictionary containing the contents of the module recursively filtered + """ + + def parameters( + self, + ) -> mx.MX_ARRAY_TREE: + """Recursively return all the :class:`mlx.core.array` members of this Module + as a dict of dicts and lists.""" + + def trainable_parameters( + self, + ) -> mx.MX_ARRAY_TREE: # -> dict[Any, Any | dict[Any, Any | dict[Any, Any] | list[Any]] | dict[Any, Any] | list[Any]]: + """Recursively return all the non frozen :class:`mlx.core.array` members of + this Module as a dict of dicts and lists.""" + + def children( + self, + ) -> mx.MX_ARRAY_TREE: # -> dict[Any, Any | dict[Any, Any | dict[Any, Any] | list[Any]] | dict[Any, Any] | list[Any]]: + """Return the direct descendants of this Module instance.""" + + def leaf_modules( + self, + ) -> mx.MX_ARRAY_TREE: # -> dict[Any, Any | dict[Any, Any | dict[Any, Any] | list[Any]] | dict[Any, Any] | list[Any]]: + """Return the submodules that do not contain other modules.""" + + def update(self, parameters: dict, strict: bool = ...) -> Module: + """Replace the parameters of this Module with the provided ones in the + dict of dicts and lists. + + Commonly used by the optimizer to change the model to the updated + (optimized) parameters. Also used by the :meth:`value_and_grad` to set the + tracers in the model in order to compute gradients. + + The passed in parameters dictionary need not be a full dictionary + similar to :meth:`parameters`. Only the provided locations will be + updated. + + Args: + parameters (dict): A complete or partial dictionary of the modules + parameters. + strict (bool): If ``True`` checks that ``parameters`` is a + subset of the module's parameters. Default: ``True``. + Returns: + The module instance after updating the parameters. + """ + + def apply( + self, + map_fn: Callable[[mx.array], mx.array], + filter_fn: Optional[Callable[[Module, str, Any], bool]] = ..., + ) -> Module: + """Map all the parameters using the provided ``map_fn`` and immediately + update the module with the mapped parameters. + + For instance running ``model.apply(lambda x: x.astype(mx.float16))`` + casts all parameters to 16 bit floats. + + Args: + map_fn (Callable): Maps an array to another array + filter_fn (Callable, optional): Filter to select which arrays to + map (default: :meth:`Module.valid_parameter_filter`). + + Returns: + The module instance after updating the parameters. + """ + + def update_modules(self, modules: dict, strict: bool = ...) -> Module: + """Replace the child modules of this :class:`Module` instance with the + provided ones in the dict of dicts and lists. + + It is the equivalent of :meth:`Module.update` but for modules instead + of parameters and allows us to flexibly edit complex architectures by + programmatically swapping layers. + + The passed in parameters dictionary need not be a full dictionary + similar to :meth:`modules`. Only the provided locations will be + updated. + + Args: + modules (dict): A complete or partial dictionary of the module's + submodules. + strict (bool): If ``True`` checks that ``modules`` is a + subset of the child modules of this instance. Default: ``True``. + Returns: + The module instance after updating the submodules. + """ + + def apply_to_modules(self, apply_fn: Callable[[str, Module], Any]) -> Module: + """Apply a function to all the modules in this instance (including this + instance). + + Args: + apply_fn (Callable): The function to apply to the modules. + + Returns: + The module instance after updating submodules. + """ + + def modules(self): # -> list[Any]: + """Return a list with all the modules in this instance. + + Returns: + A list of :class:`Module` instances. + """ + + def named_modules(self): # -> list[Any]: + """Return a list with all the modules in this instance and their name + with dot notation. + + Returns: + A list of tuples (str, :class:`Module`). + """ + + def freeze( + self, + *, + recurse: bool = ..., + keys: Optional[Union[str, List[str]]] = ..., + strict: bool = ..., + ) -> Module: + """Freeze the Module's parameters or some of them. Freezing a parameter means not + computing gradients for it. + + This function is idempotent i.e. freezing a frozen model is a no-op. + + Example: + For instance to only train the attention parameters from a Transformer: + + .. code-block:: python + + model = nn.Transformer() + model.freeze() + model.apply_to_modules(lambda k, v: v.unfreeze() if k.endswith("attention") else None) + + Args: + recurse (bool, optional): If True then freeze the parameters of the + submodules as well. Default: ``True``. + keys (str or list[str], optional): If provided then only these + parameters will be frozen otherwise all the parameters of a + module. For instance freeze all biases by calling + ``module.freeze(keys="bias")``. + strict (bool, optional): If set to ``True`` validate that the passed keys exist. + Default: ``False``. + + Returns: + The module instance after freezing the parameters. + """ + + def unfreeze( + self, + *, + recurse: bool = ..., + keys: Optional[Union[str, List[str]]] = ..., + strict: bool = ..., + ) -> Module: + """Unfreeze the Module's parameters or some of them. + + This function is idempotent ie unfreezing a model that is not frozen is + a noop. + + Example: + + For instance to only train the biases of a Transformer one can do: + + .. code-block:: python + + model = nn.Transformer() + model.freeze() + model.unfreeze(keys="bias") + + Args: + recurse (bool, optional): If True then unfreeze the parameters of the + submodules as well. Default: ``True``. + keys (str or list[str], optional): If provided then only these + parameters will be unfrozen otherwise all the parameters of a + module. For instance unfreeze all biases by calling + ``module.unfreeze(keys="bias")``. + strict (bool, optional): If set to ``True`` validate that the passed keys exist. + Default: ``False``. + + Returns: + The module instance after unfreezing the parameters. + """ + + def train(self, mode: bool = ...) -> Module: + """Set the model in or out of training mode. + + Training mode only applies to certain layers. For example + :obj:`Dropout` applies a random mask in training mode, but is the + identity in evaluation mode. + + Args: + mode (bool): Indicate if the model should be in training or + evaluation mode. Default: ``True``. + Returns: + The module instance after updating the training mode. + """ + + def eval(self) -> Module: + """Set the model to evaluation mode. + + See :func:`train`. + """ + + def set_dtype( + self, dtype: mx.Dtype, predicate: Optional[Callable[[mx.Dtype], bool]] = ... + ): # -> None: + """Set the dtype of the module's parameters. + + Args: + dtype (Dtype): The new dtype. + predicate (typing.Callable, optional): A predicate to select + parameters to cast. By default, only parameters of type + :attr:`floating` will be updated to avoid casting integer + parameters to the new dtype. + """ diff --git a/.mlx_typings/mlx/nn/layers/containers.pyi b/.mlx_typings/mlx/nn/layers/containers.pyi new file mode 100644 index 00000000..068ea179 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/containers.pyi @@ -0,0 +1,21 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Callable + +import mlx.core as mx +from base import Module + +class Sequential(Module): + """A layer that calls the passed callables in order. + + We can pass either modules or plain callables to the Sequential module. If + our functions have learnable parameters they should be implemented as + ``nn.Module`` instances. + + Args: + modules (tuple of Callables): The modules to call in order + """ + def __init__(self, *modules: Module | Callable[[mx.array], mx.array]) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/layers/convolution.pyi b/.mlx_typings/mlx/nn/layers/convolution.pyi new file mode 100644 index 00000000..c68ad289 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/convolution.pyi @@ -0,0 +1,116 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Union + +import mlx.core as mx +from base import Module + +class Conv1d(Module): + """Applies a 1-dimensional convolution over the multi-channel input sequence. + + The channels are expected to be last i.e. the input shape should be ``NLC`` where: + + * ``N`` is the batch dimension + * ``L`` is the sequence length + * ``C`` is the number of input channels + + Args: + in_channels (int): The number of input channels + out_channels (int): The number of output channels + kernel_size (int): The size of the convolution filters + stride (int, optional): The stride when applying the filter. + Default: ``1``. + padding (int, optional): How many positions to 0-pad the input with. + Default: ``0``. + dilation (int, optional): The dilation of the convolution. + groups (int, optional): The number of groups for the convolution. + Default: ``1``. + bias (bool, optional): If ``True`` add a learnable bias to the output. + Default: ``True`` + """ + def __init__( + self, + in_channels: int, + out_channels: int, + kernel_size: int, + stride: int = ..., + padding: int = ..., + dilation: int = ..., + groups: int = ..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class Conv2d(Module): + """Applies a 2-dimensional convolution over the multi-channel input image. + + The channels are expected to be last i.e. the input shape should be ``NHWC`` where: + + * ``N`` is the batch dimension + * ``H`` is the input image height + * ``W`` is the input image width + * ``C`` is the number of input channels + + Args: + in_channels (int): The number of input channels. + out_channels (int): The number of output channels. + kernel_size (int or tuple): The size of the convolution filters. + stride (int or tuple, optional): The size of the stride when + applying the filter. Default: ``1``. + padding (int or tuple, optional): How many positions to 0-pad + the input with. Default: ``0``. + dilation (int or tuple, optional): The dilation of the convolution. + groups (int, optional): The number of groups for the convolution. + Default: ``1``. + bias (bool, optional): If ``True`` add a learnable bias to the + output. Default: ``True`` + """ + def __init__( + self, + in_channels: int, + out_channels: int, + kernel_size: Union[int, tuple], + stride: Union[int, tuple] = ..., + padding: Union[int, tuple] = ..., + dilation: Union[int, tuple] = ..., + groups: int = ..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x) -> mx.array: ... + +class Conv3d(Module): + """Applies a 3-dimensional convolution over the multi-channel input image. + + The channels are expected to be last i.e. the input shape should be ``NDHWC`` where: + + * ``N`` is the batch dimension + * ``D`` is the input image depth + * ``H`` is the input image height + * ``W`` is the input image width + * ``C`` is the number of input channels + + Args: + in_channels (int): The number of input channels. + out_channels (int): The number of output channels. + kernel_size (int or tuple): The size of the convolution filters. + stride (int or tuple, optional): The size of the stride when + applying the filter. Default: ``1``. + dilation (int or tuple, optional): The dilation of the convolution. + padding (int or tuple, optional): How many positions to 0-pad + the input with. Default: ``0``. + bias (bool, optional): If ``True`` add a learnable bias to the + output. Default: ``True`` + """ + def __init__( + self, + in_channels: int, + out_channels: int, + kernel_size: Union[int, tuple], + stride: Union[int, tuple] = ..., + padding: Union[int, tuple] = ..., + dilation: Union[int, tuple] = ..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/layers/convolution_transpose.pyi b/.mlx_typings/mlx/nn/layers/convolution_transpose.pyi new file mode 100644 index 00000000..8fe11b4a --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/convolution_transpose.pyi @@ -0,0 +1,119 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Union + +import mlx.core as mx +from base import Module + +class ConvTranspose1d(Module): + """Applies a 1-dimensional transposed convolution over the multi-channel input sequence. + + The channels are expected to be last i.e. the input shape should be ``NLC`` where: + + * ``N`` is the batch dimension + * ``L`` is the sequence length + * ``C`` is the number of input channels + + Args: + in_channels (int): The number of input channels + out_channels (int): The number of output channels + kernel_size (int): The size of the convolution filters + stride (int, optional): The stride when applying the filter. + Default: ``1``. + padding (int, optional): How many positions to 0-pad the input with. + Default: ``0``. + dilation (int, optional): The dilation of the convolution. + output_padding(int, optional): Additional size added to one side of the + output shape. Default: ``0``. + bias (bool, optional): If ``True`` add a learnable bias to the output. + Default: ``True`` + """ + def __init__( + self, + in_channels: int, + out_channels: int, + kernel_size: int, + stride: int = ..., + padding: int = ..., + dilation: int = ..., + output_padding: int = ..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class ConvTranspose2d(Module): + """Applies a 2-dimensional transposed convolution over the multi-channel input image. + + The channels are expected to be last i.e. the input shape should be ``NHWC`` where: + + * ``N`` is the batch dimension + * ``H`` is the input image height + * ``W`` is the input image width + * ``C`` is the number of input channels + + Args: + in_channels (int): The number of input channels. + out_channels (int): The number of output channels. + kernel_size (int or tuple): The size of the convolution filters. + stride (int or tuple, optional): The size of the stride when + applying the filter. Default: ``1``. + padding (int or tuple, optional): How many positions to 0-pad + the input with. Default: ``0``. + dilation (int or tuple, optional): The dilation of the convolution. + output_padding(int or tuple, optional): Additional size added to one + side of the output shape. Default: ``0``. + bias (bool, optional): If ``True`` add a learnable bias to the + output. Default: ``True`` + """ + def __init__( + self, + in_channels: int, + out_channels: int, + kernel_size: Union[int, tuple], + stride: Union[int, tuple] = ..., + padding: Union[int, tuple] = ..., + dilation: Union[int, tuple] = ..., + output_padding: Union[int, tuple] = ..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class ConvTranspose3d(Module): + """Applies a 3-dimensional transposed convolution over the multi-channel input image. + + The channels are expected to be last i.e. the input shape should be ``NDHWC`` where: + + * ``N`` is the batch dimension + * ``D`` is the input image depth + * ``H`` is the input image height + * ``W`` is the input image width + * ``C`` is the number of input channels + + Args: + in_channels (int): The number of input channels. + out_channels (int): The number of output channels. + kernel_size (int or tuple): The size of the convolution filters. + stride (int or tuple, optional): The size of the stride when + applying the filter. Default: ``1``. + padding (int or tuple, optional): How many positions to 0-pad + the input with. Default: ``0``. + dilation (int or tuple, optional): The dilation of the convolution. + output_padding(int or tuple, optional): Additional size added to one + side of the output shape. Default: ``0``. + bias (bool, optional): If ``True`` add a learnable bias to the + output. Default: ``True`` + """ + def __init__( + self, + in_channels: int, + out_channels: int, + kernel_size: Union[int, tuple], + stride: Union[int, tuple] = ..., + padding: Union[int, tuple] = ..., + dilation: Union[int, tuple] = ..., + output_padding: Union[int, tuple] = ..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/layers/distributed.pyi b/.mlx_typings/mlx/nn/layers/distributed.pyi new file mode 100644 index 00000000..5be9cc4b --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/distributed.pyi @@ -0,0 +1,227 @@ +""" +This type stub file was generated by pyright. +""" + +from functools import lru_cache +from typing import Callable, Optional, Union + +import mlx.core as mx +from base import Module +from mlx.nn.layers.linear import Linear + +@lru_cache +def sum_gradients( + group: mx.distributed.Group, +) -> Callable[..., mx.array]: # -> Callable[..., Any] | Callable[..., array]: + ... +def shard_inplace( + module: Module, + sharding: str, + *, + segments: Union[int, list[int]] = ..., + group: Optional[mx.distributed.Group] = ..., +) -> None: + """Shard a module in-place by updating its parameter dictionary with the + sharded parameter dictionary. + + The ``sharding`` argument can be any callable that given the path and the + weight returns the sharding axis and optionally also the segments that + comprise the unsharded weight. For instance if the weight is a fused QKV + matrix the segments should be 3. + + .. note:: + The module doesn't change so in order for distributed communication to + happen the module needs to natively support it and for it to be enabled. + + Args: + module (Module): The parameters of this module will be sharded + in-place. + sharding (str or callable): One of "all-to-sharded" and + "sharded-to-all" or a callable that returns the sharding axis and + segments. + segments (int or list): The segments to use if ``sharding`` is a + string. Default: ``1``. + group (mlx.core.distributed.Group): The distributed group to shard + across. If not set, the global group will be used. Default: ``None``. + """ + +def shard_linear( + module: Module, + sharding: str, + *, + segments: Union[int, list[int]] = ..., + group: Optional[mx.distributed.Group] = ..., +) -> Linear: + """Create a new linear layer that has its parameters sharded and also + performs distributed communication either in the forward or backward + pass. + + .. note:: + Contrary to ``shard_inplace``, the original layer is not changed but a + new layer is returned. + + Args: + module (Module): The linear layer to be sharded. + sharding (str): One of "all-to-sharded" and + "sharded-to-all" that defines the type of sharding to perform. + segments (int or list): The segments to use. Default: ``1``. + group (mlx.core.distributed.Group): The distributed group to shard + across. If not set, the global group will be used. Default: ``None``. + """ + +class AllToShardedLinear(Module): + """Each member of the group applies part of the affine transformation such + that the result is sharded across the group. + + The gradients are automatically aggregated from each member of the group. + + Args: + input_dims (int): The dimensionality of the input features + output_dims (int): The dimensionality of the output features + bias (bool, optional): If set to ``False`` the the layer will not use a + bias. Default is ``True``. + group (mx.distributed.Group, optional): The sharding will happen across + this group. If not set then the global group is used. Default is + ``None``. + """ + def __init__( + self, + input_dims: int, + output_dims: int, + bias: bool = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + @classmethod + def from_linear( + cls, + linear_layer: Module, + *, + segments: Union[int, list[int]] = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> AllToShardedLinear: ... + +class ShardedToAllLinear(Module): + """Each member of the group applies part of the affine transformation and + then aggregates the results. + + All nodes will have the same exact result after this layer. + + :class:`ShardedToAllLinear` provides a classmethod :meth:`from_linear` to + convert linear layers to sharded :obj:`ShardedToAllLinear` layers. + + Args: + input_dims (int): The dimensionality of the input features + output_dims (int): The dimensionality of the output features + bias (bool, optional): If set to ``False`` the the layer will not use a + bias. Default is ``True``. + group (mx.distributed.Group, optional): The sharding will happen across + this group. If not set then the global group is used. Default is + ``None``. + """ + def __init__( + self, + input_dims: int, + output_dims: int, + bias: bool = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + @classmethod + def from_linear( + cls, + linear_layer: Module, + *, + segments: Union[int, list[int]] = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> ShardedToAllLinear: ... + +class QuantizedAllToShardedLinear(Module): + """Each member of the group applies part of the affine transformation with + a quantized matrix such that the result is sharded across the group. + + It is the quantized equivalent of :class:`AllToShardedLinear`. + Similar to :class:`QuantizedLinear` its parameters are frozen and + will not be included in any gradient computation. + + Args: + input_dims (int): The dimensionality of the input features. + output_dims (int): The dimensionality of the output features. + bias (bool, optional): If set to ``False`` then the layer will not use + a bias. Default: ``True``. + group_size (int, optional): The group size to use for the quantized + weight. See :func:`~mlx.core.quantize`. Default: ``64``. + bits (int, optional): The bit width to use for the quantized weight. + See :func:`~mlx.core.quantize`. Default: ``4``. + group (mx.distributed.Group, optional): The sharding will happen across + this group. If not set then the global group is used. Default is + ``None``. + """ + def __init__( + self, + input_dims: int, + output_dims: int, + bias: bool = ..., + group_size: int = ..., + bits: int = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> None: ... + def unfreeze(self, *args, **kwargs) -> None: + """Wrap unfreeze so that we unfreeze any layers we might contain but + our parameters will remain frozen.""" + + def __call__(self, x: mx.array) -> mx.array: ... + @classmethod + def from_quantized_linear( + cls, + quantized_linear_layer: Module, + *, + segments: Union[int, list[int]] = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> QuantizedAllToShardedLinear: ... + +class QuantizedShardedToAllLinear(Module): + """Each member of the group applies part of the affine transformation using + the quantized matrix and then aggregates the results. + + All nodes will have the same exact result after this layer. + + It is the quantized equivalent of :class:`ShardedToAllLinear`. + Similar to :class:`QuantizedLinear` its parameters are frozen and + will not be included in any gradient computation. + + Args: + input_dims (int): The dimensionality of the input features. + output_dims (int): The dimensionality of the output features. + bias (bool, optional): If set to ``False`` then the layer will not use + a bias. Default: ``True``. + group_size (int, optional): The group size to use for the quantized + weight. See :func:`~mlx.core.quantize`. Default: ``64``. + bits (int, optional): The bit width to use for the quantized weight. + See :func:`~mlx.core.quantize`. Default: ``4``. + group (mx.distributed.Group, optional): The sharding will happen across + this group. If not set then the global group is used. Default is + ``None``. + """ + def __init__( + self, + input_dims: int, + output_dims: int, + bias: bool = ..., + group_size: int = ..., + bits: int = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> None: ... + def unfreeze(self, *args, **kwargs): # -> None: + """Wrap unfreeze so that we unfreeze any layers we might contain but + our parameters will remain frozen.""" + + def __call__(self, x: mx.array) -> mx.array: ... + @classmethod + def from_quantized_linear( + cls, + quantized_linear_layer: Module, + *, + segments: Union[int, list[int]] = ..., + group: Optional[mx.distributed.Group] = ..., + ) -> QuantizedShardedToAllLinear: ... diff --git a/.mlx_typings/mlx/nn/layers/dropout.pyi b/.mlx_typings/mlx/nn/layers/dropout.pyi new file mode 100644 index 00000000..00ef6f01 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/dropout.pyi @@ -0,0 +1,65 @@ +""" +This type stub file was generated by pyright. +""" + +import mlx.core as mx +from base import Module + +class Dropout(Module): + r"""Randomly zero a portion of the elements during training. + + The remaining elements are multiplied with :math:`\frac{1}{1-p}` where + :math:`p` is the probability of zeroing an element. This is done so the + expected value of a given element will remain the same. + + Args: + p (float): The probability to zero an element + """ + def __init__(self, p: float = ...) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class Dropout2d(Module): + r"""Apply 2D channel-wise dropout during training. + + Randomly zero out entire channels independently with probability :math:`p`. + This layer expects the channels to be last, i.e. the input shape should be + ``NWHC`` or ``WHC`` where:``N`` is the batch dimension,``H`` is the input + image height,``W`` is the input image width, and``C`` is the number of + input channels + + The remaining channels are scaled by :math:`\frac{1}{1-p}` to + maintain the expected value of each element. Unlike traditional dropout, + which zeros individual entries, this layer zeros entire channels. This is + beneficial for early convolution layers where adjacent pixels are + correlated. In such case, traditional dropout may not effectively + regularize activations. For more details, see [1]. + + [1]: Thompson, J., Goroshin, R., Jain, A., LeCun, Y. and Bregler C., 2015. + Efficient Object Localization Using Convolutional Networks. CVPR 2015. + + Args: + p (float): Probability of zeroing a channel during training. + """ + def __init__(self, p: float = ...) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class Dropout3d(Module): + r"""Apply 3D channel-wise dropout during training. + + Randomly zero out entire channels independently with probability :math:`p`. + This layer expects the channels to be last, i.e., the input shape should be + `NDHWC` or `DHWC` where: `N` is the batch dimension, `D` is the depth, + `H` is the input image height, `W` is the input image width, and `C` is + the number of input channels. + + The remaining channels are scaled by :math:`\frac{1}{1-p}` to + maintain the expected value of each element. Unlike traditional dropout, + which zeros individual entries, this layer zeros entire channels. This is + often beneficial for convolutional layers processing 3D data, like in + medical imaging or video processing. + + Args: + p (float): Probability of zeroing a channel during training. + """ + def __init__(self, p: float = ...) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/layers/embedding.pyi b/.mlx_typings/mlx/nn/layers/embedding.pyi new file mode 100644 index 00000000..e273c801 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/embedding.pyi @@ -0,0 +1,34 @@ +""" +This type stub file was generated by pyright. +""" + +import mlx.core as mx +from base import Module + +from .quantized import QuantizedEmbedding + +class Embedding(Module): + """Implements a simple lookup table that maps each input integer to a + high-dimensional vector. + + Typically used to embed discrete tokens for processing by neural networks. + + Args: + num_embeddings (int): How many possible discrete tokens can we embed. + Usually called the vocabulary size. + dims (int): The dimensionality of the embeddings. + """ + def __init__(self, num_embeddings: int, dims: int) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + def as_linear(self, x: mx.array) -> mx.array: + """ + Call the embedding layer as a linear layer. + + Use this for example when input embedding and output projection + weights are tied. + """ + + def to_quantized( + self, group_size: int = ..., bits: int = ..., mode: str = ... + ) -> QuantizedEmbedding: + """Return a :obj:`QuantizedEmbedding` layer that approximates this embedding layer.""" diff --git a/.mlx_typings/mlx/nn/layers/linear.pyi b/.mlx_typings/mlx/nn/layers/linear.pyi new file mode 100644 index 00000000..f9c91874 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/linear.pyi @@ -0,0 +1,76 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Any + +import mlx.core as mx +from base import Module + +from .quantized import QuantizedLinear + +class Identity(Module): + r"""A placeholder identity operator that is argument-insensitive. + + Args: + args: any argument (unused) + kwargs: any keyword argument (unused) + """ + def __init__(self, *args: Any, **kwargs: Any) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class Linear(Module): + r"""Applies an affine transformation to the input. + + Concretely: + + .. math:: + + y = x W^\top + b + + where: + where :math:`W` has shape ``[output_dims, input_dims]`` and :math:`b` has shape ``[output_dims]``. + + The values are initialized from the uniform distribution :math:`\mathcal{U}(-{k}, {k})`, + where :math:`k = \frac{1}{\sqrt{D_i}}` and :math:`D_i` is equal to ``input_dims``. + + Args: + input_dims (int): The dimensionality of the input features + output_dims (int): The dimensionality of the output features + bias (bool, optional): If set to ``False`` then the layer will + not use a bias. Default is ``True``. + """ + def __init__(self, input_dims: int, output_dims: int, bias: bool = ...) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + def to_quantized( + self, group_size: int = ..., bits: int = ..., mode: str = ... + ) -> QuantizedLinear: + """Return a :obj:`QuantizedLinear` layer that approximates this layer.""" + +class Bilinear(Module): + r"""Applies a bilinear transformation to the inputs. + + Concretely: + + .. math:: + + y_i = x_1^\top W_i x_2 + b_i + + where: + :math:`W` has shape ``[output_dims, input1_dims, input2_dims]``, :math:`b` has shape ``[output_dims ]``, + and :math:`i` indexes the output dimension. + + The values are initialized from the uniform distribution :math:`\mathcal{U}(-{k}, {k})`, + where :math:`k = \frac{1}{\sqrt{D_1}}` and :math:`D_1` is ``input1_dims``. + + Args: + input1_dims (int): The dimensionality of the input1 features + input2_dims (int): The dimensionality of the input2 features + output_dims (int): The dimensionality of the output features + bias (bool, optional): If set to ``False`` then the layer will + not use a bias. Default is ``True``. + """ + def __init__( + self, input1_dims: int, input2_dims: int, output_dims: int, bias: bool = ... + ) -> None: ... + def __call__(self, x1: mx.array, x2: mx.array) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/layers/normalization.pyi b/.mlx_typings/mlx/nn/layers/normalization.pyi new file mode 100644 index 00000000..4116f860 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/normalization.pyi @@ -0,0 +1,194 @@ +""" +This type stub file was generated by pyright. +""" + +import mlx.core as mx +from base import Module + +class InstanceNorm(Module): + r"""Applies instance normalization [1] on the inputs. + + Computes + + .. math:: + + y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta, + + where :math:`\gamma` and :math:`\beta` are learned per feature dimension + parameters initialized at 1 and 0 respectively. Both are of size :attr:`dims`, + if :attr:`affine` is ``True``. + + Args: + dims (int): The number of features of the input. + eps (float): A value added to the denominator for numerical stability. Default: ``1e-5``. + affine (bool): Default: ``False``. + + Shape: + - Input: :math:`(..., C)` where :math:`C` is equal to :attr:`dims`. + - Output: Same shape as the input. + + Examples: + >>> import mlx.core as mx + >>> import mlx.nn as nn + >>> x = mx.random.normal((8, 4, 4, 16)) + >>> inorm = nn.InstanceNorm(dims=16) + >>> output = inorm(x) + + References: + [1]: https://arxiv.org/abs/1607.08022 + """ + def __init__(self, dims: int, eps: float = ..., affine: bool = ...) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class LayerNorm(Module): + r"""Applies layer normalization [1] on the inputs. + + Computes + + .. math:: + + y = \frac{x - E[x]}{\sqrt{Var[x]} + \epsilon} \gamma + \beta, + + where :math:`\gamma` and :math:`\beta` are learned per feature dimension + parameters initialized at 1 and 0 respectively. + + [1]: https://arxiv.org/abs/1607.06450 + + Args: + dims (int): The feature dimension of the input to normalize over + eps (float): A small additive constant for numerical stability + affine (bool): If True learn an affine transform to apply after the + normalization + bias (bool): If True include a translation to the affine + transformation. If set to False the transformation is not really affine + just scaling. + """ + def __init__( + self, dims: int, eps: float = ..., affine: bool = ..., bias: bool = ... + ) -> None: ... + def __call__(self, x) -> mx.array: ... + +class RMSNorm(Module): + r"""Applies Root Mean Square normalization [1] to the inputs. + + Computes + + .. math:: + + y = \frac{x}{\sqrt{E[x^2] + \epsilon}} \gamma + + where :math:`\gamma` is a learned per feature dimension parameter initialized at + 1. + + Note the accumulation for the mean is done in 32-bit precision. + + [1]: https://arxiv.org/abs/1910.07467 + + Args: + dims (int): The feature dimension of the input to normalize over + eps (float): A small additive constant for numerical stability + """ + def __init__(self, dims: int, eps: float = ...) -> None: ... + def __call__(self, x) -> mx.array: ... + +class GroupNorm(Module): + r"""Applies Group Normalization [1] to the inputs. + + Computes the same normalization as layer norm, namely + + .. math:: + + y = \frac{x - E[x]}{\sqrt{Var[x]} + \epsilon} \gamma + \beta, + + where :math:`\gamma` and :math:`\beta` are learned per feature dimension + parameters initialized at 1 and 0 respectively. However, the mean and + variance are computed over the spatial dimensions and each group of + features. In particular, the input is split into num_groups across the + feature dimension. + + The feature dimension is assumed to be the last dimension and the dimensions + that precede it (except the first) are considered the spatial dimensions. + + [1]: https://arxiv.org/abs/1803.08494 + + Args: + num_groups (int): Number of groups to separate the features into + dims (int): The feature dimensions of the input to normalize over + eps (float): A small additive constant for numerical stability + affine (bool): If True learn an affine transform to apply after the + normalization. + pytorch_compatible (bool): If True perform the group normalization in + the same order/grouping as PyTorch. + """ + def __init__( + self, + num_groups: int, + dims: int, + eps: float = ..., + affine: bool = ..., + pytorch_compatible: bool = ..., + ) -> None: ... + def __call__(self, x) -> mx.array: ... + +class BatchNorm(Module): + r"""Applies Batch Normalization over a 2D or 3D input. + + Computes + + .. math:: + + y = \frac{x - E[x]}{\sqrt{Var[x]} + \epsilon} \gamma + \beta, + + where :math:`\gamma` and :math:`\beta` are learned per feature dimension + parameters initialized at 1 and 0 respectively. + + The input shape is specified as ``NC`` or ``NLC``, where ``N`` is the + batch, ``C`` is the number of features or channels, and ``L`` is the + sequence length. The output has the same shape as the input. For + four-dimensional arrays, the shape is ``NHWC``, where ``H`` and ``W`` are + the height and width respectively. + + For more information on Batch Normalization, see the original paper `Batch + Normalization: Accelerating Deep Network Training by Reducing Internal + Covariate Shift `_. + + Args: + num_features (int): The feature dimension to normalize over. + eps (float, optional): A small additive constant for numerical + stability. Default: ``1e-5``. + momentum (float, optional): The momentum for updating the running + mean and variance. Default: ``0.1``. + affine (bool, optional): If ``True``, apply a learned affine + transformation after the normalization. Default: ``True``. + track_running_stats (bool, optional): If ``True``, track the + running mean and variance. Default: ``True``. + + Examples: + >>> import mlx.core as mx + >>> import mlx.nn as nn + >>> x = mx.random.normal((5, 4)) + >>> bn = nn.BatchNorm(num_features=4, affine=True) + >>> output = bn(x) + """ + def __init__( + self, + num_features: int, + eps: float = ..., + momentum: float = ..., + affine: bool = ..., + track_running_stats: bool = ..., + ) -> None: ... + def unfreeze(self, *args, **kwargs): # -> None: + """Wrap unfreeze to make sure that running_mean and var are always + frozen parameters.""" + + def __call__(self, x: mx.array) -> mx.array: + """ + Forward pass of BatchNorm. + + Args: + x (array): Input tensor. + + Returns: + array: Normalized output tensor. + """ diff --git a/.mlx_typings/mlx/nn/layers/pooling.pyi b/.mlx_typings/mlx/nn/layers/pooling.pyi new file mode 100644 index 00000000..36b0ca24 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/pooling.pyi @@ -0,0 +1,242 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Optional, Tuple, Union + +import mlx.core as mx +from base import Module + +class _Pool(Module): + def __init__( + self, pooling_function, kernel_size, stride, padding, padding_value + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class _Pool1d(_Pool): + def __init__( + self, + pooling_function, + padding_value, + kernel_size: Union[int, Tuple[int]], + stride: Optional[Union[int, Tuple[int]]] = ..., + padding: Union[int, Tuple[int]] = ..., + ) -> None: ... + +class _Pool2d(_Pool): + def __init__( + self, + pooling_function, + padding_value, + kernel_size: Union[int, Tuple[int, int]], + stride: Optional[Union[int, Tuple[int, int]]] = ..., + padding: Optional[Union[int, Tuple[int, int]]] = ..., + ) -> None: ... + +class _Pool3d(_Pool): + def __init__( + self, + pooling_function, + padding_value, + kernel_size: Union[int, Tuple[int, int, int]], + stride: Optional[Union[int, Tuple[int, int, int]]] = ..., + padding: Optional[Union[int, Tuple[int, int, int]]] = ..., + ) -> None: ... + +class MaxPool1d(_Pool1d): + r"""Applies 1-dimensional max pooling. + + Spatially downsamples the input by taking the maximum of a sliding window + of size ``kernel_size`` and sliding stride ``stride``. + + Args: + kernel_size (int or tuple(int)): The size of the pooling window kernel. + stride (int or tuple(int), optional): The stride of the pooling window. + Default: ``kernel_size``. + padding (int or tuple(int), optional): How much negative infinity + padding to apply to the input. The padding amount is applied to + both sides of the spatial axis. Default: ``0``. + + Examples: + >>> import mlx.core as mx + >>> import layers as nn + >>> x = mx.random.normal(shape=(4, 16, 5)) + >>> pool = nn.MaxPool1d(kernel_size=2, stride=2) + >>> pool(x) + """ + def __init__( + self, + kernel_size: Union[int, Tuple[int]], + stride: Optional[Union[int, Tuple[int]]] = ..., + padding: Union[int, Tuple[int]] = ..., + ) -> None: ... + +class AvgPool1d(_Pool1d): + r"""Applies 1-dimensional average pooling. + + Spatially downsamples the input by taking the average of a sliding window + of size ``kernel_size`` and sliding stride ``stride``. + + Args: + kernel_size (int or tuple(int)): The size of the pooling window kernel. + stride (int or tuple(int), optional): The stride of the pooling window. + Default: ``kernel_size``. + padding (int or tuple(int), optional): How much zero padding to apply to + the input. The padding amount is applied to both sides of the spatial + axis. Default: ``0``. + + Examples: + >>> import mlx.core as mx + >>> import layers as nn + >>> x = mx.random.normal(shape=(4, 16, 5)) + >>> pool = nn.AvgPool1d(kernel_size=2, stride=2) + >>> pool(x) + """ + def __init__( + self, + kernel_size: Union[int, Tuple[int]], + stride: Optional[Union[int, Tuple[int]]] = ..., + padding: Union[int, Tuple[int]] = ..., + ) -> None: ... + +class MaxPool2d(_Pool2d): + r"""Applies 2-dimensional max pooling. + + Spatially downsamples the input by taking the maximum of a sliding window + of size ``kernel_size`` and sliding stride ``stride``. + + The parameters ``kernel_size``, ``stride``, and ``padding`` can either be: + + * a single ``int`` -- in which case the same value is used for both the + height and width axis. + * a ``tuple`` of two ``int`` s -- in which case, the first ``int`` is + used for the height axis, the second ``int`` for the width axis. + + Args: + kernel_size (int or tuple(int, int)): The size of the pooling window. + stride (int or tuple(int, int), optional): The stride of the pooling + window. Default: ``kernel_size``. + padding (int or tuple(int, int), optional): How much negative infinity + padding to apply to the input. The padding is applied on both sides + of the height and width axis. Default: ``0``. + + Examples: + >>> import mlx.core as mx + >>> import layers as nn + >>> x = mx.random.normal(shape=(8, 32, 32, 4)) + >>> pool = nn.MaxPool2d(kernel_size=2, stride=2) + >>> pool(x) + """ + def __init__( + self, + kernel_size: Union[int, Tuple[int, int]], + stride: Optional[Union[int, Tuple[int, int]]] = ..., + padding: Optional[Union[int, Tuple[int, int]]] = ..., + ) -> None: ... + +class AvgPool2d(_Pool2d): + r"""Applies 2-dimensional average pooling. + + Spatially downsamples the input by taking the average of a sliding window + of size ``kernel_size`` and sliding stride ``stride``. + + The parameters ``kernel_size``, ``stride``, and ``padding`` can either be: + + * a single ``int`` -- in which case the same value is used for both the + height and width axis. + * a ``tuple`` of two ``int`` s -- in which case, the first ``int`` is + used for the height axis, the second ``int`` for the width axis. + + Args: + kernel_size (int or tuple(int, int)): The size of the pooling window. + stride (int or tuple(int, int), optional): The stride of the pooling + window. Default: ``kernel_size``. + padding (int or tuple(int, int), optional): How much zero + padding to apply to the input. The padding is applied on both sides + of the height and width axis. Default: ``0``. + + Examples: + >>> import mlx.core as mx + >>> import layers as nn + >>> x = mx.random.normal(shape=(8, 32, 32, 4)) + >>> pool = nn.AvgPool2d(kernel_size=2, stride=2) + >>> pool(x) + """ + def __init__( + self, + kernel_size: Union[int, Tuple[int, int]], + stride: Optional[Union[int, Tuple[int, int]]] = ..., + padding: Optional[Union[int, Tuple[int, int]]] = ..., + ) -> None: ... + +class MaxPool3d(_Pool3d): + r"""Applies 3-dimensional max pooling. + + Spatially downsamples the input by taking the maximum of a sliding window + of size ``kernel_size`` and sliding stride ``stride``. + + The parameters ``kernel_size``, ``stride``, and ``padding`` can either be: + + * a single ``int`` -- in which case the same value is used for the depth, + height, and width axis. + * a ``tuple`` of three ``int`` s -- in which case, the first ``int`` is used + for the depth axis, the second ``int`` for the height axis, and the third + ``int`` for the width axis. + + Args: + kernel_size (int or tuple(int, int, int)): The size of the pooling window. + stride (int or tuple(int, int, int), optional): The stride of the pooling + window. Default: ``kernel_size``. + padding (int or tuple(int, int, int), optional): How much negative infinity + padding to apply to the input. The padding is applied on both sides + of the depth, height and width axis. Default: ``0``. + + Examples: + >>> import mlx.core as mx + >>> import layers as nn + >>> x = mx.random.normal(shape=(8, 16, 32, 32, 4)) + >>> pool = nn.MaxPool3d(kernel_size=2, stride=2) + >>> pool(x) + """ + def __init__( + self, + kernel_size: Union[int, Tuple[int, int, int]], + stride: Optional[Union[int, Tuple[int, int, int]]] = ..., + padding: Optional[Union[int, Tuple[int, int, int]]] = ..., + ) -> None: ... + +class AvgPool3d(_Pool3d): + r"""Applies 3-dimensional average pooling. + + Spatially downsamples the input by taking the average of a sliding window + of size ``kernel_size`` and sliding stride ``stride``. + + The parameters ``kernel_size``, ``stride``, and ``padding`` can either be: + + * a single ``int`` -- in which case the same value is used for the depth, + height, and width axis. + * a ``tuple`` of three ``int`` s -- in which case, the first ``int`` is used + for the depth axis, the second ``int`` for the height axis, and the third + ``int`` for the width axis. + + Args: + kernel_size (int or tuple(int, int, int)): The size of the pooling window. + stride (int or tuple(int, int, int), optional): The stride of the pooling + window. Default: ``kernel_size``. + padding (int or tuple(int, int, int), optional): How much zero + padding to apply to the input. The padding is applied on both sides + of the depth, height and width axis. Default: ``0``. + + Examples: + >>> import mlx.core as mx + >>> import layers as nn + >>> x = mx.random.normal(shape=(8, 16, 32, 32, 4)) + >>> pool = nn.AvgPool3d(kernel_size=2, stride=2) + >>> pool(x) + """ + def __init__( + self, + kernel_size: Union[int, Tuple[int, int, int]], + stride: Optional[Union[int, Tuple[int, int, int]]] = ..., + padding: Optional[Union[int, Tuple[int, int, int]]] = ..., + ) -> None: ... diff --git a/.mlx_typings/mlx/nn/layers/positional_encoding.pyi b/.mlx_typings/mlx/nn/layers/positional_encoding.pyi new file mode 100644 index 00000000..14e07e14 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/positional_encoding.pyi @@ -0,0 +1,80 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Optional + +import mlx.core as mx +from base import Module + +class RoPE(Module): + """Implements the rotary positional encoding. + + The traditional implementation rotates consecutive pairs of elements in the + feature dimension while the default implementation rotates pairs with + stride half the feature dimensions for efficiency. + + For more details see `RoFormer: Enhanced Transformer with Rotary Position + Embedding `_. + + Args: + dims (int): The feature dimensions to be rotated. If the input feature + is larger than dims then the rest is left unchanged. + traditional (bool, optional): If set to ``True`` choose the traditional + implementation which is slightly less efficient. Default: ``False``. + base (float, optional): The base used to compute angular frequency for + each dimension in the positional encodings. Default: ``10000``. + scale (float, optional): The scale used to scale the positions. Default: ``1.0``. + """ + def __init__( + self, dims: int, traditional: bool = ..., base: float = ..., scale: float = ... + ) -> None: ... + def __call__(self, x, offset: int = ...) -> mx.array: ... + +class SinusoidalPositionalEncoding(Module): + r"""Implements sinusoidal positional encoding. + + For more details see the paper `Attention Is All You Need + `_. + + Args: + dims (int): The dimensionality of the resulting positional embeddings. + min_freq (float, optional): The minimum frequency expected. Default: + ``0.0001``. + max_freq (float, optional): The maximum frequency expected. Default: + ``1``. + scale (float, optional): A multiplicative scale for the embeddings. + Default: ``sqrt(2/dims)``. + cos_first (bool, optional): If ``True`` embed using ``[cos(x); sin(x)]`` + instead of the reverse. Default: ``False``. + full_turns (bool, optional): If ``True`` multiply the frequencies with + :math:`2\pi`. Default: ``False``. + """ + def __init__( + self, + dims: int, + min_freq: float = ..., + max_freq: float = ..., + scale: Optional[float] = ..., + cos_first: bool = ..., + full_turns: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + +class ALiBi(Module): + _alibi_mask_key = ... + _alibi_mask = ... + @classmethod + def create_alibi_matrix( + cls, + q_sequence_length: int, + k_sequence_length: int, + num_heads: int, + offset: int, + dtype=..., + ) -> mx.array | None: ... + @staticmethod + def create_alibi_slope(num_heads: int) -> mx.array: ... + def __call__( + self, attention_scores: mx.array, offset=..., mask=... + ) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/layers/quantized.pyi b/.mlx_typings/mlx/nn/layers/quantized.pyi new file mode 100644 index 00000000..137a4c8e --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/quantized.pyi @@ -0,0 +1,125 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Callable, Optional, Union + +import mlx.core as mx +from base import Module + +def quantize( + model: Module, + group_size: int = ..., + bits: int = ..., + *, + mode: str = ..., + class_predicate: Optional[Callable[[str, Module], Union[bool, dict]]] = ..., +): # -> None: + """Quantize the sub-modules of a module according to a predicate. + + By default all layers that define a ``to_quantized(group_size, bits)`` + method will be quantized. Both :obj:`Linear` and :obj:`Embedding` layers + will be quantized. Note also, the module is updated in-place. + + Args: + model (Module): The model whose leaf modules may be quantized. + group_size (int): The quantization group size (see + :func:`mlx.core.quantize`). Default: ``64``. + bits (int): The number of bits per parameter (see + :func:`mlx.core.quantize`). Default: ``4``. + mode (str): The quantization method to use (see + :func:`mlx.core.quantize`). Default: ``"affine"``. + class_predicate (Optional[Callable]): A callable which receives the + :obj:`Module` path and :obj:`Module` itself and returns ``True`` or a + dict of params for `to_quantized` if it should be quantized and + ``False`` otherwise. If ``None``, then all layers that define a + ``to_quantized(group_size, bits)`` method are quantized. + Default: ``None``. + """ + +class QuantizedEmbedding(Module): + """The same as :obj:`Embedding` but with a quantized weight matrix. + + :obj:`QuantizedEmbedding` also provides a :meth:`from_embedding` + classmethod to convert embedding layers to :obj:`QuantizedEmbedding` + layers. + + Args: + num_embeddings (int): How many possible discrete tokens can we embed. + Usually called the vocabulary size. + dims (int): The dimensionality of the embeddings. + group_size (int, optional): The group size to use for the quantized + weight. See :func:`~mlx.core.quantize`. Default: ``64``. + bits (int, optional): The bit width to use for the quantized weight. + See :func:`~mlx.core.quantize`. Default: ``4``. + mode (str): The quantization method to use (see + :func:`mlx.core.quantize`). Default: ``"affine"``. + """ + def __init__( + self, + num_embeddings: int, + dims: int, + group_size: int = ..., + bits: int = ..., + mode: str = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + def as_linear(self, x: mx.array) -> mx.array: + """ + Call the quantized embedding layer as a quantized linear layer. + + Use this for example when input embedding and output projection + weights are tied. + """ + + @classmethod + def from_embedding( + cls, + embedding_layer: Module, + group_size: int = ..., + bits: int = ..., + mode: str = ..., + ) -> QuantizedEmbedding: + """Create a :obj:`QuantizedEmbedding` layer from an :obj:`Embedding` layer.""" + +class QuantizedLinear(Module): + """Applies an affine transformation to the input using a quantized weight matrix. + + It is the quantized equivalent of :class:`Linear`. For now its + parameters are frozen and will not be included in any gradient computation + but this will probably change in the future. + + :obj:`QuantizedLinear` also provides a classmethod :meth:`from_linear` to + convert linear layers to :obj:`QuantizedLinear` layers. + + Args: + input_dims (int): The dimensionality of the input features. + output_dims (int): The dimensionality of the output features. + bias (bool, optional): If set to ``False`` then the layer will not use + a bias. Default: ``True``. + group_size (int, optional): The group size to use for the quantized + weight. See :func:`~mlx.core.quantize`. Default: ``64``. + bits (int, optional): The bit width to use for the quantized weight. + See :func:`~mlx.core.quantize`. Default: ``4``. + mode (str): The quantization method to use (see + :func:`mlx.core.quantize`). Default: ``"affine"``. + """ + def __init__( + self, + input_dims: int, + output_dims: int, + bias: bool = ..., + group_size: int = ..., + bits: int = ..., + mode: str = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... + @classmethod + def from_linear( + cls, + linear_layer: Module, + group_size: int = ..., + bits: int = ..., + mode: str = ..., + ) -> QuantizedLinear: + """Create a :obj:`QuantizedLinear` layer from a :obj:`Linear` layer.""" diff --git a/.mlx_typings/mlx/nn/layers/recurrent.pyi b/.mlx_typings/mlx/nn/layers/recurrent.pyi new file mode 100644 index 00000000..d31d9382 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/recurrent.pyi @@ -0,0 +1,113 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Callable, Optional + +import mlx.core as mx +from base import Module + +class RNN(Module): + r"""An Elman recurrent layer. + + The input is a sequence of shape ``NLD`` or ``LD`` where: + + * ``N`` is the optional batch dimension + * ``L`` is the sequence length + * ``D`` is the input's feature dimension + + Concretely, for each element along the sequence length axis, this + layer applies the function: + + .. math:: + + h_{t + 1} = \text{tanh} (W_{ih}x_t + W_{hh}h_t + b) + + The hidden state :math:`h` has shape ``NH`` or ``H``, depending on + whether the input is batched or not. Returns the hidden state at each + time step, of shape ``NLH`` or ``LH``. + + Args: + input_size (int): Dimension of the input, ``D``. + hidden_size (int): Dimension of the hidden state, ``H``. + bias (bool, optional): Whether to use a bias. Default: ``True``. + nonlinearity (callable, optional): Non-linearity to use. If ``None``, + then func:`tanh` is used. Default: ``None``. + """ + def __init__( + self, + input_size: int, + hidden_size: int, + bias: bool = ..., + nonlinearity: Optional[Callable] = ..., + ) -> None: ... + def __call__(self, x: mx.array, hidden=...) -> mx.array: ... + +class GRU(Module): + r"""A gated recurrent unit (GRU) RNN layer. + + The input has shape ``NLD`` or ``LD`` where: + + * ``N`` is the optional batch dimension + * ``L`` is the sequence length + * ``D`` is the input's feature dimension + + Concretely, for each element of the sequence, this layer computes: + + .. math:: + + \begin{aligned} + r_t &= \sigma (W_{xr}x_t + W_{hr}h_t + b_{r}) \\ + z_t &= \sigma (W_{xz}x_t + W_{hz}h_t + b_{z}) \\ + n_t &= \text{tanh}(W_{xn}x_t + b_{n} + r_t \odot (W_{hn}h_t + b_{hn})) \\ + h_{t + 1} &= (1 - z_t) \odot n_t + z_t \odot h_t + \end{aligned} + + The hidden state :math:`h` has shape ``NH`` or ``H`` depending on + whether the input is batched or not. Returns the hidden state at each + time step of shape ``NLH`` or ``LH``. + + Args: + input_size (int): Dimension of the input, ``D``. + hidden_size (int): Dimension of the hidden state, ``H``. + bias (bool): Whether to use biases or not. Default: ``True``. + """ + def __init__(self, input_size: int, hidden_size: int, bias: bool = ...) -> None: ... + def __call__(self, x: mx.array, hidden=...) -> mx.array: ... + +class LSTM(Module): + r"""An LSTM recurrent layer. + + The input has shape ``NLD`` or ``LD`` where: + + * ``N`` is the optional batch dimension + * ``L`` is the sequence length + * ``D`` is the input's feature dimension + + Concretely, for each element of the sequence, this layer computes: + + .. math:: + \begin{aligned} + i_t &= \sigma (W_{xi}x_t + W_{hi}h_t + b_{i}) \\ + f_t &= \sigma (W_{xf}x_t + W_{hf}h_t + b_{f}) \\ + g_t &= \text{tanh} (W_{xg}x_t + W_{hg}h_t + b_{g}) \\ + o_t &= \sigma (W_{xo}x_t + W_{ho}h_t + b_{o}) \\ + c_{t + 1} &= f_t \odot c_t + i_t \odot g_t \\ + h_{t + 1} &= o_t \text{tanh}(c_{t + 1}) + \end{aligned} + + The hidden state :math:`h` and cell state :math:`c` have shape ``NH`` + or ``H``, depending on whether the input is batched or not. + + The layer returns two arrays, the hidden state and the cell state at + each time step, both of shape ``NLH`` or ``LH``. + + Args: + input_size (int): Dimension of the input, ``D``. + hidden_size (int): Dimension of the hidden state, ``H``. + bias (bool): Whether to use biases or not. Default: ``True``. + """ + def __init__(self, input_size: int, hidden_size: int, bias: bool = ...) -> None: ... + def __call__( + self, x: mx.array, hidden=..., cell=... + ) -> tuple[mx.array, mx.array]: ... diff --git a/.mlx_typings/mlx/nn/layers/transformer.pyi b/.mlx_typings/mlx/nn/layers/transformer.pyi new file mode 100644 index 00000000..9274a823 --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/transformer.pyi @@ -0,0 +1,168 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Any, Callable, Optional + +import mlx.core as mx +from base import Module + +class MultiHeadAttention(Module): + """Implements the scaled dot product attention with multiple heads. + + Given inputs for queries, keys and values the ``MultiHeadAttention`` + produces new values by aggregating information from the input values + according to the similarities of the input queries and keys. + + All inputs as well as the output are linearly projected without biases by + default. + + ``MultiHeadAttention`` also takes an optional additive attention mask that + should be broadcastable with ``(batch, num_heads, # queries, # keys)``. The + mask should have ``-inf`` or very large negative numbers at the positions + that should *not* be attended to. + + Args: + dims (int): The model dimensions. This is also the default + value for the queries, keys, values, and the output. + num_heads (int): The number of attention heads to use. + query_input_dims (int, optional): The input dimensions of the queries. + Default: ``dims``. + key_input_dims (int, optional): The input dimensions of the keys. + Default: ``dims``. + value_input_dims (int, optional): The input dimensions of the values. + Default: ``key_input_dims``. + value_dims (int, optional): The dimensions of the values after the + projection. Default: ``dims``. + value_output_dims (int, optional): The dimensions the new values will + be projected to. Default: ``dims``. + bias (bool, optional): Whether or not to use a bias in the projections. + Default: ``False``. + """ + def __init__( + self, + dims: int, + num_heads: int, + query_input_dims: Optional[int] = ..., + key_input_dims: Optional[int] = ..., + value_input_dims: Optional[int] = ..., + value_dims: Optional[int] = ..., + value_output_dims: Optional[int] = ..., + bias: bool = ..., + ) -> None: ... + def __call__( + self, queries: mx.array, keys: mx.array, values: mx.array, mask: mx.array = ... + ) -> mx.array: ... + @staticmethod + def create_additive_causal_mask(N: int, dtype: mx.Dtype = ...) -> mx.array: ... + +class TransformerEncoderLayer(Module): + def __init__( + self, + dims: int, + num_heads: int, + mlp_dims: Optional[int] = ..., + dropout: float = ..., + activation: Callable[[Any], Any] = ..., + norm_first: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array, mask: mx.array) -> mx.array: ... + +class TransformerEncoder(Module): + def __init__( + self, + num_layers: int, + dims: int, + num_heads: int, + mlp_dims: Optional[int] = ..., + dropout: float = ..., + activation=..., + norm_first: bool = ..., + checkpoint: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array, mask: mx.array) -> mx.array: ... + +class TransformerDecoderLayer(Module): + def __init__( + self, + dims: int, + num_heads: int, + mlp_dims: Optional[int] = ..., + dropout: float = ..., + activation: Callable[[Any], Any] = ..., + norm_first: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array, memory, x_mask, memory_mask) -> mx.array: ... + +class TransformerDecoder(Module): + def __init__( + self, + num_layers: int, + dims: int, + num_heads: int, + mlp_dims: Optional[int] = ..., + dropout: float = ..., + activation=..., + norm_first: bool = ..., + checkpoint: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array, memory, x_mask, memory_mask) -> mx.array: ... + +class Transformer(Module): + """ + Implements a standard Transformer model. + + The implementation is based on `Attention Is All You Need + `_. + + The Transformer model contains an encoder and a decoder. The encoder + processes the input sequence and the decoder generates the output sequence. + The interaction between encoder and decoder happens through the attention + mechanism. + + Args: + dims (int, optional): The number of expected features in the + encoder/decoder inputs. Default: ``512``. + num_heads (int, optional): The number of attention heads. Default: + ``8``. + num_encoder_layers (int, optional): The number of encoder layers in the + Transformer encoder. Default: ``6``. + num_decoder_layers (int, optional): The number of decoder layers in the + Transformer decoder. Default: ``6``. + mlp_dims (int, optional): The hidden dimension of the MLP block in each + Transformer layer. Defaults to ``4*dims`` if not provided. Default: + ``None``. + dropout (float, optional): The dropout value for the Transformer + encoder and decoder. Dropout is used after each attention layer and + the activation in the MLP layer. Default: ``0.0``. + activation (function, optional): the activation function for the MLP + hidden layer. Default: :func:`relu`. + custom_encoder (nn.Module, optional): A custom encoder to replace the + standard Transformer encoder. Default: ``None``. + custom_decoder (nn.Module, optional): A custom decoder to replace the + standard Transformer decoder. Default: ``None``. + norm_first (bool, optional): if ``True``, encoder and decoder layers + will perform layer normalization before attention and MLP + operations, otherwise after. Default: ``True``. + checkpoint (bool, optional): if ``True`` perform gradient checkpointing + to reduce the memory usage at the expense of more computation. + Default: ``False``. + """ + def __init__( + self, + dims: int = ..., + num_heads: int = ..., + num_encoder_layers: int = ..., + num_decoder_layers: int = ..., + mlp_dims: Optional[int] = ..., + dropout: float = ..., + activation: Callable[[Any], Any] = ..., + custom_encoder: Optional[Any] = ..., + custom_decoder: Optional[Any] = ..., + norm_first: bool = ..., + checkpoint: bool = ..., + ) -> None: ... + def __call__( + self, src, tgt, src_mask, tgt_mask, memory_mask + ) -> mx.array: # -> array | Any: + ... diff --git a/.mlx_typings/mlx/nn/layers/upsample.pyi b/.mlx_typings/mlx/nn/layers/upsample.pyi new file mode 100644 index 00000000..1ef3298c --- /dev/null +++ b/.mlx_typings/mlx/nn/layers/upsample.pyi @@ -0,0 +1,87 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Literal, Tuple, Union + +import mlx.core as mx +from base import Module + +def upsample_nearest(x: mx.array, scale_factor: Tuple) -> mx.array: ... +def upsample_linear( + x: mx.array, scale_factor: Tuple, align_corners: bool = ... +): # -> int: + ... +def upsample_cubic( + x: mx.array, scale_factor: Tuple, align_corners: bool = ... +): # -> int: + ... + +class Upsample(Module): + r"""Upsample the input signal spatially. + + The spatial dimensions are by convention dimensions ``1`` to ``x.ndim - + 2``. The first is the batch dimension and the last is the feature + dimension. + + For example, an audio signal would be 3D with 1 spatial dimension, an image + 4D with 2 and so on and so forth. + + There are three upsampling algorithms implemented nearest neighbor upsampling, + linear interpolation, and cubic interpolation. All can be applied to any number + of spatial dimensions. The linear interpolation will be bilinear, trilinear etc + when applied to more than one spatial dimension. And cubic interpolation will be + bicubic when there are 2 spatial dimensions. + + .. note:: + When using one of the linear or cubic interpolation modes the ``align_corners`` + argument changes how the corners are treated in the input image. If + ``align_corners=True`` then the top and left edge of the input and + output will be matching as will the bottom right edge. + + Parameters: + scale_factor (float or tuple): The multiplier for the spatial size. + If a ``float`` is provided, it is the multiplier for all spatial dimensions. + Otherwise, the number of scale factors provided must match the + number of spatial dimensions. + mode (str, optional): The upsampling algorithm, either ``"nearest"``, + ``"linear"`` or ``"cubic"``. Default: ``"nearest"``. + align_corners (bool, optional): Changes the way the corners are treated + during ``"linear"`` and ``"cubic"`` upsampling. See the note above and the + examples below for more details. Default: ``False``. + + Examples: + >>> import mlx.core as mx + >>> import mlx.nn as nn + >>> x = mx.arange(1, 5).reshape((1, 2, 2, 1)) + >>> x + array([[[[1], + [2]], + [[3], + [4]]]], dtype=int32) + >>> n = nn.Upsample(scale_factor=2, mode='nearest') + >>> n(x).squeeze() + array([[1, 1, 2, 2], + [1, 1, 2, 2], + [3, 3, 4, 4], + [3, 3, 4, 4]], dtype=int32) + >>> b = nn.Upsample(scale_factor=2, mode='linear') + >>> b(x).squeeze() + array([[1, 1.25, 1.75, 2], + [1.5, 1.75, 2.25, 2.5], + [2.5, 2.75, 3.25, 3.5], + [3, 3.25, 3.75, 4]], dtype=float32) + >>> b = nn.Upsample(scale_factor=2, mode='linear', align_corners=True) + >>> b(x).squeeze() + array([[1, 1.33333, 1.66667, 2], + [1.66667, 2, 2.33333, 2.66667], + [2.33333, 2.66667, 3, 3.33333], + [3, 3.33333, 3.66667, 4]], dtype=float32) + """ + def __init__( + self, + scale_factor: Union[float, Tuple], + mode: Literal["nearest", "linear", "cubic"] = ..., + align_corners: bool = ..., + ) -> None: ... + def __call__(self, x: mx.array) -> mx.array: ... diff --git a/.mlx_typings/mlx/nn/losses.pyi b/.mlx_typings/mlx/nn/losses.pyi new file mode 100644 index 00000000..9b5ded9e --- /dev/null +++ b/.mlx_typings/mlx/nn/losses.pyi @@ -0,0 +1,419 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Literal, Optional + +import mlx.core as mx + +Reduction = Literal["none", "mean", "sum"] + +def cross_entropy( + logits: mx.array, + targets: mx.array, + weights: Optional[mx.array] = ..., + axis: int = ..., + label_smoothing: float = ..., + reduction: Reduction = ..., +) -> mx.array: + """ + Computes the cross entropy loss. + + Args: + logits (array): The unnormalized logits. + targets (array): The ground truth values. These can be class indices or + probabilities for each class. If the ``targets`` are class indices, + then ``targets`` shape should match the ``logits`` shape with + the ``axis`` dimension removed. If the ``targets`` are probabilities + (or one-hot encoded), then the ``targets`` shape should be the same as + the ``logits`` shape. + weights (array, optional): Optional weights for each target. Default: ``None``. + axis (int, optional): The axis over which to compute softmax. Default: ``-1``. + label_smoothing (float, optional): Label smoothing factor. Default: ``0``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed cross entropy loss. + + Examples: + >>> import mlx.core as mx + >>> import mlx.nn as nn + >>> + >>> # Class indices as targets + >>> logits = mx.array([[2.0, -1.0], [-1.0, 2.0]]) + >>> targets = mx.array([0, 1]) + >>> nn.losses.cross_entropy(logits, targets) + array([0.0485873, 0.0485873], dtype=float32) + >>> + >>> # Probabilities (or one-hot vectors) as targets + >>> logits = mx.array([[2.0, -1.0], [-1.0, 2.0]]) + >>> targets = mx.array([[0.9, 0.1], [0.1, 0.9]]) + >>> nn.losses.cross_entropy(logits, targets) + array([0.348587, 0.348587], dtype=float32) + """ + +def binary_cross_entropy( + inputs: mx.array, + targets: mx.array, + weights: Optional[mx.array] = ..., + with_logits: bool = ..., + reduction: Reduction = ..., +) -> mx.array: + """ + Computes the binary cross entropy loss. + + By default, this function takes the pre-sigmoid logits, which results in a faster + and more precise loss. For improved numerical stability when ``with_logits=False``, + the loss calculation clips the input probabilities (in log-space) to a minimum value + of ``-100``. + + Args: + inputs (array): The predicted values. If ``with_logits`` is ``True``, then + ``inputs`` are unnormalized logits. Otherwise, ``inputs`` are probabilities. + targets (array): The binary target values in {0, 1}. + with_logits (bool, optional): Whether ``inputs`` are logits. Default: ``True``. + weights (array, optional): Optional weights for each target. Default: ``None``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. + + Returns: + array: The computed binary cross entropy loss. + Examples: + >>> import mlx.core as mx + >>> import mlx.nn as nn + + >>> logits = mx.array([0.105361, 0.223144, 1.20397, 0.916291]) + >>> targets = mx.array([0, 0, 1, 1]) + >>> loss = nn.losses.binary_cross_entropy(logits, targets, reduction="mean") + >>> loss + array(0.539245, dtype=float32) + + >>> probs = mx.array([0.1, 0.1, 0.4, 0.4]) + >>> targets = mx.array([0, 0, 1, 1]) + >>> loss = nn.losses.binary_cross_entropy(probs, targets, with_logits=False, reduction="mean") + >>> loss + array(0.510826, dtype=float32) + """ + +def l1_loss( + predictions: mx.array, targets: mx.array, reduction: Reduction = ... +) -> mx.array: + """ + Computes the L1 loss. + + Args: + predictions (array): The predicted values. + targets (array): The target values. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. + + Returns: + array: The computed L1 loss. + """ + +def mse_loss( + predictions: mx.array, targets: mx.array, reduction: Reduction = ... +) -> mx.array: + """ + Computes the mean squared error loss. + + Args: + predictions (array): The predicted values. + targets (array): The target values. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. + + Returns: + array: The computed mean squared error loss. + """ + +def nll_loss( + inputs: mx.array, targets: mx.array, axis: int = ..., reduction: Reduction = ... +) -> mx.array: + """ + Computes the negative log likelihood loss. + + Args: + inputs (array): The predicted distribution in log space. + targets (array): The target values. + axis (int, optional): The distribution axis. Default: ``-1``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed NLL loss. + """ + +def gaussian_nll_loss( + inputs: mx.array, + targets: mx.array, + vars: mx.array, + full: bool = ..., + eps: float = ..., + reduction: Reduction = ..., +) -> mx.array: + r""" + Computes the negative log likelihood loss for a Gaussian distribution. + + The loss is given by: + + .. math:: + \frac{1}{2}\left(\log\left(\max\left(\text{vars}, + \ \epsilon\right)\right) + \frac{\left(\text{inputs} - \text{targets} \right)^2} + {\max\left(\text{vars}, \ \epsilon \right)}\right) + \text{const.} + + where ``inputs`` are the predicted means and ``vars`` are the the + predicted variances. + + Args: + inputs (array): The predicted expectation of the Gaussian distribution. + targets (array): The target values (samples from the Gaussian distribution). + vars (array): The predicted variance of the Gaussian distribution. + full (bool, optional): Whether to include the constant term in the loss calculation. + Default: ``False``. + eps (float, optional): Small positive constant for numerical stability. + Default: ``1e-6``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The Gaussian NLL loss. + """ + +def kl_div_loss( + inputs: mx.array, targets: mx.array, axis: int = ..., reduction: Reduction = ... +) -> mx.array: + """ + Computes the Kullback-Leibler divergence loss. + + Computes the following when ``reduction == 'none'``: + + .. code-block:: python + + mx.exp(targets) * (targets - inputs).sum(axis) + + Args: + inputs (array): Log probabilities for the predicted distribution. + targets (array): Log probabilities for the target distribution. + axis (int, optional): The distribution axis. Default: ``-1``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed Kullback-Leibler divergence loss. + """ + +def smooth_l1_loss( + predictions: mx.array, + targets: mx.array, + beta: float = ..., + reduction: Reduction = ..., +) -> mx.array: + r""" + Computes the smooth L1 loss. + + The smooth L1 loss is a variant of the L1 loss which replaces the absolute + difference with a squared difference when the absolute difference is less + than ``beta``. + + The formula for the smooth L1 Loss is: + + .. math:: + + l = \begin{cases} + 0.5 (x - y)^2 / \beta, & \text{if } |x - y| < \beta \\ + |x - y| - 0.5 \beta, & \text{otherwise} + \end{cases} + + Args: + predictions (array): Predicted values. + targets (array): Ground truth values. + beta (float, optional): The threshold after which the loss changes + from the squared to the absolute difference. Default: ``1.0``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. + + Returns: + array: The computed smooth L1 loss. + """ + +def triplet_loss( + anchors: mx.array, + positives: mx.array, + negatives: mx.array, + axis: int = ..., + p: int = ..., + margin: float = ..., + eps: float = ..., + reduction: Reduction = ..., +) -> mx.array: + r""" + Computes the triplet loss for a set of anchor, positive, and negative samples. + Margin is represented with alpha in the math section. + + .. math:: + + \max\left(\|A - P\|_p - \|A - N\|_p + \alpha, 0\right) + + Args: + anchors (array): The anchor samples. + positives (array): The positive samples. + negatives (array): The negative samples. + axis (int, optional): The distribution axis. Default: ``-1``. + p (int, optional): The norm degree for pairwise distance. Default: ``2``. + margin (float, optional): Margin for the triplet loss. Defaults to ``1.0``. + eps (float, optional): Small positive constant to prevent numerical instability. Defaults to ``1e-6``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: Computed triplet loss. If reduction is "none", returns a tensor of the same shape as input; + if reduction is "mean" or "sum", returns a scalar tensor. + """ + +def hinge_loss( + inputs: mx.array, targets: mx.array, reduction: Reduction = ... +) -> mx.array: + r""" + Computes the hinge loss between inputs and targets. + + .. math:: + + \text{hinge}(y, y_{\text{pred}}) = \max(0, 1 - y \cdot y_{\text{pred}}) + + + Args: + inputs (array): The predicted values. + targets (array): The target values. They should be -1 or 1. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed hinge loss. + """ + +def huber_loss( + inputs: mx.array, targets: mx.array, delta: float = ..., reduction: Reduction = ... +) -> mx.array: + r""" + Computes the Huber loss between inputs and targets. + + .. math:: + + l_{\delta}(a) = + \left\{ \begin{array}{ll} + \frac{1}{2} a^2 & \text{for } |a| \leq \delta, \\ + \delta \left( |a| - \frac{1}{2} \delta \right) & \text{otherwise.} + \end{array} \right. + + Args: + inputs (array): The predicted values. + targets (array): The target values. + delta (float, optional): The threshold at which to change between L1 and L2 loss. + Default: ``1.0``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed Huber loss. + """ + +def log_cosh_loss( + inputs: mx.array, targets: mx.array, reduction: Reduction = ... +) -> mx.array: + r""" + Computes the log cosh loss between inputs and targets. + + Logcosh acts like L2 loss for small errors, ensuring stable gradients, + and like the L1 loss for large errors, reducing sensitivity to outliers. This + dual behavior offers a balanced, robust approach for regression tasks. + + .. math:: + + \text{logcosh}(y_{\text{true}}, y_{\text{pred}}) = + \frac{1}{n} \sum_{i=1}^{n} + \log(\cosh(y_{\text{pred}}^{(i)} - y_{\text{true}}^{(i)})) + + + Args: + inputs (array): The predicted values. + targets (array): The target values. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed log cosh loss. + """ + +def cosine_similarity_loss( + x1: mx.array, + x2: mx.array, + axis: int = ..., + eps: float = ..., + reduction: Reduction = ..., +) -> mx.array: + r""" + Computes the cosine similarity between the two inputs. + + The cosine similarity loss is given by + + .. math:: + + \frac{x_1 \cdot x_2}{\max(\|x_1\| \cdot \|x_2\|, \epsilon)} + + Args: + x1 (mx.array): The first set of inputs. + x2 (mx.array): The second set of inputs. + axis (int, optional): The embedding axis. Default: ``1``. + eps (float, optional): The minimum value of the denominator used for + numerical stability. Default: ``1e-8``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + mx.array: The computed cosine similarity loss. + """ + +def margin_ranking_loss( + inputs1: mx.array, + inputs2: mx.array, + targets: mx.array, + margin: float = ..., + reduction: Reduction = ..., +) -> mx.array: + r""" + Calculate the margin ranking loss that loss given inputs :math:`x_1`, :math:`x_2` and a label + :math:`y` (containing 1 or -1). + + The loss is given by: + + .. math:: + \text{loss} = \max (0, -y * (x_1 - x_2) + \text{margin}) + + Where :math:`y` represents ``targets``, :math:`x_1` represents ``inputs1`` and :math:`x_2` + represents ``inputs2``. + + Args: + inputs1 (array): Scores for the first input. + inputs2 (array): Scores for the second input. + targets (array): Labels indicating whether samples in ``inputs1`` should be ranked higher + than samples in ``inputs2``. Values should be 1 or -1. + margin (float, optional): The margin by which the scores should be separated. + Default: ``0.0``. + reduction (str, optional): Specifies the reduction to apply to the output: + ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. + + Returns: + array: The computed margin ranking loss. + + Examples: + >>> import mlx.core as mx + >>> import mlx.nn as nn + >>> targets = mx.array([1, 1, -1]) + >>> inputs1 = mx.array([-0.573409, -0.765166, -0.0638]) + >>> inputs2 = mx.array([0.75596, 0.225763, 0.256995]) + >>> loss = nn.losses.margin_ranking_loss(inputs1, inputs2, targets) + >>> loss + array(0.773433, dtype=float32) + """ diff --git a/.mlx_typings/mlx/nn/utils.pyi b/.mlx_typings/mlx/nn/utils.pyi new file mode 100644 index 00000000..7df93f12 --- /dev/null +++ b/.mlx_typings/mlx/nn/utils.pyi @@ -0,0 +1,73 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Any, Callable, Optional + +import mlx.core as mx + +from .layers.base import Module + +def value_and_grad( + model: Module, fn: Callable +): # -> _Wrapped[..., Any, ..., tuple[Any, Any]]: + """Transform the passed function ``fn`` to a function that computes the + gradients of ``fn`` wrt the model's trainable parameters and also its + value. + + Args: + model (Module): The model whose trainable parameters to compute + gradients for + fn (Callable): The scalar function to compute gradients for + + Returns: + A callable that returns the value of ``fn`` and the gradients wrt the + trainable parameters of ``model`` + """ + +def checkpoint( + module: Module, fn: Optional[Callable] = ... +): # -> _Wrapped[..., Any, ..., Any]: + """Transform the passed callable to one that performs gradient + checkpointing with respect to the trainable parameters of the module (and + the callable's inputs). + + Args: + module (Module): The module for whose parameters we will be + performing gradient checkpointing. + fn (Callable, optional): The function to checkpoint. If not provided it + defaults to the provided module. + + Returns: + A callable that saves the inputs and outputs during the forward pass + and recomputes all intermediate states during the backward pass. + """ + +def average_gradients( + gradients: Any, + group: Optional[mx.distributed.Group] = ..., + all_reduce_size: int = ..., + communication_type: Optional[mx.Dtype] = ..., + communication_stream: Optional[mx.Stream] = ..., +): # -> Any: + """Average the gradients across the distributed processes in the passed group. + + This helper enables concatenating several gradients of small arrays to one + big all reduce call for better networking performance. + + Args: + gradients (Any): The Python tree containing the gradients (it should + have the same structure across processes) + group (Optional[mlx.core.distributed.Group]): The group of processes to + average the gradients. If set to ``None`` the global group is used. + Default: ``None``. + all_reduce_size (int): Group arrays until their size in bytes exceeds + this number. Perform one communication step per group of arrays. If + less or equal to 0 array grouping is disabled. Default: ``32MiB``. + communication_type (Optional[mlx.core.Dtype]): If provided cast to this + type before performing the communication. Typically cast to a + smaller float to reduce the communication size. Default: ``None``. + communication_stream (Optional[mlx.core.Stream]): The stream to usse + for the communication. If unspecified the default communication + stream is used which can vary by back-end. Default: ``None``. + """ diff --git a/.mlx_typings/mlx/utils.pyi b/.mlx_typings/mlx/utils.pyi new file mode 100644 index 00000000..43738ca7 --- /dev/null +++ b/.mlx_typings/mlx/utils.pyi @@ -0,0 +1,189 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Any, Callable, Dict, List, Optional, Tuple, Union + +from mlx.core import MX_ARRAY_TREE + +def tree_map( + fn: Callable, tree: Any, *rest: Any, is_leaf: Optional[Callable] = ... +) -> Any: + """Applies ``fn`` to the leaves of the Python tree ``tree`` and + returns a new collection with the results. + + If ``rest`` is provided, every item is assumed to be a superset of ``tree`` + and the corresponding leaves are provided as extra positional arguments to + ``fn``. In that respect, :meth:`tree_map` is closer to :func:`itertools.starmap` + than to :func:`map`. + + The keyword argument ``is_leaf`` decides what constitutes a leaf from + ``tree`` similar to :func:`tree_flatten`. + + .. code-block:: python + + import mlx.nn as nn + from mlx.utils import tree_map + + model = nn.Linear(10, 10) + print(model.parameters().keys()) + # dict_keys(['weight', 'bias']) + + # square the parameters + model.update(tree_map(lambda x: x*x, model.parameters())) + + Args: + fn (callable): The function that processes the leaves of the tree. + tree (Any): The main Python tree that will be iterated upon. + rest (tuple[Any]): Extra trees to be iterated together with ``tree``. + is_leaf (callable, optional): An optional callable that returns ``True`` + if the passed object is considered a leaf or ``False`` otherwise. + + Returns: + A Python tree with the new values returned by ``fn``. + """ + +def tree_map_with_path( + fn: Callable, + tree: Any, + *rest: Any, + is_leaf: Optional[Callable] = ..., + path: Optional[Any] = ..., +) -> Any: + """Applies ``fn`` to the path and leaves of the Python tree ``tree`` and + returns a new collection with the results. + + This function is the same :func:`tree_map` but the ``fn`` takes the path as + the first argument followed by the remaining tree nodes. + + Args: + fn (callable): The function that processes the leaves of the tree. + tree (Any): The main Python tree that will be iterated upon. + rest (tuple[Any]): Extra trees to be iterated together with ``tree``. + is_leaf (Optional[Callable]): An optional callable that returns ``True`` + if the passed object is considered a leaf or ``False`` otherwise. + path (Optional[Any]): Prefix will be added to the result. + + Returns: + A Python tree with the new values returned by ``fn``. + + Example: + >>> from mlx.utils import tree_map_with_path + >>> tree = {"model": [{"w": 0, "b": 1}, {"w": 0, "b": 1}]} + >>> new_tree = tree_map_with_path(lambda path, _: print(path), tree) + model.0.w + model.0.b + model.1.w + model.1.b + """ + +def tree_flatten( + tree: Any, + prefix: str = ..., + is_leaf: Optional[Callable] = ..., + destination: Optional[Union[List[Tuple[str, Any]], Dict[str, Any]]] = ..., +) -> Union[List[Tuple[str, Any]], Dict[str, Any]]: + """Flattens a Python tree to a list of key, value tuples. + + The keys are using the dot notation to define trees of arbitrary depth and + complexity. + + .. code-block:: python + + from mlx.utils import tree_flatten + + print(tree_flatten([[[0]]])) + # [("0.0.0", 0)] + + print(tree_flatten([[[0]]], prefix=".hello")) + # [("hello.0.0.0", 0)] + + tree_flatten({"a": {"b": 1}}, destination={}) + {"a.b": 1} + + .. note:: + Dictionaries should have keys that are valid Python identifiers. + + Args: + tree (Any): The Python tree to be flattened. + prefix (str): A prefix to use for the keys. The first character is + always discarded. + is_leaf (callable): An optional callable that returns True if the + passed object is considered a leaf or False otherwise. + destination (list or dict, optional): A list or dictionary to store the + flattened tree. If None an empty list will be used. Default: ``None``. + + Returns: + Union[List[Tuple[str, Any]], Dict[str, Any]]: The flat representation of + the Python tree. + """ + +def tree_unflatten(tree: Union[List[Tuple[str, Any]], Dict[str, Any]]) -> Any: + """Recreate a Python tree from its flat representation. + + .. code-block:: python + + from mlx.utils import tree_unflatten + + d = tree_unflatten([("hello.world", 42)]) + print(d) + # {"hello": {"world": 42}} + + d = tree_unflatten({"hello.world": 42}) + print(d) + # {"hello": {"world": 42}} + + Args: + tree (list[tuple[str, Any]] or dict[str, Any]): The flat representation of a Python tree. + For instance as returned by :meth:`tree_flatten`. + + Returns: + A Python tree. + """ + +def tree_reduce( + fn: Callable[[Any, Any], Any], + tree: list[MX_ARRAY_TREE] | tuple[MX_ARRAY_TREE, ...] | dict[str, MX_ARRAY_TREE], + initializer=..., + is_leaf=..., +) -> None: + """Applies a reduction to the leaves of a Python tree. + + This function reduces Python trees into an accumulated result by applying + the provided function ``fn`` to the leaves of the tree. + + Example: + >>> from mlx.utils import tree_reduce + >>> tree = {"a": [1, 2, 3], "b": [4, 5]} + >>> tree_reduce(lambda acc, x: acc + x, tree, 0) + 15 + + Args: + fn (callable): The reducer function that takes two arguments (accumulator, + current value) and returns the updated accumulator. + tree (Any): The Python tree to reduce. It can be any nested combination of + lists, tuples, or dictionaries. + initializer (Any, optional): The initial value to start the reduction. If + not provided, the first leaf value is used. + is_leaf (callable, optional): A function to determine if an object is a + leaf, returning ``True`` for leaf nodes and ``False`` otherwise. + + Returns: + Any: The accumulated value. + """ + +def tree_merge( + tree_a, tree_b, merge_fn=... +): # -> dict[Any, Any] | list[Any] | tuple[Any, *tuple[Any, ...]] | tuple[Any, ...]: + """Merge two Python trees in one containing the values of both. It can be + thought of as a deep dict.update method. + + Args: + tree_a (Any): The first Python tree. + tree_b (Any): The second Python tree. + merge_fn (callable, optional): A function to merge leaves. + + Returns: + The Python tree containing the values of both ``tree_a`` and + ``tree_b``. + """ diff --git a/.mlx_typings/mlx_lm/__init__.pyi b/.mlx_typings/mlx_lm/__init__.pyi new file mode 100644 index 00000000..2ed43899 --- /dev/null +++ b/.mlx_typings/mlx_lm/__init__.pyi @@ -0,0 +1,3 @@ +import models as models +import tokenizer_utils as tokenizer_utils +from generate import * diff --git a/.mlx_typings/mlx_lm/convert.pyi b/.mlx_typings/mlx_lm/convert.pyi new file mode 100644 index 00000000..aff4de7b --- /dev/null +++ b/.mlx_typings/mlx_lm/convert.pyi @@ -0,0 +1,45 @@ +""" +This type stub file was generated by pyright. +""" + +import argparse +from typing import Callable, Optional, Union + +import mlx.nn as nn + +def mixed_quant_predicate_builder( + recipe: str, model: nn.Module, group_size: int = ... +) -> Callable[[str, nn.Module, dict], Union[bool, dict]]: ... + +QUANT_RECIPES = ... +MODEL_CONVERSION_DTYPES = ... + +def convert( + hf_path: str, + mlx_path: str = ..., + quantize: bool = ..., + q_group_size: int = ..., + q_bits: int = ..., + q_mode: str = ..., + dtype: Optional[str] = ..., + upload_repo: str = ..., + revision: Optional[str] = ..., + dequantize: bool = ..., + quant_predicate: Optional[ + Union[Callable[[str, nn.Module, dict], Union[bool, dict]], str] + ] = ..., + trust_remote_code: bool = ..., +): # -> None: + ... +def configure_parser() -> argparse.ArgumentParser: + """ + Configures and returns the argument parser for the script. + + Returns: + argparse.ArgumentParser: Configured argument parser. + """ + +def main(): # -> None: + ... + +if __name__ == "__main__": ... diff --git a/.mlx_typings/mlx_lm/generate.pyi b/.mlx_typings/mlx_lm/generate.pyi new file mode 100644 index 00000000..8a957608 --- /dev/null +++ b/.mlx_typings/mlx_lm/generate.pyi @@ -0,0 +1,324 @@ +""" +This type stub file was generated by pyright. +""" + +import contextlib +from dataclasses import dataclass +from typing import Any, Callable, Generator, List, Optional, Tuple, Union + +import mlx.core as mx +import mlx.nn as nn +from transformers import PreTrainedTokenizer + +from .tokenizer_utils import TokenizerWrapper + +DEFAULT_PROMPT = ... +DEFAULT_MAX_TOKENS = ... +DEFAULT_TEMP = ... +DEFAULT_TOP_P = ... +DEFAULT_MIN_P = ... +DEFAULT_TOP_K = ... +DEFAULT_XTC_PROBABILITY = ... +DEFAULT_XTC_THRESHOLD = ... +DEFAULT_MIN_TOKENS_TO_KEEP = ... +DEFAULT_SEED = ... +DEFAULT_MODEL = ... +DEFAULT_QUANTIZED_KV_START = ... + +def str2bool(string): # -> bool: + ... +def setup_arg_parser(): # -> ArgumentParser: + """Set up and return the argument parser.""" + +generation_stream = ... + +@contextlib.contextmanager +def wired_limit( + model: nn.Module, streams: Optional[List[mx.Stream]] = ... +): # -> Generator[None, Any, None]: + """ + A context manager to temporarily change the wired limit. + + Note, the wired limit should not be changed during an async eval. If an + async eval could be running pass in the streams to synchronize with prior + to exiting the context manager. + """ +@dataclass +class GenerationResponse: + """ + The output of :func:`stream_generate`. + + Args: + text (str): The next segment of decoded text. This can be an empty string. + token (int): The next token. + from_draft (bool): Whether the token was generated by the draft model. + logprobs (mx.array): A vector of log probabilities. + prompt_tokens (int): The number of tokens in the prompt. + prompt_tps (float): The prompt processing tokens-per-second. + generation_tokens (int): The number of generated tokens. + generation_tps (float): The tokens-per-second for generation. + peak_memory (float): The peak memory used so far in GB. + finish_reason (str): The reason the response is being sent: "length", "stop" or `None` + """ + + text: str + token: int + logprobs: mx.array + from_draft: bool + prompt_tokens: int + prompt_tps: float + generation_tokens: int + generation_tps: float + peak_memory: float + finish_reason: Optional[str] = ... + +def maybe_quantize_kv_cache( + prompt_cache, quantized_kv_start, kv_group_size, kv_bits +): # -> None: + ... +def generate_step( + prompt: mx.array, + model: nn.Module, + *, + max_tokens: int = ..., + sampler: Optional[Callable[[mx.array], mx.array]] = ..., + logits_processors: Optional[List[Callable[[mx.array, mx.array], mx.array]]] = ..., + max_kv_size: Optional[int] = ..., + prompt_cache: Optional[Any] = ..., + prefill_step_size: int = ..., + kv_bits: Optional[int] = ..., + kv_group_size: int = ..., + quantized_kv_start: int = ..., + prompt_progress_callback: Optional[Callable[[int], int]] = ..., + input_embeddings: Optional[mx.array] = ..., +) -> Generator[Tuple[mx.array, mx.array], None, None]: + """ + A generator producing token ids based on the given prompt from the model. + + Args: + prompt (mx.array): The input prompt. + model (nn.Module): The model to use for generation. + max_tokens (int): The maximum number of tokens. Use``-1`` for an infinite + generator. Default: ``256``. + sampler (Callable[mx.array, mx.array], optional): A sampler for sampling a + token from a vector of log probabilities. Default: ``None``. + logits_processors (List[Callable[[mx.array, mx.array], mx.array]], optional): + A list of functions that take tokens and logits and return the processed + logits. Default: ``None``. + max_kv_size (int, optional): Maximum size of the key-value cache. Old + entries (except the first 4 tokens) will be overwritten. + prompt_cache (List[Any], optional): A pre-computed prompt cache. Note, if + provided, the cache will be updated in place. + prefill_step_size (int): Step size for processing the prompt. + kv_bits (int, optional): Number of bits to use for KV cache quantization. + None implies no cache quantization. Default: ``None``. + kv_group_size (int): Group size for KV cache quantization. Default: ``64``. + quantized_kv_start (int): Step to begin using a quantized KV cache. + when ``kv_bits`` is non-None. Default: ``0``. + prompt_progress_callback (Callable[[int], int]): A call-back which takes the + prompt tokens processed so far and the total number of prompt tokens. + input_embeddings (mx.array, optional): Input embeddings to use instead of or in + conjunction with prompt tokens. Default: ``None``. + + Yields: + Tuple[mx.array, mx.array]: One token and a vector of log probabilities. + """ + +def speculative_generate_step( + prompt: mx.array, + model: nn.Module, + draft_model: nn.Module, + *, + num_draft_tokens: int = ..., + max_tokens: int = ..., + sampler: Optional[Callable[[mx.array], mx.array]] = ..., + logits_processors: Optional[List[Callable[[mx.array, mx.array], mx.array]]] = ..., + prompt_cache: Optional[Any] = ..., + prefill_step_size: int = ..., + kv_bits: Optional[int] = ..., + kv_group_size: int = ..., + quantized_kv_start: int = ..., +) -> Generator[Tuple[mx.array, mx.array, bool], None, None]: + """ + A generator producing token ids based on the given prompt from the model. + + Args: + prompt (mx.array): The input prompt. + model (nn.Module): The model to use for generation. + draft_model (nn.Module): The draft model for speculative decoding. + num_draft_tokens (int, optional): The number of draft tokens for + speculative decoding. Default: ``2``. + max_tokens (int): The maximum number of tokens. Use``-1`` for an infinite + generator. Default: ``256``. + sampler (Callable[[mx.array], mx.array], optional): A sampler for sampling a + token from a vector of log probabilities. Default: ``None``. + logits_processors (List[Callable[[mx.array, mx.array], mx.array]], optional): + A list of functions that take tokens and logits and return the processed + logits. Default: ``None``. + prompt_cache (List[Any], optional): A pre-computed prompt cache. Note, if + provided, the cache will be updated in place. The cache must be trimmable. + prefill_step_size (int): Step size for processing the prompt. + kv_bits (int, optional): Number of bits to use for KV cache quantization. + None implies no cache quantization. Default: ``None``. + kv_group_size (int): Group size for KV cache quantization. Default: ``64``. + quantized_kv_start (int): Step to begin using a quantized KV cache. + when ``kv_bits`` is non-None. Default: ``0``. + + Yields: + Tuple[mx.array, mx.array, bool]: One token, a vector of log probabilities, + and a bool indicating if the token was generated by the draft model + """ + +def stream_generate( + model: nn.Module, + tokenizer: Union[PreTrainedTokenizer, TokenizerWrapper], + prompt: Union[str, mx.array, List[int]], + max_tokens: int = ..., + draft_model: Optional[nn.Module] = ..., + **kwargs: object, +) -> Generator[GenerationResponse, None, None]: + """ + A generator producing text based on the given prompt from the model. + + Args: + model (nn.Module): The model to use for generation. + tokenizer (PreTrainedTokenizer): The tokenizer. + prompt (Union[str, mx.array, List[int]]): The input prompt string or + integer tokens. + max_tokens (int): The maximum number of tokens to generate. + Default: ``256``. + draft_model (Optional[nn.Module]): An optional draft model. If provided + then speculative decoding is used. The draft model must use the same + tokenizer as the main model. Default: ``None``. + kwargs: The remaining options get passed to :func:`generate_step`. + See :func:`generate_step` for more details. + + Yields: + GenerationResponse: An instance containing the generated text segment and + associated metadata. See :class:`GenerationResponse` for details. + """ + +def generate( + model: nn.Module, + tokenizer: Union[PreTrainedTokenizer, TokenizerWrapper], + prompt: Union[str, List[int]], + verbose: bool = ..., + **kwargs, +) -> str: + """ + Generate a complete response from the model. + + Args: + model (nn.Module): The language model. + tokenizer (PreTrainedTokenizer): The tokenizer. + prompt (Union[str, List[int]]): The input prompt string or integer tokens. + verbose (bool): If ``True``, print tokens and timing information. + Default: ``False``. + kwargs: The remaining options get passed to :func:`stream_generate`. + See :func:`stream_generate` for more details. + """ +@dataclass +class BatchStats: + """ + An data object to hold generation stats. + + Args: + prompt_tokens (int): The number of prompt tokens processed. + prompt_tps (float): The prompt processing tokens-per-second. + prompt_time (float): The time in seconds spent in prompt processing. + generation_tokens (int): The number of generated tokens. + generation_tps (float): The tokens-per-second for generation. + generation_time (float): The time in seconds spent in generation . + peak_memory (float): The peak memory used so far in GB. + """ + + prompt_tokens: int = ... + prompt_tps: float = ... + prompt_time: float = ... + generation_tokens: int = ... + generation_tps: float = ... + generation_time: float = ... + peak_memory: float = ... + +@dataclass +class BatchResponse: + """ + An data object to hold a batch generation response. + + Args: + texts: (List[str]): The generated text for each prompt. + stats (BatchStats): Statistics about the generation. + """ + + texts: List[str] + stats: BatchStats + +@dataclass +class Batch: + uids: List[int] + y: mx.array + logprobs: mx.array + max_tokens: List[int] + num_tokens: List[int] + cache: List[Any] + def __len__(self): # -> int: + ... + def filter(self, keep_idx: List[int]): # -> None: + ... + def extend(self, other): # -> None: + ... + +class BatchGenerator: + @dataclass + class Response: + uid: int + token: int + logprobs: mx.array + finish_reason: Optional[str] + + def __init__( + self, + model, + max_tokens: int = ..., + stop_tokens: Optional[set] = ..., + sampler: Optional[Callable[[mx.array], mx.array]] = ..., + completion_batch_size: int = ..., + prefill_batch_size: int = ..., + prefill_step_size: int = ..., + ) -> None: ... + def insert( + self, prompts, max_tokens: Union[List[int], int, None] = ... + ): # -> list[Any]: + ... + def stats(self): # -> BatchStats: + ... + def next(self): # -> list[Any]: + ... + +def batch_generate( + model, + tokenizer, + prompts: List[int], + max_tokens: Union[int, List[int]] = ..., + verbose: bool = ..., + **kwargs, +) -> BatchResponse: + """ + Generate responses for the given batch of prompts. + + Args: + model (nn.Module): The language model. + tokenizer (PreTrainedTokenizer): The tokenizer. + prompt (List[List[int]]): The input prompts. + verbose (bool): If ``True``, print tokens and timing information. + Default: ``False``. + max_tokens (Union[int, List[int]): Maximum number of output tokens. This + can be per prompt if a list is provided. + kwargs: The remaining options get passed to :obj:`BatchGenerator`. + See :obj:`BatchGenerator` for more details. + """ + +def main(): # -> None: + ... + +if __name__ == "__main__": ... diff --git a/.mlx_typings/mlx_lm/models/__init__.pyi b/.mlx_typings/mlx_lm/models/__init__.pyi new file mode 100644 index 00000000..e09bd4fc --- /dev/null +++ b/.mlx_typings/mlx_lm/models/__init__.pyi @@ -0,0 +1 @@ +import cache as cache diff --git a/.mlx_typings/mlx_lm/models/base.pyi b/.mlx_typings/mlx_lm/models/base.pyi new file mode 100644 index 00000000..e549e624 --- /dev/null +++ b/.mlx_typings/mlx_lm/models/base.pyi @@ -0,0 +1,47 @@ +""" +This type stub file was generated by pyright. +""" + +from dataclasses import dataclass +from typing import Optional + +import mlx.core as mx + +@dataclass +class BaseModelArgs: + @classmethod + def from_dict(cls, params): # -> Self: + ... + +def create_causal_mask( + N: int, + offset: int = ..., + window_size: Optional[int] = ..., + right_padding: Optional[mx.array] = ..., + left_padding: Optional[mx.array] = ..., +): # -> array: + ... +def create_attention_mask( + h, cache=..., window_size: Optional[int] = ..., return_array: bool = ... +): # -> array | Literal['causal'] | None: + ... +def create_ssm_mask(h, cache=...): # -> None: + ... +def quantized_scaled_dot_product_attention( + queries: mx.array, + q_keys: tuple[mx.array, mx.array, mx.array], + q_values: tuple[mx.array, mx.array, mx.array], + scale: float, + mask: Optional[mx.array], + group_size: int = ..., + bits: int = ..., +) -> mx.array: ... +def scaled_dot_product_attention( + queries, + keys, + values, + cache, + scale: float, + mask: Optional[mx.array], + sinks: Optional[mx.array] = ..., +) -> mx.array: ... diff --git a/.mlx_typings/mlx_lm/models/bitlinear_layers.pyi b/.mlx_typings/mlx_lm/models/bitlinear_layers.pyi new file mode 100644 index 00000000..fa1caa82 --- /dev/null +++ b/.mlx_typings/mlx_lm/models/bitlinear_layers.pyi @@ -0,0 +1,26 @@ +""" +This type stub file was generated by pyright. +""" + +import mlx.nn as nn + +def bitnet_quantize(model, quantization_config: dict): ... +def make_bitlinear_kernel(): + """ + Custom Metal kernel that performs matrix multiplication directly on + packed weights and scales the output. This eliminates the need to + store unpacked weights in memory. + """ + +_bitlinear_kernel = ... + +class BitLinear(nn.Module): + """ + BitLinear module with memory-efficient weight handling. + """ + def __init__( + self, in_features, out_features, bias=..., invert_weight_scales=... + ) -> None: ... + def execute_matmul_kernel(self, x, packed_weights): ... + def __call__(self, x): # -> array: + ... diff --git a/.mlx_typings/mlx_lm/models/cache.pyi b/.mlx_typings/mlx_lm/models/cache.pyi new file mode 100644 index 00000000..37f96845 --- /dev/null +++ b/.mlx_typings/mlx_lm/models/cache.pyi @@ -0,0 +1,357 @@ +""" +This type stub file was generated by pyright. +""" + +from typing import Any, Dict, List, Optional, Protocol, Literal, Self + +import mlx.nn as nn +from mlx.core import array +import mlx.core as mx + +class Cache(Protocol): + keys: mx.array + values: mx.array + def update_and_fetch(self, keys: mx.array, values: mx.array) -> None: ... + @property + def state(self) -> tuple[mx.array, mx.array]: ... + @state.setter + def state(self, v) -> None: ... + +def make_prompt_cache( + model: nn.Module, max_kv_size: Optional[int] = ... +) -> List[Cache | Any]: + """ + Construct the model's cache for use in generation. + + This function will defer the cache construction to the model if it has a + ``make_cache`` method, otherwise it will make a default KV cache. + + Args: + model (nn.Module): The language model. + max_kv_size (Optional[int]): If provided and the model does not have a + ``make_cache`` method, a ``RotatingKVCache`` is used with a maximum + size of ``max_kv_size`` + """ + +def save_prompt_cache( + file_name: str, cache: List[Cache], metadata: Dict[str, str] = ... +) -> None: + """ + Save a pre-computed prompt cache to a file. + + Args: + file_name (str): The ``.safetensors`` file name. + cache (List[Any]): The model state. + metadata (Dict[str, str]): Optional metadata to save along with model + state. + """ + +def load_prompt_cache(file_name: str, return_metadata=...) -> array: + """ + Load a prompt cache from a file. + + Args: + file_name (str): The ``.safetensors`` file name. + return_metadata (bool): Whether or not to return metadata. + Default: ``False``. + + Returns: + List[Any] or Tuple[List[Any], Dict[str, str]]: The prompt cache and + the metadata if requested. + """ + +def can_trim_prompt_cache(cache: List[Cache]) -> bool: + """ + Check if model's cache can be trimmed. + """ + +def trim_prompt_cache(cache: List[Cache], num_tokens: int) -> List[Cache]: + """ + Trim the model's cache by the given number of tokens. + + This function will trim the cache if possible (in-place) and return the + number of tokens that were trimmed. + + Args: + cache (List[Any]): The model's cache. + num_tokens (int): The number of tokens to trim. + + Returns: + (int): The number of tokens that were trimmed. + """ + +def create_attention_mask( + N: int, offset: int, return_array: bool, window_size: Optional[int] +) -> array | Literal["causal"] | None: ... + +class _BaseCache(Cache): + keys: mx.array + values: mx.array + @property + def state(self) -> tuple[mx.array, mx.array]: ... + @state.setter + def state(self, v) -> None: ... + @property + def meta_state(self) -> Literal[""]: ... + @meta_state.setter + def meta_state(self, v) -> None: ... + def is_trimmable(self) -> Literal[False]: ... + @classmethod + def from_state(cls, state, meta_state) -> Self: ... + +class ConcatenateKVCache(_BaseCache): + """ConcatenateKVCache the simplest KV cache implementation. + + Can be used as a mock KV cache or when large blocks are being processed at + a time in which case KVCache isn't necessarily faster. Consider using the + KVCache with a larger step size before using this cache. + """ + def __init__(self) -> None: ... + def update_and_fetch(self, keys, values): # -> tuple[Any | array, Any | array]: + ... + @property + def state(self): # -> tuple[Any | array | None, Any | array | None]: + ... + @state.setter + def state(self, v): # -> None: + ... + def is_trimmable(self): # -> Literal[True]: + ... + def trim(self, n): # -> int: + ... + def make_mask(self, *args, **kwargs): # -> array | Literal['causal'] | None: + ... + +class QuantizedKVCache(_BaseCache): + step = ... + def __init__(self, group_size: int = ..., bits: int = ...) -> None: ... + def update_and_fetch(self, keys, values): # -> Any: + ... + @property + def state( + self, + ): # -> tuple[Any | tuple[array, array, array] | None, Any | tuple[array, array, array] | None] | Any: + ... + @state.setter + def state(self, v): # -> None: + ... + @property + def meta_state(self): # -> tuple[str, ...]: + ... + @meta_state.setter + def meta_state(self, v): # -> None: + ... + def is_trimmable(self): # -> Literal[True]: + ... + def trim(self, n): # -> int: + ... + def make_mask(self, *args, **kwargs): # -> array | Literal['causal'] | None: + ... + +class KVCache(_BaseCache): + step = ... + def __init__(self) -> None: ... + def update_and_fetch(self, keys, values): # -> tuple[array | Any, array | Any]: + ... + @property + def state( + self, + ) -> tuple[array, array]: ... + @state.setter + def state(self, v) -> None: ... + def is_trimmable(self): # -> Literal[True]: + ... + def trim(self, n): # -> int: + ... + def to_quantized( + self, group_size: int = ..., bits: int = ... + ) -> QuantizedKVCache: ... + def make_mask(self, *args, **kwargs): # -> array | Literal['causal'] | None: + ... + +class RotatingKVCache(_BaseCache): + step = ... + def __init__(self, max_size, keep=...) -> None: ... + def update_and_fetch( + self, keys, values + ): # -> tuple[array | Any, array | Any] | tuple[array | Any, array | Any | None]: + ... + @property + def state( + self, + ): # -> tuple[Any | array, Any | array] | tuple[Any | array | None, Any | array | None]: + ... + @state.setter + def state(self, v): # -> None: + ... + @property + def meta_state(self): # -> tuple[str, ...]: + ... + @meta_state.setter + def meta_state(self, v): # -> None: + ... + def is_trimmable(self): # -> bool: + ... + def trim(self, n): # -> int: + ... + def to_quantized( + self, group_size: int = ..., bits: int = ... + ) -> QuantizedKVCache: ... + def make_mask( + self, N: int, window_size: Optional[int] = ..., return_array: bool = ... + ): # -> array | Literal['causal'] | None: + ... + +class ArraysCache(_BaseCache): + def __init__(self, size, left_padding: Optional[List[int]] = ...) -> None: ... + def __setitem__(self, idx, value): # -> None: + ... + def __getitem__(self, idx): ... + @property + def state(self): # -> list[Any | array] | list[array]: + ... + @state.setter + def state(self, v): # -> None: + ... + def filter(self, batch_indices): # -> None: + """ + In-place filter to keep just the given indices in the cache. + """ + + def extend(self, other): # -> None: + """ + In-place extend this cache with the other cache. + """ + + def make_mask(self, N: int): # -> array | None: + ... + +class MambaCache(ArraysCache): + def __init__(self, left_padding: Optional[List[int]] = ...) -> None: ... + +class ChunkedKVCache(KVCache): + def __init__(self, chunk_size) -> None: ... + def maybe_trim_front(self): # -> None: + ... + def update_and_fetch(self, keys, values): # -> tuple[array, array]: + ... + def trim(self, n): # -> int: + ... + @property + def meta_state(self): # -> tuple[str, ...]: + ... + @meta_state.setter + def meta_state(self, v): # -> None: + ... + +class CacheList(_BaseCache): + def __init__(self, *caches) -> None: ... + def __getitem__(self, idx): ... + def is_trimmable(self): # -> bool: + ... + def trim(self, n): ... + @property + def state(self): # -> list[Any]: + ... + @state.setter + def state(self, v): # -> None: + ... + def filter(self, batch_indices): # -> None: + """ + In-place filter to keep just the given indices in the cache. + """ + + def extend(self, other): # -> None: + """ + In-place extend this cache with the other cache. + """ + +class BatchKVCache(_BaseCache): + step = ... + def __init__(self, left_padding: List[int]) -> None: + """ + The BatchKV cache expects inputs to be left-padded. + + E.g. the following prompts: + + [1, 3, 5] + [7] + [2, 6, 8, 9] + + Should be padded like so: + + [0, 1, 3, 5] + [0, 0, 0, 7] + [2, 6, 8, 9] + + And ``left_padding`` specifies the amount of padding for each. + In this case, ``left_padding = [1, 3, 0]``. + """ + + def update_and_fetch(self, keys, values): # -> tuple[array | Any, array | Any]: + ... + @property + def state( + self, + ): # -> tuple[Any | array | None, Any | array | None, array | Any, array | Any]: + ... + @state.setter + def state(self, v): # -> None: + ... + def is_trimmable(self): # -> Literal[True]: + ... + def trim(self, n): # -> int | float: + ... + def make_mask(self, N: int, return_array: bool = ..., **kwargs): # -> array: + ... + def filter(self, batch_indices): # -> None: + """ + In-place filter to keep just the given indices in the cache. + """ + + def extend(self, other): # -> None: + """ + In-place extend this cache with the other cache. + """ + +class BatchRotatingKVCache(_BaseCache): + step = ... + def __init__(self, max_size, left_padding: List[int]) -> None: ... + def update_and_fetch( + self, keys, values + ): # -> tuple[array | Any, array | Any] | tuple[array | Any, array | Any | None]: + ... + @property + def state( + self, + ): # -> tuple[Any | array | None, Any | array | None, array | Any, array | Any]: + ... + @state.setter + def state(self, v): # -> None: + ... + @property + def meta_state(self): # -> tuple[str, ...]: + ... + @meta_state.setter + def meta_state(self, v): # -> None: + ... + def is_trimmable(self): # -> bool: + ... + def trim(self, n): # -> int: + ... + def to_quantized( + self, group_size: int = ..., bits: int = ... + ) -> QuantizedKVCache: ... + def make_mask( + self, N: int, window_size: Optional[int] = ..., return_array: bool = ... + ): # -> array: + ... + def filter(self, batch_indices): # -> None: + """ + In-place filter to keep just the given indices in the cache. + """ + + def extend(self, other): # -> None: + """ + In-place extend this cache with the other cache. + """ diff --git a/.mlx_typings/mlx_lm/models/switch_layers.pyi b/.mlx_typings/mlx_lm/models/switch_layers.pyi new file mode 100644 index 00000000..c50c999a --- /dev/null +++ b/.mlx_typings/mlx_lm/models/switch_layers.pyi @@ -0,0 +1,79 @@ +""" +This type stub file was generated by pyright. +""" + +from functools import partial + +import mlx.core as mx +import mlx.nn as nn + +class QuantizedSwitchLinear(nn.Module): + def __init__( + self, + input_dims: int, + output_dims: int, + num_experts: int, + bias: bool = ..., + group_size: int = ..., + bits: int = ..., + mode: str = ..., + ) -> None: ... + @property + def input_dims(self): # -> int: + ... + @property + def output_dims(self): # -> int: + ... + @property + def num_experts(self): # -> int: + ... + def __call__(self, x, indices, sorted_indices=...): # -> array: + ... + +class SwitchLinear(nn.Module): + def __init__( + self, input_dims: int, output_dims: int, num_experts: int, bias: bool = ... + ) -> None: ... + @property + def input_dims(self): # -> int: + ... + @property + def output_dims(self): # -> int: + ... + @property + def num_experts(self): # -> int: + ... + def __call__(self, x, indices, sorted_indices=...): ... + def to_quantized( + self, group_size: int = ..., bits: int = ..., mode: str = ... + ): # -> QuantizedSwitchLinear: + ... + +@partial(mx.compile, shapeless=True) +def swiglu(x, gate): ... + +class SwiGLU(nn.Module): + def __init__(self) -> None: ... + def __call__(self, x, gate): ... + +class SwitchGLU(nn.Module): + def __init__( + self, + input_dims: int, + hidden_dims: int, + num_experts: int, + activation=..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x, indices) -> mx.array: ... + +class SwitchMLP(nn.Module): + def __init__( + self, + input_dims: int, + hidden_dims: int, + num_experts: int, + activation=..., + bias: bool = ..., + ) -> None: ... + def __call__(self, x, indices) -> mx.array: ... diff --git a/.mlx_typings/mlx_lm/sample_utils.pyi b/.mlx_typings/mlx_lm/sample_utils.pyi new file mode 100644 index 00000000..bc6955a7 --- /dev/null +++ b/.mlx_typings/mlx_lm/sample_utils.pyi @@ -0,0 +1,148 @@ +""" +This type stub file was generated by pyright. +""" + +from functools import partial +from typing import Callable, Dict, List, Optional + +import mlx.core as mx + +def make_sampler( + temp: float = ..., + top_p: float = ..., + min_p: float = ..., + min_tokens_to_keep: int = ..., + top_k: int = ..., + xtc_probability: float = ..., + xtc_threshold: float = ..., + xtc_special_tokens: List[int] = ..., +) -> Callable[[mx.array], mx.array]: + """ + Make a sampler function for use with ``generate_step``. + + Args: + temp (float): The temperature for sampling, if 0 the argmax is used. + Default: ``0``. + top_p (float, optional): Nulceus sampling, higher means model considers + more less likely words. + min_p (float, optional): The minimum value (scaled by the top token's + probability) that a token probability must have to be considered. + min_tokens_to_keep (int, optional): Minimum number of tokens that cannot + be filtered by min_p sampling. + top_k (int, optional): The top k tokens ranked by probability to constrain + the sampling to. + xtc_probability (float, optional): The probability of applying XTC + sampling. + xtc_threshold (float, optional): The threshold the probs need to reach + for being sampled. + xtc_special_tokens (list(int), optional): List of special tokens IDs to + be excluded from XTC sampling. + + + Returns: + Callable[mx.array, mx.array]: + A sampler which takes log-probabilities and returns tokens. + """ + +def make_logits_processors( + logit_bias: Optional[Dict[int, float]] = ..., + repetition_penalty: Optional[float] = ..., + repetition_context_size: Optional[int] = ..., +): # -> list[Any]: + """ + Make logits processors for use with ``generate_step``. + + Args: + repetition_penalty (float, optional): The penalty factor for repeating + tokens. + repetition_context_size (int, optional): The number of tokens to + consider for repetition penalty. Default: ``20``. + logit_bias (dictionary, optional): Additive logit bias. + + Returns: + List[Callable[[mx.array, mx.array], mx.array]]: + A list of logits processors. Each processor in the list is a + callable which takes an array of tokens and an array of logits + and returns the updated logits. + """ + +@partial(mx.compile, inputs=mx.random.state, outputs=mx.random.state) +def apply_top_k(logprobs: mx.array, top_k: int) -> mx.array: + """ + Sample from only the top K tokens ranked by probability. + + Args: + logprobs: A vector of log probabilities. + top_k (int): Top k tokens to sample from. + """ + +@partial(mx.compile, inputs=mx.random.state, outputs=mx.random.state) +def apply_min_p( + logprobs: mx.array, min_p: float, min_tokens_to_keep: int = ... +) -> mx.array: + """ + Apply min-p sampling to the logprobs. + + Min-p keeps all tokens that are above a minimum probability, scaled by the + probability of the most likely token. As a result, the filter is more + aggressive given a very high-probability token. + + Args: + logprobs: A vector of log probabilities. + min_p (float): Minimum token probability. Typical values are in the + 0.01-0.2 range, comparably selective as setting `top_p` in the + 0.99-0.8 range. + min_tokens_to_keep (int, optional): Minimum number of tokens that cannot + be filtered. Default: ``1``. + + """ + +@partial(mx.compile, inputs=mx.random.state, outputs=mx.random.state) +def apply_top_p(logprobs: mx.array, top_p: float) -> mx.array: + """ + Apply top-p (nucleus) sampling to logits. + + Args: + logprobs: A vector of log probabilities. + top_p: The cumulative probability threshold for top-p filtering. + Returns: + token selected based on the top-p criterion. + """ + +@partial(mx.compile, inputs=mx.random.state, outputs=mx.random.state) +def apply_xtc( + logits: mx.array, + xtc_probability: float, + xtc_threshold: float, + xtc_special_tokens: List[int], +) -> mx.array: + """ + Apply XTC sampling to the logits. + + Args: + logits: The logits from the model's output. + xtc_probability (float): Probability of XTC sampling to happen for each token + xtc_threshold (float): The threshold the probs need to reach for being sampled. + special_tokens_ids (list(int)): List of special tokens IDs to be excluded from XTC sampling. + """ + +@partial(mx.compile, inputs=mx.random.state, outputs=mx.random.state) +def categorical_sampling(logits, temp): # -> array: + ... +def make_repetition_penalty( + penalty: float, context_size: int = ... +): # -> Callable[..., Any]: + """ + Make repetition penalty processor. + + Paper: https://arxiv.org/abs/1909.05858 + + Args: + penalty (float): The repetition penalty factor to be applied. + context_size (int): The number of previous tokens to use. + Default: ``20``. + + Returns: + Callable[[mx.array, List[int]], mx.array]: + The repetition penalty processor. + """ diff --git a/.mlx_typings/mlx_lm/tokenizer_utils.pyi b/.mlx_typings/mlx_lm/tokenizer_utils.pyi new file mode 100644 index 00000000..a0a8355f --- /dev/null +++ b/.mlx_typings/mlx_lm/tokenizer_utils.pyi @@ -0,0 +1,168 @@ +""" +This type stub file was generated by pyright. +""" + +from functools import partial +from pathlib import Path + +from transformers import PreTrainedTokenizerFast + +class StreamingDetokenizer: + """The streaming detokenizer interface so that we can detokenize one token at a time. + + Example usage is as follows: + + detokenizer = ... + + # Reset the tokenizer state + detokenizer.reset() + + for token in generate(...): + detokenizer.add_token(token.item()) + + # Contains the whole text so far. Some tokens may not be included + # since it contains whole words usually. + detokenizer.text + + # Contains the printable segment (usually a word) since the last + # time it was accessed + detokenizer.last_segment + + # Contains all the tokens added so far + detokenizer.tokens + + # Make sure that we detokenize any remaining tokens + detokenizer.finalize() + + # Now detokenizer.text should match tokenizer.decode(detokenizer.tokens) + """ + + __slots__ = ... + def reset(self): ... + def add_token(self, token): ... + def finalize(self): ... + @property + def last_segment(self): + """Return the last segment of readable text since last time this property was accessed.""" + +class NaiveStreamingDetokenizer(StreamingDetokenizer): + """NaiveStreamingDetokenizer relies on the underlying tokenizer + implementation and should work with every tokenizer. + + Its complexity is O(T^2) where T is the longest line since it will + repeatedly detokenize the same tokens until a new line is generated. + """ + def __init__(self, tokenizer) -> None: ... + def reset(self): # -> None: + ... + def add_token(self, token): # -> None: + ... + def finalize(self): # -> None: + ... + @property + def text(self): # -> str: + ... + +class SPMStreamingDetokenizer(StreamingDetokenizer): + """A streaming detokenizer for SPM models. + + It adds tokens to the text if the next token starts with the special SPM + underscore which results in linear complexity. + """ + def __init__(self, tokenizer, trim_space=...) -> None: ... + def reset(self): # -> None: + ... + def add_token(self, token): # -> None: + ... + def finalize(self): # -> None: + ... + +class BPEStreamingDetokenizer(StreamingDetokenizer): + """A streaming detokenizer for OpenAI style BPE models. + + It adds tokens to the text if the next token starts with a space similar to + the SPM detokenizer. + """ + + _byte_decoder = ... + _space_matches = ... + def __init__(self, tokenizer) -> None: ... + def reset(self): # -> None: + ... + def add_token(self, token): # -> None: + ... + def finalize(self): # -> None: + ... + @classmethod + def make_byte_decoder(cls): # -> None: + """See https://github.com/openai/gpt-2/blob/master/src/encoder.py for the rationale.""" + +class TokenizerWrapper: + """A wrapper that combines an HF tokenizer and a detokenizer. + + Accessing any attribute other than the ``detokenizer`` is forwarded to the + huggingface tokenizer. + """ + def __init__(self, tokenizer, detokenizer_class=..., eos_token_ids=...) -> None: ... + def add_eos_token(self, token: str): # -> None: + ... + @property + def has_thinking(self): # -> bool: + ... + @property + def think_start(self): # -> str | None: + ... + @property + def think_end(self): # -> str | None: + ... + @property + def has_tool_calling(self): # -> bool: + ... + @property + def tool_call_start(self): # -> str | None: + ... + @property + def tool_call_end(self): # -> str | None: + ... + @property + def detokenizer(self): # -> NaiveStreamingDetokenizer: + """ + Get a stateful streaming detokenizer. + """ + + def __getattr__(self, attr): # -> set[Any] | Any: + ... + def __setattr__(self, attr, value): # -> None: + ... + +class NewlineTokenizer(PreTrainedTokenizerFast): + """A tokenizer that replaces newlines with and with new line.""" + def __init__(self, *args, **kwargs) -> None: ... + def encode(self, text, **kwargs): # -> list[int]: + ... + def encode_batch(self, texts, **kwargs): ... + def decode(self, *args, **kwargs): # -> str: + ... + def batch_decode(self, *args, **kwargs): # -> list[str]: + ... + +def load_tokenizer( + model_path: Path, + tokenizer_config_extra=..., + return_tokenizer=..., + eos_token_ids=..., +) -> ( + TokenizerWrapper + | type[SPMStreamingDetokenizer] + | partial[SPMStreamingDetokenizer] + | type[BPEStreamingDetokenizer] + | type[NaiveStreamingDetokenizer] +): + """Load a huggingface tokenizer and try to infer the type of streaming + detokenizer to use. + + Note, to use a fast streaming tokenizer, pass a local file path rather than + a Hugging Face repo ID. + """ + +def no_bos_or_eos(sequence: list, bos: int, eos: int) -> list: ... diff --git a/.mlx_typings/mlx_lm/utils.pyi b/.mlx_typings/mlx_lm/utils.pyi new file mode 100644 index 00000000..99b207d1 --- /dev/null +++ b/.mlx_typings/mlx_lm/utils.pyi @@ -0,0 +1,195 @@ +""" +This type stub file was generated by pyright. +""" + +import os +from pathlib import Path +from typing import Any, Callable, Dict, Optional, Tuple, Type, Union + +import mlx.nn as nn +from transformers.utils.auto_docstring import ModelArgs + +from .tokenizer_utils import TokenizerWrapper + +if os.getenv("MLXLM_USE_MODELSCOPE", "False").lower() == "true": ... +else: ... +MODEL_REMAPPING = ... +MAX_FILE_SIZE_GB = ... + +def compute_bits_per_weight(model): ... +def hf_repo_to_path(hf_repo): # -> Path: + ... +def load_config(model_path: Path) -> dict: ... +def load_model( + model_path: Path, + lazy: bool = False, + strict: bool = True, + model_config: dict[str, Any] = {}, + get_model_classes: Callable[ + [dict[str, Any]], Tuple[Type[nn.Module], Type[ModelArgs]] + ] = ..., +) -> Tuple[nn.Module, dict[str, Any]]: + """ + Load and initialize the model from a given path. + + Args: + model_path (Path): The path to load the model from. + lazy (bool): If False eval the model parameters to make sure they are + loaded in memory before returning, otherwise they will be loaded + when needed. Default: ``False`` + strict (bool): Whether or not to raise an exception if weights don't + match. Default: ``True`` + model_config (dict, optional): Optional configuration parameters for the + model. Defaults to an empty dictionary. + get_model_classes (Callable[[dict], Tuple[Type[nn.Module], Type]], optional): + A function that returns the model class and model args class given a config. + Defaults to the ``_get_classes`` function. + + Returns: + Tuple[nn.Module, dict[str, Any]]: The loaded and initialized model and config. + + Raises: + FileNotFoundError: If the weight files (.safetensors) are not found. + ValueError: If the model class or args class are not found or cannot be instantiated. + """ + +def load( + path_or_hf_repo: str, + tokenizer_config=..., + model_config=..., + adapter_path: Optional[str] = ..., + lazy: bool = ..., + return_config: bool = ..., + revision: str = ..., +) -> Union[ + Tuple[nn.Module, TokenizerWrapper], + Tuple[nn.Module, TokenizerWrapper, Dict[str, Any]], +]: + """ + Load the model and tokenizer from a given path or a huggingface repository. + + Args: + path_or_hf_repo (Path): The path or the huggingface repository to load the model from. + tokenizer_config (dict, optional): Configuration parameters specifically for the tokenizer. + Defaults to an empty dictionary. + model_config(dict, optional): Configuration parameters specifically for the model. + Defaults to an empty dictionary. + adapter_path (str, optional): Path to the LoRA adapters. If provided, applies LoRA layers + to the model. Default: ``None``. + lazy (bool): If ``False`` eval the model parameters to make sure they are + loaded in memory before returning, otherwise they will be loaded + when needed. Default: ``False`` + return_config (bool: If ``True`` return the model config as the last item.. + revision (str, optional): A revision id which can be a branch name, a tag, or a commit hash. + Returns: + Union[Tuple[nn.Module, TokenizerWrapper], Tuple[nn.Module, TokenizerWrapper, Dict[str, Any]]]: + A tuple containing the loaded model, tokenizer and, if requested, the model config. + + Raises: + FileNotFoundError: If config file or safetensors are not found. + ValueError: If model class or args class are not found. + """ + +def make_shards(weights: dict, max_file_size_gb: int = ...) -> list: + """ + Splits the weights into smaller shards. + + Args: + weights (dict): Model weights. + max_file_size_gb (int): Maximum size of each shard in gigabytes. + + Returns: + list: List of weight shards. + """ + +def create_model_card( + path: Union[str, Path], hf_path: Union[str, Path, None] +): # -> None: + """ + Uploads the model to Hugging Face hub. + + Args: + path (Union[str, Path]): Local path to the model. + hf_path (Union[str, Path, None]): Path to the original Hugging Face model. + """ + +def upload_to_hub(path: str, upload_repo: str): # -> None: + """ + Uploads the model to Hugging Face hub. + + Args: + path (str): Local path to the model. + upload_repo (str): Name of the HF repo to upload to. + """ + +def save_model( + save_path: Union[str, Path], model: nn.Module, *, donate_model: bool = ... +) -> None: + """Save model weights and metadata index into specified directory.""" + +def quantize_model( + model: nn.Module, + config: dict, + group_size: int, + bits: int, + mode: str = ..., + quant_predicate: Optional[Callable[[str, nn.Module], Union[bool, dict]]] = ..., +) -> Tuple[nn.Module, dict]: + """ + Applies quantization to the model weights. + + Args: + model (nn.Module): The model to be quantized. + config (dict): Model configuration. + group_size (int): Group size for quantization. + bits (int): Bits per weight for quantization. + mode (str): The quantization mode. + quant_predicate (Callable): A callable that decides how to quantize + each layer based on the path. Accepts the layer `path` and the + `module`. Returns either a bool to signify quantize/no quantize or + a dict of quantization parameters to pass to `to_quantized`. + + Returns: + Tuple: Tuple containing quantized model and config. + """ + +def save_config(config: dict, config_path: Union[str, Path]) -> None: + """Save the model configuration to the ``config_path``. + + The final configuration will be sorted before saving for better readability. + + Args: + config (dict): The model configuration. + config_path (Union[str, Path]): Model configuration file path. + """ + +def save( + dst_path: Union[str, Path], + src_path_or_repo: Union[str, Path], + model: nn.Module, + tokenizer: TokenizerWrapper, + config: Dict[str, Any], + donate_model: bool = ..., +): # -> None: + ... +def common_prefix_len(list1, list2): # -> int: + """ + Calculates the length of the common prefix of two lists. + + Args: + list1: The first list of strings. + list2: The second list of strings. + + Returns: + The length of the common prefix. Returns 0 if lists are empty + or do not match at the first element. + """ + +def does_model_support_input_embeddings(model: nn.Module) -> bool: + """ + Check if the model supports input_embeddings in its call signature. + Args: + model (nn.Module): The model to check. + Returns: + bool: True if the model supports input_embeddings, False otherwise. + """ diff --git a/.python-version b/.python-version new file mode 100644 index 00000000..24ee5b1b --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.13 diff --git a/.vscode/extensions.json b/.vscode/extensions.json new file mode 100644 index 00000000..3dfc2a75 --- /dev/null +++ b/.vscode/extensions.json @@ -0,0 +1,11 @@ +{ + "recommendations": [ + "detachhead.basedpyright", + "ms-python.python" + ], + "unwantedRecommendations": [ + "ms-python.vscode-pylance", + "ms-python.pyright", + "ms-python.mypy-type-checker" + ] +} \ No newline at end of file diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 00000000..31682d35 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,3 @@ +{ + "basedpyright.importStrategy": "fromEnvironment" +} \ No newline at end of file diff --git a/.zed/settings.json b/.zed/settings.json new file mode 100644 index 00000000..f885d7e7 --- /dev/null +++ b/.zed/settings.json @@ -0,0 +1,29 @@ +// Folder-specific settings +// +// For a full list of overridable settings, and general information on folder-specific settings, +// see the documentation: https://zed.dev/docs/configuring-zed#settings-files +{ + "lsp": { + "nix_python": { + "binary": { + "path": "nix", + "arguments": [ + "run", + "--quiet", + "--no-warn-dirty", + "--no-allow-import-from-derivation", + "--print-build-logs", + "never", + "${projectRoot}#python-lsp", + "--", + "--stdio" + ] + } + } + }, + "languages": { + "Python": { + "language_servers": ["nix_python"] + } + } +} diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 00000000..c54f01d1 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,5597 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aead" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d122413f284cf2d62fb1b7db97e02edb8cda96d769b16e443a4f6195e35662b0" +dependencies = [ + "crypto-common", + "generic-array", +] + +[[package]] +name = "aes" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0" +dependencies = [ + "cfg-if", + "cipher", + "cpufeatures", +] + +[[package]] +name = "aes-gcm" +version = "0.10.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "831010a0f742e1209b3bcea8fab6a8e149051ba6099432c8cb2cc117dec3ead1" +dependencies = [ + "aead", + "aes", + "cipher", + "ctr", + "ghash", + "subtle", +] + +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "once_cell", + "version_check", + "zerocopy", +] + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "allocator-api2" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" + +[[package]] +name = "android_system_properties" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311" +dependencies = [ + "libc", +] + +[[package]] +name = "anstream" +version = "0.6.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43d5b281e737544384e969a5ccad3f1cdd24b48086a0fc1b2a5262a26b8f4f4a" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5192cca8006f1fd4f7237516f40fa183bb07f8fbdfedaa0036de5ea9b0b45e78" + +[[package]] +name = "anstyle-parse" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7644824f0aa2c7b9384579234ef10eb7efb6a0deb83f9630a49594dd9c15c2" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + +[[package]] +name = "anyhow" +version = "1.0.100" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a23eb6b1614318a8071c9b2521f36b424b2c83db5eb3a0fead4a6c0809af6e61" + +[[package]] +name = "arc-swap" +version = "1.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69f7f8c3906b62b754cd5326047894316021dcfe5a194c8ea52bdd94934a3457" + +[[package]] +name = "arrayref" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb" + +[[package]] +name = "arrayvec" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50" + +[[package]] +name = "asn1-rs" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56624a96882bb8c26d61312ae18cb45868e5a9992ea73c58e45c3101e56a1e60" +dependencies = [ + "asn1-rs-derive", + "asn1-rs-impl", + "displaydoc", + "nom", + "num-traits", + "rusticata-macros", + "thiserror 2.0.17", + "time", +] + +[[package]] +name = "asn1-rs-derive" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", + "synstructure", +] + +[[package]] +name = "asn1-rs-impl" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "asn1_der" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "155a5a185e42c6b77ac7b88a15143d930a9e9727a5b7b77eed417404ab15c247" + +[[package]] +name = "async-channel" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +dependencies = [ + "concurrent-queue", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-io" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc" +dependencies = [ + "autocfg", + "cfg-if", + "concurrent-queue", + "futures-io", + "futures-lite", + "parking", + "polling", + "rustix", + "slab", + "windows-sys 0.61.2", +] + +[[package]] +name = "async-trait" +version = "0.1.89" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "asynchronous-codec" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a860072022177f903e59730004fb5dc13db9275b79bb2aef7ba8ce831956c233" +dependencies = [ + "bytes", + "futures-sink", + "futures-util", + "memchr", + "pin-project-lite", +] + +[[package]] +name = "atomic-waker" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" + +[[package]] +name = "attohttpc" +version = "0.30.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "16e2cdb6d5ed835199484bb92bb8b3edd526effe995c61732580439c1a67e2e9" +dependencies = [ + "base64", + "http", + "log", + "url", +] + +[[package]] +name = "autocfg" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" + +[[package]] +name = "base-x" +version = "0.2.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4cbbc9d0964165b47557570cce6c952866c2678457aca742aafc9fb771d30270" + +[[package]] +name = "base16ct" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4c7f02d4ea65f2c1853089ffd8d2787bdbc63de2f0d29dedbcf8ccdfa0ccd4cf" + +[[package]] +name = "base256emoji" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5e9430d9a245a77c92176e649af6e275f20839a48389859d1661e9a128d077c" +dependencies = [ + "const-str", + "match-lookup", +] + +[[package]] +name = "base64" +version = "0.22.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" + +[[package]] +name = "base64ct" +version = "1.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55248b47b0caf0546f7988906588779981c43bb1bc9d0c44087278f80cdb44ba" + +[[package]] +name = "bigdecimal" +version = "0.4.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "560f42649de9fa436b73517378a147ec21f6c997a546581df4b4b31677828934" +dependencies = [ + "autocfg", + "libm", + "num-bigint", + "num-integer", + "num-traits", +] + +[[package]] +name = "bimap" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "230c5f1ca6a325a32553f8640d31ac9b49f2411e901e427570154868b46da4f7" + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + +[[package]] +name = "bitflags" +version = "2.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "812e12b5285cc515a9c72a5c1d3b6d46a19dac5acfef5265968c166106e31dd3" + +[[package]] +name = "blake2" +version = "0.10.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "46502ad458c9a52b69d4d4d32775c788b7a1b85e8bc9d482d92250fc0e3f8efe" +dependencies = [ + "digest", +] + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "bon" +version = "3.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebeb9aaf9329dff6ceb65c689ca3db33dbf15f324909c60e4e5eef5701ce31b1" +dependencies = [ + "bon-macros", + "rustversion", +] + +[[package]] +name = "bon-macros" +version = "3.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77e9d642a7e3a318e37c2c9427b5a6a48aa1ad55dcd986f3034ab2239045a645" +dependencies = [ + "darling", + "ident_case", + "prettyplease", + "proc-macro2", + "quote", + "rustversion", + "syn 2.0.111", +] + +[[package]] +name = "bs58" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf88ba1141d185c399bee5288d850d63b8369520c1eafc32a0430b5b6c287bf4" +dependencies = [ + "tinyvec", +] + +[[package]] +name = "bumpalo" +version = "3.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "46c5e41b57b8bba42a04676d81cb89e9ee8e859a1a66f80a5a72e1cb76b34d43" + +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + +[[package]] +name = "bytes" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b35204fbdc0b3f4446b89fc1ac2cf84a8a68971995d0bf2e925ec7cd960f9cb3" +dependencies = [ + "serde", +] + +[[package]] +name = "cbor4ii" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "472931dd4dfcc785075b09be910147f9c6258883fc4591d0dac6116392b2daa6" +dependencies = [ + "serde", +] + +[[package]] +name = "cc" +version = "1.2.48" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c481bdbf0ed3b892f6f806287d72acd515b352a4ec27a208489b8c1bc839633a" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" + +[[package]] +name = "chacha20" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818" +dependencies = [ + "cfg-if", + "cipher", + "cpufeatures", +] + +[[package]] +name = "chacha20poly1305" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35" +dependencies = [ + "aead", + "chacha20", + "cipher", + "poly1305", + "zeroize", +] + +[[package]] +name = "chrono" +version = "0.4.42" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "145052bdd345b87320e369255277e3fb5152762ad123a901ef5c262dd38fe8d2" +dependencies = [ + "iana-time-zone", + "js-sys", + "num-traits", + "wasm-bindgen", + "windows-link", +] + +[[package]] +name = "cipher" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad" +dependencies = [ + "crypto-common", + "inout", + "zeroize", +] + +[[package]] +name = "clap" +version = "4.5.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9e340e012a1bf4935f5282ed1436d1489548e8f72308207ea5df0e23d2d03f8" +dependencies = [ + "clap_builder", +] + +[[package]] +name = "clap_builder" +version = "4.5.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d76b5d13eaa18c901fd2f7fca939fefe3a0727a953561fefdf3b2922b8569d00" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_lex" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1d728cc89cf3aee9ff92b05e62b19ee65a02b5702cff7d5a377e32c6ae29d8d" + +[[package]] +name = "colorchoice" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b05b61dc5112cbb17e4b6cd61790d9845d13888356391624cbe7e41efeac1e75" + +[[package]] +name = "concurrent-queue" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "const-oid" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" + +[[package]] +name = "const-str" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2f421161cb492475f1661ddc9815a745a1c894592070661180fdec3d4872e9c3" + +[[package]] +name = "convert_case" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "633458d4ef8c78b72454de2d54fd6ab2e60f9e02be22f3c6104cdc8a4e0fceb9" +dependencies = [ + "unicode-segmentation", +] + +[[package]] +name = "core-foundation" +version = "0.9.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "core-foundation-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" + +[[package]] +name = "core2" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b49ba7ef1ad6107f8824dbe97de947cbaac53c44e7f9756a1fba0d37c1eec505" +dependencies = [ + "memchr", +] + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "critical-section" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "790eea4361631c5e7d22598ecd5723ff611904e3344ce8720784c93e3d83d40b" + +[[package]] +name = "crossbeam-channel" +version = "0.5.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-deque" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9dd111b7b7f7d55b72c0a6ae361660ee5853c9af73f70c3c2ef6858b950e2e51" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28" + +[[package]] +name = "crunchy" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5" + +[[package]] +name = "crypto-bigint" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0dc92fb57ca44df6db8059111ab3af99a63d5d0f8375d9972e319a379c6bab76" +dependencies = [ + "generic-array", + "rand_core 0.6.4", + "subtle", + "zeroize", +] + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "rand_core 0.6.4", + "typenum", +] + +[[package]] +name = "ctr" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0369ee1ad671834580515889b80f2ea915f23b8be8d0daa4bbaf2ac5c7590835" +dependencies = [ + "cipher", +] + +[[package]] +name = "cuckoofilter" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b810a8449931679f64cd7eef1bbd0fa315801b6d5d9cdc1ace2804d6529eee18" +dependencies = [ + "byteorder", + "fnv", + "rand 0.7.3", +] + +[[package]] +name = "curve25519-dalek" +version = "4.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" +dependencies = [ + "cfg-if", + "cpufeatures", + "curve25519-dalek-derive", + "digest", + "fiat-crypto", + "rustc_version", + "subtle", + "zeroize", +] + +[[package]] +name = "curve25519-dalek-derive" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "darling" +version = "0.21.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9cdf337090841a411e2a7f3deb9187445851f91b309c0c0a29e05f74a00a48c0" +dependencies = [ + "darling_core", + "darling_macro", +] + +[[package]] +name = "darling_core" +version = "0.21.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1247195ecd7e3c85f83c8d2a366e4210d588e802133e1e355180a9870b517ea4" +dependencies = [ + "fnv", + "ident_case", + "proc-macro2", + "quote", + "strsim", + "syn 2.0.111", +] + +[[package]] +name = "darling_macro" +version = "0.21.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d38308df82d1080de0afee5d069fa14b0326a88c14f15c5ccda35b4a6c414c81" +dependencies = [ + "darling_core", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "data-encoding" +version = "2.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a2330da5de22e8a3cb63252ce2abb30116bf5265e89c0e01bc17015ce30a476" + +[[package]] +name = "data-encoding-macro" +version = "0.1.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47ce6c96ea0102f01122a185683611bd5ac8d99e62bc59dd12e6bda344ee673d" +dependencies = [ + "data-encoding", + "data-encoding-macro-internal", +] + +[[package]] +name = "data-encoding-macro-internal" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8d162beedaa69905488a8da94f5ac3edb4dd4788b732fadb7bd120b2625c1976" +dependencies = [ + "data-encoding", + "syn 2.0.111", +] + +[[package]] +name = "delegate" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "780eb241654bf097afb00fc5f054a09b687dad862e485fdcf8399bb056565370" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "der" +version = "0.7.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" +dependencies = [ + "const-oid", + "pem-rfc7468", + "zeroize", +] + +[[package]] +name = "der-parser" +version = "10.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6" +dependencies = [ + "asn1-rs", + "displaydoc", + "nom", + "num-bigint", + "num-traits", + "rusticata-macros", +] + +[[package]] +name = "deranged" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ececcb659e7ba858fb4f10388c250a7252eb0a27373f1a72b8748afdd248e587" +dependencies = [ + "powerfmt", +] + +[[package]] +name = "derive_more" +version = "2.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10b768e943bed7bf2cab53df09f4bc34bfd217cdb57d971e769874c9a6710618" +dependencies = [ + "derive_more-impl", +] + +[[package]] +name = "derive_more-impl" +version = "2.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6d286bfdaf75e988b4a78e013ecd79c581e06399ab53fbacd2d916c2f904f30b" +dependencies = [ + "convert_case", + "proc-macro2", + "quote", + "rustc_version", + "syn 2.0.111", + "unicode-xid", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "const-oid", + "crypto-common", + "subtle", +] + +[[package]] +name = "displaydoc" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "dtoa" +version = "1.0.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6add3b8cff394282be81f3fc1a0605db594ed69890078ca6e2cab1c408bcf04" + +[[package]] +name = "ecdsa" +version = "0.16.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee27f32b5c5292967d2d4a9d7f1e0b0aed2c15daded5a60300e4abb9d8020bca" +dependencies = [ + "der", + "digest", + "elliptic-curve", + "rfc6979", + "signature", + "spki", +] + +[[package]] +name = "ed25519" +version = "2.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" +dependencies = [ + "pkcs8", + "signature", +] + +[[package]] +name = "ed25519-dalek" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" +dependencies = [ + "curve25519-dalek", + "ed25519", + "serde", + "sha2", + "subtle", + "zeroize", +] + +[[package]] +name = "either" +version = "1.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48c757948c5ede0e46177b7add2e67155f70e33c07fea8284df6576da70b3719" + +[[package]] +name = "elliptic-curve" +version = "0.13.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5e6043086bf7973472e0c7dff2142ea0b680d30e18d9cc40f267efbf222bd47" +dependencies = [ + "base16ct", + "crypto-bigint", + "digest", + "ff", + "generic-array", + "group", + "pem-rfc7468", + "pkcs8", + "rand_core 0.6.4", + "sec1", + "subtle", + "zeroize", +] + +[[package]] +name = "enum-as-inner" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1e6a265c649f3f5979b601d26f1d05ada116434c87741c9493cb56218f76cbc" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "env_filter" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bf3c259d255ca70051b30e2e95b5446cdb8949ac4cd22c0d7fd634d89f568e2" +dependencies = [ + "log", + "regex", +] + +[[package]] +name = "env_logger" +version = "0.11.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c863f0904021b108aa8b2f55046443e6b1ebde8fd4a15c399893aae4fa069f" +dependencies = [ + "anstream", + "anstyle", + "env_filter", + "jiff", + "log", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "event-listener" +version = "5.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e13b66accf52311f30a0db42147dadea9850cb48cd070028831ae5f5d4b856ab" +dependencies = [ + "concurrent-queue", + "parking", + "pin-project-lite", +] + +[[package]] +name = "event-listener-strategy" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" +dependencies = [ + "event-listener", + "pin-project-lite", +] + +[[package]] +name = "exo_pyo3_bindings" +version = "0.0.1" +dependencies = [ + "delegate", + "derive_more", + "env_logger", + "extend", + "futures", + "impl-trait-for-tuples", + "libp2p", + "log", + "networking", + "once_cell", + "pin-project", + "pyo3", + "pyo3-async-runtimes", + "pyo3-log", + "pyo3-stub-gen", + "thiserror 2.0.17", + "thread_local", + "tokio", + "util", +] + +[[package]] +name = "extend" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "311a6d2f1f9d60bff73d2c78a0af97ed27f79672f15c238192a5bbb64db56d00" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "ff" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0b50bfb653653f9ca9095b427bed08ab8d75a137839d9ad64eb11810d5b6393" +dependencies = [ + "rand_core 0.6.4", + "subtle", +] + +[[package]] +name = "fiat-crypto" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" + +[[package]] +name = "find-msvc-tools" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a3076410a55c90011c298b04d0cfa770b00fa04e1e3c97d3f6c9de105a03844" + +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + +[[package]] +name = "foldhash" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" + +[[package]] +name = "form_urlencoded" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf" +dependencies = [ + "percent-encoding", +] + +[[package]] +name = "futures" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "65bc07b1a8bc7c85c5f2e110c476c7389b4554ba72af57d8445ea63a576b0876" +dependencies = [ + "futures-channel", + "futures-core", + "futures-executor", + "futures-io", + "futures-sink", + "futures-task", + "futures-util", +] + +[[package]] +name = "futures-bounded" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91f328e7fb845fc832912fb6a34f40cf6d1888c92f974d1893a54e97b5ff542e" +dependencies = [ + "futures-timer", + "futures-util", +] + +[[package]] +name = "futures-channel" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10" +dependencies = [ + "futures-core", + "futures-sink", +] + +[[package]] +name = "futures-core" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e" + +[[package]] +name = "futures-executor" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e28d1d997f585e54aebc3f97d39e72338912123a67330d723fdbb564d646c9f" +dependencies = [ + "futures-core", + "futures-task", + "futures-util", + "num_cpus", +] + +[[package]] +name = "futures-io" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6" + +[[package]] +name = "futures-lite" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +dependencies = [ + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "futures-macro" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "futures-rustls" +version = "0.26.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a8f2f12607f92c69b12ed746fabf9ca4f5c482cba46679c1a75b874ed7c26adb" +dependencies = [ + "futures-io", + "rustls", + "rustls-pki-types", +] + +[[package]] +name = "futures-sink" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7" + +[[package]] +name = "futures-task" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988" + +[[package]] +name = "futures-timer" +version = "3.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f288b0a4f20f9a56b5d1da57e2227c661b7b16168e2f72365f57b63326e29b24" +dependencies = [ + "gloo-timers", + "send_wrapper 0.4.0", +] + +[[package]] +name = "futures-util" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81" +dependencies = [ + "futures-channel", + "futures-core", + "futures-io", + "futures-macro", + "futures-sink", + "futures-task", + "memchr", + "pin-project-lite", + "pin-utils", + "slab", +] + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", + "zeroize", +] + +[[package]] +name = "getopts" +version = "0.2.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfe4fbac503b8d1f88e6676011885f34b7174f46e59956bba534ba83abded4df" +dependencies = [ + "unicode-width", +] + +[[package]] +name = "getrandom" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fc3cb4d91f53b50155bdcfd23f6a4c39ae1969c2ae85982b135750cccaf5fce" +dependencies = [ + "cfg-if", + "libc", + "wasi 0.9.0+wasi-snapshot-preview1", +] + +[[package]] +name = "getrandom" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "335ff9f135e4384c8150d6f27c6daed433577f86b4750418338c01a1a2528592" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "wasi 0.11.1+wasi-snapshot-preview1", + "wasm-bindgen", +] + +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "r-efi", + "wasip2", + "wasm-bindgen", +] + +[[package]] +name = "ghash" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0d8a4362ccb29cb0b265253fb0a2728f592895ee6854fd9bc13f2ffda266ff1" +dependencies = [ + "opaque-debug", + "polyval", +] + +[[package]] +name = "gloo-timers" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b995a66bb87bebce9a0f4a95aed01daca4872c050bfcb21653361c03bc35e5c" +dependencies = [ + "futures-channel", + "futures-core", + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "group" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0f9ef7462f7c099f518d754361858f86d8a07af53ba9af0fe635bbccb151a63" +dependencies = [ + "ff", + "rand_core 0.6.4", + "subtle", +] + +[[package]] +name = "h2" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3c0b69cfcb4e1b9f1bf2f53f95f766e4661169728ec61cd3fe5a0166f2d1386" +dependencies = [ + "atomic-waker", + "bytes", + "fnv", + "futures-core", + "futures-sink", + "http", + "indexmap", + "slab", + "tokio", + "tokio-util", + "tracing", +] + +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +dependencies = [ + "ahash", +] + +[[package]] +name = "hashbrown" +version = "0.15.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" +dependencies = [ + "allocator-api2", + "equivalent", + "foldhash", +] + +[[package]] +name = "hashbrown" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" + +[[package]] +name = "hashlink" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af" +dependencies = [ + "hashbrown 0.14.5", +] + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "hermit-abi" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc0fef456e4baa96da950455cd02c081ca953b141298e41db3fc7e36b1da849c" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "hex_fmt" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b07f60793ff0a4d9cef0f18e63b5357e06209987153a64648c972c1e5aff336f" + +[[package]] +name = "hickory-proto" +version = "0.25.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8a6fe56c0038198998a6f217ca4e7ef3a5e51f46163bd6dd60b5c71ca6c6502" +dependencies = [ + "async-trait", + "cfg-if", + "data-encoding", + "enum-as-inner", + "futures-channel", + "futures-io", + "futures-util", + "idna", + "ipnet", + "once_cell", + "rand 0.9.2", + "ring", + "socket2 0.5.10", + "thiserror 2.0.17", + "tinyvec", + "tokio", + "tracing", + "url", +] + +[[package]] +name = "hickory-resolver" +version = "0.25.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc62a9a99b0bfb44d2ab95a7208ac952d31060efc16241c87eaf36406fecf87a" +dependencies = [ + "cfg-if", + "futures-util", + "hickory-proto", + "ipconfig", + "moka", + "once_cell", + "parking_lot", + "rand 0.9.2", + "resolv-conf", + "smallvec", + "thiserror 2.0.17", + "tokio", + "tracing", +] + +[[package]] +name = "hkdf" +version = "0.12.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b5f8eb2ad728638ea2c7d47a21db23b7b58a72ed6a38256b8a1849f15fbbdf7" +dependencies = [ + "hmac", +] + +[[package]] +name = "hmac" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e" +dependencies = [ + "digest", +] + +[[package]] +name = "http" +version = "1.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3ba2a386d7f85a81f119ad7498ebe444d2e22c2af0b86b069416ace48b3311a" +dependencies = [ + "bytes", + "itoa", +] + +[[package]] +name = "http-body" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184" +dependencies = [ + "bytes", + "http", +] + +[[package]] +name = "http-body-util" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a" +dependencies = [ + "bytes", + "futures-core", + "http", + "http-body", + "pin-project-lite", +] + +[[package]] +name = "httparse" +version = "1.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87" + +[[package]] +name = "hyper" +version = "1.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ab2d4f250c3d7b1c9fcdff1cece94ea4e2dfbec68614f7b87cb205f24ca9d11" +dependencies = [ + "atomic-waker", + "bytes", + "futures-channel", + "futures-core", + "h2", + "http", + "http-body", + "httparse", + "itoa", + "pin-project-lite", + "pin-utils", + "smallvec", + "tokio", + "want", +] + +[[package]] +name = "hyper-util" +version = "0.1.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "727805d60e7938b76b826a6ef209eb70eaa1812794f9424d4a4e2d740662df5f" +dependencies = [ + "bytes", + "futures-channel", + "futures-core", + "futures-util", + "http", + "http-body", + "hyper", + "libc", + "pin-project-lite", + "socket2 0.6.1", + "tokio", + "tower-service", + "tracing", +] + +[[package]] +name = "iana-time-zone" +version = "0.1.64" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "33e57f83510bb73707521ebaffa789ec8caf86f9657cad665b092b581d40e9fb" +dependencies = [ + "android_system_properties", + "core-foundation-sys", + "iana-time-zone-haiku", + "js-sys", + "log", + "wasm-bindgen", + "windows-core 0.62.2", +] + +[[package]] +name = "iana-time-zone-haiku" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f" +dependencies = [ + "cc", +] + +[[package]] +name = "icu_collections" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4c6b649701667bbe825c3b7e6388cb521c23d88644678e83c0c4d0a621a34b43" +dependencies = [ + "displaydoc", + "potential_utf", + "yoke", + "zerofrom", + "zerovec", +] + +[[package]] +name = "icu_locale_core" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "edba7861004dd3714265b4db54a3c390e880ab658fec5f7db895fae2046b5bb6" +dependencies = [ + "displaydoc", + "litemap", + "tinystr", + "writeable", + "zerovec", +] + +[[package]] +name = "icu_normalizer" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f6c8828b67bf8908d82127b2054ea1b4427ff0230ee9141c54251934ab1b599" +dependencies = [ + "icu_collections", + "icu_normalizer_data", + "icu_properties", + "icu_provider", + "smallvec", + "zerovec", +] + +[[package]] +name = "icu_normalizer_data" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7aedcccd01fc5fe81e6b489c15b247b8b0690feb23304303a9e560f37efc560a" + +[[package]] +name = "icu_properties" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e93fcd3157766c0c8da2f8cff6ce651a31f0810eaa1c51ec363ef790bbb5fb99" +dependencies = [ + "icu_collections", + "icu_locale_core", + "icu_properties_data", + "icu_provider", + "zerotrie", + "zerovec", +] + +[[package]] +name = "icu_properties_data" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02845b3647bb045f1100ecd6480ff52f34c35f82d9880e029d329c21d1054899" + +[[package]] +name = "icu_provider" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85962cf0ce02e1e0a629cc34e7ca3e373ce20dda4c4d7294bbd0bf1fdb59e614" +dependencies = [ + "displaydoc", + "icu_locale_core", + "writeable", + "yoke", + "zerofrom", + "zerotrie", + "zerovec", +] + +[[package]] +name = "ident_case" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39" + +[[package]] +name = "idna" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de" +dependencies = [ + "idna_adapter", + "smallvec", + "utf8_iter", +] + +[[package]] +name = "idna_adapter" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3acae9609540aa318d1bc588455225fb2085b9ed0c4f6bd0d9d5bcd86f1a0344" +dependencies = [ + "icu_normalizer", + "icu_properties", +] + +[[package]] +name = "if-addrs" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cabb0019d51a643781ff15c9c8a3e5dedc365c47211270f4e8f82812fedd8f0a" +dependencies = [ + "libc", + "windows-sys 0.48.0", +] + +[[package]] +name = "if-watch" +version = "3.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdf9d64cfcf380606e64f9a0bcf493616b65331199f984151a6fa11a7b3cde38" +dependencies = [ + "async-io", + "core-foundation", + "fnv", + "futures", + "if-addrs", + "ipnet", + "log", + "netlink-packet-core", + "netlink-packet-route", + "netlink-proto", + "netlink-sys", + "rtnetlink", + "system-configuration", + "tokio", + "windows 0.53.0", +] + +[[package]] +name = "igd-next" +version = "0.16.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "516893339c97f6011282d5825ac94fc1c7aad5cad26bdc2d0cee068c0bf97f97" +dependencies = [ + "async-trait", + "attohttpc", + "bytes", + "futures", + "http", + "http-body-util", + "hyper", + "hyper-util", + "log", + "rand 0.9.2", + "tokio", + "url", + "xmltree", +] + +[[package]] +name = "impl-trait-for-tuples" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0eb5a3343abf848c0984fe4604b2b105da9539376e24fc0a3b0007411ae4fd9" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "indexmap" +version = "2.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ad4bb2b565bca0645f4d68c5c9af97fba094e9791da685bf83cb5f3ce74acf2" +dependencies = [ + "equivalent", + "hashbrown 0.16.1", +] + +[[package]] +name = "indoc" +version = "2.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706" +dependencies = [ + "rustversion", +] + +[[package]] +name = "inout" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01" +dependencies = [ + "generic-array", +] + +[[package]] +name = "internment" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "636d4b0f6a39fd684effe2a73f5310df16a3fa7954c26d36833e98f44d1977a2" +dependencies = [ + "hashbrown 0.15.5", +] + +[[package]] +name = "inventory" +version = "0.3.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc61209c082fbeb19919bee74b176221b27223e27b65d781eb91af24eb1fb46e" +dependencies = [ + "rustversion", +] + +[[package]] +name = "ipconfig" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b58db92f96b720de98181bbbe63c831e87005ab460c1bf306eb2622b4707997f" +dependencies = [ + "socket2 0.5.10", + "widestring", + "windows-sys 0.48.0", + "winreg", +] + +[[package]] +name = "ipnet" +version = "2.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "469fb0b9cefa57e3ef31275ee7cacb78f2fdca44e4765491884a2b119d4eb130" + +[[package]] +name = "is-macro" +version = "0.3.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d57a3e447e24c22647738e4607f1df1e0ec6f72e16182c4cd199f647cdfb0e4" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "itertools" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1c173a5686ce8bfa551b3563d0c2170bf24ca44da99c7ca4bfdab5418c3fe57" +dependencies = [ + "either", +] + +[[package]] +name = "itertools" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2b192c782037fadd9cfa75548310488aabdbf3d2da73885b31bd0abd03351285" +dependencies = [ + "either", +] + +[[package]] +name = "itoa" +version = "1.0.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4a5f13b858c8d314ee3e8f639011f7ccefe71f97f96e50151fb991f267928e2c" + +[[package]] +name = "jiff" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49cce2b81f2098e7e3efc35bc2e0a6b7abec9d34128283d7a26fa8f32a6dbb35" +dependencies = [ + "jiff-static", + "log", + "portable-atomic", + "portable-atomic-util", + "serde_core", +] + +[[package]] +name = "jiff-static" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "980af8b43c3ad5d8d349ace167ec8170839f753a42d233ba19e08afe1850fa69" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "js-sys" +version = "0.3.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "464a3709c7f55f1f721e5389aa6ea4e3bc6aba669353300af094b29ffbdde1d8" +dependencies = [ + "once_cell", + "wasm-bindgen", +] + +[[package]] +name = "k256" +version = "0.13.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f6e3919bbaa2945715f0bb6d3934a173d1e9a59ac23767fbaaef277265a7411b" +dependencies = [ + "cfg-if", + "ecdsa", + "elliptic-curve", + "once_cell", + "sha2", + "signature", +] + +[[package]] +name = "keccak" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecc2af9a1119c51f12a14607e783cb977bde58bc069ff0c3da1095e635d70654" +dependencies = [ + "cpufeatures", +] + +[[package]] +name = "keccak-const" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57d8d8ce877200136358e0bbff3a77965875db3af755a11e1fa6b1b3e2df13ea" + +[[package]] +name = "lalrpop-util" +version = "0.20.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "507460a910eb7b32ee961886ff48539633b788a36b65692b95f225b844c82553" + +[[package]] +name = "lazy_static" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" + +[[package]] +name = "libc" +version = "0.2.178" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "37c93d8daa9d8a012fd8ab92f088405fb202ea0b6ab73ee2482ae66af4f42091" + +[[package]] +name = "libm" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f9fbbcab51052fe104eb5e5d351cf728d30a5be1fe14d9be8a3b097481fb97de" + +[[package]] +name = "libp2p" +version = "0.56.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce71348bf5838e46449ae240631117b487073d5f347c06d434caddcb91dceb5a" +dependencies = [ + "bytes", + "either", + "futures", + "futures-timer", + "getrandom 0.2.16", + "libp2p-allow-block-list", + "libp2p-autonat", + "libp2p-connection-limits", + "libp2p-core", + "libp2p-dcutr", + "libp2p-dns", + "libp2p-floodsub", + "libp2p-gossipsub", + "libp2p-identify", + "libp2p-identity", + "libp2p-kad", + "libp2p-mdns", + "libp2p-memory-connection-limits", + "libp2p-metrics", + "libp2p-noise", + "libp2p-ping", + "libp2p-plaintext", + "libp2p-pnet", + "libp2p-quic", + "libp2p-relay", + "libp2p-rendezvous", + "libp2p-request-response", + "libp2p-swarm", + "libp2p-tcp", + "libp2p-tls", + "libp2p-uds", + "libp2p-upnp", + "libp2p-webrtc-websys", + "libp2p-websocket", + "libp2p-websocket-websys", + "libp2p-webtransport-websys", + "libp2p-yamux", + "multiaddr", + "pin-project", + "rw-stream-sink", + "thiserror 2.0.17", +] + +[[package]] +name = "libp2p-allow-block-list" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d16ccf824ee859ca83df301e1c0205270206223fd4b1f2e512a693e1912a8f4a" +dependencies = [ + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", +] + +[[package]] +name = "libp2p-autonat" +version = "0.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fab5e25c49a7d48dac83d95d8f3bac0a290d8a5df717012f6e34ce9886396c0b" +dependencies = [ + "async-trait", + "asynchronous-codec", + "either", + "futures", + "futures-bounded", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-request-response", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "rand_core 0.6.4", + "thiserror 2.0.17", + "tracing", + "web-time", +] + +[[package]] +name = "libp2p-connection-limits" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a18b8b607cf3bfa2f8c57db9c7d8569a315d5cc0a282e6bfd5ebfc0a9840b2a0" +dependencies = [ + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", +] + +[[package]] +name = "libp2p-core" +version = "0.43.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4d28e2d2def7c344170f5c6450c0dbe3dfef655610dbfde2f6ac28a527abbe36" +dependencies = [ + "either", + "fnv", + "futures", + "futures-timer", + "libp2p-identity", + "multiaddr", + "multihash", + "multistream-select", + "parking_lot", + "pin-project", + "quick-protobuf", + "rand 0.8.5", + "rw-stream-sink", + "thiserror 2.0.17", + "tracing", + "unsigned-varint 0.8.0", + "web-time", +] + +[[package]] +name = "libp2p-dcutr" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f4f0eec23bc79cabfdf6934718f161fc42a1d98e2c9d44007c80eb91534200c" +dependencies = [ + "asynchronous-codec", + "either", + "futures", + "futures-bounded", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "lru", + "quick-protobuf", + "quick-protobuf-codec", + "thiserror 2.0.17", + "tracing", + "web-time", +] + +[[package]] +name = "libp2p-dns" +version = "0.44.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b770c1c8476736ca98c578cba4b505104ff8e842c2876b528925f9766379f9a" +dependencies = [ + "async-trait", + "futures", + "hickory-resolver", + "libp2p-core", + "libp2p-identity", + "parking_lot", + "smallvec", + "tracing", +] + +[[package]] +name = "libp2p-floodsub" +version = "0.47.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0914997f56315c83bc64ffb721cd4e764ad819370582db287232c5791469697" +dependencies = [ + "asynchronous-codec", + "bytes", + "cuckoofilter", + "fnv", + "futures", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "smallvec", + "thiserror 2.0.17", + "tracing", +] + +[[package]] +name = "libp2p-gossipsub" +version = "0.49.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7f58e37d8d6848e5c4c9e3c35c6f61133235bff2960c9c00a663b0849301221" +dependencies = [ + "async-channel", + "asynchronous-codec", + "base64", + "byteorder", + "bytes", + "either", + "fnv", + "futures", + "futures-timer", + "getrandom 0.2.16", + "hashlink", + "hex_fmt", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "regex", + "serde", + "sha2", + "tracing", + "web-time", +] + +[[package]] +name = "libp2p-identify" +version = "0.47.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ab792a8b68fdef443a62155b01970c81c3aadab5e659621b063ef252a8e65e8" +dependencies = [ + "asynchronous-codec", + "either", + "futures", + "futures-bounded", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "smallvec", + "thiserror 2.0.17", + "tracing", +] + +[[package]] +name = "libp2p-identity" +version = "0.2.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3104e13b51e4711ff5738caa1fb54467c8604c2e94d607e27745bcf709068774" +dependencies = [ + "asn1_der", + "bs58", + "ed25519-dalek", + "hkdf", + "k256", + "multihash", + "p256", + "quick-protobuf", + "rand 0.8.5", + "ring", + "sec1", + "serde", + "sha2", + "thiserror 2.0.17", + "tracing", + "zeroize", +] + +[[package]] +name = "libp2p-kad" +version = "0.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13d3fd632a5872ec804d37e7413ceea20588f69d027a0fa3c46f82574f4dee60" +dependencies = [ + "asynchronous-codec", + "bytes", + "either", + "fnv", + "futures", + "futures-bounded", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "serde", + "sha2", + "smallvec", + "thiserror 2.0.17", + "tracing", + "uint", + "web-time", +] + +[[package]] +name = "libp2p-mdns" +version = "0.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c66872d0f1ffcded2788683f76931be1c52e27f343edb93bc6d0bcd8887be443" +dependencies = [ + "futures", + "hickory-proto", + "if-watch", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "rand 0.8.5", + "smallvec", + "socket2 0.5.10", + "tokio", + "tracing", +] + +[[package]] +name = "libp2p-memory-connection-limits" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f9d052a767edd0235d5c29dacf46013955eabce1085781ce0d12a4fc66bf87cd" +dependencies = [ + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "memory-stats", + "sysinfo", + "tracing", +] + +[[package]] +name = "libp2p-metrics" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "805a555148522cb3414493a5153451910cb1a146c53ffbf4385708349baf62b7" +dependencies = [ + "futures", + "libp2p-core", + "libp2p-dcutr", + "libp2p-gossipsub", + "libp2p-identify", + "libp2p-identity", + "libp2p-kad", + "libp2p-ping", + "libp2p-relay", + "libp2p-swarm", + "pin-project", + "prometheus-client", + "web-time", +] + +[[package]] +name = "libp2p-noise" +version = "0.46.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc73eacbe6462a0eb92a6527cac6e63f02026e5407f8831bde8293f19217bfbf" +dependencies = [ + "asynchronous-codec", + "bytes", + "futures", + "libp2p-core", + "libp2p-identity", + "multiaddr", + "multihash", + "quick-protobuf", + "rand 0.8.5", + "snow", + "static_assertions", + "thiserror 2.0.17", + "tracing", + "x25519-dalek", + "zeroize", +] + +[[package]] +name = "libp2p-ping" +version = "0.47.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "74bb7fcdfd9fead4144a3859da0b49576f171a8c8c7c0bfc7c541921d25e60d3" +dependencies = [ + "futures", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "rand 0.8.5", + "tracing", + "web-time", +] + +[[package]] +name = "libp2p-plaintext" +version = "0.43.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e659439578fc6d305da8303834beb9d62f155f40e7f5b9d81c9f2b2c69d1926" +dependencies = [ + "asynchronous-codec", + "bytes", + "futures", + "libp2p-core", + "libp2p-identity", + "quick-protobuf", + "quick-protobuf-codec", + "tracing", +] + +[[package]] +name = "libp2p-pnet" +version = "0.26.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf240b834dfa3f8b48feb2c4b87bb2cf82751543001b6ee86077f48183b18d52" +dependencies = [ + "futures", + "pin-project", + "rand 0.8.5", + "salsa20", + "sha3", + "tracing", +] + +[[package]] +name = "libp2p-quic" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8dc448b2de9f4745784e3751fe8bc6c473d01b8317edd5ababcb0dec803d843f" +dependencies = [ + "futures", + "futures-timer", + "if-watch", + "libp2p-core", + "libp2p-identity", + "libp2p-tls", + "quinn", + "rand 0.8.5", + "ring", + "rustls", + "socket2 0.5.10", + "thiserror 2.0.17", + "tokio", + "tracing", +] + +[[package]] +name = "libp2p-relay" +version = "0.21.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "551b24ae04c63859bf5e25644acdd6aa469deb5c5cd872ca21c2c9b45a5a5192" +dependencies = [ + "asynchronous-codec", + "bytes", + "either", + "futures", + "futures-bounded", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "static_assertions", + "thiserror 2.0.17", + "tracing", + "web-time", +] + +[[package]] +name = "libp2p-rendezvous" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15285d828c2b4a34cb660c2e74cd6938116daceab1f4357bae933d5b08cca933" +dependencies = [ + "async-trait", + "asynchronous-codec", + "bimap", + "futures", + "futures-timer", + "libp2p-core", + "libp2p-identity", + "libp2p-request-response", + "libp2p-swarm", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "thiserror 2.0.17", + "tracing", + "web-time", +] + +[[package]] +name = "libp2p-request-response" +version = "0.29.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9f1cca83488b90102abac7b67d5c36fc65bc02ed47620228af7ed002e6a1478" +dependencies = [ + "async-trait", + "cbor4ii", + "futures", + "futures-bounded", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm", + "rand 0.8.5", + "serde", + "serde_json", + "smallvec", + "tracing", +] + +[[package]] +name = "libp2p-swarm" +version = "0.47.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6aa762e5215919a34e31c35d4b18bf2e18566ecab7f8a3d39535f4a3068f8b62" +dependencies = [ + "either", + "fnv", + "futures", + "futures-timer", + "getrandom 0.2.16", + "libp2p-core", + "libp2p-identity", + "libp2p-swarm-derive", + "lru", + "multistream-select", + "rand 0.8.5", + "smallvec", + "tokio", + "tracing", + "wasm-bindgen-futures", + "web-time", +] + +[[package]] +name = "libp2p-swarm-derive" +version = "0.35.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dd297cf53f0cb3dee4d2620bb319ae47ef27c702684309f682bdb7e55a18ae9c" +dependencies = [ + "heck", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "libp2p-tcp" +version = "0.44.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "65b4e030c52c46c8d01559b2b8ca9b7c4185f10576016853129ca1fe5cd1a644" +dependencies = [ + "futures", + "futures-timer", + "if-watch", + "libc", + "libp2p-core", + "socket2 0.5.10", + "tokio", + "tracing", +] + +[[package]] +name = "libp2p-tls" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96ff65a82e35375cbc31ebb99cacbbf28cb6c4fefe26bf13756ddcf708d40080" +dependencies = [ + "futures", + "futures-rustls", + "libp2p-core", + "libp2p-identity", + "rcgen", + "ring", + "rustls", + "rustls-webpki", + "thiserror 2.0.17", + "x509-parser", + "yasna", +] + +[[package]] +name = "libp2p-uds" +version = "0.43.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0413aa7a1cc51c409358186a46a198ad9195a782dae6b9a95ea3acf5db67569d" +dependencies = [ + "futures", + "libp2p-core", + "tracing", +] + +[[package]] +name = "libp2p-upnp" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4757e65fe69399c1a243bbb90ec1ae5a2114b907467bf09f3575e899815bb8d3" +dependencies = [ + "futures", + "futures-timer", + "igd-next", + "libp2p-core", + "libp2p-swarm", + "tokio", + "tracing", +] + +[[package]] +name = "libp2p-webrtc-utils" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "490abff5ee5f9a7a77f0145c79cc97c76941231a3626f4dee18ebf2abb95618f" +dependencies = [ + "asynchronous-codec", + "bytes", + "futures", + "hex", + "libp2p-core", + "libp2p-identity", + "libp2p-noise", + "quick-protobuf", + "quick-protobuf-codec", + "rand 0.8.5", + "serde", + "sha2", + "tinytemplate", + "tracing", +] + +[[package]] +name = "libp2p-webrtc-websys" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3830f0bf6f0f16ded2c735599fe70baea43a8c1a2d76152216693329217301dd" +dependencies = [ + "bytes", + "futures", + "getrandom 0.2.16", + "hex", + "js-sys", + "libp2p-core", + "libp2p-identity", + "libp2p-webrtc-utils", + "send_wrapper 0.6.0", + "thiserror 2.0.17", + "tracing", + "wasm-bindgen", + "wasm-bindgen-futures", + "web-sys", +] + +[[package]] +name = "libp2p-websocket" +version = "0.45.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "520e29066a48674c007bc11defe5dce49908c24cafd8fad2f5e1a6a8726ced53" +dependencies = [ + "either", + "futures", + "futures-rustls", + "libp2p-core", + "libp2p-identity", + "parking_lot", + "pin-project-lite", + "rw-stream-sink", + "soketto", + "thiserror 2.0.17", + "tracing", + "url", + "webpki-roots 0.26.11", +] + +[[package]] +name = "libp2p-websocket-websys" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e73d85b4dc8c2044f58508461bd8bb12f541217c0038ade8cce0ddc1607b8f72" +dependencies = [ + "bytes", + "futures", + "js-sys", + "libp2p-core", + "send_wrapper 0.6.0", + "thiserror 2.0.17", + "tracing", + "wasm-bindgen", + "web-sys", +] + +[[package]] +name = "libp2p-webtransport-websys" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34bc528d7fa278e1324a88978114a610deaa9e75c8e2230cd868321c512b3f43" +dependencies = [ + "futures", + "js-sys", + "libp2p-core", + "libp2p-identity", + "libp2p-noise", + "multiaddr", + "multihash", + "send_wrapper 0.6.0", + "thiserror 2.0.17", + "tracing", + "wasm-bindgen", + "wasm-bindgen-futures", + "web-sys", +] + +[[package]] +name = "libp2p-yamux" +version = "0.47.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f15df094914eb4af272acf9adaa9e287baa269943f32ea348ba29cfb9bfc60d8" +dependencies = [ + "either", + "futures", + "libp2p-core", + "thiserror 2.0.17", + "tracing", + "yamux 0.12.1", + "yamux 0.13.8", +] + +[[package]] +name = "linux-raw-sys" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df1d3c3b53da64cf5760482273a98e575c651a67eec7f77df96b5b642de8f039" + +[[package]] +name = "litemap" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77" + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "log" +version = "0.4.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897" + +[[package]] +name = "lru" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "234cf4f4a04dc1f57e24b96cc0cd600cf2af460d4161ac5ecdd0af8e1f3b2a38" +dependencies = [ + "hashbrown 0.15.5", +] + +[[package]] +name = "lru-slab" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" + +[[package]] +name = "maplit" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d" + +[[package]] +name = "match-lookup" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1265724d8cb29dbbc2b0f06fffb8bf1a8c0cf73a78eede9ba73a4a66c52a981e" +dependencies = [ + "proc-macro2", + "quote", + "syn 1.0.109", +] + +[[package]] +name = "matchers" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9" +dependencies = [ + "regex-automata", +] + +[[package]] +name = "matrixmultiply" +version = "0.3.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a06de3016e9fae57a36fd14dba131fccf49f74b40b7fbdb472f96e361ec71a08" +dependencies = [ + "autocfg", + "rawpointer", +] + +[[package]] +name = "memchr" +version = "2.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f52b00d39961fc5b2736ea853c9cc86238e165017a493d1d5c8eac6bdc4cc273" + +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + +[[package]] +name = "memory-stats" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c73f5c649995a115e1a0220b35e4df0a1294500477f97a91d0660fb5abeb574a" +dependencies = [ + "libc", + "windows-sys 0.52.0", +] + +[[package]] +name = "minimal-lexical" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" + +[[package]] +name = "mio" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a69bcab0ad47271a0234d9422b131806bf3968021e5dc9328caf2d4cd58557fc" +dependencies = [ + "libc", + "wasi 0.11.1+wasi-snapshot-preview1", + "windows-sys 0.61.2", +] + +[[package]] +name = "moka" +version = "0.12.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8261cd88c312e0004c1d51baad2980c66528dfdb2bee62003e643a4d8f86b077" +dependencies = [ + "crossbeam-channel", + "crossbeam-epoch", + "crossbeam-utils", + "equivalent", + "parking_lot", + "portable-atomic", + "rustc_version", + "smallvec", + "tagptr", + "uuid", +] + +[[package]] +name = "multiaddr" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fe6351f60b488e04c1d21bc69e56b89cb3f5e8f5d22557d6e8031bdfd79b6961" +dependencies = [ + "arrayref", + "byteorder", + "data-encoding", + "libp2p-identity", + "multibase", + "multihash", + "percent-encoding", + "serde", + "static_assertions", + "unsigned-varint 0.8.0", + "url", +] + +[[package]] +name = "multibase" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8694bb4835f452b0e3bb06dbebb1d6fc5385b6ca1caf2e55fd165c042390ec77" +dependencies = [ + "base-x", + "base256emoji", + "data-encoding", + "data-encoding-macro", +] + +[[package]] +name = "multihash" +version = "0.19.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6b430e7953c29dd6a09afc29ff0bb69c6e306329ee6794700aee27b76a1aea8d" +dependencies = [ + "core2", + "serde", + "unsigned-varint 0.8.0", +] + +[[package]] +name = "multistream-select" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea0df8e5eec2298a62b326ee4f0d7fe1a6b90a09dfcf9df37b38f947a8c42f19" +dependencies = [ + "bytes", + "futures", + "log", + "pin-project", + "smallvec", + "unsigned-varint 0.7.2", +] + +[[package]] +name = "ndarray" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c7c9125e8f6f10c9da3aad044cc918cf8784fa34de857b1aa68038eb05a50a9" +dependencies = [ + "matrixmultiply", + "num-complex", + "num-integer", + "num-traits", + "portable-atomic", + "portable-atomic-util", + "rawpointer", +] + +[[package]] +name = "netlink-packet-core" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72724faf704479d67b388da142b186f916188505e7e0b26719019c525882eda4" +dependencies = [ + "anyhow", + "byteorder", + "netlink-packet-utils", +] + +[[package]] +name = "netlink-packet-route" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053998cea5a306971f88580d0829e90f270f940befd7cf928da179d4187a5a66" +dependencies = [ + "anyhow", + "bitflags 1.3.2", + "byteorder", + "libc", + "netlink-packet-core", + "netlink-packet-utils", +] + +[[package]] +name = "netlink-packet-utils" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ede8a08c71ad5a95cdd0e4e52facd37190977039a4704eb82a283f713747d34" +dependencies = [ + "anyhow", + "byteorder", + "paste", + "thiserror 1.0.69", +] + +[[package]] +name = "netlink-proto" +version = "0.11.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72452e012c2f8d612410d89eea01e2d9b56205274abb35d53f60200b2ec41d60" +dependencies = [ + "bytes", + "futures", + "log", + "netlink-packet-core", + "netlink-sys", + "thiserror 2.0.17", +] + +[[package]] +name = "netlink-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "16c903aa70590cb93691bf97a767c8d1d6122d2cc9070433deb3bbf36ce8bd23" +dependencies = [ + "bytes", + "futures", + "libc", + "log", + "tokio", +] + +[[package]] +name = "networking" +version = "0.0.1" +dependencies = [ + "delegate", + "derive_more", + "either", + "extend", + "futures", + "futures-timer", + "impl-trait-for-tuples", + "keccak-const", + "libp2p", + "log", + "thiserror 2.0.17", + "tokio", + "tracing-subscriber", + "util", +] + +[[package]] +name = "nix" +version = "0.26.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "598beaf3cc6fdd9a5dfb1630c2800c7acd31df7aaf0f565796fba2b53ca1af1b" +dependencies = [ + "bitflags 1.3.2", + "cfg-if", + "libc", +] + +[[package]] +name = "nohash-hasher" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2bf50223579dc7cdcfb3bfcacf7069ff68243f8c363f62ffa99cf000a6b9c451" + +[[package]] +name = "nom" +version = "7.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a" +dependencies = [ + "memchr", + "minimal-lexical", +] + +[[package]] +name = "ntapi" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8a3895c6391c39d7fe7ebc444a87eb2991b2a0bc718fdabd071eec617fc68e4" +dependencies = [ + "winapi", +] + +[[package]] +name = "nu-ansi-term" +version = "0.50.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "num-bigint" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a5e44f723f1133c9deac646763579fdb3ac745e418f2a7af9cd0c431da1f20b9" +dependencies = [ + "num-integer", + "num-traits", +] + +[[package]] +name = "num-complex" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-conv" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "51d515d32fb182ee37cda2ccdcb92950d6a3c2893aa280e540671c2cd0f3b1d9" + +[[package]] +name = "num-integer" +version = "0.1.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-rational" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824" +dependencies = [ + "num-bigint", + "num-integer", + "num-traits", +] + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "num_cpus" +version = "1.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91df4bbde75afed763b708b7eee1e8e7651e02d97f6d5dd763e89367e957b23b" +dependencies = [ + "hermit-abi", + "libc", +] + +[[package]] +name = "numpy" +version = "0.27.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7aac2e6a6e4468ffa092ad43c39b81c79196c2bb773b8db4085f695efe3bba17" +dependencies = [ + "libc", + "ndarray", + "num-complex", + "num-integer", + "num-traits", + "pyo3", + "pyo3-build-config", + "rustc-hash 2.1.1", +] + +[[package]] +name = "oid-registry" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7" +dependencies = [ + "asn1-rs", +] + +[[package]] +name = "once_cell" +version = "1.21.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d" +dependencies = [ + "critical-section", + "portable-atomic", +] + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "opaque-debug" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381" + +[[package]] +name = "ordered-float" +version = "5.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f4779c6901a562440c3786d08192c6fbda7c1c2060edd10006b05ee35d10f2d" +dependencies = [ + "num-traits", +] + +[[package]] +name = "p256" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9863ad85fa8f4460f9c48cb909d38a0d689dba1f6f6988a5e3e0d31071bcd4b" +dependencies = [ + "ecdsa", + "elliptic-curve", + "primeorder", + "sha2", +] + +[[package]] +name = "parking" +version = "2.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" + +[[package]] +name = "parking_lot" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-link", +] + +[[package]] +name = "paste" +version = "1.0.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" + +[[package]] +name = "pem" +version = "3.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be" +dependencies = [ + "base64", + "serde_core", +] + +[[package]] +name = "pem-rfc7468" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412" +dependencies = [ + "base64ct", +] + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "phf" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078" +dependencies = [ + "phf_shared", +] + +[[package]] +name = "phf_codegen" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a" +dependencies = [ + "phf_generator", + "phf_shared", +] + +[[package]] +name = "phf_generator" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d" +dependencies = [ + "phf_shared", + "rand 0.8.5", +] + +[[package]] +name = "phf_shared" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5" +dependencies = [ + "siphasher", +] + +[[package]] +name = "pin-project" +version = "1.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "677f1add503faace112b9f1373e43e9e054bfdd22ff1a63c1bc485eaec6a6a8a" +dependencies = [ + "pin-project-internal", +] + +[[package]] +name = "pin-project-internal" +version = "1.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e918e4ff8c4549eb882f14b3a4bc8c8bc93de829416eacf579f1207a8fbf861" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "pin-project-lite" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b" + +[[package]] +name = "pin-utils" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" + +[[package]] +name = "pkcs8" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" +dependencies = [ + "der", + "spki", +] + +[[package]] +name = "polling" +version = "3.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218" +dependencies = [ + "cfg-if", + "concurrent-queue", + "hermit-abi", + "pin-project-lite", + "rustix", + "windows-sys 0.61.2", +] + +[[package]] +name = "poly1305" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf" +dependencies = [ + "cpufeatures", + "opaque-debug", + "universal-hash", +] + +[[package]] +name = "polyval" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d1fe60d06143b2430aa532c94cfe9e29783047f06c0d7fd359a9a51b729fa25" +dependencies = [ + "cfg-if", + "cpufeatures", + "opaque-debug", + "universal-hash", +] + +[[package]] +name = "portable-atomic" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483" + +[[package]] +name = "portable-atomic-util" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8a2f0d8d040d7848a709caf78912debcc3f33ee4b3cac47d73d1e1069e83507" +dependencies = [ + "portable-atomic", +] + +[[package]] +name = "potential_utf" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b73949432f5e2a09657003c25bca5e19a0e9c84f8058ca374f49e0ebe605af77" +dependencies = [ + "zerovec", +] + +[[package]] +name = "powerfmt" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "prettyplease" +version = "0.2.37" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b" +dependencies = [ + "proc-macro2", + "syn 2.0.111", +] + +[[package]] +name = "primeorder" +version = "0.13.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "353e1ca18966c16d9deb1c69278edbc5f194139612772bd9537af60ac231e1e6" +dependencies = [ + "elliptic-curve", +] + +[[package]] +name = "proc-macro2" +version = "1.0.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ee95bc4ef87b8d5ba32e8b7714ccc834865276eab0aed5c9958d00ec45f49e8" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "prometheus-client" +version = "0.23.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf41c1a7c32ed72abe5082fb19505b969095c12da9f5732a4bc9878757fd087c" +dependencies = [ + "dtoa", + "itoa", + "parking_lot", + "prometheus-client-derive-encode", +] + +[[package]] +name = "prometheus-client-derive-encode" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "440f724eba9f6996b75d63681b0a92b06947f1457076d503a4d2e2c8f56442b8" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "pyo3" +version = "0.27.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ab53c047fcd1a1d2a8820fe84f05d6be69e9526be40cb03b73f86b6b03e6d87d" +dependencies = [ + "bigdecimal", + "either", + "hashbrown 0.16.1", + "indexmap", + "indoc", + "inventory", + "libc", + "lock_api", + "memoffset", + "num-bigint", + "num-complex", + "num-rational", + "num-traits", + "once_cell", + "ordered-float", + "parking_lot", + "portable-atomic", + "pyo3-build-config", + "pyo3-ffi", + "pyo3-macros", + "rust_decimal", + "smallvec", + "unindent", +] + +[[package]] +name = "pyo3-async-runtimes" +version = "0.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57ddb5b570751e93cc6777e81fee8087e59cd53b5043292f2a6d59d5bd80fdfd" +dependencies = [ + "clap", + "futures", + "inventory", + "once_cell", + "pin-project-lite", + "pyo3", + "pyo3-async-runtimes-macros", + "tokio", +] + +[[package]] +name = "pyo3-async-runtimes-macros" +version = "0.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bcd7d70ee0ca1661c40407e6f84e4463ef2658c90a9e2fbbd4515b2bcdfcaeca" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "pyo3-build-config" +version = "0.27.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b455933107de8642b4487ed26d912c2d899dec6114884214a0b3bb3be9261ea6" +dependencies = [ + "target-lexicon", +] + +[[package]] +name = "pyo3-ffi" +version = "0.27.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c85c9cbfaddf651b1221594209aed57e9e5cff63c4d11d1feead529b872a089" +dependencies = [ + "libc", + "pyo3-build-config", +] + +[[package]] +name = "pyo3-log" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2f8bae9ad5ba08b0b0ed2bb9c2bdbaeccc69cafca96d78cf0fbcea0d45d122bb" +dependencies = [ + "arc-swap", + "log", + "pyo3", +] + +[[package]] +name = "pyo3-macros" +version = "0.27.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0a5b10c9bf9888125d917fb4d2ca2d25c8df94c7ab5a52e13313a07e050a3b02" +dependencies = [ + "proc-macro2", + "pyo3-macros-backend", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "pyo3-macros-backend" +version = "0.27.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03b51720d314836e53327f5871d4c0cfb4fb37cc2c4a11cc71907a86342c40f9" +dependencies = [ + "heck", + "proc-macro2", + "pyo3-build-config", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "pyo3-stub-gen" +version = "0.17.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "398b833826a83ca72c1e26d1b2c7c71f9ca7c3bfc74eacc663901895c362ae33" +dependencies = [ + "anyhow", + "chrono", + "either", + "indexmap", + "inventory", + "itertools 0.14.0", + "log", + "maplit", + "num-complex", + "numpy", + "ordered-float", + "pyo3", + "pyo3-stub-gen-derive", + "serde", + "toml", +] + +[[package]] +name = "pyo3-stub-gen-derive" +version = "0.17.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2426ba759d848787239d80f9fdb1f223786976f87fb6c3da8188ca7c17744b28" +dependencies = [ + "heck", + "indexmap", + "proc-macro2", + "quote", + "rustpython-parser", + "syn 2.0.111", +] + +[[package]] +name = "quick-protobuf" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d6da84cc204722a989e01ba2f6e1e276e190f22263d0cb6ce8526fcdb0d2e1f" +dependencies = [ + "byteorder", +] + +[[package]] +name = "quick-protobuf-codec" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15a0580ab32b169745d7a39db2ba969226ca16738931be152a3209b409de2474" +dependencies = [ + "asynchronous-codec", + "bytes", + "quick-protobuf", + "thiserror 1.0.69", + "unsigned-varint 0.8.0", +] + +[[package]] +name = "quinn" +version = "0.11.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20" +dependencies = [ + "bytes", + "cfg_aliases", + "futures-io", + "pin-project-lite", + "quinn-proto", + "quinn-udp", + "rustc-hash 2.1.1", + "rustls", + "socket2 0.6.1", + "thiserror 2.0.17", + "tokio", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-proto" +version = "0.11.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1906b49b0c3bc04b5fe5d86a77925ae6524a19b816ae38ce1e426255f1d8a31" +dependencies = [ + "bytes", + "getrandom 0.3.4", + "lru-slab", + "rand 0.9.2", + "ring", + "rustc-hash 2.1.1", + "rustls", + "rustls-pki-types", + "slab", + "thiserror 2.0.17", + "tinyvec", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-udp" +version = "0.5.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "addec6a0dcad8a8d96a771f815f0eaf55f9d1805756410b39f5fa81332574cbd" +dependencies = [ + "cfg_aliases", + "libc", + "once_cell", + "socket2 0.6.1", + "tracing", + "windows-sys 0.60.2", +] + +[[package]] +name = "quote" +version = "1.0.42" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a338cc41d27e6cc6dce6cefc13a0729dfbb81c262b1f519331575dd80ef3067f" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + +[[package]] +name = "rand" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a6b1679d49b24bbfe0c803429aa1874472f50d9b363131f0e89fc356b544d03" +dependencies = [ + "getrandom 0.1.16", + "libc", + "rand_chacha 0.2.2", + "rand_core 0.5.1", + "rand_hc", +] + +[[package]] +name = "rand" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404" +dependencies = [ + "libc", + "rand_chacha 0.3.1", + "rand_core 0.6.4", +] + +[[package]] +name = "rand" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1" +dependencies = [ + "rand_chacha 0.9.0", + "rand_core 0.9.3", +] + +[[package]] +name = "rand_chacha" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4c8ed856279c9737206bf725bf36935d8666ead7aa69b52be55af369d193402" +dependencies = [ + "ppv-lite86", + "rand_core 0.5.1", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core 0.6.4", +] + +[[package]] +name = "rand_chacha" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb" +dependencies = [ + "ppv-lite86", + "rand_core 0.9.3", +] + +[[package]] +name = "rand_core" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19" +dependencies = [ + "getrandom 0.1.16", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom 0.2.16", +] + +[[package]] +name = "rand_core" +version = "0.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "99d9a13982dcf210057a8a78572b2217b667c3beacbf3a0d8b454f6f82837d38" +dependencies = [ + "getrandom 0.3.4", +] + +[[package]] +name = "rand_hc" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c" +dependencies = [ + "rand_core 0.5.1", +] + +[[package]] +name = "rawpointer" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3" + +[[package]] +name = "rayon" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "368f01d005bf8fd9b1206fb6fa653e6c4a81ceb1466406b81792d87c5677a58f" +dependencies = [ + "either", + "rayon-core", +] + +[[package]] +name = "rayon-core" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" +dependencies = [ + "crossbeam-deque", + "crossbeam-utils", +] + +[[package]] +name = "rcgen" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75e669e5202259b5314d1ea5397316ad400819437857b90861765f24c4cf80a2" +dependencies = [ + "pem", + "ring", + "rustls-pki-types", + "time", + "yasna", +] + +[[package]] +name = "recursion" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9dba2197bf7b1d87b4dd460c195f4edeb45a94e82e8054f8d5f317c1f0e93ca1" + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags 2.10.0", +] + +[[package]] +name = "regex" +version = "1.12.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58" + +[[package]] +name = "resolv-conf" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e061d1b48cb8d38042de4ae0a7a6401009d6143dc80d2e2d6f31f0bdd6470c7" + +[[package]] +name = "rfc6979" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dd2a808d456c4a54e300a23e9f5a67e122c3024119acbfd73e3bf664491cb2" +dependencies = [ + "hmac", + "subtle", +] + +[[package]] +name = "ring" +version = "0.17.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7" +dependencies = [ + "cc", + "cfg-if", + "getrandom 0.2.16", + "libc", + "untrusted", + "windows-sys 0.52.0", +] + +[[package]] +name = "rtnetlink" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a552eb82d19f38c3beed3f786bd23aa434ceb9ac43ab44419ca6d67a7e186c0" +dependencies = [ + "futures", + "log", + "netlink-packet-core", + "netlink-packet-route", + "netlink-packet-utils", + "netlink-proto", + "netlink-sys", + "nix", + "thiserror 1.0.69", + "tokio", +] + +[[package]] +name = "rust_decimal" +version = "1.39.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35affe401787a9bd846712274d97654355d21b2a2c092a3139aabe31e9022282" +dependencies = [ + "arrayvec", + "num-traits", +] + +[[package]] +name = "rustc-hash" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2" + +[[package]] +name = "rustc-hash" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d" + +[[package]] +name = "rustc_version" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" +dependencies = [ + "semver", +] + +[[package]] +name = "rusticata-macros" +version = "4.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632" +dependencies = [ + "nom", +] + +[[package]] +name = "rustix" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd15f8a2c5551a84d56efdc1cd049089e409ac19a3072d5037a17fd70719ff3e" +dependencies = [ + "bitflags 2.10.0", + "errno", + "libc", + "linux-raw-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "rustls" +version = "0.23.35" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "533f54bc6a7d4f647e46ad909549eda97bf5afc1585190ef692b4286b198bd8f" +dependencies = [ + "once_cell", + "ring", + "rustls-pki-types", + "rustls-webpki", + "subtle", + "zeroize", +] + +[[package]] +name = "rustls-pki-types" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "708c0f9d5f54ba0272468c1d306a52c495b31fa155e91bc25371e6df7996908c" +dependencies = [ + "web-time", + "zeroize", +] + +[[package]] +name = "rustls-webpki" +version = "0.103.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ffdfa2f5286e2247234e03f680868ac2815974dc39e00ea15adc445d0aafe52" +dependencies = [ + "ring", + "rustls-pki-types", + "untrusted", +] + +[[package]] +name = "rustpython-ast" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4cdaf8ee5c1473b993b398c174641d3aa9da847af36e8d5eb8291930b72f31a5" +dependencies = [ + "is-macro", + "num-bigint", + "rustpython-parser-core", + "static_assertions", +] + +[[package]] +name = "rustpython-parser" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "868f724daac0caf9bd36d38caf45819905193a901e8f1c983345a68e18fb2abb" +dependencies = [ + "anyhow", + "is-macro", + "itertools 0.11.0", + "lalrpop-util", + "log", + "num-bigint", + "num-traits", + "phf", + "phf_codegen", + "rustc-hash 1.1.0", + "rustpython-ast", + "rustpython-parser-core", + "tiny-keccak", + "unic-emoji-char", + "unic-ucd-ident", + "unicode_names2", +] + +[[package]] +name = "rustpython-parser-core" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b4b6c12fa273825edc7bccd9a734f0ad5ba4b8a2f4da5ff7efe946f066d0f4ad" +dependencies = [ + "is-macro", + "memchr", + "rustpython-parser-vendored", +] + +[[package]] +name = "rustpython-parser-vendored" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "04fcea49a4630a3a5d940f4d514dc4f575ed63c14c3e3ed07146634aed7f67a6" +dependencies = [ + "memchr", + "once_cell", +] + +[[package]] +name = "rustversion" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" + +[[package]] +name = "rw-stream-sink" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8c9026ff5d2f23da5e45bbc283f156383001bfb09c4e44256d02c1a685fe9a1" +dependencies = [ + "futures", + "pin-project", + "static_assertions", +] + +[[package]] +name = "ryu" +version = "1.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28d3b2b1366ec20994f1fd18c3c594f05c5dd4bc44d8bb0c1c632c8d6829481f" + +[[package]] +name = "salsa20" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97a22f5af31f73a954c10289c93e8a50cc23d971e80ee446f1f6f7137a088213" +dependencies = [ + "cipher", +] + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "sec1" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3e97a565f76233a6003f9f5c54be1d9c5bdfa3eccfb189469f11ec4901c47dc" +dependencies = [ + "base16ct", + "der", + "generic-array", + "pkcs8", + "subtle", + "zeroize", +] + +[[package]] +name = "semver" +version = "1.0.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2" + +[[package]] +name = "send_wrapper" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f638d531eccd6e23b980caf34876660d38e265409d8e99b397ab71eb3612fad0" + +[[package]] +name = "send_wrapper" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd0b0ec5f1c1ca621c432a25813d8d60c88abe6d3e08a3eb9cf37d97a0fe3d73" +dependencies = [ + "futures-core", +] + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "serde_json" +version = "1.0.145" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "402a6f66d8c709116cf22f558eab210f5a50187f702eb4d7e5ef38d9a7f1c79c" +dependencies = [ + "itoa", + "memchr", + "ryu", + "serde", + "serde_core", +] + +[[package]] +name = "serde_spanned" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e24345aa0fe688594e73770a5f6d1b216508b4f93484c0026d521acd30134392" +dependencies = [ + "serde_core", +] + +[[package]] +name = "sha1" +version = "0.10.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "sha3" +version = "0.10.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75872d278a8f37ef87fa0ddbda7802605cb18344497949862c0d4dcb291eba60" +dependencies = [ + "digest", + "keccak", +] + +[[package]] +name = "sharded-slab" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6" +dependencies = [ + "lazy_static", +] + +[[package]] +name = "shlex" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" + +[[package]] +name = "signal-hook-registry" +version = "1.4.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7664a098b8e616bdfcc2dc0e9ac44eb231eedf41db4e9fe95d8d32ec728dedad" +dependencies = [ + "libc", +] + +[[package]] +name = "signature" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" +dependencies = [ + "digest", + "rand_core 0.6.4", +] + +[[package]] +name = "siphasher" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56199f7ddabf13fe5074ce809e7d3f42b42ae711800501b5b16ea82ad029c39d" + +[[package]] +name = "slab" +version = "0.4.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2ae44ef20feb57a68b23d846850f861394c2e02dc425a50098ae8c90267589" + +[[package]] +name = "smallvec" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" + +[[package]] +name = "snow" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "850948bee068e713b8ab860fe1adc4d109676ab4c3b621fd8147f06b261f2f85" +dependencies = [ + "aes-gcm", + "blake2", + "chacha20poly1305", + "curve25519-dalek", + "rand_core 0.6.4", + "ring", + "rustc_version", + "sha2", + "subtle", +] + +[[package]] +name = "socket2" +version = "0.5.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e22376abed350d73dd1cd119b57ffccad95b4e585a7cda43e286245ce23c0678" +dependencies = [ + "libc", + "windows-sys 0.52.0", +] + +[[package]] +name = "socket2" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "17129e116933cf371d018bb80ae557e889637989d8638274fb25622827b03881" +dependencies = [ + "libc", + "windows-sys 0.60.2", +] + +[[package]] +name = "soketto" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2e859df029d160cb88608f5d7df7fb4753fd20fdfb4de5644f3d8b8440841721" +dependencies = [ + "base64", + "bytes", + "futures", + "httparse", + "log", + "rand 0.8.5", + "sha1", +] + +[[package]] +name = "spki" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +dependencies = [ + "base64ct", + "der", +] + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "static_assertions" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "subtle" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" + +[[package]] +name = "syn" +version = "1.0.109" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72b64191b275b66ffe2469e8af2c1cfe3bafa67b529ead792a6d0160888b4237" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "2.0.111" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "390cc9a294ab71bdb1aa2e99d13be9c753cd2d7bd6560c77118597410c4d2e87" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "synstructure" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "sysinfo" +version = "0.33.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fc858248ea01b66f19d8e8a6d55f41deaf91e9d495246fd01368d99935c6c01" +dependencies = [ + "core-foundation-sys", + "libc", + "memchr", + "ntapi", + "rayon", + "windows 0.57.0", +] + +[[package]] +name = "system-configuration" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3c879d448e9d986b661742763247d3693ed13609438cf3d006f51f5368a5ba6b" +dependencies = [ + "bitflags 2.10.0", + "core-foundation", + "system-configuration-sys", +] + +[[package]] +name = "system-configuration-sys" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "system_custodian" +version = "0.0.1" +dependencies = [ + "delegate", + "derive_more", + "either", + "extend", + "futures", + "futures-timer", + "impl-trait-for-tuples", + "keccak-const", + "log", + "thiserror 2.0.17", + "tokio", + "tracing-subscriber", + "util", +] + +[[package]] +name = "tagptr" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b2093cf4c8eb1e67749a6762251bc9cd836b6fc171623bd0a9d324d37af2417" + +[[package]] +name = "target-lexicon" +version = "0.13.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df7f62577c25e07834649fc3b39fafdc597c0a3527dc1c60129201ccfcbaa50c" + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl 1.0.69", +] + +[[package]] +name = "thiserror" +version = "2.0.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f63587ca0f12b72a0600bcba1d40081f830876000bb46dd2337a3051618f4fc8" +dependencies = [ + "thiserror-impl 2.0.17", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ff15c8ecd7de3849db632e14d18d2571fa09dfc5ed93479bc4485c7a517c913" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "thread_local" +version = "1.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f60246a4944f24f6e018aa17cdeffb7818b76356965d03b07d6a9886e8962185" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "time" +version = "0.3.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91e7d9e3bb61134e77bde20dd4825b97c010155709965fedf0f49bb138e52a9d" +dependencies = [ + "deranged", + "itoa", + "num-conv", + "powerfmt", + "serde", + "time-core", + "time-macros", +] + +[[package]] +name = "time-core" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40868e7c1d2f0b8d73e4a8c7f0ff63af4f6d19be117e90bd73eb1d62cf831c6b" + +[[package]] +name = "time-macros" +version = "0.2.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "30cfb0125f12d9c277f35663a0a33f8c30190f4e4574868a330595412d34ebf3" +dependencies = [ + "num-conv", + "time-core", +] + +[[package]] +name = "tiny-keccak" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2c9d3793400a45f954c52e73d068316d76b6f4e36977e3fcebb13a2721e80237" +dependencies = [ + "crunchy", +] + +[[package]] +name = "tinystr" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "42d3e9c45c09de15d06dd8acf5f4e0e399e85927b7f00711024eb7ae10fa4869" +dependencies = [ + "displaydoc", + "zerovec", +] + +[[package]] +name = "tinytemplate" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "be4d6b5f19ff7664e8c98d03e2139cb510db9b0a60b55f8e8709b689d939b6bc" +dependencies = [ + "serde", + "serde_json", +] + +[[package]] +name = "tinyvec" +version = "1.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bfa5fdc3bce6191a1dbc8c02d5c8bffcf557bafa17c124c5264a458f1b0613fa" +dependencies = [ + "tinyvec_macros", +] + +[[package]] +name = "tinyvec_macros" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" + +[[package]] +name = "tokio" +version = "1.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff360e02eab121e0bc37a2d3b4d4dc622e6eda3a8e5253d5435ecf5bd4c68408" +dependencies = [ + "bytes", + "libc", + "mio", + "parking_lot", + "pin-project-lite", + "signal-hook-registry", + "socket2 0.6.1", + "tokio-macros", + "tracing", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-macros" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af407857209536a95c8e56f8231ef2c2e2aff839b22e07a1ffcbc617e9db9fa5" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "tokio-util" +version = "0.7.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2efa149fe76073d6e8fd97ef4f4eca7b67f599660115591483572e406e165594" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "toml" +version = "0.9.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0dc8b1fb61449e27716ec0e1bdf0f6b8f3e8f6b05391e8497b8b6d7804ea6d8" +dependencies = [ + "indexmap", + "serde_core", + "serde_spanned", + "toml_datetime", + "toml_parser", + "toml_writer", + "winnow", +] + +[[package]] +name = "toml_datetime" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2cdb639ebbc97961c51720f858597f7f24c4fc295327923af55b74c3c724533" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_parser" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0cbe268d35bdb4bb5a56a2de88d0ad0eb70af5384a99d648cd4b3d04039800e" +dependencies = [ + "winnow", +] + +[[package]] +name = "toml_writer" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df8b2b54733674ad286d16267dcfc7a71ed5c776e4ac7aa3c3e2561f7c637bf2" + +[[package]] +name = "tower-service" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3" + +[[package]] +name = "tracing" +version = "0.1.43" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2d15d90a0b5c19378952d479dc858407149d7bb45a14de0142f6c534b16fc647" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "tracing-core" +version = "0.1.35" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a04e24fab5c89c6a36eb8558c9656f30d81de51dfa4d3b45f26b21d61fa0a6c" +dependencies = [ + "once_cell", + "valuable", +] + +[[package]] +name = "tracing-log" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3" +dependencies = [ + "log", + "once_cell", + "tracing-core", +] + +[[package]] +name = "tracing-subscriber" +version = "0.3.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2f30143827ddab0d256fd843b7a66d164e9f271cfa0dde49142c5ca0ca291f1e" +dependencies = [ + "matchers", + "nu-ansi-term", + "once_cell", + "regex-automata", + "sharded-slab", + "smallvec", + "thread_local", + "tracing", + "tracing-core", + "tracing-log", +] + +[[package]] +name = "try-lock" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" + +[[package]] +name = "typenum" +version = "1.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb" + +[[package]] +name = "uint" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "909988d098b2f738727b161a106cfc7cab00c539c2687a8836f8e565976fb53e" +dependencies = [ + "byteorder", + "crunchy", + "hex", + "static_assertions", +] + +[[package]] +name = "unic-char-property" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a8c57a407d9b6fa02b4795eb81c5b6652060a15a7903ea981f3d723e6c0be221" +dependencies = [ + "unic-char-range", +] + +[[package]] +name = "unic-char-range" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0398022d5f700414f6b899e10b8348231abf9173fa93144cbc1a43b9793c1fbc" + +[[package]] +name = "unic-common" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "80d7ff825a6a654ee85a63e80f92f054f904f21e7d12da4e22f9834a4aaa35bc" + +[[package]] +name = "unic-emoji-char" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b07221e68897210270a38bde4babb655869637af0f69407f96053a34f76494d" +dependencies = [ + "unic-char-property", + "unic-char-range", + "unic-ucd-version", +] + +[[package]] +name = "unic-ucd-ident" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e230a37c0381caa9219d67cf063aa3a375ffed5bf541a452db16e744bdab6987" +dependencies = [ + "unic-char-property", + "unic-char-range", + "unic-ucd-version", +] + +[[package]] +name = "unic-ucd-version" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96bd2f2237fe450fcd0a1d2f5f4e91711124f7857ba2e964247776ebeeb7b0c4" +dependencies = [ + "unic-common", +] + +[[package]] +name = "unicode-ident" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9312f7c4f6ff9069b165498234ce8be658059c6728633667c526e27dc2cf1df5" + +[[package]] +name = "unicode-segmentation" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f6ccf251212114b54433ec949fd6a7841275f9ada20dddd2f29e9ceea4501493" + +[[package]] +name = "unicode-width" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254" + +[[package]] +name = "unicode-xid" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" + +[[package]] +name = "unicode_names2" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1673eca9782c84de5f81b82e4109dcfb3611c8ba0d52930ec4a9478f547b2dd" +dependencies = [ + "phf", + "unicode_names2_generator", +] + +[[package]] +name = "unicode_names2_generator" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b91e5b84611016120197efd7dc93ef76774f4e084cd73c9fb3ea4a86c570c56e" +dependencies = [ + "getopts", + "log", + "phf_codegen", + "rand 0.8.5", +] + +[[package]] +name = "unindent" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3" + +[[package]] +name = "universal-hash" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc1de2c688dc15305988b563c3854064043356019f97a4b46276fe734c4f07ea" +dependencies = [ + "crypto-common", + "subtle", +] + +[[package]] +name = "unsigned-varint" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6889a77d49f1f013504cec6bf97a2c730394adedaeb1deb5ea08949a50541105" + +[[package]] +name = "unsigned-varint" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eb066959b24b5196ae73cb057f45598450d2c5f71460e98c49b738086eff9c06" + +[[package]] +name = "untrusted" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1" + +[[package]] +name = "url" +version = "2.5.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08bc136a29a3d1758e07a9cca267be308aeebf5cfd5a10f3f67ab2097683ef5b" +dependencies = [ + "form_urlencoded", + "idna", + "percent-encoding", + "serde", +] + +[[package]] +name = "utf8_iter" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "util" +version = "0.0.1" +dependencies = [ + "bon", + "derive_more", + "extend", + "internment", + "once_cell", + "recursion", + "thiserror 2.0.17", +] + +[[package]] +name = "uuid" +version = "1.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e2e054861b4bd027cd373e18e8d8d8e6548085000e41290d95ce0c373a654b4a" +dependencies = [ + "getrandom 0.3.4", + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "valuable" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "want" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e" +dependencies = [ + "try-lock", +] + +[[package]] +name = "wasi" +version = "0.9.0+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519" + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasip2" +version = "1.0.1+wasi-0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0562428422c63773dad2c345a1882263bbf4d65cf3f42e90921f787ef5ad58e7" +dependencies = [ + "wit-bindgen", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d759f433fa64a2d763d1340820e46e111a7a5ab75f993d1852d70b03dbb80fd" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-futures" +version = "0.4.56" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "836d9622d604feee9e5de25ac10e3ea5f2d65b41eac0d9ce72eb5deae707ce7c" +dependencies = [ + "cfg-if", + "js-sys", + "once_cell", + "wasm-bindgen", + "web-sys", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48cb0d2638f8baedbc542ed444afc0644a29166f1595371af4fecf8ce1e7eeb3" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cefb59d5cd5f92d9dcf80e4683949f15ca4b511f4ac0a6e14d4e1ac60c6ecd40" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 2.0.111", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cbc538057e648b67f72a982e708d485b2efa771e1ac05fec311f9f63e5800db4" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "web-sys" +version = "0.3.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b32828d774c412041098d182a8b38b16ea816958e07cf40eec2bc080ae137ac" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "web-time" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "webpki-roots" +version = "0.26.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9" +dependencies = [ + "webpki-roots 1.0.4", +] + +[[package]] +name = "webpki-roots" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2878ef029c47c6e8cf779119f20fcf52bde7ad42a731b2a304bc221df17571e" +dependencies = [ + "rustls-pki-types", +] + +[[package]] +name = "widestring" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471" + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "windows" +version = "0.53.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "efc5cf48f83140dcaab716eeaea345f9e93d0018fb81162753a3f76c3397b538" +dependencies = [ + "windows-core 0.53.0", + "windows-targets 0.52.6", +] + +[[package]] +name = "windows" +version = "0.57.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12342cb4d8e3b046f3d80effd474a7a02447231330ef77d71daa6fbc40681143" +dependencies = [ + "windows-core 0.57.0", + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-core" +version = "0.53.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9dcc5b895a6377f1ab9fa55acedab1fd5ac0db66ad1e6c7f47e28a22e446a5dd" +dependencies = [ + "windows-result 0.1.2", + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-core" +version = "0.57.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2ed2439a290666cd67ecce2b0ffaad89c2a56b976b736e6ece670297897832d" +dependencies = [ + "windows-implement 0.57.0", + "windows-interface 0.57.0", + "windows-result 0.1.2", + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-core" +version = "0.62.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb" +dependencies = [ + "windows-implement 0.60.2", + "windows-interface 0.59.3", + "windows-link", + "windows-result 0.4.1", + "windows-strings", +] + +[[package]] +name = "windows-implement" +version = "0.57.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9107ddc059d5b6fbfbffdfa7a7fe3e22a226def0b2608f72e9d552763d3e1ad7" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "windows-implement" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "windows-interface" +version = "0.57.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29bee4b38ea3cde66011baa44dba677c432a78593e202392d1e9070cf2a7fca7" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "windows-interface" +version = "0.59.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-result" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e383302e8ec8515204254685643de10811af0ed97ea37210dc26fb0032647f8" +dependencies = [ + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-result" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-strings" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-sys" +version = "0.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9" +dependencies = [ + "windows-targets 0.48.5", +] + +[[package]] +name = "windows-sys" +version = "0.52.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" +dependencies = [ + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-sys" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb" +dependencies = [ + "windows-targets 0.53.5", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-targets" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c" +dependencies = [ + "windows_aarch64_gnullvm 0.48.5", + "windows_aarch64_msvc 0.48.5", + "windows_i686_gnu 0.48.5", + "windows_i686_msvc 0.48.5", + "windows_x86_64_gnu 0.48.5", + "windows_x86_64_gnullvm 0.48.5", + "windows_x86_64_msvc 0.48.5", +] + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm 0.52.6", + "windows_aarch64_msvc 0.52.6", + "windows_i686_gnu 0.52.6", + "windows_i686_gnullvm 0.52.6", + "windows_i686_msvc 0.52.6", + "windows_x86_64_gnu 0.52.6", + "windows_x86_64_gnullvm 0.52.6", + "windows_x86_64_msvc 0.52.6", +] + +[[package]] +name = "windows-targets" +version = "0.53.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3" +dependencies = [ + "windows-link", + "windows_aarch64_gnullvm 0.53.1", + "windows_aarch64_msvc 0.53.1", + "windows_i686_gnu 0.53.1", + "windows_i686_gnullvm 0.53.1", + "windows_i686_msvc 0.53.1", + "windows_x86_64_gnu 0.53.1", + "windows_x86_64_gnullvm 0.53.1", + "windows_x86_64_msvc 0.53.1", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8" + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" + +[[package]] +name = "windows_i686_gnu" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e" + +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" + +[[package]] +name = "windows_i686_msvc" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406" + +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + +[[package]] +name = "windows_i686_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" + +[[package]] +name = "winnow" +version = "0.7.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a5364e9d77fcdeeaa6062ced926ee3381faa2ee02d3eb83a5c27a8825540829" + +[[package]] +name = "winreg" +version = "0.50.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "524e57b2c537c0f9b1e69f1965311ec12182b4122e45035b1508cd24d2adadb1" +dependencies = [ + "cfg-if", + "windows-sys 0.48.0", +] + +[[package]] +name = "wit-bindgen" +version = "0.46.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f17a85883d4e6d00e8a97c586de764dabcc06133f7f1d55dce5cdc070ad7fe59" + +[[package]] +name = "writeable" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9edde0db4769d2dc68579893f2306b26c6ecfbe0ef499b013d731b7b9247e0b9" + +[[package]] +name = "x25519-dalek" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7e468321c81fb07fa7f4c636c3972b9100f0346e5b6a9f2bd0603a52f7ed277" +dependencies = [ + "curve25519-dalek", + "rand_core 0.6.4", + "serde", + "zeroize", +] + +[[package]] +name = "x509-parser" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4569f339c0c402346d4a75a9e39cf8dad310e287eef1ff56d4c68e5067f53460" +dependencies = [ + "asn1-rs", + "data-encoding", + "der-parser", + "lazy_static", + "nom", + "oid-registry", + "rusticata-macros", + "thiserror 2.0.17", + "time", +] + +[[package]] +name = "xml-rs" +version = "0.8.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ae8337f8a065cfc972643663ea4279e04e7256de865aa66fe25cec5fb912d3f" + +[[package]] +name = "xmltree" +version = "0.10.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7d8a75eaf6557bb84a65ace8609883db44a29951042ada9b393151532e41fcb" +dependencies = [ + "xml-rs", +] + +[[package]] +name = "yamux" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed0164ae619f2dc144909a9f082187ebb5893693d8c0196e8085283ccd4b776" +dependencies = [ + "futures", + "log", + "nohash-hasher", + "parking_lot", + "pin-project", + "rand 0.8.5", + "static_assertions", +] + +[[package]] +name = "yamux" +version = "0.13.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "deab71f2e20691b4728b349c6cee8fc7223880fa67b6b4f92225ec32225447e5" +dependencies = [ + "futures", + "log", + "nohash-hasher", + "parking_lot", + "pin-project", + "rand 0.9.2", + "static_assertions", + "web-time", +] + +[[package]] +name = "yasna" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e17bb3549cc1321ae1296b9cdc2698e2b6cb1992adfa19a8c72e5b7a738f44cd" +dependencies = [ + "time", +] + +[[package]] +name = "yoke" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72d6e5c6afb84d73944e5cedb052c4680d5657337201555f9f2a16b7406d4954" +dependencies = [ + "stable_deref_trait", + "yoke-derive", + "zerofrom", +] + +[[package]] +name = "yoke-derive" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b659052874eb698efe5b9e8cf382204678a0086ebf46982b79d6ca3182927e5d" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", + "synstructure", +] + +[[package]] +name = "zerocopy" +version = "0.8.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fd74ec98b9250adb3ca554bdde269adf631549f51d8a8f8f0a10b50f1cb298c3" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8a8d209fdf45cf5138cbb5a506f6b52522a25afccc534d1475dad8e31105c6a" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "zerofrom" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "50cc42e0333e05660c3587f3bf9d0478688e15d870fab3346451ce7f8c9fbea5" +dependencies = [ + "zerofrom-derive", +] + +[[package]] +name = "zerofrom-derive" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d71e5d6e06ab090c67b5e44993ec16b72dcbaabc526db883a360057678b48502" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", + "synstructure", +] + +[[package]] +name = "zeroize" +version = "1.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" +dependencies = [ + "zeroize_derive", +] + +[[package]] +name = "zeroize_derive" +version = "1.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce36e65b0d2999d2aafac989fb249189a141aee1f53c612c1f37d72631959f69" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] + +[[package]] +name = "zerotrie" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a59c17a5562d507e4b54960e8569ebee33bee890c70aa3fe7b97e85a9fd7851" +dependencies = [ + "displaydoc", + "yoke", + "zerofrom", +] + +[[package]] +name = "zerovec" +version = "0.11.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c28719294829477f525be0186d13efa9a3c602f7ec202ca9e353d310fb9a002" +dependencies = [ + "yoke", + "zerofrom", + "zerovec-derive", +] + +[[package]] +name = "zerovec-derive" +version = "0.11.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eadce39539ca5cb3985590102671f2567e659fca9666581ad3411d59207951f3" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.111", +] diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 00000000..e16c7b67 --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,165 @@ +[workspace] +resolver = "3" +members = [ + "rust/networking", + "rust/exo_pyo3_bindings", + "rust/system_custodian", + "rust/util", +] + +[workspace.package] +version = "0.0.1" +edition = "2024" + +[profile.dev] +opt-level = 1 +debug = true + +[profile.release] +opt-level = 3 + +# Common shared dependendencies configured once at the workspace +# level, to be re-used more easily across workspace member crates. +# +# Common configurations include versions, paths, features, etc. +[workspace.dependencies] +## Crate members as common dependencies +networking = { path = "rust/networking" } +system_custodian = { path = "rust/system_custodian" } +util = { path = "rust/util" } + +# Proc-macro authoring tools +syn = "2.0" +quote = "1.0" +proc-macro2 = "1.0" +darling = "0.20" + +# Macro dependecies +extend = "1.2" +delegate = "0.13" +impl-trait-for-tuples = "0.2" +clap = "4.5" +derive_more = { version = "2.0.1", features = ["display"] } +pin-project = "1" + +# Utility dependencies +itertools = "0.14" +thiserror = "2" +internment = "0.8" +recursion = "0.5" +regex = "1.11" +once_cell = "1.21" +thread_local = "1.1" +bon = "3.4" +generativity = "1.1" +anyhow = "1.0" +keccak-const = "0.2" + +# Functional generics/lenses frameworks +frunk_core = "0.4" +frunk = "0.4" +frunk_utils = "0.2" +frunk-enum-core = "0.3" + +# Async dependencies +tokio = "1.46" +futures = "0.3" +futures-util = "0.3" +futures-timer = "3.0" + +# Data structures +either = "1.15" +ordered-float = "5.0" +ahash = "0.8" + +# Tracing/logging +log = "0.4" + +# networking +libp2p = "0.56" +libp2p-tcp = "0.44" + +[workspace.lints.rust] +static_mut_refs = "warn" # Or use "warn" instead of deny +incomplete_features = "allow" + +# Clippy's lint category level configurations; +# every member crate needs to inherit these by adding +# +# ```toml +# [lints] +# workspace = true +# ``` +# +# to their `Cargo.toml` files +[workspace.lints.clippy] +# Clippy lint categories meant to be enabled all at once +correctness = { level = "deny", priority = -1 } +suspicious = { level = "warn", priority = -1 } +style = { level = "warn", priority = -1 } +complexity = { level = "warn", priority = -1 } +perf = { level = "warn", priority = -1 } +pedantic = { level = "warn", priority = -1 } +nursery = { level = "warn", priority = -1 } +cargo = { level = "warn", priority = -1 } + +# Individual Clippy lints from the `restriction` category +arithmetic_side_effects = "warn" +as_conversions = "warn" +assertions_on_result_states = "warn" +clone_on_ref_ptr = "warn" +decimal_literal_representation = "warn" +default_union_representation = "warn" +deref_by_slicing = "warn" +disallowed_script_idents = "deny" +else_if_without_else = "warn" +empty_enum_variants_with_brackets = "warn" +empty_structs_with_brackets = "warn" +error_impl_error = "warn" +exit = "deny" +expect_used = "warn" +float_cmp_const = "warn" +get_unwrap = "warn" +if_then_some_else_none = "warn" +impl_trait_in_params = "warn" +indexing_slicing = "warn" +infinite_loop = "warn" +let_underscore_must_use = "warn" +let_underscore_untyped = "warn" +lossy_float_literal = "warn" +mem_forget = "warn" +missing_inline_in_public_items = "warn" +multiple_inherent_impl = "warn" +multiple_unsafe_ops_per_block = "warn" +mutex_atomic = "warn" +non_zero_suggestions = "warn" +panic = "warn" +partial_pub_fields = "warn" +pattern_type_mismatch = "warn" +pub_without_shorthand = "warn" +rc_buffer = "warn" +rc_mutex = "warn" +redundant_type_annotations = "warn" +renamed_function_params = "warn" +rest_pat_in_fully_bound_structs = "warn" +same_name_method = "warn" +self_named_module_files = "deny" +semicolon_inside_block = "warn" +shadow_same = "warn" +shadow_unrelated = "warn" +str_to_string = "warn" +string_add = "warn" +string_lit_chars_any = "warn" +string_to_string = "warn" +tests_outside_test_module = "warn" +todo = "warn" +try_err = "warn" +undocumented_unsafe_blocks = "warn" +unnecessary_safety_comment = "warn" +unnecessary_safety_doc = "warn" +unneeded_field_pattern = "warn" +unseparated_literal_suffix = "warn" +unused_result_ok = "warn" +unused_trait_names = "warn" +unwrap_used = "warn" +verbose_file_reads = "warn" diff --git a/RULES.md b/RULES.md new file mode 100644 index 00000000..6524ee4b --- /dev/null +++ b/RULES.md @@ -0,0 +1,84 @@ +# Repository Rules + +* if you see any code that violates these rules, raise it with me directly rather than trying to fix. + * where applicable, file a GitHub Issue. +* adhere to these rules strictly. + +## General Rules + +* if its possible to eliminate an extra try-catch or if-statement at runtime using type-level discipline, do it! +* name your types, functions, and classes appropriately. + * no three-letter acronyms. + * no non-standard contractions. + * each data type has a meaning, pick a name which is accurate and descriptive. + * the average layman should be able to easily understand what your function does using the function signature alone! + * sometimes, there will be exceptions. eg, when you're using specific technical terms that are well understood (saga, event, etc). + * usually, you'll think that your code is an exception to the rules, but it won't be. + +## State, Functions and Classes + +* every function, given the same inputs, should produce the same outputs. ie, no hidden state. +* use classes to prevent fixed state from being mutated arbitrarily (unsafely); methods provide a safe way of interfacing with state. +* if your logic doesn't mutate fixed state, it probably belongs in a standalone function rather than a class. +* functions shouldn't usually produce side-effects (they should be computationally pure). + * if, for example, you're updating a state using an event (computationally pure), and you want to trigger a saga (computational side-effect), store the logic for triggering the saga into an effect handler (a function, capable of producing side-effects, that you pass into an otherwise computationally pure function, so that it may trigger side-effects safely). + +## Pydantic + +* read the Pydantic docs. +* respect the Pydantic docs. +* pydantic is all you need. +* declare and re-use a central `ConfigDict` for your use-case, you'll usually want `frozen` and `strict` to be `True`. + +## Unique ID (UUID) Generation + +* inherit from Pydantic's `UUID4` class to create your own UUID class. +* use `uuid.uuid4()` to initialize your class with a fresh UUID where possible. +* ensure that idempotency tags are generated by taking the salted hash of persisted state. + * rationale: if a node crashes and resumes from an older state, it should not accidentally re-publish the same event twice under different idempotency tags. + * every distinct function should feature a unique salt, so that there are no accidental collisions in idempotency tags. + +## Type Wrappers + +* reuse types that already exist in the Python standard library. +* when two distinct data types are structurally identical (for example, different IDs which are both UUIDs but shouldn't never mixed up), make sure they can't be conflated by the type system. + * if you're working with a primitive data type (`str`, `int`, etc), use `NewType` (it has zero runtime overhead). + * if you're working with serializable data objects, consider adding a field (type `str`) that states its type. + +## Type Discipline + +* do not bypass the type-checker, preserve strict typing by any means necessary. +* by default, use literal types (like `Literal['one', 'two']`) where an enum seems appropriate. + +pro-tip: Python's type system is quite complex and feature-rich, so reading the documentation is often advisable; Matt discovered that Python `typing` library allows you to check that you've implemented a `match` exhaustively using `Literal` and `get_args(type)` after reading the docs. + +## Use of `@final`, Freezing + +* use wherever applicable. + +## Error Handling + +* don't try-catch for no reason. +* make sure that you always know where and when the exceptions your code produces are meant to be handled, so that it's never a nasty surprise. + * always write the rationale for your error-handling down in the docstring! + * communicate the details to your colleagues when appropriate. + +## Dependencies + +* don't introduce any new dependencies without asking. +* don't ask for any dependencies that aren't ubiquitous within production environments. + +## Commit Messages + +* use the imperative mood in the subject line. +* prefix the subject line with a change type. our change types are: + * `documentation`: documentation changes. + * `feature`: a new feature. + * `refactor`: a code change that neither fixes a bug nor adds a feature. + * `bugfix`: a bug fix. + * `chore`: routine tasks, maintenance, or tooling changes. + * `test`: adding or correcting tests. +* restrict the subject line to fifty characters or less. +* capitalize the subject line. +* do not end the subject line with a period. +* separate subject from body with a blank line. \ No newline at end of file diff --git a/TODO.md b/TODO.md new file mode 100644 index 00000000..89d7a525 --- /dev/null +++ b/TODO.md @@ -0,0 +1,27 @@ +2. Currently a lot of requests from the API are timing out, but we still process those requests internally. If an API request times out, we should cancel all corresponding tasks to that API request (why process a request with nobody listening). +3. Task cancellation. When API http request gets cancelled, it should cancel corresponding task. +4. I'd like to see profiled network latency / bandwidth. +5. I'd like to see how much bandwidth each link is using. +6. We should handle the case where one machine doesn't have the model downloaded and then other machines are waiting on it. In this case we get loads of timeout errors because the others are waiting for the one that needs to download the model. +7. Solve the problem of in continuous batching when a new prompt comes in, it will block decode of the current batch until the prefill is complete. +8. We want people to be able to copy models over to a new device without ever connecting EXO to the internet. Right now EXO require internet connection once to cache some files to check if a download is complete. Instead, we should simply check if there is a non-empty model folder locally with no .partial files. This indicates it's a fully downloaded model that can be loaded. +10. More granular control over how to deploy instances. +12. Nix is great but installing it is a pain and we have ended up in a lot of cases having PATH issues or installation issues. For example, after rebooting mike it seemed to no longer have a nix installation and needed reinstalling. It has a bunch of broken symlinks left over from nix that caused ssh to fail, making it even harder to debug. We need consistent environments (perhaps MDM) so we can guarantee nix is installed properly on each machine. +13. Memory pressure instead of memory used. +14. Show the type of each connection (TB5, Ethernet, etc.) in the UI. Refer to old exo: https://github.com/exo-explore/exo/blob/56f783b38dc6b08ce606b07a5386dc40dae00330/exo/helpers.py#L251 +15. Prioritise certain connection types (or by latency). TB5 > Ethernet > WiFi. Refer to old exo: https://github.com/exo-explore/exo/blob/56f783b38dc6b08ce606b07a5386dc40dae00330/exo/helpers.py#L251 +16. Dynamically switch to higher priority connection when it becomes available. Probably bring back InstanceReplacedAtomically. +17. Faster model loads by streaming model from other devices in cluster. +18. Add support for specifying the type of network connection to use in a test. Depends on 15/16. +20. Add chat completion cancellations (e.g OpenWebUI has something for cancelling an ongoing request). +23. Do we need cache_limit? We went back and forth on that a lot because we thought it might be causing issues. One problem is it sets it relative to model size. So if you have multiple models loaded in it will take the most recent model size for the cache_limit. This is problematic if you launch DeepSeek -> Llama for example. +24. further openai/lmstudio api compatibility +25. Rethink retry logic +26. Task cancellation. When API http request gets cancelled, it should cancel corresponding task. +27. Log cleanup - per-module log filters and default to DEBUG log levels + +Potential refactors: + +2. Topology can be simplified + +Random errors we've run into: diff --git a/dashboard/exo-logo-hq-square-black-bg.jpg b/dashboard/exo-logo-hq-square-black-bg.jpg new file mode 100644 index 00000000..e72eaf0d Binary files /dev/null and b/dashboard/exo-logo-hq-square-black-bg.jpg differ diff --git a/dashboard/exo-logo-hq-square-black-bg.png b/dashboard/exo-logo-hq-square-black-bg.png new file mode 100644 index 00000000..7ff00135 Binary files /dev/null and b/dashboard/exo-logo-hq-square-black-bg.png differ diff --git a/dashboard/exo-logo-hq-square-black-bg.webp b/dashboard/exo-logo-hq-square-black-bg.webp new file mode 100644 index 00000000..9a67219a Binary files /dev/null and b/dashboard/exo-logo-hq-square-black-bg.webp differ diff --git a/dashboard/exo-logo.png b/dashboard/exo-logo.png new file mode 100644 index 00000000..199bcfdd Binary files /dev/null and b/dashboard/exo-logo.png differ diff --git a/dashboard/favicon.ico b/dashboard/favicon.ico new file mode 100644 index 00000000..c0ae2099 Binary files /dev/null and b/dashboard/favicon.ico differ diff --git a/dashboard/package-lock.json b/dashboard/package-lock.json new file mode 100644 index 00000000..e075d621 --- /dev/null +++ b/dashboard/package-lock.json @@ -0,0 +1,3058 @@ +{ + "name": "exo-dashboard", + "version": "1.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "exo-dashboard", + "version": "1.0.0", + "dependencies": { + "highlight.js": "^11.11.1", + "mode-watcher": "^1.1.0" + }, + "devDependencies": { + "@sveltejs/adapter-static": "^3.0.10", + "@sveltejs/kit": "^2.48.4", + "@sveltejs/vite-plugin-svelte": "^5.0.0", + "@tailwindcss/vite": "^4.0.0", + "@types/d3": "^7.4.3", + "@types/node": "^22", + "d3": "^7.9.0", + "svelte": "^5.0.0", + "svelte-check": "^4.0.0", + "tailwindcss": "^4.0.0", + "tw-animate-css": "^1.3.5", + "typescript": "^5.0.0", + "vite": "^6.0.0" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.12.tgz", + "integrity": "sha512-Hhmwd6CInZ3dwpuGTF8fJG6yoWmsToE+vYgD4nytZVxcu1ulHpUQRAB1UJ8+N1Am3Mz4+xOByoQoSZf4D+CpkA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.12.tgz", + "integrity": "sha512-VJ+sKvNA/GE7Ccacc9Cha7bpS8nyzVv0jdVgwNDaR4gDMC/2TTRc33Ip8qrNYUcpkOHUT5OZ0bUcNNVZQ9RLlg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.12.tgz", + "integrity": "sha512-6AAmLG7zwD1Z159jCKPvAxZd4y/VTO0VkprYy+3N2FtJ8+BQWFXU+OxARIwA46c5tdD9SsKGZ/1ocqBS/gAKHg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.12.tgz", + "integrity": "sha512-5jbb+2hhDHx5phYR2By8GTWEzn6I9UqR11Kwf22iKbNpYrsmRB18aX/9ivc5cabcUiAT/wM+YIZ6SG9QO6a8kg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.12.tgz", + "integrity": "sha512-N3zl+lxHCifgIlcMUP5016ESkeQjLj/959RxxNYIthIg+CQHInujFuXeWbWMgnTo4cp5XVHqFPmpyu9J65C1Yg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.12.tgz", + "integrity": "sha512-HQ9ka4Kx21qHXwtlTUVbKJOAnmG1ipXhdWTmNXiPzPfWKpXqASVcWdnf2bnL73wgjNrFXAa3yYvBSd9pzfEIpA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.12.tgz", + "integrity": "sha512-gA0Bx759+7Jve03K1S0vkOu5Lg/85dou3EseOGUes8flVOGxbhDDh/iZaoek11Y8mtyKPGF3vP8XhnkDEAmzeg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.12.tgz", + "integrity": "sha512-TGbO26Yw2xsHzxtbVFGEXBFH0FRAP7gtcPE7P5yP7wGy7cXK2oO7RyOhL5NLiqTlBh47XhmIUXuGciXEqYFfBQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.12.tgz", + "integrity": "sha512-lPDGyC1JPDou8kGcywY0YILzWlhhnRjdof3UlcoqYmS9El818LLfJJc3PXXgZHrHCAKs/Z2SeZtDJr5MrkxtOw==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.12.tgz", + "integrity": "sha512-8bwX7a8FghIgrupcxb4aUmYDLp8pX06rGh5HqDT7bB+8Rdells6mHvrFHHW2JAOPZUbnjUpKTLg6ECyzvas2AQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.12.tgz", + "integrity": "sha512-0y9KrdVnbMM2/vG8KfU0byhUN+EFCny9+8g202gYqSSVMonbsCfLjUO+rCci7pM0WBEtz+oK/PIwHkzxkyharA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.12.tgz", + "integrity": "sha512-h///Lr5a9rib/v1GGqXVGzjL4TMvVTv+s1DPoxQdz7l/AYv6LDSxdIwzxkrPW438oUXiDtwM10o9PmwS/6Z0Ng==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.12.tgz", + "integrity": "sha512-iyRrM1Pzy9GFMDLsXn1iHUm18nhKnNMWscjmp4+hpafcZjrr2WbT//d20xaGljXDBYHqRcl8HnxbX6uaA/eGVw==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.12.tgz", + "integrity": "sha512-9meM/lRXxMi5PSUqEXRCtVjEZBGwB7P/D4yT8UG/mwIdze2aV4Vo6U5gD3+RsoHXKkHCfSxZKzmDssVlRj1QQA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.12.tgz", + "integrity": "sha512-Zr7KR4hgKUpWAwb1f3o5ygT04MzqVrGEGXGLnj15YQDJErYu/BGg+wmFlIDOdJp0PmB0lLvxFIOXZgFRrdjR0w==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.12.tgz", + "integrity": "sha512-MsKncOcgTNvdtiISc/jZs/Zf8d0cl/t3gYWX8J9ubBnVOwlk65UIEEvgBORTiljloIWnBzLs4qhzPkJcitIzIg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.12.tgz", + "integrity": "sha512-uqZMTLr/zR/ed4jIGnwSLkaHmPjOjJvnm6TVVitAa08SLS9Z0VM8wIRx7gWbJB5/J54YuIMInDquWyYvQLZkgw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.12.tgz", + "integrity": "sha512-xXwcTq4GhRM7J9A8Gv5boanHhRa/Q9KLVmcyXHCTaM4wKfIpWkdXiMog/KsnxzJ0A1+nD+zoecuzqPmCRyBGjg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.12.tgz", + "integrity": "sha512-Ld5pTlzPy3YwGec4OuHh1aCVCRvOXdH8DgRjfDy/oumVovmuSzWfnSJg+VtakB9Cm0gxNO9BzWkj6mtO1FMXkQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.12.tgz", + "integrity": "sha512-fF96T6KsBo/pkQI950FARU9apGNTSlZGsv1jZBAlcLL1MLjLNIWPBkj5NlSz8aAzYKg+eNqknrUJ24QBybeR5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.12.tgz", + "integrity": "sha512-MZyXUkZHjQxUvzK7rN8DJ3SRmrVrke8ZyRusHlP+kuwqTcfWLyqMOE3sScPPyeIXN/mDJIfGXvcMqCgYKekoQw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.25.12.tgz", + "integrity": "sha512-rm0YWsqUSRrjncSXGA7Zv78Nbnw4XL6/dzr20cyrQf7ZmRcsovpcRBdhD43Nuk3y7XIoW2OxMVvwuRvk9XdASg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.12.tgz", + "integrity": "sha512-3wGSCDyuTHQUzt0nV7bocDy72r2lI33QL3gkDNGkod22EsYl04sMf0qLb8luNKTOmgF/eDEDP5BFNwoBKH441w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.12.tgz", + "integrity": "sha512-rMmLrur64A7+DKlnSuwqUdRKyd3UE7oPJZmnljqEptesKM8wx9J8gx5u0+9Pq0fQQW8vqeKebwNXdfOyP+8Bsg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.12.tgz", + "integrity": "sha512-HkqnmmBoCbCwxUKKNPBixiWDGCpQGVsrQfJoVGYLPT41XWF8lHuE5N6WhVia2n4o5QK5M4tYr21827fNhi4byQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.12.tgz", + "integrity": "sha512-alJC0uCZpTFrSL0CCDjcgleBXPnCrEAhTBILpeAp7M/OFgoqtAetfBzX0xM00MUsVVPpVjlPuMbREqnZCXaTnA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@jridgewell/gen-mapping": { + "version": "0.3.13", + "resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.13.tgz", + "integrity": "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==", + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.0", + "@jridgewell/trace-mapping": "^0.3.24" + } + }, + "node_modules/@jridgewell/remapping": { + "version": "2.3.5", + "resolved": "https://registry.npmjs.org/@jridgewell/remapping/-/remapping-2.3.5.tgz", + "integrity": "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ==", + "license": "MIT", + "dependencies": { + "@jridgewell/gen-mapping": "^0.3.5", + "@jridgewell/trace-mapping": "^0.3.24" + } + }, + "node_modules/@jridgewell/resolve-uri": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", + "integrity": "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==", + "license": "MIT", + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@jridgewell/sourcemap-codec": { + "version": "1.5.5", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.5.tgz", + "integrity": "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==", + "license": "MIT" + }, + "node_modules/@jridgewell/trace-mapping": { + "version": "0.3.31", + "resolved": "https://registry.npmjs.org/@jridgewell/trace-mapping/-/trace-mapping-0.3.31.tgz", + "integrity": "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw==", + "license": "MIT", + "dependencies": { + "@jridgewell/resolve-uri": "^3.1.0", + "@jridgewell/sourcemap-codec": "^1.4.14" + } + }, + "node_modules/@polka/url": { + "version": "1.0.0-next.29", + "resolved": "https://registry.npmjs.org/@polka/url/-/url-1.0.0-next.29.tgz", + "integrity": "sha512-wwQAWhWSuHaag8c4q/KN/vCoeOJYshAIvMQwD4GpSb3OiZklFfvAgmj0VCBBImRpuF/aFgIRzllXlVX93Jevww==", + "dev": true, + "license": "MIT" + }, + "node_modules/@rollup/rollup-android-arm-eabi": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.53.3.tgz", + "integrity": "sha512-mRSi+4cBjrRLoaal2PnqH82Wqyb+d3HsPUN/W+WslCXsZsyHa9ZeQQX/pQsZaVIWDkPcpV6jJ+3KLbTbgnwv8w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-android-arm64": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.53.3.tgz", + "integrity": "sha512-CbDGaMpdE9sh7sCmTrTUyllhrg65t6SwhjlMJsLr+J8YjFuPmCEjbBSx4Z/e4SmDyH3aB5hGaJUP2ltV/vcs4w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-darwin-arm64": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.53.3.tgz", + "integrity": "sha512-Nr7SlQeqIBpOV6BHHGZgYBuSdanCXuw09hon14MGOLGmXAFYjx1wNvquVPmpZnl0tLjg25dEdr4IQ6GgyToCUA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-darwin-x64": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.53.3.tgz", + "integrity": "sha512-DZ8N4CSNfl965CmPktJ8oBnfYr3F8dTTNBQkRlffnUarJ2ohudQD17sZBa097J8xhQ26AwhHJ5mvUyQW8ddTsQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-freebsd-arm64": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.53.3.tgz", + "integrity": "sha512-yMTrCrK92aGyi7GuDNtGn2sNW+Gdb4vErx4t3Gv/Tr+1zRb8ax4z8GWVRfr3Jw8zJWvpGHNpss3vVlbF58DZ4w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-freebsd-x64": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.53.3.tgz", + "integrity": "sha512-lMfF8X7QhdQzseM6XaX0vbno2m3hlyZFhwcndRMw8fbAGUGL3WFMBdK0hbUBIUYcEcMhVLr1SIamDeuLBnXS+Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-linux-arm-gnueabihf": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.53.3.tgz", + "integrity": "sha512-k9oD15soC/Ln6d2Wv/JOFPzZXIAIFLp6B+i14KhxAfnq76ajt0EhYc5YPeX6W1xJkAdItcVT+JhKl1QZh44/qw==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm-musleabihf": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.53.3.tgz", + "integrity": "sha512-vTNlKq+N6CK/8UktsrFuc+/7NlEYVxgaEgRXVUVK258Z5ymho29skzW1sutgYjqNnquGwVUObAaxae8rZ6YMhg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.53.3.tgz", + "integrity": "sha512-RGrFLWgMhSxRs/EWJMIFM1O5Mzuz3Xy3/mnxJp/5cVhZ2XoCAxJnmNsEyeMJtpK+wu0FJFWz+QF4mjCA7AUQ3w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-musl": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.53.3.tgz", + "integrity": "sha512-kASyvfBEWYPEwe0Qv4nfu6pNkITLTb32p4yTgzFCocHnJLAHs+9LjUu9ONIhvfT/5lv4YS5muBHyuV84epBo/A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loong64-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loong64-gnu/-/rollup-linux-loong64-gnu-4.53.3.tgz", + "integrity": "sha512-JiuKcp2teLJwQ7vkJ95EwESWkNRFJD7TQgYmCnrPtlu50b4XvT5MOmurWNrCj3IFdyjBQ5p9vnrX4JM6I8OE7g==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-gnu/-/rollup-linux-ppc64-gnu-4.53.3.tgz", + "integrity": "sha512-EoGSa8nd6d3T7zLuqdojxC20oBfNT8nexBbB/rkxgKj5T5vhpAQKKnD+h3UkoMuTyXkP5jTjK/ccNRmQrPNDuw==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.53.3.tgz", + "integrity": "sha512-4s+Wped2IHXHPnAEbIB0YWBv7SDohqxobiiPA1FIWZpX+w9o2i4LezzH/NkFUl8LRci/8udci6cLq+jJQlh+0g==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-musl": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-musl/-/rollup-linux-riscv64-musl-4.53.3.tgz", + "integrity": "sha512-68k2g7+0vs2u9CxDt5ktXTngsxOQkSEV/xBbwlqYcUrAVh6P9EgMZvFsnHy4SEiUl46Xf0IObWVbMvPrr2gw8A==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-s390x-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.53.3.tgz", + "integrity": "sha512-VYsFMpULAz87ZW6BVYw3I6sWesGpsP9OPcyKe8ofdg9LHxSbRMd7zrVrr5xi/3kMZtpWL/wC+UIJWJYVX5uTKg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.53.3.tgz", + "integrity": "sha512-3EhFi1FU6YL8HTUJZ51imGJWEX//ajQPfqWLI3BQq4TlvHy4X0MOr5q3D2Zof/ka0d5FNdPwZXm3Yyib/UEd+w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-musl": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.53.3.tgz", + "integrity": "sha512-eoROhjcc6HbZCJr+tvVT8X4fW3/5g/WkGvvmwz/88sDtSJzO7r/blvoBDgISDiCjDRZmHpwud7h+6Q9JxFwq1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-openharmony-arm64": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-openharmony-arm64/-/rollup-openharmony-arm64-4.53.3.tgz", + "integrity": "sha512-OueLAWgrNSPGAdUdIjSWXw+u/02BRTcnfw9PN41D2vq/JSEPnJnVuBgw18VkN8wcd4fjUs+jFHVM4t9+kBSNLw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ] + }, + "node_modules/@rollup/rollup-win32-arm64-msvc": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.53.3.tgz", + "integrity": "sha512-GOFuKpsxR/whszbF/bzydebLiXIHSgsEUp6M0JI8dWvi+fFa1TD6YQa4aSZHtpmh2/uAlj/Dy+nmby3TJ3pkTw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-ia32-msvc": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.53.3.tgz", + "integrity": "sha512-iah+THLcBJdpfZ1TstDFbKNznlzoxa8fmnFYK4V67HvmuNYkVdAywJSoteUszvBQ9/HqN2+9AZghbajMsFT+oA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-gnu": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-gnu/-/rollup-win32-x64-gnu-4.53.3.tgz", + "integrity": "sha512-J9QDiOIZlZLdcot5NXEepDkstocktoVjkaKUtqzgzpt2yWjGlbYiKyp05rWwk4nypbYUNoFAztEgixoLaSETkg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-msvc": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.53.3.tgz", + "integrity": "sha512-UhTd8u31dXadv0MopwGgNOBpUVROFKWVQgAg5N1ESyCz8AuBcMqm4AuTjrwgQKGDfoFuz02EuMRHQIw/frmYKQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@standard-schema/spec": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.0.0.tgz", + "integrity": "sha512-m2bOd0f2RT9k8QJx1JN85cZYyH1RqFBdlwtkSlf4tBDYLCiiZnv1fIIwacK6cqwXavOydf0NPToMQgpKq+dVlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@sveltejs/acorn-typescript": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/@sveltejs/acorn-typescript/-/acorn-typescript-1.0.8.tgz", + "integrity": "sha512-esgN+54+q0NjB0Y/4BomT9samII7jGwNy/2a3wNZbT2A2RpmXsXwUt24LvLhx6jUq2gVk4cWEvcRO6MFQbOfNA==", + "license": "MIT", + "peerDependencies": { + "acorn": "^8.9.0" + } + }, + "node_modules/@sveltejs/adapter-static": { + "version": "3.0.10", + "resolved": "https://registry.npmjs.org/@sveltejs/adapter-static/-/adapter-static-3.0.10.tgz", + "integrity": "sha512-7D9lYFWJmB7zxZyTE/qxjksvMqzMuYrrsyh1f4AlZqeZeACPRySjbC3aFiY55wb1tWUaKOQG9PVbm74JcN2Iew==", + "dev": true, + "license": "MIT", + "peerDependencies": { + "@sveltejs/kit": "^2.0.0" + } + }, + "node_modules/@sveltejs/kit": { + "version": "2.49.0", + "resolved": "https://registry.npmjs.org/@sveltejs/kit/-/kit-2.49.0.tgz", + "integrity": "sha512-oH8tXw7EZnie8FdOWYrF7Yn4IKrqTFHhXvl8YxXxbKwTMcD/5NNCryUSEXRk2ZR4ojnub0P8rNrsVGHXWqIDtA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@standard-schema/spec": "^1.0.0", + "@sveltejs/acorn-typescript": "^1.0.5", + "@types/cookie": "^0.6.0", + "acorn": "^8.14.1", + "cookie": "^0.6.0", + "devalue": "^5.3.2", + "esm-env": "^1.2.2", + "kleur": "^4.1.5", + "magic-string": "^0.30.5", + "mrmime": "^2.0.0", + "sade": "^1.8.1", + "set-cookie-parser": "^2.6.0", + "sirv": "^3.0.0" + }, + "bin": { + "svelte-kit": "svelte-kit.js" + }, + "engines": { + "node": ">=18.13" + }, + "peerDependencies": { + "@opentelemetry/api": "^1.0.0", + "@sveltejs/vite-plugin-svelte": "^3.0.0 || ^4.0.0-next.1 || ^5.0.0 || ^6.0.0-next.0", + "svelte": "^4.0.0 || ^5.0.0-next.0", + "vite": "^5.0.3 || ^6.0.0 || ^7.0.0-beta.0" + }, + "peerDependenciesMeta": { + "@opentelemetry/api": { + "optional": true + } + } + }, + "node_modules/@sveltejs/vite-plugin-svelte": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/@sveltejs/vite-plugin-svelte/-/vite-plugin-svelte-5.1.1.tgz", + "integrity": "sha512-Y1Cs7hhTc+a5E9Va/xwKlAJoariQyHY+5zBgCZg4PFWNYQ1nMN9sjK1zhw1gK69DuqVP++sht/1GZg1aRwmAXQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@sveltejs/vite-plugin-svelte-inspector": "^4.0.1", + "debug": "^4.4.1", + "deepmerge": "^4.3.1", + "kleur": "^4.1.5", + "magic-string": "^0.30.17", + "vitefu": "^1.0.6" + }, + "engines": { + "node": "^18.0.0 || ^20.0.0 || >=22" + }, + "peerDependencies": { + "svelte": "^5.0.0", + "vite": "^6.0.0" + } + }, + "node_modules/@sveltejs/vite-plugin-svelte-inspector": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/@sveltejs/vite-plugin-svelte-inspector/-/vite-plugin-svelte-inspector-4.0.1.tgz", + "integrity": "sha512-J/Nmb2Q2y7mck2hyCX4ckVHcR5tu2J+MtBEQqpDrrgELZ2uvraQcK/ioCV61AqkdXFgriksOKIceDcQmqnGhVw==", + "dev": true, + "license": "MIT", + "dependencies": { + "debug": "^4.3.7" + }, + "engines": { + "node": "^18.0.0 || ^20.0.0 || >=22" + }, + "peerDependencies": { + "@sveltejs/vite-plugin-svelte": "^5.0.0", + "svelte": "^5.0.0", + "vite": "^6.0.0" + } + }, + "node_modules/@tailwindcss/node": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/node/-/node-4.1.17.tgz", + "integrity": "sha512-csIkHIgLb3JisEFQ0vxr2Y57GUNYh447C8xzwj89U/8fdW8LhProdxvnVH6U8M2Y73QKiTIH+LWbK3V2BBZsAg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/remapping": "^2.3.4", + "enhanced-resolve": "^5.18.3", + "jiti": "^2.6.1", + "lightningcss": "1.30.2", + "magic-string": "^0.30.21", + "source-map-js": "^1.2.1", + "tailwindcss": "4.1.17" + } + }, + "node_modules/@tailwindcss/oxide": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide/-/oxide-4.1.17.tgz", + "integrity": "sha512-F0F7d01fmkQhsTjXezGBLdrl1KresJTcI3DB8EkScCldyKp3Msz4hub4uyYaVnk88BAS1g5DQjjF6F5qczheLA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 10" + }, + "optionalDependencies": { + "@tailwindcss/oxide-android-arm64": "4.1.17", + "@tailwindcss/oxide-darwin-arm64": "4.1.17", + "@tailwindcss/oxide-darwin-x64": "4.1.17", + "@tailwindcss/oxide-freebsd-x64": "4.1.17", + "@tailwindcss/oxide-linux-arm-gnueabihf": "4.1.17", + "@tailwindcss/oxide-linux-arm64-gnu": "4.1.17", + "@tailwindcss/oxide-linux-arm64-musl": "4.1.17", + "@tailwindcss/oxide-linux-x64-gnu": "4.1.17", + "@tailwindcss/oxide-linux-x64-musl": "4.1.17", + "@tailwindcss/oxide-wasm32-wasi": "4.1.17", + "@tailwindcss/oxide-win32-arm64-msvc": "4.1.17", + "@tailwindcss/oxide-win32-x64-msvc": "4.1.17" + } + }, + "node_modules/@tailwindcss/oxide-android-arm64": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-android-arm64/-/oxide-android-arm64-4.1.17.tgz", + "integrity": "sha512-BMqpkJHgOZ5z78qqiGE6ZIRExyaHyuxjgrJ6eBO5+hfrfGkuya0lYfw8fRHG77gdTjWkNWEEm+qeG2cDMxArLQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-darwin-arm64": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-arm64/-/oxide-darwin-arm64-4.1.17.tgz", + "integrity": "sha512-EquyumkQweUBNk1zGEU/wfZo2qkp/nQKRZM8bUYO0J+Lums5+wl2CcG1f9BgAjn/u9pJzdYddHWBiFXJTcxmOg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-darwin-x64": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-x64/-/oxide-darwin-x64-4.1.17.tgz", + "integrity": "sha512-gdhEPLzke2Pog8s12oADwYu0IAw04Y2tlmgVzIN0+046ytcgx8uZmCzEg4VcQh+AHKiS7xaL8kGo/QTiNEGRog==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-freebsd-x64": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-freebsd-x64/-/oxide-freebsd-x64-4.1.17.tgz", + "integrity": "sha512-hxGS81KskMxML9DXsaXT1H0DyA+ZBIbyG/sSAjWNe2EDl7TkPOBI42GBV3u38itzGUOmFfCzk1iAjDXds8Oh0g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-linux-arm-gnueabihf": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm-gnueabihf/-/oxide-linux-arm-gnueabihf-4.1.17.tgz", + "integrity": "sha512-k7jWk5E3ldAdw0cNglhjSgv501u7yrMf8oeZ0cElhxU6Y2o7f8yqelOp3fhf7evjIS6ujTI3U8pKUXV2I4iXHQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-linux-arm64-gnu": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-gnu/-/oxide-linux-arm64-gnu-4.1.17.tgz", + "integrity": "sha512-HVDOm/mxK6+TbARwdW17WrgDYEGzmoYayrCgmLEw7FxTPLcp/glBisuyWkFz/jb7ZfiAXAXUACfyItn+nTgsdQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-linux-arm64-musl": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-musl/-/oxide-linux-arm64-musl-4.1.17.tgz", + "integrity": "sha512-HvZLfGr42i5anKtIeQzxdkw/wPqIbpeZqe7vd3V9vI3RQxe3xU1fLjss0TjyhxWcBaipk7NYwSrwTwK1hJARMg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-linux-x64-gnu": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-gnu/-/oxide-linux-x64-gnu-4.1.17.tgz", + "integrity": "sha512-M3XZuORCGB7VPOEDH+nzpJ21XPvK5PyjlkSFkFziNHGLc5d6g3di2McAAblmaSUNl8IOmzYwLx9NsE7bplNkwQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-linux-x64-musl": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-musl/-/oxide-linux-x64-musl-4.1.17.tgz", + "integrity": "sha512-k7f+pf9eXLEey4pBlw+8dgfJHY4PZ5qOUFDyNf7SI6lHjQ9Zt7+NcscjpwdCEbYi6FI5c2KDTDWyf2iHcCSyyQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-wasm32-wasi/-/oxide-wasm32-wasi-4.1.17.tgz", + "integrity": "sha512-cEytGqSSoy7zK4JRWiTCx43FsKP/zGr0CsuMawhH67ONlH+T79VteQeJQRO/X7L0juEUA8ZyuYikcRBf0vsxhg==", + "bundleDependencies": [ + "@napi-rs/wasm-runtime", + "@emnapi/core", + "@emnapi/runtime", + "@tybys/wasm-util", + "@emnapi/wasi-threads", + "tslib" + ], + "cpu": [ + "wasm32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/core": "^1.6.0", + "@emnapi/runtime": "^1.6.0", + "@emnapi/wasi-threads": "^1.1.0", + "@napi-rs/wasm-runtime": "^1.0.7", + "@tybys/wasm-util": "^0.10.1", + "tslib": "^2.4.0" + }, + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/@tailwindcss/oxide-win32-arm64-msvc": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.1.17.tgz", + "integrity": "sha512-JU5AHr7gKbZlOGvMdb4722/0aYbU+tN6lv1kONx0JK2cGsh7g148zVWLM0IKR3NeKLv+L90chBVYcJ8uJWbC9A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/oxide-win32-x64-msvc": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-x64-msvc/-/oxide-win32-x64-msvc-4.1.17.tgz", + "integrity": "sha512-SKWM4waLuqx0IH+FMDUw6R66Hu4OuTALFgnleKbqhgGU30DY20NORZMZUKgLRjQXNN2TLzKvh48QXTig4h4bGw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tailwindcss/vite": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/@tailwindcss/vite/-/vite-4.1.17.tgz", + "integrity": "sha512-4+9w8ZHOiGnpcGI6z1TVVfWaX/koK7fKeSYF3qlYg2xpBtbteP2ddBxiarL+HVgfSJGeK5RIxRQmKm4rTJJAwA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@tailwindcss/node": "4.1.17", + "@tailwindcss/oxide": "4.1.17", + "tailwindcss": "4.1.17" + }, + "peerDependencies": { + "vite": "^5.2.0 || ^6 || ^7" + } + }, + "node_modules/@types/cookie": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/@types/cookie/-/cookie-0.6.0.tgz", + "integrity": "sha512-4Kh9a6B2bQciAhf7FSuMRRkUWecJgJu9nPnx3yzpsfXX/c50REIqpHY4C82bXP90qrLtXtkDxTZosYO3UpOwlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3": { + "version": "7.4.3", + "resolved": "https://registry.npmjs.org/@types/d3/-/d3-7.4.3.tgz", + "integrity": "sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-array": "*", + "@types/d3-axis": "*", + "@types/d3-brush": "*", + "@types/d3-chord": "*", + "@types/d3-color": "*", + "@types/d3-contour": "*", + "@types/d3-delaunay": "*", + "@types/d3-dispatch": "*", + "@types/d3-drag": "*", + "@types/d3-dsv": "*", + "@types/d3-ease": "*", + "@types/d3-fetch": "*", + "@types/d3-force": "*", + "@types/d3-format": "*", + "@types/d3-geo": "*", + "@types/d3-hierarchy": "*", + "@types/d3-interpolate": "*", + "@types/d3-path": "*", + "@types/d3-polygon": "*", + "@types/d3-quadtree": "*", + "@types/d3-random": "*", + "@types/d3-scale": "*", + "@types/d3-scale-chromatic": "*", + "@types/d3-selection": "*", + "@types/d3-shape": "*", + "@types/d3-time": "*", + "@types/d3-time-format": "*", + "@types/d3-timer": "*", + "@types/d3-transition": "*", + "@types/d3-zoom": "*" + } + }, + "node_modules/@types/d3-array": { + "version": "3.2.2", + "resolved": "https://registry.npmjs.org/@types/d3-array/-/d3-array-3.2.2.tgz", + "integrity": "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-axis": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-axis/-/d3-axis-3.0.6.tgz", + "integrity": "sha512-pYeijfZuBd87T0hGn0FO1vQ/cgLk6E1ALJjfkC0oJ8cbwkZl3TpgS8bVBLZN+2jjGgg38epgxb2zmoGtSfvgMw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-brush": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-brush/-/d3-brush-3.0.6.tgz", + "integrity": "sha512-nH60IZNNxEcrh6L1ZSMNA28rj27ut/2ZmI3r96Zd+1jrZD++zD3LsMIjWlvg4AYrHn/Pqz4CF3veCxGjtbqt7A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-chord": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-chord/-/d3-chord-3.0.6.tgz", + "integrity": "sha512-LFYWWd8nwfwEmTZG9PfQxd17HbNPksHBiJHaKuY1XeqscXacsS2tyoo6OdRsjf+NQYeB6XrNL3a25E3gH69lcg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-color": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-3.1.3.tgz", + "integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-contour": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-contour/-/d3-contour-3.0.6.tgz", + "integrity": "sha512-BjzLgXGnCWjUSYGfH1cpdo41/hgdWETu4YxpezoztawmqsvCeep+8QGfiY6YbDvfgHz/DkjeIkkZVJavB4a3rg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-array": "*", + "@types/geojson": "*" + } + }, + "node_modules/@types/d3-delaunay": { + "version": "6.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-delaunay/-/d3-delaunay-6.0.4.tgz", + "integrity": "sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-dispatch": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/@types/d3-dispatch/-/d3-dispatch-3.0.7.tgz", + "integrity": "sha512-5o9OIAdKkhN1QItV2oqaE5KMIiXAvDWBDPrD85e58Qlz1c1kI/J0NcqbEG88CoTwJrYe7ntUCVfeUl2UJKbWgA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-drag": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/@types/d3-drag/-/d3-drag-3.0.7.tgz", + "integrity": "sha512-HE3jVKlzU9AaMazNufooRJ5ZpWmLIoc90A37WU2JMmeq28w1FQqCZswHZ3xR+SuxYftzHq6WU6KJHvqxKzTxxQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-dsv": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/@types/d3-dsv/-/d3-dsv-3.0.7.tgz", + "integrity": "sha512-n6QBF9/+XASqcKK6waudgL0pf/S5XHPPI8APyMLLUHd8NqouBGLsU8MgtO7NINGtPBtk9Kko/W4ea0oAspwh9g==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-ease": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz", + "integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-fetch": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/@types/d3-fetch/-/d3-fetch-3.0.7.tgz", + "integrity": "sha512-fTAfNmxSb9SOWNB9IoG5c8Hg6R+AzUHDRlsXsDZsNp6sxAEOP0tkP3gKkNSO/qmHPoBFTxNrjDprVHDQDvo5aA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-dsv": "*" + } + }, + "node_modules/@types/d3-force": { + "version": "3.0.10", + "resolved": "https://registry.npmjs.org/@types/d3-force/-/d3-force-3.0.10.tgz", + "integrity": "sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-format": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-format/-/d3-format-3.0.4.tgz", + "integrity": "sha512-fALi2aI6shfg7vM5KiR1wNJnZ7r6UuggVqtDA+xiEdPZQwy/trcQaHnwShLuLdta2rTymCNpxYTiMZX/e09F4g==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-geo": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@types/d3-geo/-/d3-geo-3.1.0.tgz", + "integrity": "sha512-856sckF0oP/diXtS4jNsiQw/UuK5fQG8l/a9VVLeSouf1/PPbBE1i1W852zVwKwYCBkFJJB7nCFTbk6UMEXBOQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/geojson": "*" + } + }, + "node_modules/@types/d3-hierarchy": { + "version": "3.1.7", + "resolved": "https://registry.npmjs.org/@types/d3-hierarchy/-/d3-hierarchy-3.1.7.tgz", + "integrity": "sha512-tJFtNoYBtRtkNysX1Xq4sxtjK8YgoWUNpIiUee0/jHGRwqvzYxkq0hGVbbOGSz+JgFxxRu4K8nb3YpG3CMARtg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-interpolate": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz", + "integrity": "sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-color": "*" + } + }, + "node_modules/@types/d3-path": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/@types/d3-path/-/d3-path-3.1.1.tgz", + "integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-polygon": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-polygon/-/d3-polygon-3.0.2.tgz", + "integrity": "sha512-ZuWOtMaHCkN9xoeEMr1ubW2nGWsp4nIql+OPQRstu4ypeZ+zk3YKqQT0CXVe/PYqrKpZAi+J9mTs05TKwjXSRA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-quadtree": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-quadtree/-/d3-quadtree-3.0.6.tgz", + "integrity": "sha512-oUzyO1/Zm6rsxKRHA1vH0NEDG58HrT5icx/azi9MF1TWdtttWl0UIUsjEQBBh+SIkrpd21ZjEv7ptxWys1ncsg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-random": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/@types/d3-random/-/d3-random-3.0.3.tgz", + "integrity": "sha512-Imagg1vJ3y76Y2ea0871wpabqp613+8/r0mCLEBfdtqC7xMSfj9idOnmBYyMoULfHePJyxMAw3nWhJxzc+LFwQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-scale": { + "version": "4.0.9", + "resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.9.tgz", + "integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-time": "*" + } + }, + "node_modules/@types/d3-scale-chromatic": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@types/d3-scale-chromatic/-/d3-scale-chromatic-3.1.0.tgz", + "integrity": "sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-selection": { + "version": "3.0.11", + "resolved": "https://registry.npmjs.org/@types/d3-selection/-/d3-selection-3.0.11.tgz", + "integrity": "sha512-bhAXu23DJWsrI45xafYpkQ4NtcKMwWnAC/vKrd2l+nxMFuvOT3XMYTIj2opv8vq8AO5Yh7Qac/nSeP/3zjTK0w==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-shape": { + "version": "3.1.7", + "resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.7.tgz", + "integrity": "sha512-VLvUQ33C+3J+8p+Daf+nYSOsjB4GXp19/S/aGo60m9h1v6XaxjiT82lKVWJCfzhtuZ3yD7i/TPeC/fuKLLOSmg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-path": "*" + } + }, + "node_modules/@types/d3-time": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.4.tgz", + "integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-time-format": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/@types/d3-time-format/-/d3-time-format-4.0.3.tgz", + "integrity": "sha512-5xg9rC+wWL8kdDj153qZcsJ0FWiFt0J5RB6LYUNZjwSnesfblqrI/bJ1wBdJ8OQfncgbJG5+2F+qfqnqyzYxyg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-timer": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-timer/-/d3-timer-3.0.2.tgz", + "integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-transition": { + "version": "3.0.9", + "resolved": "https://registry.npmjs.org/@types/d3-transition/-/d3-transition-3.0.9.tgz", + "integrity": "sha512-uZS5shfxzO3rGlu0cC3bjmMFKsXv+SmZZcgp0KD22ts4uGXp5EVYGzu/0YdwZeKmddhcAccYtREJKkPfXkZuCg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-zoom": { + "version": "3.0.8", + "resolved": "https://registry.npmjs.org/@types/d3-zoom/-/d3-zoom-3.0.8.tgz", + "integrity": "sha512-iqMC4/YlFCSlO8+2Ii1GGGliCAY4XdeG748w5vQUbevlbDu0zSjH/+jojorQVBK/se0j6DUFNPBGSqD3YWYnDw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-interpolate": "*", + "@types/d3-selection": "*" + } + }, + "node_modules/@types/estree": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz", + "integrity": "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==", + "license": "MIT" + }, + "node_modules/@types/geojson": { + "version": "7946.0.16", + "resolved": "https://registry.npmjs.org/@types/geojson/-/geojson-7946.0.16.tgz", + "integrity": "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/node": { + "version": "22.19.1", + "resolved": "https://registry.npmjs.org/@types/node/-/node-22.19.1.tgz", + "integrity": "sha512-LCCV0HdSZZZb34qifBsyWlUmok6W7ouER+oQIGBScS8EsZsQbrtFTUrDX4hOl+CS6p7cnNC4td+qrSVGSCTUfQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/acorn": { + "version": "8.15.0", + "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz", + "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==", + "license": "MIT", + "bin": { + "acorn": "bin/acorn" + }, + "engines": { + "node": ">=0.4.0" + } + }, + "node_modules/aria-query": { + "version": "5.3.2", + "resolved": "https://registry.npmjs.org/aria-query/-/aria-query-5.3.2.tgz", + "integrity": "sha512-COROpnaoap1E2F000S62r6A60uHZnmlvomhfyT2DlTcrY1OrBKn2UhH7qn5wTC9zMvD0AY7csdPSNwKP+7WiQw==", + "license": "Apache-2.0", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/axobject-query": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/axobject-query/-/axobject-query-4.1.0.tgz", + "integrity": "sha512-qIj0G9wZbMGNLjLmg1PT6v2mE9AH2zlnADJD/2tC6E00hgmhUOfEB6greHPAfLRSufHqROIUTkw6E+M3lH0PTQ==", + "license": "Apache-2.0", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/chokidar": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-4.0.3.tgz", + "integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==", + "dev": true, + "license": "MIT", + "dependencies": { + "readdirp": "^4.0.1" + }, + "engines": { + "node": ">= 14.16.0" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/clsx": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz", + "integrity": "sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==", + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/commander": { + "version": "7.2.0", + "resolved": "https://registry.npmjs.org/commander/-/commander-7.2.0.tgz", + "integrity": "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 10" + } + }, + "node_modules/cookie": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.6.0.tgz", + "integrity": "sha512-U71cyTamuh1CRNCfpGY6to28lxvNwPG4Guz/EVjgf3Jmzv0vlDp1atT9eS5dDjMYHucpHbWns6Lwf3BKz6svdw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/d3": { + "version": "7.9.0", + "resolved": "https://registry.npmjs.org/d3/-/d3-7.9.0.tgz", + "integrity": "sha512-e1U46jVP+w7Iut8Jt8ri1YsPOvFpg46k+K8TpCb0P+zjCkjkPnV7WzfDJzMHy1LnA+wj5pLT1wjO901gLXeEhA==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "3", + "d3-axis": "3", + "d3-brush": "3", + "d3-chord": "3", + "d3-color": "3", + "d3-contour": "4", + "d3-delaunay": "6", + "d3-dispatch": "3", + "d3-drag": "3", + "d3-dsv": "3", + "d3-ease": "3", + "d3-fetch": "3", + "d3-force": "3", + "d3-format": "3", + "d3-geo": "3", + "d3-hierarchy": "3", + "d3-interpolate": "3", + "d3-path": "3", + "d3-polygon": "3", + "d3-quadtree": "3", + "d3-random": "3", + "d3-scale": "4", + "d3-scale-chromatic": "3", + "d3-selection": "3", + "d3-shape": "3", + "d3-time": "3", + "d3-time-format": "4", + "d3-timer": "3", + "d3-transition": "3", + "d3-zoom": "3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-array": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/d3-array/-/d3-array-3.2.4.tgz", + "integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==", + "dev": true, + "license": "ISC", + "dependencies": { + "internmap": "1 - 2" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-axis": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/d3-axis/-/d3-axis-3.0.0.tgz", + "integrity": "sha512-IH5tgjV4jE/GhHkRV0HiVYPDtvfjHQlQfJHs0usq7M30XcSBvOotpmH1IgkcXsO/5gEQZD43B//fc7SRT5S+xw==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-brush": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/d3-brush/-/d3-brush-3.0.0.tgz", + "integrity": "sha512-ALnjWlVYkXsVIGlOsuWH1+3udkYFI48Ljihfnh8FZPF2QS9o+PzGLBslO0PjzVoHLZ2KCVgAM8NVkXPJB2aNnQ==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-drag": "2 - 3", + "d3-interpolate": "1 - 3", + "d3-selection": "3", + "d3-transition": "3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-chord": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-chord/-/d3-chord-3.0.1.tgz", + "integrity": "sha512-VE5S6TNa+j8msksl7HwjxMHDM2yNK3XCkusIlpX5kwauBfXuyLAtNg9jCp/iHH61tgI4sb6R/EIMWCqEIdjT/g==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-path": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-color": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz", + "integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-contour": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/d3-contour/-/d3-contour-4.0.2.tgz", + "integrity": "sha512-4EzFTRIikzs47RGmdxbeUvLWtGedDUNkTcmzoeyg4sP/dvCexO47AaQL7VKy/gul85TOxw+IBgA8US2xwbToNA==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "^3.2.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-delaunay": { + "version": "6.0.4", + "resolved": "https://registry.npmjs.org/d3-delaunay/-/d3-delaunay-6.0.4.tgz", + "integrity": "sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==", + "dev": true, + "license": "ISC", + "dependencies": { + "delaunator": "5" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-dispatch": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-dispatch/-/d3-dispatch-3.0.1.tgz", + "integrity": "sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-drag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/d3-drag/-/d3-drag-3.0.0.tgz", + "integrity": "sha512-pWbUJLdETVA8lQNJecMxoXfH6x+mO2UQo8rSmZ+QqxcbyA3hfeprFgIT//HW2nlHChWeIIMwS2Fq+gEARkhTkg==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-selection": "3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-dsv": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-dsv/-/d3-dsv-3.0.1.tgz", + "integrity": "sha512-UG6OvdI5afDIFP9w4G0mNq50dSOsXHJaRE8arAS5o9ApWnIElp8GZw1Dun8vP8OyHOZ/QJUKUJwxiiCCnUwm+Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "commander": "7", + "iconv-lite": "0.6", + "rw": "1" + }, + "bin": { + "csv2json": "bin/dsv2json.js", + "csv2tsv": "bin/dsv2dsv.js", + "dsv2dsv": "bin/dsv2dsv.js", + "dsv2json": "bin/dsv2json.js", + "json2csv": "bin/json2dsv.js", + "json2dsv": "bin/json2dsv.js", + "json2tsv": "bin/json2dsv.js", + "tsv2csv": "bin/dsv2dsv.js", + "tsv2json": "bin/dsv2json.js" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-ease": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz", + "integrity": "sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-fetch": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-fetch/-/d3-fetch-3.0.1.tgz", + "integrity": "sha512-kpkQIM20n3oLVBKGg6oHrUchHM3xODkTzjMoj7aWQFq5QEM+R6E4WkzT5+tojDY7yjez8KgCBRoj4aEr99Fdqw==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dsv": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-force": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/d3-force/-/d3-force-3.0.0.tgz", + "integrity": "sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-quadtree": "1 - 3", + "d3-timer": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-format": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-format/-/d3-format-3.1.0.tgz", + "integrity": "sha512-YyUI6AEuY/Wpt8KWLgZHsIU86atmikuoOmCfommt0LYHiQSPjvX2AcFc38PX0CBpr2RCyZhjex+NS/LPOv6YqA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-geo": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/d3-geo/-/d3-geo-3.1.1.tgz", + "integrity": "sha512-637ln3gXKXOwhalDzinUgY83KzNWZRKbYubaG+fGVuc/dxO64RRljtCTnf5ecMyE1RIdtqpkVcq0IbtU2S8j2Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "2.5.0 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-hierarchy": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/d3-hierarchy/-/d3-hierarchy-3.1.2.tgz", + "integrity": "sha512-FX/9frcub54beBdugHjDCdikxThEqjnR93Qt7PvQTOHxyiNCAlvMrHhclk3cD5VeAaq9fxmfRp+CnWw9rEMBuA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-interpolate": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-3.0.1.tgz", + "integrity": "sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-path": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-path/-/d3-path-3.1.0.tgz", + "integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-polygon": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-3.0.1.tgz", + "integrity": "sha512-3vbA7vXYwfe1SYhED++fPUQlWSYTTGmFmQiany/gdbiWgU/iEyQzyymwL9SkJjFFuCS4902BSzewVGsHHmHtXg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-quadtree": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-quadtree/-/d3-quadtree-3.0.1.tgz", + "integrity": "sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-random": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-random/-/d3-random-3.0.1.tgz", + "integrity": "sha512-FXMe9GfxTxqd5D6jFsQ+DJ8BJS4E/fT5mqqdjovykEB2oFbTMDVdg1MGFxfQW+FBOGoB++k8swBrgwSHT1cUXQ==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-scale": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/d3-scale/-/d3-scale-4.0.2.tgz", + "integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "2.10.0 - 3", + "d3-format": "1 - 3", + "d3-interpolate": "1.2.0 - 3", + "d3-time": "2.1.1 - 3", + "d3-time-format": "2 - 4" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-scale-chromatic": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-scale-chromatic/-/d3-scale-chromatic-3.1.0.tgz", + "integrity": "sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3", + "d3-interpolate": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-selection": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/d3-selection/-/d3-selection-3.0.0.tgz", + "integrity": "sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-shape": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz", + "integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-path": "^3.1.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-time": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-time/-/d3-time-3.1.0.tgz", + "integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "2 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-time-format": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/d3-time-format/-/d3-time-format-4.1.0.tgz", + "integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-time": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-timer": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-timer/-/d3-timer-3.0.1.tgz", + "integrity": "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-transition": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-transition/-/d3-transition-3.0.1.tgz", + "integrity": "sha512-ApKvfjsSR6tg06xrL434C0WydLr7JewBB3V+/39RMHsaXTOG0zmt/OAXeng5M5LBm0ojmxJrpomQVZ1aPvBL4w==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3", + "d3-dispatch": "1 - 3", + "d3-ease": "1 - 3", + "d3-interpolate": "1 - 3", + "d3-timer": "1 - 3" + }, + "engines": { + "node": ">=12" + }, + "peerDependencies": { + "d3-selection": "2 - 3" + } + }, + "node_modules/d3-zoom": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/d3-zoom/-/d3-zoom-3.0.0.tgz", + "integrity": "sha512-b8AmV3kfQaqWAuacbPuNbL6vahnOJflOhexLzMMNLga62+/nh0JzvJ0aO/5a5MVgUFGS7Hu1P9P03o3fJkDCyw==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-drag": "2 - 3", + "d3-interpolate": "1 - 3", + "d3-selection": "2 - 3", + "d3-transition": "2 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/deepmerge": { + "version": "4.3.1", + "resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-4.3.1.tgz", + "integrity": "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/delaunator": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/delaunator/-/delaunator-5.0.1.tgz", + "integrity": "sha512-8nvh+XBe96aCESrGOqMp/84b13H9cdKbG5P2ejQCh4d4sK9RL4371qou9drQjMhvnPmhWl5hnmqbEE0fXr9Xnw==", + "dev": true, + "license": "ISC", + "dependencies": { + "robust-predicates": "^3.0.2" + } + }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=8" + } + }, + "node_modules/devalue": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/devalue/-/devalue-5.5.0.tgz", + "integrity": "sha512-69sM5yrHfFLJt0AZ9QqZXGCPfJ7fQjvpln3Rq5+PS03LD32Ost1Q9N+eEnaQwGRIriKkMImXD56ocjQmfjbV3w==", + "license": "MIT" + }, + "node_modules/enhanced-resolve": { + "version": "5.18.3", + "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.18.3.tgz", + "integrity": "sha512-d4lC8xfavMeBjzGr2vECC3fsGXziXZQyJxD868h2M/mBI3PwAuODxAkLkq5HYuvrPYcUtiLzsTo8U3PgX3Ocww==", + "dev": true, + "license": "MIT", + "dependencies": { + "graceful-fs": "^4.2.4", + "tapable": "^2.2.0" + }, + "engines": { + "node": ">=10.13.0" + } + }, + "node_modules/esbuild": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz", + "integrity": "sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.25.12", + "@esbuild/android-arm": "0.25.12", + "@esbuild/android-arm64": "0.25.12", + "@esbuild/android-x64": "0.25.12", + "@esbuild/darwin-arm64": "0.25.12", + "@esbuild/darwin-x64": "0.25.12", + "@esbuild/freebsd-arm64": "0.25.12", + "@esbuild/freebsd-x64": "0.25.12", + "@esbuild/linux-arm": "0.25.12", + "@esbuild/linux-arm64": "0.25.12", + "@esbuild/linux-ia32": "0.25.12", + "@esbuild/linux-loong64": "0.25.12", + "@esbuild/linux-mips64el": "0.25.12", + "@esbuild/linux-ppc64": "0.25.12", + "@esbuild/linux-riscv64": "0.25.12", + "@esbuild/linux-s390x": "0.25.12", + "@esbuild/linux-x64": "0.25.12", + "@esbuild/netbsd-arm64": "0.25.12", + "@esbuild/netbsd-x64": "0.25.12", + "@esbuild/openbsd-arm64": "0.25.12", + "@esbuild/openbsd-x64": "0.25.12", + "@esbuild/openharmony-arm64": "0.25.12", + "@esbuild/sunos-x64": "0.25.12", + "@esbuild/win32-arm64": "0.25.12", + "@esbuild/win32-ia32": "0.25.12", + "@esbuild/win32-x64": "0.25.12" + } + }, + "node_modules/esm-env": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/esm-env/-/esm-env-1.2.2.tgz", + "integrity": "sha512-Epxrv+Nr/CaL4ZcFGPJIYLWFom+YeV1DqMLHJoEd9SYRxNbaFruBwfEX/kkHUJf55j2+TUbmDcmuilbP1TmXHA==", + "license": "MIT" + }, + "node_modules/esrap": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/esrap/-/esrap-2.2.1.tgz", + "integrity": "sha512-GiYWG34AN/4CUyaWAgunGt0Rxvr1PTMlGC0vvEov/uOQYWne2bpN03Um+k8jT+q3op33mKouP2zeJ6OlM+qeUg==", + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.4.15" + } + }, + "node_modules/fdir": { + "version": "6.5.0", + "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz", + "integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12.0.0" + }, + "peerDependencies": { + "picomatch": "^3 || ^4" + }, + "peerDependenciesMeta": { + "picomatch": { + "optional": true + } + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/graceful-fs": { + "version": "4.2.11", + "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz", + "integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==", + "dev": true, + "license": "ISC" + }, + "node_modules/highlight.js": { + "version": "11.11.1", + "resolved": "https://registry.npmjs.org/highlight.js/-/highlight.js-11.11.1.tgz", + "integrity": "sha512-Xwwo44whKBVCYoliBQwaPvtd/2tYFkRQtXDWj1nackaV2JPXx3L0+Jvd8/qCJ2p+ML0/XVkJ2q+Mr+UVdpJK5w==", + "license": "BSD-3-Clause", + "engines": { + "node": ">=12.0.0" + } + }, + "node_modules/iconv-lite": { + "version": "0.6.3", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz", + "integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==", + "dev": true, + "license": "MIT", + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/inline-style-parser": { + "version": "0.2.7", + "resolved": "https://registry.npmjs.org/inline-style-parser/-/inline-style-parser-0.2.7.tgz", + "integrity": "sha512-Nb2ctOyNR8DqQoR0OwRG95uNWIC0C1lCgf5Naz5H6Ji72KZ8OcFZLz2P5sNgwlyoJ8Yif11oMuYs5pBQa86csA==", + "license": "MIT" + }, + "node_modules/internmap": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz", + "integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/is-reference": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/is-reference/-/is-reference-3.0.3.tgz", + "integrity": "sha512-ixkJoqQvAP88E6wLydLGGqCJsrFUnqoH6HnaczB8XmDH1oaWU+xxdptvikTgaEhtZ53Ky6YXiBuUI2WXLMCwjw==", + "license": "MIT", + "dependencies": { + "@types/estree": "^1.0.6" + } + }, + "node_modules/jiti": { + "version": "2.6.1", + "resolved": "https://registry.npmjs.org/jiti/-/jiti-2.6.1.tgz", + "integrity": "sha512-ekilCSN1jwRvIbgeg/57YFh8qQDNbwDb9xT/qu2DAHbFFZUicIl4ygVaAvzveMhMVr3LnpSKTNnwt8PoOfmKhQ==", + "dev": true, + "license": "MIT", + "bin": { + "jiti": "lib/jiti-cli.mjs" + } + }, + "node_modules/kleur": { + "version": "4.1.5", + "resolved": "https://registry.npmjs.org/kleur/-/kleur-4.1.5.tgz", + "integrity": "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/lightningcss": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.30.2.tgz", + "integrity": "sha512-utfs7Pr5uJyyvDETitgsaqSyjCb2qNRAtuqUeWIAKztsOYdcACf2KtARYXg2pSvhkt+9NfoaNY7fxjl6nuMjIQ==", + "dev": true, + "license": "MPL-2.0", + "dependencies": { + "detect-libc": "^2.0.3" + }, + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + }, + "optionalDependencies": { + "lightningcss-android-arm64": "1.30.2", + "lightningcss-darwin-arm64": "1.30.2", + "lightningcss-darwin-x64": "1.30.2", + "lightningcss-freebsd-x64": "1.30.2", + "lightningcss-linux-arm-gnueabihf": "1.30.2", + "lightningcss-linux-arm64-gnu": "1.30.2", + "lightningcss-linux-arm64-musl": "1.30.2", + "lightningcss-linux-x64-gnu": "1.30.2", + "lightningcss-linux-x64-musl": "1.30.2", + "lightningcss-win32-arm64-msvc": "1.30.2", + "lightningcss-win32-x64-msvc": "1.30.2" + } + }, + "node_modules/lightningcss-android-arm64": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.30.2.tgz", + "integrity": "sha512-BH9sEdOCahSgmkVhBLeU7Hc9DWeZ1Eb6wNS6Da8igvUwAe0sqROHddIlvU06q3WyXVEOYDZ6ykBZQnjTbmo4+A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-darwin-arm64": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.30.2.tgz", + "integrity": "sha512-ylTcDJBN3Hp21TdhRT5zBOIi73P6/W0qwvlFEk22fkdXchtNTOU4Qc37SkzV+EKYxLouZ6M4LG9NfZ1qkhhBWA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-darwin-x64": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.30.2.tgz", + "integrity": "sha512-oBZgKchomuDYxr7ilwLcyms6BCyLn0z8J0+ZZmfpjwg9fRVZIR5/GMXd7r9RH94iDhld3UmSjBM6nXWM2TfZTQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-freebsd-x64": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.30.2.tgz", + "integrity": "sha512-c2bH6xTrf4BDpK8MoGG4Bd6zAMZDAXS569UxCAGcA7IKbHNMlhGQ89eRmvpIUGfKWNVdbhSbkQaWhEoMGmGslA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm-gnueabihf": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.30.2.tgz", + "integrity": "sha512-eVdpxh4wYcm0PofJIZVuYuLiqBIakQ9uFZmipf6LF/HRj5Bgm0eb3qL/mr1smyXIS1twwOxNWndd8z0E374hiA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm64-gnu": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.30.2.tgz", + "integrity": "sha512-UK65WJAbwIJbiBFXpxrbTNArtfuznvxAJw4Q2ZGlU8kPeDIWEX1dg3rn2veBVUylA2Ezg89ktszWbaQnxD/e3A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm64-musl": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.30.2.tgz", + "integrity": "sha512-5Vh9dGeblpTxWHpOx8iauV02popZDsCYMPIgiuw97OJ5uaDsL86cnqSFs5LZkG3ghHoX5isLgWzMs+eD1YzrnA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-x64-gnu": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.30.2.tgz", + "integrity": "sha512-Cfd46gdmj1vQ+lR6VRTTadNHu6ALuw2pKR9lYq4FnhvgBc4zWY1EtZcAc6EffShbb1MFrIPfLDXD6Xprbnni4w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-x64-musl": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.30.2.tgz", + "integrity": "sha512-XJaLUUFXb6/QG2lGIW6aIk6jKdtjtcffUT0NKvIqhSBY3hh9Ch+1LCeH80dR9q9LBjG3ewbDjnumefsLsP6aiA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-win32-arm64-msvc": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.30.2.tgz", + "integrity": "sha512-FZn+vaj7zLv//D/192WFFVA0RgHawIcHqLX9xuWiQt7P0PtdFEVaxgF9rjM/IRYHQXNnk61/H/gb2Ei+kUQ4xQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-win32-x64-msvc": { + "version": "1.30.2", + "resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.30.2.tgz", + "integrity": "sha512-5g1yc73p+iAkid5phb4oVFMB45417DkRevRbt/El/gKXJk4jid+vPFF/AXbxn05Aky8PapwzZrdJShv5C0avjw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/locate-character": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/locate-character/-/locate-character-3.0.0.tgz", + "integrity": "sha512-SW13ws7BjaeJ6p7Q6CO2nchbYEc3X3J6WrmTTDto7yMPqVSZTUyY5Tjbid+Ab8gLnATtygYtiDIJGQRRn2ZOiA==", + "license": "MIT" + }, + "node_modules/magic-string": { + "version": "0.30.21", + "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", + "integrity": "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==", + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.5" + } + }, + "node_modules/mode-watcher": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/mode-watcher/-/mode-watcher-1.1.0.tgz", + "integrity": "sha512-mUT9RRGPDYenk59qJauN1rhsIMKBmWA3xMF+uRwE8MW/tjhaDSCCARqkSuDTq8vr4/2KcAxIGVjACxTjdk5C3g==", + "license": "MIT", + "dependencies": { + "runed": "^0.25.0", + "svelte-toolbelt": "^0.7.1" + }, + "peerDependencies": { + "svelte": "^5.27.0" + } + }, + "node_modules/mri": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/mri/-/mri-1.2.0.tgz", + "integrity": "sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=4" + } + }, + "node_modules/mrmime": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/mrmime/-/mrmime-2.0.1.tgz", + "integrity": "sha512-Y3wQdFg2Va6etvQ5I82yUhGdsKrcYox6p7FfL1LbK2J4V01F9TGlepTIhnK24t7koZibmg82KGglhA1XK5IsLQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=10" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/nanoid": { + "version": "3.3.11", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz", + "integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "bin": { + "nanoid": "bin/nanoid.cjs" + }, + "engines": { + "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" + } + }, + "node_modules/picocolors": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, + "license": "ISC" + }, + "node_modules/picomatch": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz", + "integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, + "node_modules/postcss": { + "version": "8.5.6", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.6.tgz", + "integrity": "sha512-3Ybi1tAuwAP9s0r1UQ2J4n5Y0G05bJkpUIO0/bI9MhwmD70S5aTWbXGBwxHrelT+XM1k6dM0pk+SwNkpTRN7Pg==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/postcss/" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/postcss" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "nanoid": "^3.3.11", + "picocolors": "^1.1.1", + "source-map-js": "^1.2.1" + }, + "engines": { + "node": "^10 || ^12 || >=14" + } + }, + "node_modules/readdirp": { + "version": "4.1.2", + "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-4.1.2.tgz", + "integrity": "sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 14.18.0" + }, + "funding": { + "type": "individual", + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/robust-predicates": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/robust-predicates/-/robust-predicates-3.0.2.tgz", + "integrity": "sha512-IXgzBWvWQwE6PrDI05OvmXUIruQTcoMDzRsOd5CDvHCVLcLHMTSYvOK5Cm46kWqlV3yAbuSpBZdJ5oP5OUoStg==", + "dev": true, + "license": "Unlicense" + }, + "node_modules/rollup": { + "version": "4.53.3", + "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.53.3.tgz", + "integrity": "sha512-w8GmOxZfBmKknvdXU1sdM9NHcoQejwF/4mNgj2JuEEdRaHwwF12K7e9eXn1nLZ07ad+du76mkVsyeb2rKGllsA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "1.0.8" + }, + "bin": { + "rollup": "dist/bin/rollup" + }, + "engines": { + "node": ">=18.0.0", + "npm": ">=8.0.0" + }, + "optionalDependencies": { + "@rollup/rollup-android-arm-eabi": "4.53.3", + "@rollup/rollup-android-arm64": "4.53.3", + "@rollup/rollup-darwin-arm64": "4.53.3", + "@rollup/rollup-darwin-x64": "4.53.3", + "@rollup/rollup-freebsd-arm64": "4.53.3", + "@rollup/rollup-freebsd-x64": "4.53.3", + "@rollup/rollup-linux-arm-gnueabihf": "4.53.3", + "@rollup/rollup-linux-arm-musleabihf": "4.53.3", + "@rollup/rollup-linux-arm64-gnu": "4.53.3", + "@rollup/rollup-linux-arm64-musl": "4.53.3", + "@rollup/rollup-linux-loong64-gnu": "4.53.3", + "@rollup/rollup-linux-ppc64-gnu": "4.53.3", + "@rollup/rollup-linux-riscv64-gnu": "4.53.3", + "@rollup/rollup-linux-riscv64-musl": "4.53.3", + "@rollup/rollup-linux-s390x-gnu": "4.53.3", + "@rollup/rollup-linux-x64-gnu": "4.53.3", + "@rollup/rollup-linux-x64-musl": "4.53.3", + "@rollup/rollup-openharmony-arm64": "4.53.3", + "@rollup/rollup-win32-arm64-msvc": "4.53.3", + "@rollup/rollup-win32-ia32-msvc": "4.53.3", + "@rollup/rollup-win32-x64-gnu": "4.53.3", + "@rollup/rollup-win32-x64-msvc": "4.53.3", + "fsevents": "~2.3.2" + } + }, + "node_modules/runed": { + "version": "0.25.0", + "resolved": "https://registry.npmjs.org/runed/-/runed-0.25.0.tgz", + "integrity": "sha512-7+ma4AG9FT2sWQEA0Egf6mb7PBT2vHyuHail1ie8ropfSjvZGtEAx8YTmUjv/APCsdRRxEVvArNjALk9zFSOrg==", + "funding": [ + "https://github.com/sponsors/huntabyte", + "https://github.com/sponsors/tglide" + ], + "dependencies": { + "esm-env": "^1.0.0" + }, + "peerDependencies": { + "svelte": "^5.7.0" + } + }, + "node_modules/rw": { + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/rw/-/rw-1.3.3.tgz", + "integrity": "sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/sade": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/sade/-/sade-1.8.1.tgz", + "integrity": "sha512-xal3CZX1Xlo/k4ApwCFrHVACi9fBqJ7V+mwhBsuf/1IOKbBy098Fex+Wa/5QMubw09pSZ/u8EY8PWgevJsXp1A==", + "dev": true, + "license": "MIT", + "dependencies": { + "mri": "^1.1.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/safer-buffer": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", + "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", + "dev": true, + "license": "MIT" + }, + "node_modules/set-cookie-parser": { + "version": "2.7.2", + "resolved": "https://registry.npmjs.org/set-cookie-parser/-/set-cookie-parser-2.7.2.tgz", + "integrity": "sha512-oeM1lpU/UvhTxw+g3cIfxXHyJRc/uidd3yK1P242gzHds0udQBYzs3y8j4gCCW+ZJ7ad0yctld8RYO+bdurlvw==", + "dev": true, + "license": "MIT" + }, + "node_modules/sirv": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/sirv/-/sirv-3.0.2.tgz", + "integrity": "sha512-2wcC/oGxHis/BoHkkPwldgiPSYcpZK3JU28WoMVv55yHJgcZ8rlXvuG9iZggz+sU1d4bRgIGASwyWqjxu3FM0g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@polka/url": "^1.0.0-next.24", + "mrmime": "^2.0.0", + "totalist": "^3.0.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/source-map-js": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/style-to-object": { + "version": "1.0.14", + "resolved": "https://registry.npmjs.org/style-to-object/-/style-to-object-1.0.14.tgz", + "integrity": "sha512-LIN7rULI0jBscWQYaSswptyderlarFkjQ+t79nzty8tcIAceVomEVlLzH5VP4Cmsv6MtKhs7qaAiwlcp+Mgaxw==", + "license": "MIT", + "dependencies": { + "inline-style-parser": "0.2.7" + } + }, + "node_modules/svelte": { + "version": "5.45.3", + "resolved": "https://registry.npmjs.org/svelte/-/svelte-5.45.3.tgz", + "integrity": "sha512-ngKXNhNvwPzF43QqEhDOue7TQTrG09em1sd4HBxVF0Wr2gopAmdEWan+rgbdgK4fhBtSOTJO8bYU4chUG7VXZQ==", + "license": "MIT", + "dependencies": { + "@jridgewell/remapping": "^2.3.4", + "@jridgewell/sourcemap-codec": "^1.5.0", + "@sveltejs/acorn-typescript": "^1.0.5", + "@types/estree": "^1.0.5", + "acorn": "^8.12.1", + "aria-query": "^5.3.1", + "axobject-query": "^4.1.0", + "clsx": "^2.1.1", + "devalue": "^5.5.0", + "esm-env": "^1.2.1", + "esrap": "^2.2.0", + "is-reference": "^3.0.3", + "locate-character": "^3.0.0", + "magic-string": "^0.30.11", + "zimmerframe": "^1.1.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/svelte-check": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/svelte-check/-/svelte-check-4.3.4.tgz", + "integrity": "sha512-DVWvxhBrDsd+0hHWKfjP99lsSXASeOhHJYyuKOFYJcP7ThfSCKgjVarE8XfuMWpS5JV3AlDf+iK1YGGo2TACdw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/trace-mapping": "^0.3.25", + "chokidar": "^4.0.1", + "fdir": "^6.2.0", + "picocolors": "^1.0.0", + "sade": "^1.7.4" + }, + "bin": { + "svelte-check": "bin/svelte-check" + }, + "engines": { + "node": ">= 18.0.0" + }, + "peerDependencies": { + "svelte": "^4.0.0 || ^5.0.0-next.0", + "typescript": ">=5.0.0" + } + }, + "node_modules/svelte-toolbelt": { + "version": "0.7.1", + "resolved": "https://registry.npmjs.org/svelte-toolbelt/-/svelte-toolbelt-0.7.1.tgz", + "integrity": "sha512-HcBOcR17Vx9bjaOceUvxkY3nGmbBmCBBbuWLLEWO6jtmWH8f/QoWmbyUfQZrpDINH39en1b8mptfPQT9VKQ1xQ==", + "funding": [ + "https://github.com/sponsors/huntabyte" + ], + "dependencies": { + "clsx": "^2.1.1", + "runed": "^0.23.2", + "style-to-object": "^1.0.8" + }, + "engines": { + "node": ">=18", + "pnpm": ">=8.7.0" + }, + "peerDependencies": { + "svelte": "^5.0.0" + } + }, + "node_modules/svelte-toolbelt/node_modules/runed": { + "version": "0.23.4", + "resolved": "https://registry.npmjs.org/runed/-/runed-0.23.4.tgz", + "integrity": "sha512-9q8oUiBYeXIDLWNK5DfCWlkL0EW3oGbk845VdKlPeia28l751VpfesaB/+7pI6rnbx1I6rqoZ2fZxptOJLxILA==", + "funding": [ + "https://github.com/sponsors/huntabyte", + "https://github.com/sponsors/tglide" + ], + "dependencies": { + "esm-env": "^1.0.0" + }, + "peerDependencies": { + "svelte": "^5.7.0" + } + }, + "node_modules/tailwindcss": { + "version": "4.1.17", + "resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.1.17.tgz", + "integrity": "sha512-j9Ee2YjuQqYT9bbRTfTZht9W/ytp5H+jJpZKiYdP/bpnXARAuELt9ofP0lPnmHjbga7SNQIxdTAXCmtKVYjN+Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/tapable": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/tapable/-/tapable-2.3.0.tgz", + "integrity": "sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/webpack" + } + }, + "node_modules/tinyglobby": { + "version": "0.2.15", + "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz", + "integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "fdir": "^6.5.0", + "picomatch": "^4.0.3" + }, + "engines": { + "node": ">=12.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/SuperchupuDev" + } + }, + "node_modules/totalist": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/totalist/-/totalist-3.0.1.tgz", + "integrity": "sha512-sf4i37nQ2LBx4m3wB74y+ubopq6W/dIzXg0FDGjsYnZHVa1Da8FH853wlL2gtUhg+xJXjfk3kUZS3BRoQeoQBQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/tw-animate-css": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/tw-animate-css/-/tw-animate-css-1.4.0.tgz", + "integrity": "sha512-7bziOlRqH0hJx80h/3mbicLW7o8qLsH5+RaLR2t+OHM3D0JlWGODQKQ4cxbK7WlvmUxpcj6Kgu6EKqjrGFe3QQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/Wombosvideo" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/vite": { + "version": "6.4.1", + "resolved": "https://registry.npmjs.org/vite/-/vite-6.4.1.tgz", + "integrity": "sha512-+Oxm7q9hDoLMyJOYfUYBuHQo+dkAloi33apOPP56pzj+vsdJDzr+j1NISE5pyaAuKL4A3UD34qd0lx5+kfKp2g==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "^0.25.0", + "fdir": "^6.4.4", + "picomatch": "^4.0.2", + "postcss": "^8.5.3", + "rollup": "^4.34.9", + "tinyglobby": "^0.2.13" + }, + "bin": { + "vite": "bin/vite.js" + }, + "engines": { + "node": "^18.0.0 || ^20.0.0 || >=22.0.0" + }, + "funding": { + "url": "https://github.com/vitejs/vite?sponsor=1" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + }, + "peerDependencies": { + "@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0", + "jiti": ">=1.21.0", + "less": "*", + "lightningcss": "^1.21.0", + "sass": "*", + "sass-embedded": "*", + "stylus": "*", + "sugarss": "*", + "terser": "^5.16.0", + "tsx": "^4.8.1", + "yaml": "^2.4.2" + }, + "peerDependenciesMeta": { + "@types/node": { + "optional": true + }, + "jiti": { + "optional": true + }, + "less": { + "optional": true + }, + "lightningcss": { + "optional": true + }, + "sass": { + "optional": true + }, + "sass-embedded": { + "optional": true + }, + "stylus": { + "optional": true + }, + "sugarss": { + "optional": true + }, + "terser": { + "optional": true + }, + "tsx": { + "optional": true + }, + "yaml": { + "optional": true + } + } + }, + "node_modules/vitefu": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/vitefu/-/vitefu-1.1.1.tgz", + "integrity": "sha512-B/Fegf3i8zh0yFbpzZ21amWzHmuNlLlmJT6n7bu5e+pCHUKQIfXSYokrqOBGEMMe9UG2sostKQF9mml/vYaWJQ==", + "dev": true, + "license": "MIT", + "workspaces": [ + "tests/deps/*", + "tests/projects/*", + "tests/projects/workspace/packages/*" + ], + "peerDependencies": { + "vite": "^3.0.0 || ^4.0.0 || ^5.0.0 || ^6.0.0 || ^7.0.0-beta.0" + }, + "peerDependenciesMeta": { + "vite": { + "optional": true + } + } + }, + "node_modules/zimmerframe": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/zimmerframe/-/zimmerframe-1.1.4.tgz", + "integrity": "sha512-B58NGBEoc8Y9MWWCQGl/gq9xBCe4IiKM0a2x7GZdQKOW5Exr8S1W24J6OgM1njK8xCRGvAJIL/MxXHf6SkmQKQ==", + "license": "MIT" + } + } +} diff --git a/dashboard/package.json b/dashboard/package.json new file mode 100644 index 00000000..c9c27630 --- /dev/null +++ b/dashboard/package.json @@ -0,0 +1,33 @@ +{ + "name": "exo-dashboard", + "private": true, + "version": "1.0.0", + "type": "module", + "scripts": { + "dev": "vite dev", + "build": "vite build", + "preview": "vite preview", + "prepare": "svelte-kit sync || echo ''", + "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json" + }, + "devDependencies": { + "@sveltejs/adapter-static": "^3.0.10", + "@sveltejs/kit": "^2.48.4", + "@sveltejs/vite-plugin-svelte": "^5.0.0", + "@tailwindcss/vite": "^4.0.0", + "@types/d3": "^7.4.3", + "@types/node": "^22", + "d3": "^7.9.0", + "svelte": "^5.0.0", + "svelte-check": "^4.0.0", + "tailwindcss": "^4.0.0", + "tw-animate-css": "^1.3.5", + "typescript": "^5.0.0", + "vite": "^6.0.0" + }, + "dependencies": { + "highlight.js": "^11.11.1", + "mode-watcher": "^1.1.0" + } +} + diff --git a/dashboard/src/app.css b/dashboard/src/app.css new file mode 100644 index 00000000..fc532578 --- /dev/null +++ b/dashboard/src/app.css @@ -0,0 +1,322 @@ +@import 'tailwindcss'; +@import 'tw-animate-css'; + +@custom-variant dark (&:is(.dark *)); + +:root { + /* EXO Brand Colors - Command Center Theme (neutral dark greys) */ + --exo-black: oklch(0.12 0 0); + --exo-dark-gray: oklch(0.16 0 0); + --exo-medium-gray: oklch(0.22 0 0); + --exo-light-gray: oklch(0.6 0 0); + --exo-yellow: oklch(0.85 0.18 85); + --exo-yellow-darker: oklch(0.7 0.16 85); + --exo-yellow-glow: oklch(0.9 0.2 85); + + /* Gotham-inspired accent colors */ + --exo-grid: oklch(0.25 0 0); + --exo-scanline: oklch(0.15 0 0); + --exo-glow-yellow: 0 0 20px oklch(0.85 0.18 85 / 0.3); + --exo-glow-yellow-strong: 0 0 40px oklch(0.85 0.18 85 / 0.5); + + /* Theme Variables */ + --radius: 0.375rem; + --background: var(--exo-black); + --foreground: oklch(0.9 0 0); + --card: var(--exo-dark-gray); + --card-foreground: oklch(0.9 0 0); + --popover: var(--exo-dark-gray); + --popover-foreground: oklch(0.9 0 0); + --primary: var(--exo-yellow); + --primary-foreground: var(--exo-black); + --secondary: var(--exo-medium-gray); + --secondary-foreground: oklch(0.9 0 0); + --muted: var(--exo-medium-gray); + --muted-foreground: var(--exo-light-gray); + --accent: var(--exo-medium-gray); + --accent-foreground: oklch(0.9 0 0); + --destructive: oklch(0.6 0.25 25); + --border: oklch(0.22 0 0); + --input: oklch(0.22 0 0); + --ring: var(--exo-yellow); +} + +@theme inline { + --radius-sm: calc(var(--radius) - 2px); + --radius-md: var(--radius); + --radius-lg: calc(var(--radius) + 2px); + --radius-xl: calc(var(--radius) + 4px); + --color-background: var(--background); + --color-foreground: var(--foreground); + --color-card: var(--card); + --color-card-foreground: var(--card-foreground); + --color-popover: var(--popover); + --color-popover-foreground: var(--popover-foreground); + --color-primary: var(--primary); + --color-primary-foreground: var(--primary-foreground); + --color-secondary: var(--secondary); + --color-secondary-foreground: var(--secondary-foreground); + --color-muted: var(--muted); + --color-muted-foreground: var(--muted-foreground); + --color-accent: var(--accent); + --color-accent-foreground: var(--accent-foreground); + --color-destructive: var(--destructive); + --color-border: var(--border); + --color-input: var(--input); + --color-ring: var(--ring); + + /* Custom EXO colors */ + --color-exo-yellow: var(--exo-yellow); + --color-exo-yellow-darker: var(--exo-yellow-darker); + --color-exo-black: var(--exo-black); + --color-exo-dark-gray: var(--exo-dark-gray); + --color-exo-medium-gray: var(--exo-medium-gray); + --color-exo-light-gray: var(--exo-light-gray); +} + +@layer base { + * { + @apply border-border outline-ring/50; + } + html, body { + @apply bg-background text-foreground; + font-family: 'SF Mono', 'Fira Code', 'Monaco', 'Consolas', 'Liberation Mono', monospace; + letter-spacing: 0.02em; + } +} + +@layer utilities { + .scrollbar-hide { + &::-webkit-scrollbar { + display: none; + } + -ms-overflow-style: none; + scrollbar-width: none; + } + + /* CRT Scanline effect */ + .scanlines { + position: relative; + &::before { + content: ''; + position: absolute; + inset: 0; + background: repeating-linear-gradient( + 0deg, + transparent, + transparent 2px, + oklch(0 0 0 / 0.03) 2px, + oklch(0 0 0 / 0.03) 4px + ); + pointer-events: none; + z-index: 100; + } + } + + /* Command panel styling */ + .command-panel { + background: linear-gradient( + 180deg, + oklch(0.16 0 0 / 0.95) 0%, + oklch(0.12 0 0 / 0.98) 100% + ); + border: 1px solid oklch(0.25 0 0); + box-shadow: + inset 0 1px 0 oklch(1 0 0 / 0.03), + 0 4px 20px oklch(0 0 0 / 0.5); + } + + /* Glow text */ + .glow-text { + text-shadow: + 0 0 10px oklch(0.85 0.18 85 / 0.5), + 0 0 20px oklch(0.85 0.18 85 / 0.3), + 0 0 40px oklch(0.85 0.18 85 / 0.1); + } + + /* Status indicator pulse */ + .status-pulse { + animation: statusPulse 2s ease-in-out infinite; + } + + /* Grid background */ + .grid-bg { + background-image: + linear-gradient(oklch(0.2 0 0 / 0.3) 1px, transparent 1px), + linear-gradient(90deg, oklch(0.2 0 0 / 0.3) 1px, transparent 1px); + background-size: 40px 40px; + } +} + +/* Animations */ +@keyframes flowAnimation { + from { + stroke-dashoffset: 0; + } + to { + stroke-dashoffset: -16; + } +} + +@keyframes statusPulse { + 0%, 100% { + opacity: 1; + } + 50% { + opacity: 0.5; + } +} + +@keyframes radarSweep { + from { + transform: rotate(0deg); + } + to { + transform: rotate(360deg); + } +} + +@keyframes glowPulse { + 0%, 100% { + box-shadow: 0 0 5px oklch(0.85 0.18 85 / 0.3), 0 0 10px oklch(0.85 0.18 85 / 0.1); + } + 50% { + box-shadow: 0 0 15px oklch(0.85 0.18 85 / 0.5), 0 0 30px oklch(0.85 0.18 85 / 0.2); + } +} + +@keyframes dataPulse { + 0%, 100% { + opacity: 0.6; + } + 50% { + opacity: 1; + } +} + +.graph-link { + stroke: oklch(0.85 0.18 85 / 0.4); + stroke-width: 1.5px; + stroke-dasharray: 8, 8; + animation: flowAnimation 1s linear infinite; + filter: drop-shadow(0 0 3px oklch(0.85 0.18 85 / 0.5)); +} + +.graph-link-active { + stroke: oklch(0.85 0.18 85 / 0.8); + stroke-width: 2px; + filter: drop-shadow(0 0 6px oklch(0.85 0.18 85 / 0.8)); +} + +/* CRT Screen effect for topology */ +.crt-screen { + position: relative; + border-radius: 50%; + background: radial-gradient( + ellipse at center, + oklch(0.16 0 0) 0%, + oklch(0.12 0 0) 50%, + oklch(0.09 0 0) 100% + ); + box-shadow: + inset 0 0 100px oklch(0 0 0 / 0.5), + 0 0 50px oklch(0.85 0.18 85 / 0.1); +} + +/* Data readout styling */ +.data-readout { + font-family: 'SF Mono', 'Fira Code', monospace; + font-size: 11px; + letter-spacing: 0.05em; + text-transform: uppercase; +} + +/* Terminal cursor blink */ +.cursor-blink { + animation: cursorBlink 1s step-end infinite; +} + +@keyframes cursorBlink { + 0%, 100% { opacity: 1; } + 50% { opacity: 0; } +} + +/* Custom scrollbar for command center */ +::-webkit-scrollbar { + width: 6px; + height: 6px; +} + +::-webkit-scrollbar-track { + background: oklch(0.1 0 0); +} + +::-webkit-scrollbar-thumb { + background: oklch(0.3 0 0); + border-radius: 3px; +} + +::-webkit-scrollbar-thumb:hover { + background: oklch(0.85 0.18 85 / 0.5); +} + +/* Remove focus outline/border for inputs */ +input:focus, textarea:focus { + outline: none; + box-shadow: none; +} + +/* Shooting Stars Animation */ +.shooting-stars { + position: fixed; + inset: 0; + overflow: hidden; + pointer-events: none; + z-index: 0; +} + +.shooting-star { + position: absolute; + width: 3px; + height: 3px; + background: oklch(0.85 0.18 85 / 1); + border-radius: 50%; + box-shadow: 0 0 6px oklch(0.85 0.18 85 / 0.8); + animation: shootingStar var(--duration, 3s) linear infinite; + animation-delay: var(--delay, 0s); + opacity: 0; +} + +.shooting-star::before { + content: ''; + position: absolute; + width: 80px; + height: 2px; + background: linear-gradient(90deg, oklch(0.85 0.18 85 / 0), oklch(0.85 0.18 85 / 0.6)); + transform: rotate(45deg); + transform-origin: right center; + top: 0; + right: 2px; +} + +@keyframes shootingStar { + 0% { + opacity: 0; + transform: translate(0, 0); + } + 0.5% { + opacity: 1; + } + 2.5% { + opacity: 0.8; + transform: translate(300px, 300px); + } + 3.5% { + opacity: 0; + transform: translate(400px, 400px); + } + 100% { + opacity: 0; + transform: translate(400px, 400px); + } +} diff --git a/dashboard/src/app.d.ts b/dashboard/src/app.d.ts new file mode 100644 index 00000000..b111beb0 --- /dev/null +++ b/dashboard/src/app.d.ts @@ -0,0 +1,14 @@ +// See https://svelte.dev/docs/kit/types#app.d.ts +// for information about these interfaces +declare global { + namespace App { + // interface Error {} + // interface Locals {} + // interface PageData {} + // interface PageState {} + // interface Platform {} + } +} + +export {}; + diff --git a/dashboard/src/app.html b/dashboard/src/app.html new file mode 100644 index 00000000..a974a968 --- /dev/null +++ b/dashboard/src/app.html @@ -0,0 +1,14 @@ + + + + + + + EXO + %sveltekit.head% + + +
%sveltekit.body%
+ + + diff --git a/dashboard/src/lib/components/ChatAttachments.svelte b/dashboard/src/lib/components/ChatAttachments.svelte new file mode 100644 index 00000000..f56e23e3 --- /dev/null +++ b/dashboard/src/lib/components/ChatAttachments.svelte @@ -0,0 +1,75 @@ + + +{#if files.length > 0} +
+ {#each files as file (file.id)} +
+ + {#if file.preview && getFileCategory(file.type, file.name) === 'image'} + {file.name} + {:else} + {getFileIcon(file)} + {/if} + + +
+ + {truncateName(file.name)} + + + {formatFileSize(file.size)} + +
+ + + {#if !readonly && onRemove} + + {/if} +
+ {/each} +
+{/if} + diff --git a/dashboard/src/lib/components/ChatForm.svelte b/dashboard/src/lib/components/ChatForm.svelte new file mode 100644 index 00000000..95d023c3 --- /dev/null +++ b/dashboard/src/lib/components/ChatForm.svelte @@ -0,0 +1,398 @@ + + + + + +
{ e.preventDefault(); handleSubmit(); }} + class="w-full {className}" + ondragover={handleDragOver} + ondragleave={handleDragLeave} + ondrop={handleDrop} +> +
+ +
+ + + {#if isDragOver} +
+
+ DROP FILES HERE +
+
+ {/if} + + + {#if showModelSelector && availableModels().length > 0} +
+
+ MODEL: + +
+ +
+ + + +
+
+ + {#if isModelDropdownOpen} + + + + +
+
+ {#each availableModels() as model} + + {/each} +
+
+ {/if} +
+ + {#if currentTtft !== null || currentTps !== null} +
+ {#if currentTtft !== null} + + TTFT {currentTtft.toFixed(1)}ms + + {/if} + {#if currentTps !== null} + + TPS {currentTps.toFixed(1)} tok/s + ({(1000 / currentTps).toFixed(1)} ms/tok) + + {/if} +
+ {/if} +
+ {/if} + + + {#if uploadedFiles.length > 0} +
+ +
+ {/if} + + +
+ + + + + ▶ + + + + +
+ + +
+
+ + {#if showHelperText} +

+ ENTER + TO SEND + | + SHIFT+ENTER + NEW LINE + | + DRAG & DROP OR PASTE FILES +

+ {/if} +
diff --git a/dashboard/src/lib/components/ChatMessages.svelte b/dashboard/src/lib/components/ChatMessages.svelte new file mode 100644 index 00000000..baaf43f7 --- /dev/null +++ b/dashboard/src/lib/components/ChatMessages.svelte @@ -0,0 +1,462 @@ + + +
+ {#each messageList as message (message.id)} +
+
+ {#if message.role === 'assistant'} + +
+
+ EXO + {formatTimestamp(message.timestamp)} + {#if message.ttftMs || message.tps} + + {#if message.ttftMs}TTFT {message.ttftMs.toFixed(0)}ms{/if}{#if message.ttftMs && message.tps}•{/if}{#if message.tps}{message.tps.toFixed(1)} tok/s{/if} + + {/if} +
+ {:else} + +
+ {formatTimestamp(message.timestamp)} + QUERY +
+
+ {/if} + + {#if deleteConfirmId === message.id} + +
+

Delete this message{message.role === 'user' ? ' and all responses after it' : ''}?

+
+ + +
+
+ {:else if editingMessageId === message.id} + +
+ +
+ + +
+
+ {:else} +
+ + {#if message.role === 'user'} + +
+ + {#if message.attachments && message.attachments.length > 0} +
+ {#each message.attachments as attachment} +
+ {#if attachment.type === 'image' && attachment.preview} + {attachment.name} + {:else} + {getAttachmentIcon(attachment)} + {/if} + {truncateName(attachment.name)} +
+ {/each} +
+ {/if} + + {#if message.content} +
+ {message.content} +
+ {/if} +
+ {:else} + +
+ {#if message.thinking && message.thinking.trim().length > 0} +
+ + {#if isThinkingExpanded(message.id)} +
+ {message.thinking.trim()} +
+ {/if} +
+ {/if} +
+ {message.content || (loading ? response : '')} + {#if loading && !message.content} + + {/if} +
+
+ {/if} +
+ + +
+ + + + + {#if message.role === 'user'} + + {/if} + + + {#if message.role === 'assistant' && isLastAssistantMessage(message.id) && !loading} + + {/if} + + + +
+ {/if} +
+
+ {/each} + + {#if messageList.length === 0} +
+
+
+
+
+
+

AWAITING INPUT

+

ENTER A QUERY TO BEGIN

+
+ {/if} + + +
+
diff --git a/dashboard/src/lib/components/ChatSidebar.svelte b/dashboard/src/lib/components/ChatSidebar.svelte new file mode 100644 index 00000000..87e06059 --- /dev/null +++ b/dashboard/src/lib/components/ChatSidebar.svelte @@ -0,0 +1,430 @@ + + + + diff --git a/dashboard/src/lib/components/HeaderNav.svelte b/dashboard/src/lib/components/HeaderNav.svelte new file mode 100644 index 00000000..4ec770d6 --- /dev/null +++ b/dashboard/src/lib/components/HeaderNav.svelte @@ -0,0 +1,57 @@ + + +
+ + + + +
+ {#if showHome} + + {/if} + + + + + + + Downloads + +
+
diff --git a/dashboard/src/lib/components/ModelCard.svelte b/dashboard/src/lib/components/ModelCard.svelte new file mode 100644 index 00000000..ee5f07ab --- /dev/null +++ b/dashboard/src/lib/components/ModelCard.svelte @@ -0,0 +1,660 @@ + + +
+ +
+
+
+
+ +
+ +
+
+
+
+ {model.name || model.id} +
+ {#if huggingFaceModelId} + + + + + + + + {/if} + {#if tags.length > 0} +
+ {#each tags as tag} + + {tag} + + {/each} +
+ {/if} +
+ {#if model.name && model.name !== model.id} +
+ {model.id} +
+ {/if} +
+
+
+ {estimatedMemory}GB +
+
+
+ + +
+ + {sharding} + + + {runtime === 'MlxRing' ? 'MLX Ring' : runtime === 'MlxIbv' || runtime === 'MlxJaccl' ? 'MLX RDMA' : runtime} + +
+ + + {#if placementPreview().nodes.length > 0} + {@const preview = placementPreview()} +
+ +
+ + + + + + + + + + + + + + + + + + + + + {#if preview.nodes.length > 1} + {#each preview.nodes as node, i} + {#each preview.nodes.slice(i + 1) as node2} + + {/each} + {/each} + {/if} + + {#each preview.nodes as node} + + + {#if node.deviceType === 'macbook'} + + + + + + + + + + {#if node.modelUsageGB > 0 && node.isUsed} + + {/if} + + + + {:else if node.deviceType === 'studio'} + + + + + + + + + {#if node.modelUsageGB > 0 && node.isUsed} + + {/if} + + {:else if node.deviceType === 'mini'} + + + + + + + + + {#if node.modelUsageGB > 0 && node.isUsed} + + {/if} + + {:else} + + + + + {/if} + + + 90 ? '#f87171' : '#FFD700') : '#4B5563'} + > + {node.newPercent.toFixed(0)}% + + + {/each} + +
+ {/if} + + + +
+
+ + diff --git a/dashboard/src/lib/components/TopologyGraph.svelte b/dashboard/src/lib/components/TopologyGraph.svelte new file mode 100644 index 00000000..e45ca080 --- /dev/null +++ b/dashboard/src/lib/components/TopologyGraph.svelte @@ -0,0 +1,971 @@ + + + + + diff --git a/dashboard/src/lib/components/index.ts b/dashboard/src/lib/components/index.ts new file mode 100644 index 00000000..bd750839 --- /dev/null +++ b/dashboard/src/lib/components/index.ts @@ -0,0 +1,7 @@ +export { default as TopologyGraph } from './TopologyGraph.svelte'; +export { default as ChatForm } from './ChatForm.svelte'; +export { default as ChatMessages } from './ChatMessages.svelte'; +export { default as ChatAttachments } from './ChatAttachments.svelte'; +export { default as ChatSidebar } from './ChatSidebar.svelte'; +export { default as ModelCard } from './ModelCard.svelte'; + diff --git a/dashboard/src/lib/stores/app.svelte.ts b/dashboard/src/lib/stores/app.svelte.ts new file mode 100644 index 00000000..ffeb1aa1 --- /dev/null +++ b/dashboard/src/lib/stores/app.svelte.ts @@ -0,0 +1,1395 @@ +/** + * AppStore - Central state management for the EXO dashboard + * + * Manages: + * - Chat state (whether a conversation has started) + * - Topology data from the EXO server + * - UI state for the topology/chat transition + */ + +import { browser } from '$app/environment'; + +// UUID generation fallback for browsers without crypto.randomUUID +function generateUUID(): string { + if (typeof crypto !== 'undefined' && typeof crypto.randomUUID === 'function') { + return crypto.randomUUID(); + } + // Fallback implementation + return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, (c) => { + const r = Math.random() * 16 | 0; + const v = c === 'x' ? r : (r & 0x3 | 0x8); + return v.toString(16); + }); +} + +export interface NodeInfo { + system_info?: { + model_id?: string; + chip?: string; + memory?: number; + }; + network_interfaces?: Array<{ + name?: string; + addresses?: string[]; + }>; + ip_to_interface?: Record; + macmon_info?: { + memory?: { + ram_usage: number; + ram_total: number; + }; + temp?: { + gpu_temp_avg: number; + }; + gpu_usage?: [number, number]; + sys_power?: number; + }; + last_macmon_update: number; + friendly_name?: string; +} + +export interface TopologyEdge { + source: string; + target: string; + sendBackIp?: string; + sendBackInterface?: string; +} + +export interface TopologyData { + nodes: Record; + edges: TopologyEdge[]; +} + +export interface Instance { + shardAssignments?: { + modelId?: string; + runnerToShard?: Record; + nodeToRunner?: Record; + }; +} + +interface RawNodeProfile { + modelId?: string; + chipId?: string; + friendlyName?: string; + networkInterfaces?: Array<{ + name?: string; + ipAddress?: string; + addresses?: Array<{ address?: string } | string>; + ipv4?: string; + ipv6?: string; + ipAddresses?: string[]; + ips?: string[]; + }>; + memory?: { + ramTotal?: { inBytes: number }; + ramAvailable?: { inBytes: number }; + swapTotal?: { inBytes: number }; + swapAvailable?: { inBytes: number }; + }; + system?: { + gpuUsage?: number; + temp?: number; + sysPower?: number; + }; +} + +interface RawTopologyNode { + nodeId: string; + nodeProfile: RawNodeProfile; +} + +interface RawTopologyConnection { + localNodeId: string; + sendBackNodeId: string; + sendBackMultiaddr?: { multiaddr?: string; address?: string; ip_address?: string } | string; +} + +interface RawTopology { + nodes: RawTopologyNode[]; + connections?: RawTopologyConnection[]; +} + +type RawNodeProfiles = Record; + +export interface DownloadProgress { + totalBytes: number; + downloadedBytes: number; + speed: number; + etaMs: number; + percentage: number; + completedFiles: number; + totalFiles: number; + files: Array<{ + name: string; + totalBytes: number; + downloadedBytes: number; + speed: number; + etaMs: number; + percentage: number; + }>; +} + +export interface ModelDownloadStatus { + isDownloading: boolean; + progress: DownloadProgress | null; + nodeDetails: Array<{ + nodeId: string; + nodeName: string; + progress: DownloadProgress; + }>; +} + +// Placement preview from the API +export interface PlacementPreview { + model_id: string; + sharding: 'Pipeline' | 'Tensor'; + instance_meta: 'MlxRing' | 'MlxIbv' | 'MlxJaccl'; + instance: unknown | null; + memory_delta_by_node: Record | null; + error: string | null; +} + +export interface PlacementPreviewResponse { + previews: PlacementPreview[]; +} + +interface RawStateResponse { + topology?: RawTopology; + instances?: Record; + runners?: Record; + downloads?: Record; + nodeProfiles?: RawNodeProfiles; +} + +export interface MessageAttachment { + type: 'image' | 'text' | 'file'; + name: string; + content?: string; + preview?: string; + mimeType?: string; +} + +export interface Message { + id: string; + role: 'user' | 'assistant' | 'system'; + content: string; + timestamp: number; + thinking?: string; + attachments?: MessageAttachment[]; + ttftMs?: number; // Time to first token in ms (for assistant messages) + tps?: number; // Tokens per second (for assistant messages) +} + +export interface Conversation { + id: string; + name: string; + messages: Message[]; + createdAt: number; + updatedAt: number; + modelId: string | null; + sharding: string | null; + instanceType: string | null; +} + +const STORAGE_KEY = 'exo-conversations'; + +function transformTopology(raw: RawTopology, profiles?: RawNodeProfiles): TopologyData { + const nodes: Record = {}; + const edges: TopologyEdge[] = []; + + for (const node of raw.nodes || []) { + const mergedProfile = profiles?.[node.nodeId]; + const profile = { ...(node.nodeProfile ?? {}), ...(mergedProfile ?? {}) }; + const ramTotal = profile?.memory?.ramTotal?.inBytes ?? 0; + const ramAvailable = profile?.memory?.ramAvailable?.inBytes ?? 0; + const ramUsage = Math.max(ramTotal - ramAvailable, 0); + + const networkInterfaces = (profile?.networkInterfaces || []).map((iface) => { + const addresses: string[] = []; + if (iface.ipAddress && typeof iface.ipAddress === 'string') { + addresses.push(iface.ipAddress); + } + if (Array.isArray(iface.addresses)) { + for (const addr of iface.addresses) { + if (typeof addr === 'string') addresses.push(addr); + else if (addr && typeof addr === 'object' && addr.address) addresses.push(addr.address); + } + } + if (Array.isArray(iface.ipAddresses)) { + addresses.push(...iface.ipAddresses.filter((a): a is string => typeof a === 'string')); + } + if (Array.isArray(iface.ips)) { + addresses.push(...iface.ips.filter((a): a is string => typeof a === 'string')); + } + if (iface.ipv4 && typeof iface.ipv4 === 'string') addresses.push(iface.ipv4); + if (iface.ipv6 && typeof iface.ipv6 === 'string') addresses.push(iface.ipv6); + + return { + name: iface.name, + addresses: Array.from(new Set(addresses)) + }; + }); + + const ipToInterface: Record = {}; + for (const iface of networkInterfaces) { + for (const addr of iface.addresses || []) { + ipToInterface[addr] = iface.name ?? ''; + } + } + + nodes[node.nodeId] = { + system_info: { + model_id: profile?.modelId ?? 'Unknown', + chip: profile?.chipId, + memory: ramTotal + }, + network_interfaces: networkInterfaces, + ip_to_interface: ipToInterface, + macmon_info: { + memory: { + ram_usage: ramUsage, + ram_total: ramTotal + }, + temp: profile?.system?.temp !== undefined ? { gpu_temp_avg: profile.system.temp } : undefined, + gpu_usage: profile?.system?.gpuUsage !== undefined ? [0, profile.system.gpuUsage] : undefined, + sys_power: profile?.system?.sysPower + }, + last_macmon_update: Date.now() / 1000, + friendly_name: profile?.friendlyName + }; + } + + for (const conn of raw.connections || []) { + if (!conn.localNodeId || !conn.sendBackNodeId) continue; + if (conn.localNodeId === conn.sendBackNodeId) continue; + if (!nodes[conn.localNodeId] || !nodes[conn.sendBackNodeId]) continue; + + let sendBackIp: string | undefined; + if (conn.sendBackMultiaddr) { + const multi = conn.sendBackMultiaddr; + if (typeof multi === 'string') { + sendBackIp = extractIpFromMultiaddr(multi); + } else { + sendBackIp = multi.ip_address || extractIpFromMultiaddr(multi.multiaddr) || extractIpFromMultiaddr(multi.address); + } + } + + edges.push({ + source: conn.localNodeId, + target: conn.sendBackNodeId, + sendBackIp + }); + } + + return { nodes, edges }; +} + +function extractIpFromMultiaddr(ma?: string): string | undefined { + if (!ma) return undefined; + const parts = ma.split('/'); + const ip4Idx = parts.indexOf('ip4'); + const ip6Idx = parts.indexOf('ip6'); + const idx = ip4Idx >= 0 ? ip4Idx : ip6Idx; + if (idx >= 0 && parts.length > idx + 1) { + return parts[idx + 1]; + } + return undefined; +} + +class AppStore { + // Conversation state + conversations = $state([]); + activeConversationId = $state(null); + + // Chat state + hasStartedChat = $state(false); + messages = $state([]); + currentResponse = $state(''); + isLoading = $state(false); + + // Performance metrics + ttftMs = $state(null); // Time to first token in ms + tps = $state(null); // Tokens per second + totalTokens = $state(0); // Total tokens in current response + + // Topology state + topologyData = $state(null); + instances = $state>({}); + runners = $state>({}); + downloads = $state>({}); + placementPreviews = $state([]); + selectedPreviewModelId = $state(null); + isLoadingPreviews = $state(false); + lastUpdate = $state(null); + + // UI state + isTopologyMinimized = $state(false); + isSidebarOpen = $state(false); // Hidden by default, shown when in chat mode + debugMode = $state(false); + + private fetchInterval: ReturnType | null = null; + private previewsInterval: ReturnType | null = null; + private lastConversationPersistTs = 0; + + constructor() { + if (browser) { + this.startPolling(); + this.loadConversationsFromStorage(); + this.loadDebugModeFromStorage(); + } + } + + /** + * Load conversations from localStorage + */ + private loadConversationsFromStorage() { + try { + const stored = localStorage.getItem(STORAGE_KEY); + if (stored) { + const parsed = JSON.parse(stored) as Array>; + this.conversations = parsed.map((conversation) => ({ + id: conversation.id ?? generateUUID(), + name: conversation.name ?? 'Chat', + messages: conversation.messages ?? [], + createdAt: conversation.createdAt ?? Date.now(), + updatedAt: conversation.updatedAt ?? Date.now(), + modelId: conversation.modelId ?? null, + sharding: conversation.sharding ?? null, + instanceType: conversation.instanceType ?? null + })); + } + } catch (error) { + console.error('Failed to load conversations:', error); + } + } + + /** + * Save conversations to localStorage + */ + private saveConversationsToStorage() { + try { + localStorage.setItem(STORAGE_KEY, JSON.stringify(this.conversations)); + } catch (error) { + console.error('Failed to save conversations:', error); + } + } + + private loadDebugModeFromStorage() { + try { + const stored = localStorage.getItem('exo-debug-mode'); + if (stored !== null) { + this.debugMode = stored === 'true'; + } + } catch (error) { + console.error('Failed to load debug mode:', error); + } + } + + private saveDebugModeToStorage() { + try { + localStorage.setItem('exo-debug-mode', this.debugMode ? 'true' : 'false'); + } catch (error) { + console.error('Failed to save debug mode:', error); + } + } + + /** + * Create a new conversation + */ + createConversation(name?: string): string { + const id = generateUUID(); + const now = Date.now(); + + // Try to derive model and strategy immediately from selected model or running instances + let derivedModelId = this.selectedChatModel || null; + let derivedInstanceType: string | null = null; + let derivedSharding: string | null = null; + + // If no selected model, fall back to the first running instance + if (!derivedModelId) { + const firstInstance = Object.values(this.instances)[0]; + if (firstInstance) { + const candidateModel = this.extractInstanceModelId(firstInstance); + derivedModelId = candidateModel ?? null; + const details = this.describeInstance(firstInstance); + derivedInstanceType = details.instanceType; + derivedSharding = details.sharding; + } + } else { + // If selected model is set, attempt to get its details from instances + for (const [, instanceWrapper] of Object.entries(this.instances)) { + const candidateModelId = this.extractInstanceModelId(instanceWrapper); + if (candidateModelId === derivedModelId) { + const details = this.describeInstance(instanceWrapper); + derivedInstanceType = details.instanceType; + derivedSharding = details.sharding; + break; + } + } + } + + const conversation: Conversation = { + id, + name: name || `Chat ${new Date(now).toLocaleString('en-US', { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' })}`, + messages: [], + createdAt: now, + updatedAt: now, + modelId: derivedModelId, + sharding: derivedSharding, + instanceType: derivedInstanceType + }; + + this.conversations.unshift(conversation); + this.activeConversationId = id; + this.messages = []; + this.hasStartedChat = true; + this.isTopologyMinimized = true; + this.isSidebarOpen = true; // Auto-open sidebar when chatting + + this.saveConversationsToStorage(); + return id; + } + + /** + * Load a conversation by ID + */ + loadConversation(id: string): boolean { + const conversation = this.conversations.find(c => c.id === id); + if (!conversation) return false; + + this.activeConversationId = id; + this.messages = [...conversation.messages]; + this.hasStartedChat = true; + this.isTopologyMinimized = true; + this.isSidebarOpen = true; // Auto-open sidebar when chatting + this.refreshConversationModelFromInstances(); + + return true; + } + + /** + * Delete a conversation by ID + */ + deleteConversation(id: string) { + this.conversations = this.conversations.filter(c => c.id !== id); + + if (this.activeConversationId === id) { + this.activeConversationId = null; + this.messages = []; + this.hasStartedChat = false; + this.isTopologyMinimized = false; + } + + this.saveConversationsToStorage(); + } + + /** + * Delete all conversations + */ + deleteAllConversations() { + this.conversations = []; + this.activeConversationId = null; + this.messages = []; + this.hasStartedChat = false; + this.isTopologyMinimized = false; + this.saveConversationsToStorage(); + } + + /** + * Rename a conversation + */ + renameConversation(id: string, newName: string) { + const conversation = this.conversations.find(c => c.id === id); + if (conversation) { + conversation.name = newName; + conversation.updatedAt = Date.now(); + this.saveConversationsToStorage(); + } + } + + private getTaggedValue(obj: unknown): [string | null, unknown] { + if (!obj || typeof obj !== 'object') return [null, null]; + const keys = Object.keys(obj as Record); + if (keys.length === 1) { + return [keys[0], (obj as Record)[keys[0]]]; + } + return [null, null]; + } + + private extractInstanceModelId(instanceWrapped: unknown): string | null { + const [, instance] = this.getTaggedValue(instanceWrapped); + if (!instance || typeof instance !== 'object') return null; + const inst = instance as { shardAssignments?: { modelId?: string } }; + return inst.shardAssignments?.modelId ?? null; + } + + private describeInstance(instanceWrapped: unknown): { sharding: string | null; instanceType: string | null } { + const [instanceTag, instance] = this.getTaggedValue(instanceWrapped); + if (!instance || typeof instance !== 'object') { + return { sharding: null, instanceType: null }; + } + + let instanceType: string | null = null; + if (instanceTag === 'MlxRingInstance') instanceType = 'MLX Ring'; + else if (instanceTag === 'MlxIbvInstance' || instanceTag === 'MlxJacclInstance') instanceType = 'MLX RDMA'; + + let sharding: string | null = null; + const inst = instance as { shardAssignments?: { runnerToShard?: Record } }; + const runnerToShard = inst.shardAssignments?.runnerToShard || {}; + const firstShardWrapped = Object.values(runnerToShard)[0]; + if (firstShardWrapped) { + const [shardTag] = this.getTaggedValue(firstShardWrapped); + if (shardTag === 'PipelineShardMetadata') sharding = 'Pipeline'; + else if (shardTag === 'TensorShardMetadata') sharding = 'Tensor'; + else if (shardTag === 'PrefillDecodeShardMetadata') sharding = 'Prefill/Decode'; + } + + return { sharding, instanceType }; + } + + private buildConversationModelInfo(modelId: string): { modelId: string; sharding: string | null; instanceType: string | null } { + let sharding: string | null = null; + let instanceType: string | null = null; + + for (const [, instanceWrapper] of Object.entries(this.instances)) { + const candidateModelId = this.extractInstanceModelId(instanceWrapper); + if (candidateModelId === modelId) { + const details = this.describeInstance(instanceWrapper); + sharding = details.sharding; + instanceType = details.instanceType; + break; + } + } + + return { modelId, sharding, instanceType }; + } + + private applyConversationModelInfo(info: { modelId: string; sharding: string | null; instanceType: string | null }) { + if (!this.activeConversationId) return; + const conversation = this.conversations.find(c => c.id === this.activeConversationId); + if (!conversation) return; + + // Keep the first known modelId stable; only backfill if missing + if (!conversation.modelId) { + conversation.modelId = info.modelId; + } + conversation.sharding = info.sharding; + conversation.instanceType = info.instanceType; + this.saveConversationsToStorage(); + } + + private getModelTail(modelId: string): string { + const parts = modelId.split('/'); + return (parts[parts.length - 1] || modelId).toLowerCase(); + } + + private isBetterModelId(currentId: string | null, candidateId: string | null): boolean { + if (!candidateId) return false; + if (!currentId) return true; + const currentTail = this.getModelTail(currentId); + const candidateTail = this.getModelTail(candidateId); + return candidateTail.length > currentTail.length && candidateTail.startsWith(currentTail); + } + + private refreshConversationModelFromInstances() { + if (!this.activeConversationId) return; + const conversation = this.conversations.find(c => c.id === this.activeConversationId); + if (!conversation) return; + + // Prefer stored model; do not replace it once set. Only backfill when missing. + let modelId = conversation.modelId; + + // If missing, try the selected model + if (!modelId && this.selectedChatModel) { + modelId = this.selectedChatModel; + } + + // If still missing, fall back to first instance model + if (!modelId) { + const firstInstance = Object.values(this.instances)[0]; + if (firstInstance) { + modelId = this.extractInstanceModelId(firstInstance); + } + } + + if (!modelId) return; + + // If a more specific instance modelId is available (e.g., adds "-4bit"), prefer it + let preferredModelId = modelId; + for (const [, instanceWrapper] of Object.entries(this.instances)) { + const candidate = this.extractInstanceModelId(instanceWrapper); + if (!candidate) continue; + if (candidate === preferredModelId) { + break; + } + if (this.isBetterModelId(preferredModelId, candidate)) { + preferredModelId = candidate; + } + } + + if (this.isBetterModelId(conversation.modelId, preferredModelId)) { + conversation.modelId = preferredModelId; + } + + const info = this.buildConversationModelInfo(preferredModelId); + const hasNewInfo = Boolean(info.sharding || info.instanceType || !conversation.modelId); + if (hasNewInfo) { + this.applyConversationModelInfo(info); + } + } + + getDebugMode(): boolean { + return this.debugMode; + } + + /** + * Update the active conversation with current messages + */ + private updateActiveConversation() { + if (!this.activeConversationId) return; + + const conversation = this.conversations.find(c => c.id === this.activeConversationId); + if (conversation) { + conversation.messages = [...this.messages]; + conversation.updatedAt = Date.now(); + + // Auto-generate name from first user message if still has default name + if (conversation.name.startsWith('Chat ')) { + const firstUserMsg = conversation.messages.find(m => m.role === 'user' && m.content.trim()); + if (firstUserMsg) { + // Clean up the content - remove file context markers and whitespace + let content = firstUserMsg.content + .replace(/\[File:.*?\][\s\S]*?```[\s\S]*?```/g, '') // Remove file attachments + .trim(); + + if (content) { + const preview = content.slice(0, 50); + conversation.name = preview.length < content.length ? preview + '...' : preview; + } + } + } + + this.saveConversationsToStorage(); + } + } + + private persistActiveConversation(throttleMs = 400) { + const now = Date.now(); + if (now - this.lastConversationPersistTs < throttleMs) return; + this.lastConversationPersistTs = now; + this.updateActiveConversation(); + } + + /** + * Toggle sidebar visibility + */ + toggleSidebar() { + this.isSidebarOpen = !this.isSidebarOpen; + } + + setDebugMode(enabled: boolean) { + this.debugMode = enabled; + this.saveDebugModeToStorage(); + } + + toggleDebugMode() { + this.debugMode = !this.debugMode; + this.saveDebugModeToStorage(); + } + + startPolling() { + this.fetchState(); + this.fetchInterval = setInterval(() => this.fetchState(), 1000); + } + + stopPolling() { + if (this.fetchInterval) { + clearInterval(this.fetchInterval); + this.fetchInterval = null; + } + this.stopPreviewsPolling(); + } + + async fetchState() { + try { + const response = await fetch('/state'); + if (!response.ok) { + throw new Error(`Failed to fetch state: ${response.status}`); + } + const data: RawStateResponse = await response.json(); + + if (data.topology) { + this.topologyData = transformTopology(data.topology, data.nodeProfiles); + } + if (data.instances) { + this.instances = data.instances; + this.refreshConversationModelFromInstances(); + } + if (data.runners) { + this.runners = data.runners; + } + if (data.downloads) { + this.downloads = data.downloads; + } + this.lastUpdate = Date.now(); + } catch (error) { + console.error('Error fetching state:', error); + } + } + + async fetchPlacementPreviews(modelId: string, showLoading = true) { + if (!modelId) return; + + if (showLoading) { + this.isLoadingPreviews = true; + } + this.selectedPreviewModelId = modelId; + + try { + const response = await fetch(`/instance/previews?model_id=${encodeURIComponent(modelId)}`); + if (!response.ok) { + throw new Error(`Failed to fetch placement previews: ${response.status}`); + } + const data: PlacementPreviewResponse = await response.json(); + this.placementPreviews = data.previews; + } catch (error) { + console.error('Error fetching placement previews:', error); + this.placementPreviews = []; + } finally { + if (showLoading) { + this.isLoadingPreviews = false; + } + } + } + + startPreviewsPolling(modelId: string) { + // Stop any existing preview polling + this.stopPreviewsPolling(); + + // Fetch immediately + this.fetchPlacementPreviews(modelId); + + // Then poll every 15 seconds (don't show loading spinner for subsequent fetches) + this.previewsInterval = setInterval(() => { + if (this.selectedPreviewModelId) { + this.fetchPlacementPreviews(this.selectedPreviewModelId, false); + } + }, 15000); + } + + stopPreviewsPolling() { + if (this.previewsInterval) { + clearInterval(this.previewsInterval); + this.previewsInterval = null; + } + } + + selectPreviewModel(modelId: string | null) { + if (modelId) { + this.startPreviewsPolling(modelId); + } else { + this.stopPreviewsPolling(); + this.selectedPreviewModelId = null; + this.placementPreviews = []; + } + } + + /** + * Starts a chat conversation - triggers the topology minimization animation + * Creates a new conversation if none is active + */ + startChat() { + if (!this.activeConversationId) { + this.createConversation(); + } else { + this.hasStartedChat = true; + this.isSidebarOpen = true; // Auto-open sidebar when chatting + // Small delay before minimizing for a nice visual effect + setTimeout(() => { + this.isTopologyMinimized = true; + }, 100); + } + } + + /** + * Add a message to the conversation + */ + addMessage(role: 'user' | 'assistant', content: string) { + const message: Message = { + id: generateUUID(), + role, + content, + timestamp: Date.now() + }; + this.messages.push(message); + return message; + } + + /** + * Delete a message and all subsequent messages + */ + deleteMessage(messageId: string) { + const messageIndex = this.messages.findIndex(m => m.id === messageId); + if (messageIndex === -1) return; + + // Remove this message and all subsequent messages + this.messages = this.messages.slice(0, messageIndex); + this.updateActiveConversation(); + } + + /** + * Edit a user message content (does not regenerate response) + */ + editMessage(messageId: string, newContent: string) { + const message = this.messages.find(m => m.id === messageId); + if (!message) return; + + message.content = newContent; + message.timestamp = Date.now(); + this.updateActiveConversation(); + } + + /** + * Edit a user message and regenerate the response + */ + async editAndRegenerate(messageId: string, newContent: string): Promise { + const messageIndex = this.messages.findIndex(m => m.id === messageId); + if (messageIndex === -1) return; + + const message = this.messages[messageIndex]; + if (message.role !== 'user') return; + + // Update the message content + message.content = newContent; + message.timestamp = Date.now(); + + // Remove all messages after this one (including the assistant response) + this.messages = this.messages.slice(0, messageIndex + 1); + + // Regenerate the response + await this.regenerateLastResponse(); + } + + /** + * Regenerate the last assistant response + */ + async regenerateLastResponse(): Promise { + if (this.isLoading) return; + + // Find the last user message + let lastUserIndex = -1; + for (let i = this.messages.length - 1; i >= 0; i--) { + if (this.messages[i].role === 'user') { + lastUserIndex = i; + break; + } + } + + if (lastUserIndex === -1) return; + + const lastUserMessage = this.messages[lastUserIndex]; + + // Remove any messages after the user message + this.messages = this.messages.slice(0, lastUserIndex + 1); + + // Resend the message to get a new response + this.isLoading = true; + this.currentResponse = ''; + + // Create placeholder for assistant message + const assistantMessage = this.addMessage('assistant', ''); + + try { + const systemPrompt = { + role: 'system' as const, + content: 'You are a helpful AI assistant. Respond directly and concisely. Do not show your reasoning or thought process.' + }; + + const apiMessages = [ + systemPrompt, + ...this.messages.slice(0, -1).map((m) => { + return { role: m.role, content: m.content }; + }) + ]; + + // Determine which model to use + let modelToUse = this.selectedChatModel; + if (!modelToUse) { + const firstInstanceKey = Object.keys(this.instances)[0]; + if (firstInstanceKey) { + const instance = this.instances[firstInstanceKey] as Record | undefined; + if (instance) { + const keys = Object.keys(instance); + if (keys.length === 1) { + const inst = instance[keys[0]] as { shardAssignments?: { modelId?: string } } | undefined; + modelToUse = inst?.shardAssignments?.modelId || ''; + } + } + } + } + + if (!modelToUse) { + assistantMessage.content = 'Error: No model available. Please launch an instance first.'; + this.isLoading = false; + this.updateActiveConversation(); + return; + } + + const response = await fetch('/v1/chat/completions', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + model: modelToUse, + messages: apiMessages, + stream: true + }) + }); + + if (!response.ok) { + const errorText = await response.text(); + assistantMessage.content = `Error: ${response.status} - ${errorText}`; + this.isLoading = false; + this.updateActiveConversation(); + return; + } + + const reader = response.body?.getReader(); + if (!reader) { + assistantMessage.content = 'Error: No response stream available'; + this.isLoading = false; + this.updateActiveConversation(); + return; + } + + const decoder = new TextDecoder(); + let fullContent = ''; + let partialLine = ''; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + const chunk = decoder.decode(value, { stream: true }); + const lines = (partialLine + chunk).split('\n'); + partialLine = lines.pop() || ''; + + for (const line of lines) { + const trimmed = line.trim(); + if (!trimmed || trimmed === 'data: [DONE]') continue; + + if (trimmed.startsWith('data: ')) { + try { + const json = JSON.parse(trimmed.slice(6)); + const delta = json.choices?.[0]?.delta?.content; + if (delta) { + fullContent += delta; + const { displayContent } = this.stripThinkingTags(fullContent); + this.currentResponse = displayContent; + assistantMessage.content = displayContent; + } + } catch { + // Skip malformed JSON + } + } + } + } + + const { displayContent } = this.stripThinkingTags(fullContent); + assistantMessage.content = displayContent; + this.currentResponse = ''; + this.updateActiveConversation(); + + } catch (error) { + assistantMessage.content = `Error: ${error instanceof Error ? error.message : 'Unknown error'}`; + this.updateActiveConversation(); + } finally { + this.isLoading = false; + } + } + + /** + * Selected model for chat (can be set by the UI) + */ + selectedChatModel = $state(''); + + /** + * Set the model to use for chat + */ + setSelectedModel(modelId: string) { + this.selectedChatModel = modelId; + // Clear stats when model changes + this.ttftMs = null; + this.tps = null; + } + + /** + * Strip thinking tags from content for display. + * Handles both complete ... blocks and in-progress ... blocks during streaming. + */ + private stripThinkingTags(content: string): { displayContent: string; thinkingContent: string } { + const extracted: string[] = []; + let displayContent = content; + + // Extract complete ... blocks + const completeBlockRegex = /([\s\S]*?)<\/think>/gi; + let match: RegExpExecArray | null; + while ((match = completeBlockRegex.exec(content)) !== null) { + const inner = match[1]?.trim(); + if (inner) extracted.push(inner); + } + displayContent = displayContent.replace(completeBlockRegex, ''); + + // Handle in-progress thinking block (has but no closing yet) + const openTagIndex = displayContent.lastIndexOf(''); + if (openTagIndex !== -1) { + const inProgressThinking = displayContent.slice(openTagIndex + 7).trim(); + if (inProgressThinking) { + extracted.push(inProgressThinking); + } + displayContent = displayContent.slice(0, openTagIndex); + } + + return { displayContent: displayContent.trim(), thinkingContent: extracted.join('\n\n') }; + } + + /** + * Send a message to the LLM and stream the response + */ + async sendMessage(content: string, files?: { id: string; name: string; type: string; textContent?: string; preview?: string }[]): Promise { + if ((!content.trim() && (!files || files.length === 0)) || this.isLoading) return; + + if (!this.hasStartedChat) { + this.startChat(); + } + + this.isLoading = true; + this.currentResponse = ''; + this.ttftMs = null; + this.tps = null; + this.totalTokens = 0; + + // Build attachments from files + const attachments: MessageAttachment[] = []; + let fileContext = ''; + + if (files && files.length > 0) { + for (const file of files) { + const isImage = file.type.startsWith('image/'); + + if (isImage && file.preview) { + attachments.push({ + type: 'image', + name: file.name, + preview: file.preview, + mimeType: file.type + }); + } else if (file.textContent) { + attachments.push({ + type: 'text', + name: file.name, + content: file.textContent, + mimeType: file.type + }); + // Add text file content to the message context + fileContext += `\n\n[File: ${file.name}]\n\`\`\`\n${file.textContent}\n\`\`\``; + } else { + attachments.push({ + type: 'file', + name: file.name, + mimeType: file.type + }); + } + } + } + + // Combine content with file context + const fullContent = content + fileContext; + + // Add user message with attachments + const userMessage: Message = { + id: generateUUID(), + role: 'user', + content: content, // Store original content for display + timestamp: Date.now(), + attachments: attachments.length > 0 ? attachments : undefined + }; + this.messages.push(userMessage); + + // Create placeholder for assistant message + const assistantMessage = this.addMessage('assistant', ''); + this.updateActiveConversation(); + + try { + // Build the messages array for the API with system prompt + const systemPrompt = { + role: 'system' as const, + content: 'You are a helpful AI assistant. Respond directly and concisely. Do not show your reasoning or thought process. When files are shared with you, analyze them and respond helpfully.' + }; + + // Build API messages - include file content for text files + const apiMessages = [ + systemPrompt, + ...this.messages.slice(0, -1).map((m) => { + // Build content including any text file attachments + let msgContent = m.content; + + // Add text attachments as context + if (m.attachments) { + for (const attachment of m.attachments) { + if (attachment.type === 'text' && attachment.content) { + msgContent += `\n\n[File: ${attachment.name}]\n\`\`\`\n${attachment.content}\n\`\`\``; + } + } + } + + return { + role: m.role, + content: msgContent + }; + }) + ]; + + // Determine the model to use - prefer selectedChatModel, otherwise try to get from instances + let modelToUse = this.selectedChatModel; + if (!modelToUse) { + // Try to get model from first running instance + for (const [, instanceWrapper] of Object.entries(this.instances)) { + if (instanceWrapper && typeof instanceWrapper === 'object') { + const keys = Object.keys(instanceWrapper as Record); + if (keys.length === 1) { + const instance = (instanceWrapper as Record)[keys[0]] as { shardAssignments?: { modelId?: string } }; + if (instance?.shardAssignments?.modelId) { + modelToUse = instance.shardAssignments.modelId; + break; + } + } + } + } + } + + if (!modelToUse) { + throw new Error('No model selected and no running instances available. Please launch an instance first.'); + } + + const conversationModelInfo = this.buildConversationModelInfo(modelToUse); + this.applyConversationModelInfo(conversationModelInfo); + + // Start timing for TTFT measurement + const requestStartTime = performance.now(); + let firstTokenTime: number | null = null; + let tokenCount = 0; + + const response = await fetch('/v1/chat/completions', { + method: 'POST', + headers: { + 'Content-Type': 'application/json' + }, + body: JSON.stringify({ + model: modelToUse, + messages: apiMessages, + temperature: 0.7, + stream: true + }) + }); + + if (!response.ok) { + const errorText = await response.text(); + throw new Error(`API error: ${response.status} - ${errorText}`); + } + + const reader = response.body?.getReader(); + if (!reader) { + throw new Error('No response body'); + } + + const decoder = new TextDecoder(); + let fullContent = ''; + let buffer = ''; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + buffer += decoder.decode(value, { stream: true }); + + // Process complete lines + const lines = buffer.split('\n'); + buffer = lines.pop() || ''; // Keep incomplete line in buffer + + for (const line of lines) { + const trimmed = line.trim(); + if (!trimmed) continue; + + if (trimmed.startsWith('data: ')) { + const data = trimmed.slice(6); + if (data === '[DONE]') continue; + + try { + const parsed = JSON.parse(data); + const tokenContent = parsed.choices?.[0]?.delta?.content; + if (tokenContent) { + // Track first token for TTFT + if (firstTokenTime === null) { + firstTokenTime = performance.now(); + this.ttftMs = firstTokenTime - requestStartTime; + } + + // Count tokens (each SSE chunk is typically one token) + tokenCount += 1; + this.totalTokens = tokenCount; + + // Update real-time TPS during streaming + if (firstTokenTime !== null && tokenCount > 1) { + const elapsed = performance.now() - firstTokenTime; + this.tps = (tokenCount / elapsed) * 1000; + } + + fullContent += tokenContent; + + // Strip thinking tags for display and extract thinking content + const { displayContent, thinkingContent } = this.stripThinkingTags(fullContent); + this.currentResponse = displayContent; + + // Update the assistant message in place + const idx = this.messages.findIndex(m => m.id === assistantMessage.id); + if (idx !== -1) { + this.messages[idx].content = displayContent; + this.messages[idx].thinking = thinkingContent || undefined; + } + this.persistActiveConversation(); + } + } catch { + // Skip invalid JSON lines + } + } + } + } + + // Process any remaining buffer + if (buffer.trim()) { + const trimmed = buffer.trim(); + if (trimmed.startsWith('data: ') && trimmed.slice(6) !== '[DONE]') { + try { + const parsed = JSON.parse(trimmed.slice(6)); + const tokenContent = parsed.choices?.[0]?.delta?.content; + if (tokenContent) { + fullContent += tokenContent; + this.persistActiveConversation(); + } + } catch { + // Skip + } + } + } + + // Calculate final TPS + if (firstTokenTime !== null && tokenCount > 1) { + const totalGenerationTime = performance.now() - firstTokenTime; + this.tps = (tokenCount / totalGenerationTime) * 1000; // tokens per second + } + + // Final cleanup of the message + const { displayContent, thinkingContent } = this.stripThinkingTags(fullContent); + const idx = this.messages.findIndex(m => m.id === assistantMessage.id); + if (idx !== -1) { + this.messages[idx].content = displayContent; + this.messages[idx].thinking = thinkingContent || undefined; + // Store performance metrics on the message + if (this.ttftMs !== null) { + this.messages[idx].ttftMs = this.ttftMs; + } + if (this.tps !== null) { + this.messages[idx].tps = this.tps; + } + } + this.persistActiveConversation(); + + } catch (error) { + console.error('Error sending message:', error); + // Update the assistant message with error + const idx = this.messages.findIndex(m => m.id === assistantMessage.id); + if (idx !== -1) { + this.messages[idx].content = `Error: ${error instanceof Error ? error.message : 'Failed to get response'}`; + } + this.persistActiveConversation(); + } finally { + this.isLoading = false; + this.currentResponse = ''; + this.updateActiveConversation(); + } + } + + /** + * Clear current chat and go back to welcome state + */ + clearChat() { + this.activeConversationId = null; + this.messages = []; + this.hasStartedChat = false; + this.isTopologyMinimized = false; + this.currentResponse = ''; + // Clear performance stats + this.ttftMs = null; + this.tps = null; + } + + /** + * Get the active conversation + */ + getActiveConversation(): Conversation | null { + if (!this.activeConversationId) return null; + return this.conversations.find(c => c.id === this.activeConversationId) || null; + } +} + +export const appStore = new AppStore(); + +// Reactive exports +export const hasStartedChat = () => appStore.hasStartedChat; +export const messages = () => appStore.messages; +export const currentResponse = () => appStore.currentResponse; +export const isLoading = () => appStore.isLoading; +export const ttftMs = () => appStore.ttftMs; +export const tps = () => appStore.tps; +export const totalTokens = () => appStore.totalTokens; +export const topologyData = () => appStore.topologyData; +export const instances = () => appStore.instances; +export const runners = () => appStore.runners; +export const downloads = () => appStore.downloads; +export const placementPreviews = () => appStore.placementPreviews; +export const selectedPreviewModelId = () => appStore.selectedPreviewModelId; +export const isLoadingPreviews = () => appStore.isLoadingPreviews; +export const lastUpdate = () => appStore.lastUpdate; +export const isTopologyMinimized = () => appStore.isTopologyMinimized; +export const selectedChatModel = () => appStore.selectedChatModel; +export const debugMode = () => appStore.getDebugMode(); + +// Actions +export const startChat = () => appStore.startChat(); +export const sendMessage = (content: string, files?: { id: string; name: string; type: string; textContent?: string; preview?: string }[]) => appStore.sendMessage(content, files); +export const clearChat = () => appStore.clearChat(); +export const setSelectedChatModel = (modelId: string) => appStore.setSelectedModel(modelId); +export const selectPreviewModel = (modelId: string | null) => appStore.selectPreviewModel(modelId); +export const deleteMessage = (messageId: string) => appStore.deleteMessage(messageId); +export const editMessage = (messageId: string, newContent: string) => appStore.editMessage(messageId, newContent); +export const editAndRegenerate = (messageId: string, newContent: string) => appStore.editAndRegenerate(messageId, newContent); +export const regenerateLastResponse = () => appStore.regenerateLastResponse(); + +// Conversation actions +export const conversations = () => appStore.conversations; +export const activeConversationId = () => appStore.activeConversationId; +export const createConversation = (name?: string) => appStore.createConversation(name); +export const loadConversation = (id: string) => appStore.loadConversation(id); +export const deleteConversation = (id: string) => appStore.deleteConversation(id); +export const deleteAllConversations = () => appStore.deleteAllConversations(); +export const renameConversation = (id: string, name: string) => appStore.renameConversation(id, name); +export const getActiveConversation = () => appStore.getActiveConversation(); + +// Sidebar actions +export const isSidebarOpen = () => appStore.isSidebarOpen; +export const toggleSidebar = () => appStore.toggleSidebar(); +export const toggleDebugMode = () => appStore.toggleDebugMode(); +export const setDebugMode = (enabled: boolean) => appStore.setDebugMode(enabled); +export const refreshState = () => appStore.fetchState(); + diff --git a/dashboard/src/lib/types/files.ts b/dashboard/src/lib/types/files.ts new file mode 100644 index 00000000..b92e269e --- /dev/null +++ b/dashboard/src/lib/types/files.ts @@ -0,0 +1,169 @@ +/** + * File attachment types for the chat interface + */ + +export interface ChatUploadedFile { + id: string; + name: string; + size: number; + type: string; + file: File; + preview?: string; + textContent?: string; +} + +export interface ChatAttachment { + type: 'image' | 'text' | 'pdf' | 'audio'; + name: string; + content?: string; + base64Url?: string; + mimeType?: string; +} + +export type FileCategory = 'image' | 'text' | 'pdf' | 'audio' | 'unknown'; + +export const IMAGE_EXTENSIONS = ['.jpg', '.jpeg', '.png', '.gif', '.webp', '.svg']; +export const IMAGE_MIME_TYPES = ['image/jpeg', 'image/png', 'image/gif', 'image/webp', 'image/svg+xml']; + +export const TEXT_EXTENSIONS = [ + '.txt', '.md', '.json', '.xml', '.yaml', '.yml', '.csv', '.log', + '.js', '.ts', '.jsx', '.tsx', '.py', '.java', '.cpp', '.c', '.h', + '.css', '.html', '.htm', '.sql', '.sh', '.bat', '.rs', '.go', + '.rb', '.php', '.swift', '.kt', '.scala', '.r', '.dart', '.vue', '.svelte' +]; +export const TEXT_MIME_TYPES = [ + 'text/plain', 'text/markdown', 'text/csv', 'text/html', 'text/css', + 'application/json', 'application/xml', 'text/xml', 'application/javascript', + 'text/javascript', 'application/typescript' +]; + +export const PDF_EXTENSIONS = ['.pdf']; +export const PDF_MIME_TYPES = ['application/pdf']; + +export const AUDIO_EXTENSIONS = ['.mp3', '.wav', '.ogg', '.m4a']; +export const AUDIO_MIME_TYPES = ['audio/mpeg', 'audio/wav', 'audio/ogg', 'audio/mp4']; + +/** + * Get file category based on MIME type and extension + */ +export function getFileCategory(mimeType: string, fileName: string): FileCategory { + const extension = fileName.toLowerCase().slice(fileName.lastIndexOf('.')); + + if (IMAGE_MIME_TYPES.includes(mimeType) || IMAGE_EXTENSIONS.includes(extension)) { + return 'image'; + } + if (PDF_MIME_TYPES.includes(mimeType) || PDF_EXTENSIONS.includes(extension)) { + return 'pdf'; + } + if (AUDIO_MIME_TYPES.includes(mimeType) || AUDIO_EXTENSIONS.includes(extension)) { + return 'audio'; + } + if (TEXT_MIME_TYPES.includes(mimeType) || TEXT_EXTENSIONS.includes(extension) || mimeType.startsWith('text/')) { + return 'text'; + } + return 'unknown'; +} + +/** + * Get accept string for file input based on categories + */ +export function getAcceptString(categories: FileCategory[]): string { + const accepts: string[] = []; + + for (const category of categories) { + switch (category) { + case 'image': + accepts.push(...IMAGE_EXTENSIONS, ...IMAGE_MIME_TYPES); + break; + case 'text': + accepts.push(...TEXT_EXTENSIONS, ...TEXT_MIME_TYPES); + break; + case 'pdf': + accepts.push(...PDF_EXTENSIONS, ...PDF_MIME_TYPES); + break; + case 'audio': + accepts.push(...AUDIO_EXTENSIONS, ...AUDIO_MIME_TYPES); + break; + } + } + + return accepts.join(','); +} + +/** + * Format file size for display + */ +export function formatFileSize(bytes: number): string { + if (bytes === 0) return '0 B'; + const k = 1024; + const sizes = ['B', 'KB', 'MB', 'GB']; + const i = Math.floor(Math.log(bytes) / Math.log(k)); + return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i]; +} + +/** + * Read file as data URL (base64) + */ +export function readFileAsDataURL(file: File): Promise { + return new Promise((resolve, reject) => { + const reader = new FileReader(); + reader.onload = () => resolve(reader.result as string); + reader.onerror = () => reject(reader.error); + reader.readAsDataURL(file); + }); +} + +/** + * Read file as text + */ +export function readFileAsText(file: File): Promise { + return new Promise((resolve, reject) => { + const reader = new FileReader(); + reader.onload = () => resolve(reader.result as string); + reader.onerror = () => reject(reader.error); + reader.readAsText(file); + }); +} + +/** + * Process uploaded files into ChatUploadedFile format + */ +export async function processUploadedFiles(files: File[]): Promise { + const results: ChatUploadedFile[] = []; + + for (const file of files) { + const id = Date.now().toString() + Math.random().toString(36).substring(2, 9); + const category = getFileCategory(file.type, file.name); + + const base: ChatUploadedFile = { + id, + name: file.name, + size: file.size, + type: file.type, + file + }; + + try { + if (category === 'image') { + const preview = await readFileAsDataURL(file); + results.push({ ...base, preview }); + } else if (category === 'text' || category === 'unknown') { + const textContent = await readFileAsText(file); + results.push({ ...base, textContent }); + } else if (category === 'pdf') { + results.push(base); + } else if (category === 'audio') { + const preview = await readFileAsDataURL(file); + results.push({ ...base, preview }); + } else { + results.push(base); + } + } catch (error) { + console.error('Error processing file:', file.name, error); + results.push(base); + } + } + + return results; +} + diff --git a/dashboard/src/routes/+layout.svelte b/dashboard/src/routes/+layout.svelte new file mode 100644 index 00000000..7e75b676 --- /dev/null +++ b/dashboard/src/routes/+layout.svelte @@ -0,0 +1,15 @@ + + + + EXO + + + +
+ {@render children?.()} +
+ diff --git a/dashboard/src/routes/+page.svelte b/dashboard/src/routes/+page.svelte new file mode 100644 index 00000000..082d1138 --- /dev/null +++ b/dashboard/src/routes/+page.svelte @@ -0,0 +1,1840 @@ + + + + + +
+ +
+ + +
+
+
+
+
+ + + + +
+ +
+ +
+ + {#if !chatStarted} + +
+ + +
+ + +
+ + + +
+ + +
+
+ +
+
+
+ + + + +
+ {:else} + +
+ +
+
+
+ +
+
+ +
+
+ +
+
+
+ + + {#if minimized} + + {/if} +
+ {/if} +
+ +
diff --git a/dashboard/src/routes/downloads/+page.svelte b/dashboard/src/routes/downloads/+page.svelte new file mode 100644 index 00000000..81e29ed9 --- /dev/null +++ b/dashboard/src/routes/downloads/+page.svelte @@ -0,0 +1,441 @@ + + +
+ +
+
+
+

Downloads

+

Overview of models on each node

+
+
+ +
+ Last update: {lastUpdateTs ? new Date(lastUpdateTs).toLocaleTimeString() : 'n/a'} +
+
+
+ + {#if !hasDownloads} +
+
No downloads found. Start a model download to see progress here.
+
+ Download keys detected: {downloadKeys.length === 0 ? 'none' : downloadKeys.join(', ')} +
+
+ {:else} +
+ {#each downloadOverview as node} +
+
+
+
{node.nodeName}
+
{node.nodeId}
+
+
+ {node.models.filter(m => m.status === 'completed').length} /{node.models.length} models +
+
+ + {#each node.models as model} + {@const key = `${node.nodeId}|${model.modelId}`} + {@const pct = clampPercent(model.percentage)} + {@const gradient = getBarGradient(pct)} + {@const isExpanded = expanded.has(key)} +
+
+
+
{model.prettyName ?? model.modelId}
+
+ {model.modelId} +
+
+ {formatBytes(model.downloadedBytes)} / {formatBytes(model.totalBytes)} +
+
+
+ + {pct.toFixed(1)}% + + +
+
+ +
+
+
+ +
+ {model.status === 'completed' ? 'Completed' : `${formatSpeed(model.speed)} • ETA ${formatEta(model.etaMs)}`} + {#if model.status !== 'completed'} + {model.files.length} file{model.files.length === 1 ? '' : 's'} + {/if} +
+ + {#if isExpanded} +
+ {#if model.files.length === 0} +
No file details reported.
+ {:else} + {#each model.files as f} + {@const fpct = clampPercent(f.percentage)} + {@const fgradient = getBarGradient(fpct)} +
+
+ {f.name} + {fpct.toFixed(1)}% +
+
+
+
+
+ {formatBytes(f.downloadedBytes)} / {formatBytes(f.totalBytes)} + {formatSpeed(f.speed)} • ETA {formatEta(f.etaMs)} +
+
+ {/each} + {/if} +
+ {/if} +
+ {/each} +
+ {/each} +
+ {/if} + +
+
+ + diff --git a/dashboard/static/exo-logo.png b/dashboard/static/exo-logo.png new file mode 100644 index 00000000..199bcfdd Binary files /dev/null and b/dashboard/static/exo-logo.png differ diff --git a/dashboard/static/favicon.ico b/dashboard/static/favicon.ico new file mode 100644 index 00000000..c0ae2099 Binary files /dev/null and b/dashboard/static/favicon.ico differ diff --git a/dashboard/svelte.config.js b/dashboard/svelte.config.js new file mode 100644 index 00000000..991b07b0 --- /dev/null +++ b/dashboard/svelte.config.js @@ -0,0 +1,28 @@ +import adapter from '@sveltejs/adapter-static'; +import { vitePreprocess } from '@sveltejs/vite-plugin-svelte'; + +/** @type {import('@sveltejs/kit').Config} */ +const config = { + preprocess: [vitePreprocess()], + + kit: { + paths: { + relative: true + }, + router: { type: 'hash' }, + adapter: adapter({ + pages: 'build', + assets: 'build', + fallback: 'index.html', + precompress: false, + strict: true + }), + alias: { + $lib: 'src/lib', + $components: 'src/lib/components' + } + } +}; + +export default config; + diff --git a/dashboard/tsconfig.json b/dashboard/tsconfig.json new file mode 100644 index 00000000..51db996c --- /dev/null +++ b/dashboard/tsconfig.json @@ -0,0 +1,15 @@ +{ + "extends": "./.svelte-kit/tsconfig.json", + "compilerOptions": { + "allowJs": true, + "checkJs": true, + "esModuleInterop": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "skipLibCheck": true, + "sourceMap": true, + "strict": true, + "moduleResolution": "bundler" + } +} + diff --git a/dashboard/vite.config.ts b/dashboard/vite.config.ts new file mode 100644 index 00000000..4d22f688 --- /dev/null +++ b/dashboard/vite.config.ts @@ -0,0 +1,16 @@ +import tailwindcss from '@tailwindcss/vite'; +import { sveltekit } from '@sveltejs/kit/vite'; +import { defineConfig } from 'vite'; + +export default defineConfig({ + plugins: [tailwindcss(), sveltekit()], + server: { + proxy: { + '/v1': 'http://localhost:8000', + '/state': 'http://localhost:8000', + '/models': 'http://localhost:8000', + '/instance': 'http://localhost:8000' + } + } +}); + diff --git a/flake.lock b/flake.lock new file mode 100644 index 00000000..869ba848 --- /dev/null +++ b/flake.lock @@ -0,0 +1,121 @@ +{ + "nodes": { + "fenix": { + "inputs": { + "nixpkgs": [ + "nixpkgs" + ], + "rust-analyzer-src": "rust-analyzer-src" + }, + "locked": { + "lastModified": 1761893049, + "narHash": "sha256-1TtFDPhC+ZsrOOtBnry1EZC+WipTTvsOVjIEVugqji8=", + "owner": "nix-community", + "repo": "fenix", + "rev": "c2ac9a5c0d6d16630c3b225b874bd14528d1abe6", + "type": "github" + }, + "original": { + "owner": "nix-community", + "repo": "fenix", + "type": "github" + } + }, + "flake-utils": { + "inputs": { + "systems": "systems" + }, + "locked": { + "lastModified": 1731533236, + "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", + "owner": "numtide", + "repo": "flake-utils", + "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "flake-utils", + "type": "github" + } + }, + "nixpkgs": { + "locked": { + "lastModified": 1761672384, + "narHash": "sha256-o9KF3DJL7g7iYMZq9SWgfS1BFlNbsm6xplRjVlOCkXI=", + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "08dacfca559e1d7da38f3cf05f1f45ee9bfd213c", + "type": "github" + }, + "original": { + "owner": "NixOS", + "ref": "nixos-unstable", + "repo": "nixpkgs", + "type": "github" + } + }, + "root": { + "inputs": { + "fenix": "fenix", + "flake-utils": "flake-utils", + "nixpkgs": "nixpkgs", + "treefmt-nix": "treefmt-nix" + } + }, + "rust-analyzer-src": { + "flake": false, + "locked": { + "lastModified": 1761849405, + "narHash": "sha256-igXdvC+WCUN+3gnfk+ptT7rMmxQuY6WbIg1rXMUN1DM=", + "owner": "rust-lang", + "repo": "rust-analyzer", + "rev": "f7de8ae045a5fe80f1203c5a1c3015b05f7c3550", + "type": "github" + }, + "original": { + "owner": "rust-lang", + "ref": "nightly", + "repo": "rust-analyzer", + "type": "github" + } + }, + "systems": { + "locked": { + "lastModified": 1681028828, + "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", + "owner": "nix-systems", + "repo": "default", + "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", + "type": "github" + }, + "original": { + "owner": "nix-systems", + "repo": "default", + "type": "github" + } + }, + "treefmt-nix": { + "inputs": { + "nixpkgs": [ + "nixpkgs" + ] + }, + "locked": { + "lastModified": 1762938485, + "narHash": "sha256-AlEObg0syDl+Spi4LsZIBrjw+snSVU4T8MOeuZJUJjM=", + "owner": "numtide", + "repo": "treefmt-nix", + "rev": "5b4ee75aeefd1e2d5a1cc43cf6ba65eba75e83e4", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "treefmt-nix", + "type": "github" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/flake.nix b/flake.nix new file mode 100644 index 00000000..9d3ade75 --- /dev/null +++ b/flake.nix @@ -0,0 +1,111 @@ +{ + description = "The development environment for Exo"; + + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; + flake-utils.url = "github:numtide/flake-utils"; + # Provides Rust dev-env integration: + fenix = { + url = "github:nix-community/fenix"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + # Provides formatting infrastructure: + treefmt-nix = { + url = "github:numtide/treefmt-nix"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + }; + + # TODO: figure out caching story + # nixConfig = { + # # nix community cachix + # extra-trusted-public-keys = "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="; + # extra-substituters = "https://nix-community.cachix.org"; + # }; + + outputs = + inputs: + let + systems = [ + "x86_64-linux" + "aarch64-darwin" + "aarch64-linux" + ]; + fenixToolchain = system: inputs.fenix.packages.${system}.complete; + in + inputs.flake-utils.lib.eachSystem systems ( + system: + let + pkgs = import inputs.nixpkgs { + inherit system; + overlays = [ inputs.fenix.overlays.default ]; + }; + treefmtEval = inputs.treefmt-nix.lib.evalModule pkgs { + projectRootFile = "flake.nix"; + programs.ruff-format.enable = true; + programs.ruff-format.excludes = [ "rust/exo_pyo3_bindings/exo_pyo3_bindings.pyi" ]; + programs.rustfmt.enable = true; + programs.rustfmt.package = (fenixToolchain system).rustfmt; + programs.nixpkgs-fmt.enable = true; + }; + in + { + formatter = treefmtEval.config.build.wrapper; + checks.formatting = treefmtEval.config.build.check inputs.self; + checks.lint = pkgs.runCommand "lint-check" { } '' + export RUFF_CACHE_DIR="$TMPDIR/ruff-cache" + ${pkgs.ruff}/bin/ruff check ${inputs.self}/ + touch $out + ''; + + devShells.default = pkgs.mkShell { + packages = + with pkgs; + [ + # PYTHON + python313 + uv + ruff + basedpyright + + # RUST + ((fenixToolchain system).withComponents [ + "cargo" + "rustc" + "clippy" + "rustfmt" + "rust-src" + ]) + rustup # Just here to make RustRover happy + + # NIX + nixpkgs-fmt + + # SVELTE + nodejs + + # MISC + just + jq + ] + ++ (pkgs.lib.optionals pkgs.stdenv.isLinux [ + # IFCONFIG + unixtools.ifconfig + ]) + ++ (pkgs.lib.optionals pkgs.stdenv.isDarwin [ + # MACMON + macmon + ]); + + shellHook = '' + # PYTHON + export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${pkgs.python313}/lib" + echo + echo "🍎🍎 Run 'just ' to get started" + just --list + ''; + + }; + } + ); +} diff --git a/justfile b/justfile new file mode 100644 index 00000000..0a82d616 --- /dev/null +++ b/justfile @@ -0,0 +1,38 @@ +fmt: + nix fmt + +lint: + uv run ruff check --fix + +test: + uv run pytest src + +check: + uv run basedpyright --project pyproject.toml + +sync: + uv sync --all-packages + +sync-clean: + uv sync --all-packages --force-reinstall --no-cache + +rust-rebuild: + cargo run --bin stub_gen + just sync-clean + +build-dashboard: + #!/usr/bin/env bash + cd dashboard + npm install + npm run build + +package: + uv run pyinstaller packaging/pyinstaller/exo.spec + +clean: + rm -rf **/__pycache__ + rm -rf target/ + rm -rf .venv + rm -rf dashboard/node_modules + rm -rf dashboard/.svelte-kit + rm -rf dashboard/build diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 00000000..d9c4715e --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,135 @@ +[project] +name = "exo" +version = "0.3.0" +description = "Exo" +readme = "README.md" +requires-python = ">=3.13" +dependencies = [ + "aiofiles>=24.1.0", + "aiohttp>=3.12.14", + "types-aiofiles>=24.1.0.20250708", + "typeguard>=4.4.4", + "pydantic>=2.11.7", + "base58>=2.1.1", + "cryptography>=45.0.5", + "fastapi>=0.116.1", + "filelock>=3.18.0", + "aiosqlite>=0.21.0", + "networkx>=3.5", + "pathlib>=1.0.1", + "protobuf>=6.32.0", + "rich>=14.1.0", + "rustworkx>=0.17.1", + "sqlmodel>=0.0.24", + "sqlalchemy[asyncio]>=2.0.43", + "greenlet>=3.2.4", + "huggingface-hub>=0.33.4", + "psutil>=7.0.0", + "loguru>=0.7.3", + "textual>=5.3.0", + "exo_pyo3_bindings", # rust bindings + "anyio==4.11.0", + "bidict>=0.23.1", + "mlx>=0.29.3", + "mlx-lm>=0.28.3", + "tiktoken>=0.12.0", # required for kimi k2 tokenizer + "hypercorn>=0.18.0", +] + +[project.scripts] +exo-master = "exo.master.main:main" +exo-worker = "exo.worker.main:main" +exo = "exo.main:main" + +# dependencies only required for development +[dependency-groups] +dev = [ + "pytest>=8.4.0", + "pytest-asyncio>=1.0.0", + "pytest-env", + "ruff>=0.11.13", +] + +# mlx[cuda] requires a newer version of mlx. the ideal on linux is: default to mlx[cpu] unless[cuda] specified. +[project.optional-dependencies] +# cuda = [ +# "mlx[cuda]==0.26.3", +# ] + +### +# workspace configuration +### + +[tool.uv.workspace] +members = [ + "rust/exo_pyo3_bindings", +] + +[tool.uv.sources] +exo_pyo3_bindings = { workspace = true } +# Uncomment to use local mlx/mlx-lm development versions: +# mlx = { path = "/Users/Shared/mlx", editable=true } +# mlx-lm = { path = "/Users/Shared/mlx-lm", editable=true } + +[build-system] +requires = ["uv_build>=0.8.9,<0.9.0"] +build-backend = "uv_build" + +### +# type-checker configuration +### + +[tool.basedpyright] +include = [".venv/lib/mlx", ".venv/lib/mlx_lm", "src"] +typeCheckingMode = "strict" +failOnWarnings = true + +reportAny = "error" +reportUnknownVariableType = "error" +reportUnknownParameterType = "error" +reportMissingParameterType = "error" +reportMissingTypeStubs = "error" +reportInvalidCast = "error" +reportUnnecessaryCast = "error" +reportUnnecessaryTypeIgnoreComment = "error" + +pythonVersion = "3.13" +pythonPlatform = "Darwin" + +exclude = ["**/.venv", "**/venv", "**/__pycache__", "**/exo_scripts", "**/.direnv", "**/rust", "**/.github"] +stubPath = ".mlx_typings" + +[[tool.basedpyright.executionEnvironments]] +root = "src" + +### +# uv configuration +### + +# supported platforms for this project +[tool.uv] +environments = [ + "sys_platform == 'darwin'", + "sys_platform == 'linux'", +] + +### +# ruff configuration +### + +[tool.ruff] +extend-exclude = ["shared/protobufs/**", "*mlx_typings/**", "rust/exo_pyo3_bindings/**"] + +[tool.ruff.lint] +extend-select = ["I", "N", "B", "A", "PIE", "SIM"] + +[tool.pytest.ini_options] +pythonpath = "." +asyncio_mode = "auto" +markers = [ + "slow: marks tests as slow (deselected by default)" +] +env = [ + "EXO_TESTS=1" +] +addopts = "-m 'not slow'" diff --git a/rust/clippy.toml b/rust/clippy.toml new file mode 100644 index 00000000..6d5a6187 --- /dev/null +++ b/rust/clippy.toml @@ -0,0 +1,2 @@ +# we can manually exclude false-positive lint errors for dual packages (if in dependencies) +#allowed-duplicate-crates = ["hashbrown"] \ No newline at end of file diff --git a/rust/exo_pyo3_bindings/Cargo.toml b/rust/exo_pyo3_bindings/Cargo.toml new file mode 100644 index 00000000..12803ab4 --- /dev/null +++ b/rust/exo_pyo3_bindings/Cargo.toml @@ -0,0 +1,77 @@ +[package] +name = "exo_pyo3_bindings" +version = { workspace = true } +edition = { workspace = true } +publish = false + +[lib] +doctest = false +path = "src/lib.rs" +name = "exo_pyo3_bindings" + +# "cdylib" needed to produce shared library for Python to import +# "rlib" needed for stub-gen to run +crate-type = ["cdylib", "rlib"] + +[[bin]] +path = "src/bin/stub_gen.rs" +name = "stub_gen" +doc = false + +[lints] +workspace = true + +[dependencies] +networking = { workspace = true } + +# interop +pyo3 = { version = "0.27.1", features = [ + # "abi3-py311", # tells pyo3 (and maturin) to build using the stable ABI with minimum Python version 3.11 + "nightly", # enables better-supported GIL integration + "experimental-async", # async support in #[pyfunction] & #[pymethods] + #"experimental-inspect", # inspection of generated binary => easier to automate type-hint generation + #"py-clone", # adding Clone-ing of `Py` without GIL (may cause panics - remove if panics happen) + "multiple-pymethods", # allows multiple #[pymethods] sections per class + + # integrations with other libraries + "arc_lock", "bigdecimal", "either", "hashbrown", "indexmap", "num-bigint", "num-complex", "num-rational", + "ordered-float", "rust_decimal", "smallvec", + # "anyhow", "chrono", "chrono-local", "chrono-tz", "eyre", "jiff-02", "lock_api", "parking-lot", "time", "serde", +] } +pyo3-stub-gen = { version = "0.17.2" } +pyo3-async-runtimes = { version = "0.27.0", features = ["attributes", "tokio-runtime", "testing"] } +pyo3-log = "0.13.2" + +# macro dependencies +extend = { workspace = true } +delegate = { workspace = true } +impl-trait-for-tuples = { workspace = true } +derive_more = { workspace = true } +pin-project = { workspace = true } + +# async runtime +tokio = { workspace = true, features = ["full", "tracing"] } +futures = { workspace = true } + +# utility dependencies +once_cell = "1.21.3" +thread_local = "1.1.9" +util = { workspace = true } +thiserror = { workspace = true } +#internment = { workspace = true } +#recursion = { workspace = true } +#generativity = { workspace = true } +#itertools = { workspace = true } + + +# Tracing +#tracing = "0.1" +#tracing-subscriber = "0.3" +#console-subscriber = "0.1.5" +#tracing-log = "0.2.0" +log = { workspace = true } +env_logger = "0.11" + + +# Networking +libp2p = { workspace = true, features = ["full"] } diff --git a/rust/exo_pyo3_bindings/README.md b/rust/exo_pyo3_bindings/README.md new file mode 100644 index 00000000..e739dd89 --- /dev/null +++ b/rust/exo_pyo3_bindings/README.md @@ -0,0 +1 @@ +TODO: do something here.... diff --git a/rust/exo_pyo3_bindings/exo_pyo3_bindings.pyi b/rust/exo_pyo3_bindings/exo_pyo3_bindings.pyi new file mode 100644 index 00000000..fa6700ff --- /dev/null +++ b/rust/exo_pyo3_bindings/exo_pyo3_bindings.pyi @@ -0,0 +1,221 @@ +# This file is automatically generated by pyo3_stub_gen +# ruff: noqa: E501, F401 + +import builtins +import enum +import typing + +@typing.final +class AllQueuesFullError(builtins.Exception): + def __new__(cls, *args: typing.Any) -> AllQueuesFullError: ... + def __repr__(self) -> builtins.str: ... + def __str__(self) -> builtins.str: ... + +@typing.final +class ConnectionUpdate: + @property + def update_type(self) -> ConnectionUpdateType: + r""" + Whether this is a connection or disconnection event + """ + @property + def peer_id(self) -> PeerId: + r""" + Identity of the peer that we have connected to or disconnected from. + """ + @property + def remote_ipv4(self) -> builtins.str: + r""" + Remote connection's IPv4 address. + """ + @property + def remote_tcp_port(self) -> builtins.int: + r""" + Remote connection's TCP port. + """ + +@typing.final +class Keypair: + r""" + Identity keypair of a node. + """ + @staticmethod + def generate_ed25519() -> Keypair: + r""" + Generate a new Ed25519 keypair. + """ + @staticmethod + def generate_ecdsa() -> Keypair: + r""" + Generate a new ECDSA keypair. + """ + @staticmethod + def generate_secp256k1() -> Keypair: + r""" + Generate a new Secp256k1 keypair. + """ + @staticmethod + def from_protobuf_encoding(bytes: bytes) -> Keypair: + r""" + Decode a private key from a protobuf structure and parse it as a `Keypair`. + """ + @staticmethod + def rsa_from_pkcs8(bytes: bytes) -> Keypair: + r""" + Decode an keypair from a DER-encoded secret key in PKCS#8 `PrivateKeyInfo` + format (i.e. unencrypted) as defined in [RFC5208]. + + [RFC5208]: https://tools.ietf.org/html/rfc5208#section-5 + """ + @staticmethod + def secp256k1_from_der(bytes: bytes) -> Keypair: + r""" + Decode a keypair from a DER-encoded Secp256k1 secret key in an `ECPrivateKey` + structure as defined in [RFC5915]. + + [RFC5915]: https://tools.ietf.org/html/rfc5915 + """ + @staticmethod + def ed25519_from_bytes(bytes: bytes) -> Keypair: ... + def to_protobuf_encoding(self) -> bytes: + r""" + Encode a private key as protobuf structure. + """ + def to_peer_id(self) -> PeerId: + r""" + Convert the `Keypair` into the corresponding `PeerId`. + """ + +@typing.final +class Multiaddr: + r""" + Representation of a Multiaddr. + """ + @staticmethod + def empty() -> Multiaddr: + r""" + Create a new, empty multiaddress. + """ + @staticmethod + def with_capacity(n: builtins.int) -> Multiaddr: + r""" + Create a new, empty multiaddress with the given capacity. + """ + @staticmethod + def from_bytes(bytes: bytes) -> Multiaddr: + r""" + Parse a `Multiaddr` value from its byte slice representation. + """ + @staticmethod + def from_string(string: builtins.str) -> Multiaddr: + r""" + Parse a `Multiaddr` value from its string representation. + """ + def len(self) -> builtins.int: + r""" + Return the length in bytes of this multiaddress. + """ + def is_empty(self) -> builtins.bool: + r""" + Returns true if the length of this multiaddress is 0. + """ + def to_bytes(self) -> bytes: + r""" + Return a copy of this [`Multiaddr`]'s byte representation. + """ + def to_string(self) -> builtins.str: + r""" + Convert a Multiaddr to a string. + """ + +@typing.final +class NetworkingHandle: + def __new__(cls, identity: Keypair) -> NetworkingHandle: ... + async def connection_update_recv(self) -> ConnectionUpdate: + r""" + Receives the next `ConnectionUpdate` from networking. + """ + async def connection_update_recv_many(self, limit: builtins.int) -> builtins.list[ConnectionUpdate]: + r""" + Receives at most `limit` `ConnectionUpdate`s from networking and returns them. + + For `limit = 0`, an empty collection of `ConnectionUpdate`s will be returned immediately. + For `limit > 0`, if there are no `ConnectionUpdate`s in the channel's queue this method + will sleep until a `ConnectionUpdate`s is sent. + """ + async def gossipsub_subscribe(self, topic: builtins.str) -> builtins.bool: + r""" + Subscribe to a `GossipSub` topic. + + Returns `True` if the subscription worked. Returns `False` if we were already subscribed. + """ + async def gossipsub_unsubscribe(self, topic: builtins.str) -> builtins.bool: + r""" + Unsubscribes from a `GossipSub` topic. + + Returns `True` if we were subscribed to this topic. Returns `False` if we were not subscribed. + """ + async def gossipsub_publish(self, topic: builtins.str, data: bytes) -> None: + r""" + Publishes a message with multiple topics to the `GossipSub` network. + + If no peers are found that subscribe to this topic, throws `NoPeersSubscribedToTopicError` exception. + """ + async def gossipsub_recv(self) -> tuple[builtins.str, bytes]: + r""" + Receives the next message from the `GossipSub` network. + """ + async def gossipsub_recv_many(self, limit: builtins.int) -> builtins.list[tuple[builtins.str, bytes]]: + r""" + Receives at most `limit` messages from the `GossipSub` network and returns them. + + For `limit = 0`, an empty collection of messages will be returned immediately. + For `limit > 0`, if there are no messages in the channel's queue this method + will sleep until a message is sent. + """ + +@typing.final +class NoPeersSubscribedToTopicError(builtins.Exception): + def __new__(cls, *args: typing.Any) -> NoPeersSubscribedToTopicError: ... + def __repr__(self) -> builtins.str: ... + def __str__(self) -> builtins.str: ... + +@typing.final +class PeerId: + r""" + Identifier of a peer of the network. + + The data is a `CIDv0` compatible multihash of the protobuf encoded public key of the peer + as specified in [specs/peer-ids](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md). + """ + @staticmethod + def random() -> PeerId: + r""" + Generates a random peer ID from a cryptographically secure PRNG. + + This is useful for randomly walking on a DHT, or for testing purposes. + """ + @staticmethod + def from_bytes(bytes: bytes) -> PeerId: + r""" + Parses a `PeerId` from bytes. + """ + def to_bytes(self) -> bytes: + r""" + Returns a raw bytes representation of this `PeerId`. + """ + def to_base58(self) -> builtins.str: + r""" + Returns a base-58 encoded string of this `PeerId`. + """ + def __repr__(self) -> builtins.str: ... + def __str__(self) -> builtins.str: ... + +@typing.final +class ConnectionUpdateType(enum.Enum): + r""" + Connection or disconnection event discriminant type. + """ + Connected = ... + Disconnected = ... + diff --git a/rust/exo_pyo3_bindings/pyproject.toml b/rust/exo_pyo3_bindings/pyproject.toml new file mode 100644 index 00000000..fbe53a84 --- /dev/null +++ b/rust/exo_pyo3_bindings/pyproject.toml @@ -0,0 +1,32 @@ +[build-system] +requires = ["maturin>=1.0,<2.0"] +build-backend = "maturin" + +[project] +name = "exo_pyo3_bindings" +version = "0.1.0" +description = "Add your description here" +readme = "README.md" +authors = [ + { name = "Andrei Cravtov", email = "the.andrei.cravtov@gmail.com" } +] +requires-python = ">=3.13" +dependencies = [] + +[dependency-groups] +dev = [ + "exo_pyo3_bindings", + "pytest>=8.4.0", + "pytest-asyncio>=1.0.0", +] + +[tool.maturin] +#purelib = true +#python-source = "python" +module-name = "exo_pyo3_bindings" +features = ["pyo3/extension-module", "pyo3/experimental-async"] + +[tool.pytest.ini_options] +log_cli = true +log_cli_level = "INFO" +asyncio_mode = "auto" diff --git a/rust/exo_pyo3_bindings/src/allow_threading.rs b/rust/exo_pyo3_bindings/src/allow_threading.rs new file mode 100644 index 00000000..3106e535 --- /dev/null +++ b/rust/exo_pyo3_bindings/src/allow_threading.rs @@ -0,0 +1,40 @@ +//! SEE: https://pyo3.rs/v0.26.0/async-await.html#detaching-from-the-interpreter-across-await +//! + +use pin_project::pin_project; +use pyo3::marker::Ungil; +use pyo3::prelude::*; +use std::{ + future::Future, + pin::{Pin, pin}, + task::{Context, Poll}, +}; + +/// SEE: https://pyo3.rs/v0.26.0/async-await.html#detaching-from-the-interpreter-across-await +#[pin_project] +#[repr(transparent)] +pub(crate) struct AllowThreads(#[pin] F); + +impl AllowThreads +where + Self: Future, +{ + pub fn new(f: F) -> Self { + Self(f) + } +} + +impl Future for AllowThreads +where + F: Future + Ungil, + F::Output: Ungil, +{ + type Output = F::Output; + + fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll { + let waker = cx.waker(); + Python::with_gil(|py| { + py.allow_threads(|| self.project().0.poll(&mut Context::from_waker(waker))) + }) + } +} diff --git a/rust/exo_pyo3_bindings/src/bin/stub_gen.rs b/rust/exo_pyo3_bindings/src/bin/stub_gen.rs new file mode 100644 index 00000000..3e30f493 --- /dev/null +++ b/rust/exo_pyo3_bindings/src/bin/stub_gen.rs @@ -0,0 +1,8 @@ +use pyo3_stub_gen::Result; + +fn main() -> Result<()> { + env_logger::Builder::from_env(env_logger::Env::default().filter_or("RUST_LOG", "info")).init(); + let stub = exo_pyo3_bindings::stub_info()?; + stub.generate()?; + Ok(()) +} diff --git a/rust/exo_pyo3_bindings/src/examples/mod.rs b/rust/exo_pyo3_bindings/src/examples/mod.rs new file mode 100644 index 00000000..bde14199 --- /dev/null +++ b/rust/exo_pyo3_bindings/src/examples/mod.rs @@ -0,0 +1,240 @@ +//! This module exists to hold examples of some pyo3 patterns that may be too complex to +//! re-create from scratch, but too inhomogenous to create an abstraction/wrapper around. +//! +//! Pattern examples include: +//! - Async task handles: with GC-integrated cleanup +//! - Sync/async callbacks from python: with propper eventloop handling +//! +//! Mutability pattern: https://pyo3.rs/v0.26.0/async-await.html#send--static-constraint +//! - Store mutable fields in tokio's `Mutex` +//! - For async code: take `&self` and `.lock().await` +//! - For sync code: take `&mut self` and `.get_mut()` + +use crate::ext::{PyResultExt as _, ResultExt as _, TokioRuntimeExt as _}; +use futures::FutureExt as _; +use futures::future::BoxFuture; +use pyo3::exceptions::PyRuntimeError; +use pyo3::prelude::{PyModule, PyModuleMethods as _}; +use pyo3::{ + Bound, Py, PyAny, PyErr, PyResult, PyTraverseError, PyVisit, Python, pyclass, pymethods, +}; +use std::time::Duration; +use tokio::sync::mpsc; +use tokio::sync::mpsc::error::TryRecvError; + +fn needs_tokio_runtime() { + tokio::runtime::Handle::current(); +} + +type SyncCallback = Box; +type AsyncCallback = Box BoxFuture<'static, ()> + Send + Sync>; + +enum AsyncTaskMessage { + SyncCallback(SyncCallback), + AsyncCallback(AsyncCallback), +} + +async fn async_task( + sender: mpsc::UnboundedSender<()>, + mut receiver: mpsc::UnboundedReceiver, +) { + log::info!("RUST: async task started"); + + // task state + let mut interval = tokio::time::interval(Duration::from_secs(1)); + + let mut sync_cbs: Vec = vec![]; + let mut async_cbs: Vec = vec![]; + + loop { + tokio::select! { + // handle incoming messages from task-handle + message = receiver.recv() => { + // handle closed channel by exiting + let Some(message) = message else { + log::info!("RUST: channel closed"); + break; + }; + + // dispatch incoming event + match message { + AsyncTaskMessage::SyncCallback(cb) => { + sync_cbs.push(cb); + } + AsyncTaskMessage::AsyncCallback(cb) => { + async_cbs.push(cb); + } + } + } + + // handle all other events + _ = interval.tick() => { + log::info!("RUST: async task tick"); + + // call back all sync callbacks + for cb in &sync_cbs { + cb(); + } + + // call back all async callbacks + for cb in &async_cbs { + cb().await; + } + + // send event on unbounded channel + sender.send(()).expect("handle receiver cannot be closed/dropped"); + } + } + } + + log::info!("RUST: async task stopped"); +} + +// #[gen_stub_pyclass] +#[pyclass(name = "AsyncTaskHandle")] +#[derive(Debug)] +struct PyAsyncTaskHandle { + sender: Option>, + receiver: mpsc::UnboundedReceiver<()>, +} + +#[allow(clippy::expect_used)] +impl PyAsyncTaskHandle { + const fn sender(&self) -> &mpsc::UnboundedSender { + self.sender + .as_ref() + .expect("The sender should only be None after de-initialization.") + } + + const fn sender_mut(&mut self) -> &mpsc::UnboundedSender { + self.sender + .as_mut() + .expect("The sender should only be None after de-initialization.") + } + + const fn new( + sender: mpsc::UnboundedSender, + receiver: mpsc::UnboundedReceiver<()>, + ) -> Self { + Self { + sender: Some(sender), + receiver, + } + } +} + +// #[gen_stub_pymethods] +#[pymethods] +impl PyAsyncTaskHandle { + #[new] + fn py_new(py: Python<'_>) -> PyResult { + use pyo3_async_runtimes::tokio::get_runtime; + + // create communication channel TOWARDS our task + let (h_sender, t_receiver) = mpsc::unbounded_channel::(); + + // create communication channel FROM our task + let (t_sender, h_receiver) = mpsc::unbounded_channel::<()>(); + + // perform necessary setup within tokio context - or it crashes + let () = get_runtime().block_on(async { needs_tokio_runtime() }); + + // spawn tokio task with this thread's task-locals - without this, async callbacks on the new threads will not work!! + _ = get_runtime().spawn_with_scope(py, async move { + async_task(t_sender, t_receiver).await; + }); + Ok(Self::new(h_sender, h_receiver)) + } + + /// NOTE: exceptions in callbacks are silently ignored until end of execution + fn add_sync_callback( + &self, + // #[gen_stub(override_type( + // type_repr="collections.abc.Callable[[], None]", + // imports=("collections.abc") + // ))] + callback: Py, + ) -> PyResult<()> { + // blocking call to async method -> can do non-blocking if needed + self.sender() + .send(AsyncTaskMessage::SyncCallback(Box::new(move || { + _ = Python::with_gil(|py| callback.call0(py).write_unraisable_with(py)); + }))) + .pyerr()?; + Ok(()) + } + + /// NOTE: exceptions in callbacks are silently ignored until end of execution + fn add_async_callback( + &self, + // #[gen_stub(override_type( + // type_repr="collections.abc.Callable[[], collections.abc.Awaitable[None]]", + // imports=("collections.abc") + // ))] + callback: Py, + ) -> PyResult<()> { + // blocking call to async method -> can do non-blocking if needed + self.sender() + .send(AsyncTaskMessage::AsyncCallback(Box::new(move || { + let c = Python::with_gil(|py| callback.clone_ref(py)); + async move { + if let Some(f) = Python::with_gil(|py| { + let coroutine = c.call0(py).write_unraisable_with(py)?; + pyo3_async_runtimes::tokio::into_future(coroutine.into_bound(py)) + .write_unraisable_with(py) + }) { + _ = f.await.write_unraisable(); + } + } + .boxed() + }))) + .pyerr()?; + Ok(()) + } + + async fn receive_unit(&mut self) -> PyResult<()> { + self.receiver + .recv() + .await + .ok_or(PyErr::new::( + "cannot receive unit on closed channel", + )) + } + + fn drain_units(&mut self) -> PyResult { + let mut cnt = 0; + loop { + match self.receiver.try_recv() { + Err(TryRecvError::Disconnected) => { + return Err(PyErr::new::( + "cannot receive unit on closed channel", + )); + } + Err(TryRecvError::Empty) => return Ok(cnt), + Ok(()) => { + cnt += 1; + continue; + } + } + } + } + + // #[gen_stub(skip)] + const fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { + Ok(()) // This is needed purely so `__clear__` can work + } + + // #[gen_stub(skip)] + fn __clear__(&mut self) { + // TODO: may or may not need to await a "kill-signal" oneshot channel message, + // to ensure that the networking task is done BEFORE exiting the clear function... + // but this may require GIL?? and it may not be safe to call GIL here?? + self.sender = None; // Using Option as a trick to force `sender` channel to be dropped + } +} + +pub fn examples_submodule(m: &Bound<'_, PyModule>) -> PyResult<()> { + m.add_class::()?; + + Ok(()) +} diff --git a/rust/exo_pyo3_bindings/src/lib.rs b/rust/exo_pyo3_bindings/src/lib.rs new file mode 100644 index 00000000..4f591b8c --- /dev/null +++ b/rust/exo_pyo3_bindings/src/lib.rs @@ -0,0 +1,217 @@ +//! TODO: crate documentation +//! +//! this is here as a placeholder documentation +//! +//! + +// enable Rust-unstable features for convenience +#![feature(trait_alias)] +#![feature(tuple_trait)] +#![feature(unboxed_closures)] +// #![feature(stmt_expr_attributes)] +// #![feature(assert_matches)] +// #![feature(async_fn_in_dyn_trait)] +// #![feature(async_for_loop)] +// #![feature(auto_traits)] +// #![feature(negative_impls)] + +extern crate core; +mod allow_threading; +mod examples; +pub(crate) mod networking; +pub(crate) mod pylibp2p; + +use crate::networking::networking_submodule; +use crate::pylibp2p::ident::ident_submodule; +use crate::pylibp2p::multiaddr::multiaddr_submodule; +use pyo3::prelude::PyModule; +use pyo3::prelude::*; +use pyo3::{Bound, PyResult, pyclass, pymodule}; +use pyo3_stub_gen::define_stub_info_gatherer; + +/// Namespace for all the constants used by this crate. +pub(crate) mod r#const { + pub const MPSC_CHANNEL_SIZE: usize = 1024; +} + +/// Namespace for all the type/trait aliases used by this crate. +pub(crate) mod alias { + use std::error::Error; + use std::marker::Tuple; + + pub trait SendFn = + Fn + Send + 'static; + + pub type AnyError = Box; + pub type AnyResult = Result; +} + +/// Namespace for crate-wide extension traits/methods +pub(crate) mod ext { + use crate::allow_threading::AllowThreads; + use extend::ext; + use pyo3::exceptions::{PyConnectionError, PyRuntimeError}; + use pyo3::marker::Ungil; + use pyo3::types::PyBytes; + use pyo3::{Py, PyErr, PyResult, Python}; + use tokio::runtime::Runtime; + use tokio::sync::mpsc; + use tokio::sync::mpsc::error::TryRecvError; + use tokio::task::JoinHandle; + + #[ext(pub, name = ByteArrayExt)] + impl [u8] { + fn pybytes(&self) -> Py { + Python::with_gil(|py| PyBytes::new(py, self).unbind()) + } + } + + #[ext(pub, name = ResultExt)] + impl Result + where + E: ToString, + { + fn pyerr(self) -> PyResult { + self.map_err(|e| PyRuntimeError::new_err(e.to_string())) + } + } + + pub trait FutureExt: Future + Sized { + /// SEE: https://pyo3.rs/v0.26.0/async-await.html#detaching-from-the-interpreter-across-await + fn allow_threads_py(self) -> AllowThreads + where + AllowThreads: Future, + { + AllowThreads::new(self) + } + } + + impl FutureExt for T {} + + #[ext(pub, name = PyErrExt)] + impl PyErr { + fn receiver_channel_closed() -> Self { + PyConnectionError::new_err("Receiver channel closed unexpectedly") + } + } + + #[ext(pub, name = PyResultExt)] + impl PyResult { + fn write_unraisable(self) -> Option { + Python::with_gil(|py| self.write_unraisable_with(py)) + } + + fn write_unraisable_with(self, py: Python<'_>) -> Option { + match self { + Ok(v) => Some(v), + Err(e) => { + // write error back to python + e.write_unraisable(py, None); + None + } + } + } + } + + #[ext(pub, name = TokioRuntimeExt)] + impl Runtime { + fn spawn_with_scope(&self, py: Python<'_>, future: F) -> PyResult> + where + F: Future + Send + 'static, + F::Output: Send + 'static, + { + let locals = pyo3_async_runtimes::tokio::get_current_locals(py)?; + Ok(self.spawn(pyo3_async_runtimes::tokio::scope(locals, future))) + } + } + + #[ext(pub, name = TokioMpscSenderExt)] + impl mpsc::Sender { + /// Sends a value, waiting until there is capacity. + /// + /// A successful send occurs when it is determined that the other end of the + /// channel has not hung up already. An unsuccessful send would be one where + /// the corresponding receiver has already been closed. + async fn send_py(&self, value: T) -> PyResult<()> { + self.send(value) + .await + .map_err(|_| PyErr::receiver_channel_closed()) + } + } + + #[ext(pub, name = TokioMpscReceiverExt)] + impl mpsc::Receiver { + /// Receives the next value for this receiver. + async fn recv_py(&mut self) -> PyResult { + self.recv().await.ok_or_else(PyErr::receiver_channel_closed) + } + + /// Receives at most `limit` values for this receiver and returns them. + /// + /// For `limit = 0`, an empty collection of messages will be returned immediately. + /// For `limit > 0`, if there are no messages in the channel's queue this method + /// will sleep until a message is sent. + async fn recv_many_py(&mut self, limit: usize) -> PyResult> { + // get updates from receiver channel + let mut updates = Vec::with_capacity(limit); + let received = self.recv_many(&mut updates, limit).await; + + // if we received zero items, then the channel was unexpectedly closed + if limit != 0 && received == 0 { + return Err(PyErr::receiver_channel_closed()); + } + + Ok(updates) + } + + /// Tries to receive the next value for this receiver. + fn try_recv_py(&mut self) -> PyResult> { + match self.try_recv() { + Ok(v) => Ok(Some(v)), + Err(TryRecvError::Empty) => Ok(None), + Err(TryRecvError::Disconnected) => Err(PyErr::receiver_channel_closed()), + } + } + } +} + +pub(crate) mod private { + use std::marker::Sized; + + /// Sealed traits support + pub trait Sealed {} + impl Sealed for T {} +} + +/// A wrapper around [`Py`] that implements [`Clone`] using [`Python::with_gil`]. +#[repr(transparent)] +pub(crate) struct ClonePy(pub Py); + +impl Clone for ClonePy { + fn clone(&self) -> Self { + Python::with_gil(|py| Self(self.0.clone_ref(py))) + } +} + +/// A Python module implemented in Rust. The name of this function must match +/// the `lib.name` setting in the `Cargo.toml`, else Python will not be able to +/// import the module. +#[pymodule(name = "exo_pyo3_bindings")] +fn main_module(m: &Bound<'_, PyModule>) -> PyResult<()> { + // install logger + pyo3_log::init(); + + // TODO: for now this is all NOT a submodule, but figure out how to make the submodule system + // work with maturin, where the types generate correctly, in the right folder, without + // too many importing issues... + ident_submodule(m)?; + multiaddr_submodule(m)?; + networking_submodule(m)?; + + // top-level constructs + // TODO: ... + + Ok(()) +} + +define_stub_info_gatherer!(stub_info); diff --git a/rust/exo_pyo3_bindings/src/networking.rs b/rust/exo_pyo3_bindings/src/networking.rs new file mode 100644 index 00000000..e2f88f2b --- /dev/null +++ b/rust/exo_pyo3_bindings/src/networking.rs @@ -0,0 +1,571 @@ +#![allow( + clippy::multiple_inherent_impl, + clippy::unnecessary_wraps, + clippy::unused_self, + clippy::needless_pass_by_value +)] + +use crate::r#const::MPSC_CHANNEL_SIZE; +use crate::ext::{ByteArrayExt as _, FutureExt, PyErrExt as _}; +use crate::ext::{ResultExt as _, TokioMpscReceiverExt as _, TokioMpscSenderExt as _}; +use crate::pyclass; +use crate::pylibp2p::ident::{PyKeypair, PyPeerId}; +use libp2p::futures::StreamExt as _; +use libp2p::gossipsub::{IdentTopic, Message, MessageId, PublishError}; +use libp2p::swarm::SwarmEvent; +use libp2p::{gossipsub, mdns}; +use networking::discovery; +use networking::swarm::create_swarm; +use pyo3::prelude::{PyModule, PyModuleMethods as _}; +use pyo3::types::PyBytes; +use pyo3::{Bound, Py, PyErr, PyResult, PyTraverseError, PyVisit, Python, pymethods}; +use pyo3_stub_gen::derive::{gen_stub_pyclass, gen_stub_pyclass_enum, gen_stub_pymethods}; +use std::net::IpAddr; +use tokio::sync::{Mutex, mpsc, oneshot}; +use util::ext::VecExt as _; + +mod exception { + use pyo3::types::PyTuple; + use pyo3::{PyErrArguments, exceptions::PyException, prelude::*}; + use pyo3_stub_gen::derive::*; + + #[gen_stub_pyclass] + #[pyclass(frozen, extends=PyException, name="NoPeersSubscribedToTopicError")] + pub struct PyNoPeersSubscribedToTopicError {} + + impl PyNoPeersSubscribedToTopicError { + const MSG: &'static str = "\ + No peers are currently subscribed to receive messages on this topic. \ + Wait for peers to subscribe or check your network connectivity."; + + /// Creates a new [ `PyErr` ] of this type. + /// + /// [`PyErr`] : https://docs.rs/pyo3/latest/pyo3/struct.PyErr.html "PyErr in pyo3" + pub(crate) fn new_err() -> PyErr { + PyErr::new::(()) // TODO: check if this needs to be replaced??? + } + } + + #[gen_stub_pymethods] + #[pymethods] + impl PyNoPeersSubscribedToTopicError { + #[new] + #[pyo3(signature = (*args))] + #[allow(unused_variables)] + pub(crate) fn new(args: &Bound<'_, PyTuple>) -> Self { + Self {} + } + + fn __repr__(&self) -> String { + format!("PeerId(\"{}\")", Self::MSG) + } + + fn __str__(&self) -> String { + Self::MSG.to_string() + } + } + + #[gen_stub_pyclass] + #[pyclass(frozen, extends=PyException, name="AllQueuesFullError")] + pub struct PyAllQueuesFullError {} + + impl PyAllQueuesFullError { + const MSG: &'static str = + "All libp2p peers are unresponsive, resend the message or reconnect."; + + /// Creates a new [ `PyErr` ] of this type. + /// + /// [`PyErr`] : https://docs.rs/pyo3/latest/pyo3/struct.PyErr.html "PyErr in pyo3" + pub(crate) fn new_err() -> PyErr { + PyErr::new::(()) // TODO: check if this needs to be replaced??? + } + } + + #[gen_stub_pymethods] + #[pymethods] + impl PyAllQueuesFullError { + #[new] + #[pyo3(signature = (*args))] + #[allow(unused_variables)] + pub(crate) fn new(args: &Bound<'_, PyTuple>) -> Self { + Self {} + } + + fn __repr__(&self) -> String { + format!("PeerId(\"{}\")", Self::MSG) + } + + fn __str__(&self) -> String { + Self::MSG.to_string() + } + } +} + +/// Connection or disconnection event discriminant type. +#[gen_stub_pyclass_enum] +#[pyclass(eq, eq_int, name = "ConnectionUpdateType")] +#[derive(Debug, Clone, PartialEq)] +enum PyConnectionUpdateType { + Connected = 0, + Disconnected, +} + +#[gen_stub_pyclass] +#[pyclass(frozen, name = "ConnectionUpdate")] +#[derive(Debug, Clone)] +struct PyConnectionUpdate { + /// Whether this is a connection or disconnection event + #[pyo3(get)] + update_type: PyConnectionUpdateType, + + /// Identity of the peer that we have connected to or disconnected from. + #[pyo3(get)] + peer_id: PyPeerId, + + /// Remote connection's IPv4 address. + #[pyo3(get)] + remote_ipv4: String, + + /// Remote connection's TCP port. + #[pyo3(get)] + remote_tcp_port: u16, +} + +enum ToTask { + GossipsubSubscribe { + topic: String, + result_tx: oneshot::Sender>, + }, + GossipsubUnsubscribe { + topic: String, + result_tx: oneshot::Sender, + }, + GossipsubPublish { + topic: String, + data: Vec, + result_tx: oneshot::Sender>, + }, +} + +#[allow(clippy::enum_glob_use)] +async fn networking_task( + mut swarm: networking::swarm::Swarm, + mut to_task_rx: mpsc::Receiver, + connection_update_tx: mpsc::Sender, + gossipsub_message_tx: mpsc::Sender<(String, Vec)>, +) { + use SwarmEvent::*; + use ToTask::*; + use mdns::Event::*; + use networking::swarm::BehaviourEvent::*; + + log::info!("RUST: networking task started"); + + loop { + tokio::select! { + message = to_task_rx.recv() => { + // handle closed channel + let Some(message) = message else { + log::info!("RUST: channel closed"); + break; + }; + + // dispatch incoming messages + match message { + GossipsubSubscribe { topic, result_tx } => { + // try to subscribe + let result = swarm.behaviour_mut() + .gossipsub.subscribe(&IdentTopic::new(topic)); + + // send response oneshot + if let Err(e) = result_tx.send(result.pyerr()) { + log::error!("RUST: could not subscribe to gossipsub topic since channel already closed: {e:?}"); + continue; + } + } + GossipsubUnsubscribe { topic, result_tx } => { + // try to unsubscribe from the topic + let result = swarm.behaviour_mut() + .gossipsub.unsubscribe(&IdentTopic::new(topic)); + + // send response oneshot (or exit if connection closed) + if let Err(e) = result_tx.send(result) { + log::error!("RUST: could not unsubscribe from gossipsub topic since channel already closed: {e:?}"); + continue; + } + } + GossipsubPublish { topic, data, result_tx } => { + // try to publish the data -> catch NoPeersSubscribedToTopic error & convert to correct exception + let result = swarm.behaviour_mut().gossipsub.publish( + IdentTopic::new(topic), data); + let pyresult: PyResult = if let Err(PublishError::NoPeersSubscribedToTopic) = result { + Err(exception::PyNoPeersSubscribedToTopicError::new_err()) + } else if let Err(PublishError::AllQueuesFull(_)) = result { + Err(exception::PyAllQueuesFullError::new_err()) + } else { + result.pyerr() + }; + + // send response oneshot (or exit if connection closed) + if let Err(e) = result_tx.send(pyresult) { + log::error!("RUST: could not publish gossipsub message since channel already closed: {e:?}"); + continue; + } + } + } + } + + // architectural solution to this problem: + // create keep_alive behavior who's job it is to dial peers discovered by mDNS (and drop when expired) + // -> it will emmit TRUE connected/disconnected events consumable elsewhere + // + // gossipsub will feed off-of dial attempts created by networking, and that will bootstrap its' peers list + // then for actual communication it will dial those peers if need-be + swarm_event = swarm.select_next_some() => { + match swarm_event { + Behaviour(Gossipsub(gossipsub::Event::Message { + message: Message { + topic, + data, + .. + }, + .. + })) => { + // topic-ID is just the topic hash!!! (since we used identity hasher) + let message = (topic.into_string(), data); + + // send incoming message to channel (or exit if connection closed) + if let Err(e) = gossipsub_message_tx.send(message).await { + log::error!("RUST: could not send incoming gossipsub message since channel already closed: {e}"); + continue; + } + }, + Behaviour(Discovery(discovery::Event::ConnectionEstablished { peer_id, remote_ip, remote_tcp_port, .. })) => { + // grab IPv4 string + let remote_ipv4 = match remote_ip { + IpAddr::V4(ip) => ip.to_string(), + IpAddr::V6(ip) => { + log::warn!("RUST: ignoring connection to IPv6 address: {ip}"); + continue; + } + }; + + // send connection event to channel (or exit if connection closed) + if let Err(e) = connection_update_tx.send(PyConnectionUpdate { + update_type: PyConnectionUpdateType::Connected, + peer_id: PyPeerId(peer_id), + remote_ipv4, + remote_tcp_port, + }).await { + log::error!("RUST: could not send connection update since channel already closed: {e}"); + continue; + } + }, + Behaviour(Discovery(discovery::Event::ConnectionClosed { peer_id, remote_ip, remote_tcp_port, .. })) => { + // grab IPv4 string + let remote_ipv4 = match remote_ip { + IpAddr::V4(ip) => ip.to_string(), + IpAddr::V6(ip) => { + log::warn!("RUST: ignoring disconnection from IPv6 address: {ip}"); + continue; + } + }; + + // send disconnection event to channel (or exit if connection closed) + if let Err(e) = connection_update_tx.send(PyConnectionUpdate { + update_type: PyConnectionUpdateType::Disconnected, + peer_id: PyPeerId(peer_id), + remote_ipv4, + remote_tcp_port, + }).await { + log::error!("RUST: could not send connection update since channel already closed: {e}"); + continue; + } + }, + e => { + log::info!("RUST: other event {e:?}"); + } + } + } + } + } + + log::info!("RUST: networking task stopped"); +} + +#[gen_stub_pyclass] +#[pyclass(name = "NetworkingHandle")] +#[derive(Debug)] +struct PyNetworkingHandle { + // channels + to_task_tx: Option>, + connection_update_rx: Mutex>, + gossipsub_message_rx: Mutex)>>, +} + +impl Drop for PyNetworkingHandle { + fn drop(&mut self) { + // TODO: may or may not need to await a "kill-signal" oneshot channel message, + // to ensure that the networking task is done BEFORE exiting the clear function... + // but this may require GIL?? and it may not be safe to call GIL here?? + self.to_task_tx = None; // Using Option as a trick to force channel to be dropped + } +} + +#[allow(clippy::expect_used)] +impl PyNetworkingHandle { + fn new( + to_task_tx: mpsc::Sender, + connection_update_rx: mpsc::Receiver, + gossipsub_message_rx: mpsc::Receiver<(String, Vec)>, + ) -> Self { + Self { + to_task_tx: Some(to_task_tx), + connection_update_rx: Mutex::new(connection_update_rx), + gossipsub_message_rx: Mutex::new(gossipsub_message_rx), + } + } + + const fn to_task_tx(&self) -> &mpsc::Sender { + self.to_task_tx + .as_ref() + .expect("The sender should only be None after de-initialization.") + } +} + +#[gen_stub_pymethods] +#[pymethods] +impl PyNetworkingHandle { + // NOTE: `async fn`s here that use `.await` will wrap the future in `.allow_threads_py()` + // immediately beforehand to release the interpreter. + // SEE: https://pyo3.rs/v0.26.0/async-await.html#detaching-from-the-interpreter-across-await + + // ---- Lifecycle management methods ---- + + #[new] + fn py_new(identity: Bound<'_, PyKeypair>) -> PyResult { + use pyo3_async_runtimes::tokio::get_runtime; + + // create communication channels + let (to_task_tx, to_task_rx) = mpsc::channel(MPSC_CHANNEL_SIZE); + let (connection_update_tx, connection_update_rx) = mpsc::channel(MPSC_CHANNEL_SIZE); + let (gossipsub_message_tx, gossipsub_message_rx) = mpsc::channel(MPSC_CHANNEL_SIZE); + + // get identity + let identity = identity.borrow().0.clone(); + + // create networking swarm (within tokio context!! or it crashes) + let swarm = get_runtime() + .block_on(async { create_swarm(identity) }) + .pyerr()?; + + // spawn tokio task running the networking logic + get_runtime().spawn(async move { + networking_task( + swarm, + to_task_rx, + connection_update_tx, + gossipsub_message_tx, + ) + .await; + }); + Ok(Self::new( + to_task_tx, + connection_update_rx, + gossipsub_message_rx, + )) + } + + #[gen_stub(skip)] + const fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { + Ok(()) // This is needed purely so `__clear__` can work + } + + #[gen_stub(skip)] + fn __clear__(&mut self) { + // TODO: may or may not need to await a "kill-signal" oneshot channel message, + // to ensure that the networking task is done BEFORE exiting the clear function... + // but this may require GIL?? and it may not be safe to call GIL here?? + self.to_task_tx = None; // Using Option as a trick to force channel to be dropped + } + + // ---- Connection update receiver methods ---- + + /// Receives the next `ConnectionUpdate` from networking. + async fn connection_update_recv(&self) -> PyResult { + self.connection_update_rx + .lock() + .allow_threads_py() // allow-threads-aware async call + .await + .recv_py() + .allow_threads_py() // allow-threads-aware async call + .await + } + + /// Receives at most `limit` `ConnectionUpdate`s from networking and returns them. + /// + /// For `limit = 0`, an empty collection of `ConnectionUpdate`s will be returned immediately. + /// For `limit > 0`, if there are no `ConnectionUpdate`s in the channel's queue this method + /// will sleep until a `ConnectionUpdate`s is sent. + async fn connection_update_recv_many(&self, limit: usize) -> PyResult> { + self.connection_update_rx + .lock() + .allow_threads_py() // allow-threads-aware async call + .await + .recv_many_py(limit) + .allow_threads_py() // allow-threads-aware async call + .await + } + + // TODO: rn this blocks main thread if anything else is awaiting the channel (bc its a mutex) + // so its too dangerous to expose just yet. figure out a better semantics for handling this, + // so things don't randomly block + // /// Tries to receive the next `ConnectionUpdate` from networking. + // fn connection_update_try_recv(&self) -> PyResult> { + // self.connection_update_rx.blocking_lock().try_recv_py() + // } + // + // /// Checks if the `ConnectionUpdate` channel is empty. + // fn connection_update_is_empty(&self) -> bool { + // self.connection_update_rx.blocking_lock().is_empty() + // } + // + // /// Returns the number of `ConnectionUpdate`s in the channel. + // fn connection_update_len(&self) -> usize { + // self.connection_update_rx.blocking_lock().len() + // } + + // ---- Gossipsub management methods ---- + + /// Subscribe to a `GossipSub` topic. + /// + /// Returns `True` if the subscription worked. Returns `False` if we were already subscribed. + async fn gossipsub_subscribe(&self, topic: String) -> PyResult { + let (tx, rx) = oneshot::channel(); + + // send off request to subscribe + self.to_task_tx() + .send_py(ToTask::GossipsubSubscribe { + topic, + result_tx: tx, + }) + .allow_threads_py() // allow-threads-aware async call + .await?; + + // wait for response & return any errors + rx.allow_threads_py() // allow-threads-aware async call + .await + .map_err(|_| PyErr::receiver_channel_closed())? + } + + /// Unsubscribes from a `GossipSub` topic. + /// + /// Returns `True` if we were subscribed to this topic. Returns `False` if we were not subscribed. + async fn gossipsub_unsubscribe(&self, topic: String) -> PyResult { + let (tx, rx) = oneshot::channel(); + + // send off request to unsubscribe + self.to_task_tx() + .send_py(ToTask::GossipsubUnsubscribe { + topic, + result_tx: tx, + }) + .allow_threads_py() // allow-threads-aware async call + .await?; + + // wait for response & convert any errors + rx.allow_threads_py() // allow-threads-aware async call + .await + .map_err(|_| PyErr::receiver_channel_closed()) + } + + /// Publishes a message with multiple topics to the `GossipSub` network. + /// + /// If no peers are found that subscribe to this topic, throws `NoPeersSubscribedToTopicError` exception. + async fn gossipsub_publish(&self, topic: String, data: Py) -> PyResult<()> { + let (tx, rx) = oneshot::channel(); + + // send off request to subscribe + let data = Python::with_gil(|py| Vec::from(data.as_bytes(py))); + self.to_task_tx() + .send_py(ToTask::GossipsubPublish { + topic, + data, + result_tx: tx, + }) + .allow_threads_py() // allow-threads-aware async call + .await?; + + // wait for response & return any errors => ignore messageID for now!!! + let _ = rx + .allow_threads_py() // allow-threads-aware async call + .await + .map_err(|_| PyErr::receiver_channel_closed())??; + Ok(()) + } + + // ---- Gossipsub message receiver methods ---- + + /// Receives the next message from the `GossipSub` network. + async fn gossipsub_recv(&self) -> PyResult<(String, Py)> { + self.gossipsub_message_rx + .lock() + .allow_threads_py() // allow-threads-aware async call + .await + .recv_py() + .allow_threads_py() // allow-threads-aware async call + .await + .map(|(t, d)| (t, d.pybytes())) + } + + /// Receives at most `limit` messages from the `GossipSub` network and returns them. + /// + /// For `limit = 0`, an empty collection of messages will be returned immediately. + /// For `limit > 0`, if there are no messages in the channel's queue this method + /// will sleep until a message is sent. + async fn gossipsub_recv_many(&self, limit: usize) -> PyResult)>> { + Ok(self + .gossipsub_message_rx + .lock() + .allow_threads_py() // allow-threads-aware async call + .await + .recv_many_py(limit) + .allow_threads_py() // allow-threads-aware async call + .await? + .map(|(t, d)| (t, d.pybytes()))) + } + + // TODO: rn this blocks main thread if anything else is awaiting the channel (bc its a mutex) + // so its too dangerous to expose just yet. figure out a better semantics for handling this, + // so things don't randomly block + // /// Tries to receive the next message from the `GossipSub` network. + // fn gossipsub_try_recv(&self) -> PyResult)>> { + // Ok(self + // .gossipsub_message_rx + // .blocking_lock() + // .try_recv_py()? + // .map(|(t, d)| (t, d.pybytes()))) + // } + // + // /// Checks if the `GossipSub` message channel is empty. + // fn gossipsub_is_empty(&self) -> bool { + // self.gossipsub_message_rx.blocking_lock().is_empty() + // } + // + // /// Returns the number of `GossipSub` messages in the channel. + // fn gossipsub_len(&self) -> usize { + // self.gossipsub_message_rx.blocking_lock().len() + // } +} + +pub fn networking_submodule(m: &Bound<'_, PyModule>) -> PyResult<()> { + m.add_class::()?; + m.add_class::()?; + + m.add_class::()?; + m.add_class::()?; + m.add_class::()?; + m.add_class::()?; + + Ok(()) +} diff --git a/rust/exo_pyo3_bindings/src/pylibp2p/ident.rs b/rust/exo_pyo3_bindings/src/pylibp2p/ident.rs new file mode 100644 index 00000000..3c27526a --- /dev/null +++ b/rust/exo_pyo3_bindings/src/pylibp2p/ident.rs @@ -0,0 +1,159 @@ +use crate::ext::ResultExt as _; +use libp2p::PeerId; +use libp2p::identity::Keypair; +use pyo3::prelude::{PyBytesMethods as _, PyModule, PyModuleMethods as _}; +use pyo3::types::PyBytes; +use pyo3::{Bound, PyResult, Python, pyclass, pymethods}; +use pyo3_stub_gen::derive::{gen_stub_pyclass, gen_stub_pymethods}; + +/// Identity keypair of a node. +#[gen_stub_pyclass] +#[pyclass(name = "Keypair", frozen)] +#[repr(transparent)] +pub struct PyKeypair(pub Keypair); + +#[gen_stub_pymethods] +#[pymethods] +#[allow(clippy::needless_pass_by_value)] +impl PyKeypair { + /// Generate a new Ed25519 keypair. + #[staticmethod] + fn generate_ed25519() -> Self { + Self(Keypair::generate_ed25519()) + } + + /// Generate a new ECDSA keypair. + #[staticmethod] + fn generate_ecdsa() -> Self { + Self(Keypair::generate_ecdsa()) + } + + /// Generate a new Secp256k1 keypair. + #[staticmethod] + fn generate_secp256k1() -> Self { + Self(Keypair::generate_secp256k1()) + } + + /// Decode a private key from a protobuf structure and parse it as a `Keypair`. + #[staticmethod] + fn from_protobuf_encoding(bytes: Bound<'_, PyBytes>) -> PyResult { + let bytes = Vec::from(bytes.as_bytes()); + Ok(Self(Keypair::from_protobuf_encoding(&bytes).pyerr()?)) + } + + /// Decode an keypair from a DER-encoded secret key in PKCS#8 `PrivateKeyInfo` + /// format (i.e. unencrypted) as defined in [RFC5208]. + /// + /// [RFC5208]: https://tools.ietf.org/html/rfc5208#section-5 + #[staticmethod] + fn rsa_from_pkcs8(bytes: Bound<'_, PyBytes>) -> PyResult { + let mut bytes = Vec::from(bytes.as_bytes()); + Ok(Self(Keypair::rsa_from_pkcs8(&mut bytes).pyerr()?)) + } + + /// Decode a keypair from a DER-encoded Secp256k1 secret key in an `ECPrivateKey` + /// structure as defined in [RFC5915]. + /// + /// [RFC5915]: https://tools.ietf.org/html/rfc5915 + #[staticmethod] + fn secp256k1_from_der(bytes: Bound<'_, PyBytes>) -> PyResult { + let mut bytes = Vec::from(bytes.as_bytes()); + Ok(Self(Keypair::secp256k1_from_der(&mut bytes).pyerr()?)) + } + + #[staticmethod] + fn ed25519_from_bytes(bytes: Bound<'_, PyBytes>) -> PyResult { + let mut bytes = Vec::from(bytes.as_bytes()); + Ok(Self(Keypair::ed25519_from_bytes(&mut bytes).pyerr()?)) + } + + /// Encode a private key as protobuf structure. + fn to_protobuf_encoding<'py>(&self, py: Python<'py>) -> PyResult> { + let bytes = self.0.to_protobuf_encoding().pyerr()?; + Ok(PyBytes::new(py, &bytes)) + } + + /// Convert the `Keypair` into the corresponding `PeerId`. + fn to_peer_id(&self) -> PyPeerId { + PyPeerId(self.0.public().to_peer_id()) + } + + // /// Hidden constructor for pickling support. TODO: figure out how to do pickling... + // #[gen_stub(skip)] + // #[new] + // fn py_new(bytes: Bound<'_, PyBytes>) -> PyResult { + // Self::from_protobuf_encoding(bytes) + // } + // + // #[gen_stub(skip)] + // fn __setstate__(&mut self, state: Bound<'_, PyBytes>) -> PyResult<()> { + // *self = Self::from_protobuf_encoding(state)?; + // Ok(()) + // } + // + // #[gen_stub(skip)] + // fn __getstate__<'py>(&self, py: Python<'py>) -> PyResult> { + // self.to_protobuf_encoding(py) + // } + // + // #[gen_stub(skip)] + // pub fn __getnewargs__<'py>(&self, py: Python<'py>) -> PyResult<(Bound<'py, PyBytes>,)> { + // Ok((self.to_protobuf_encoding(py)?,)) + // } +} + +/// Identifier of a peer of the network. +/// +/// The data is a `CIDv0` compatible multihash of the protobuf encoded public key of the peer +/// as specified in [specs/peer-ids](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md). +#[gen_stub_pyclass] +#[pyclass(name = "PeerId", frozen)] +#[derive(Debug, Clone)] +#[repr(transparent)] +pub struct PyPeerId(pub PeerId); + +#[gen_stub_pymethods] +#[pymethods] +#[allow(clippy::needless_pass_by_value)] +impl PyPeerId { + /// Generates a random peer ID from a cryptographically secure PRNG. + /// + /// This is useful for randomly walking on a DHT, or for testing purposes. + #[staticmethod] + fn random() -> Self { + Self(PeerId::random()) + } + + /// Parses a `PeerId` from bytes. + #[staticmethod] + fn from_bytes(bytes: Bound<'_, PyBytes>) -> PyResult { + let bytes = Vec::from(bytes.as_bytes()); + Ok(Self(PeerId::from_bytes(&bytes).pyerr()?)) + } + + /// Returns a raw bytes representation of this `PeerId`. + fn to_bytes<'py>(&self, py: Python<'py>) -> Bound<'py, PyBytes> { + let bytes = self.0.to_bytes(); + PyBytes::new(py, &bytes) + } + + /// Returns a base-58 encoded string of this `PeerId`. + fn to_base58(&self) -> String { + self.0.to_base58() + } + + fn __repr__(&self) -> String { + format!("PeerId({})", self.to_base58()) + } + + fn __str__(&self) -> String { + self.to_base58() + } +} + +pub fn ident_submodule(m: &Bound<'_, PyModule>) -> PyResult<()> { + m.add_class::()?; + m.add_class::()?; + + Ok(()) +} diff --git a/rust/exo_pyo3_bindings/src/pylibp2p/mod.rs b/rust/exo_pyo3_bindings/src/pylibp2p/mod.rs new file mode 100644 index 00000000..8eb1bdc0 --- /dev/null +++ b/rust/exo_pyo3_bindings/src/pylibp2p/mod.rs @@ -0,0 +1,8 @@ +//! A module for exposing Rust's libp2p datatypes over Pyo3 +//! +//! TODO: right now we are coupled to libp2p's identity, but eventually we want to create our own +//! independent identity type of some kind or another. This may require handshaking. +//! + +pub mod ident; +pub mod multiaddr; diff --git a/rust/exo_pyo3_bindings/src/pylibp2p/multiaddr.rs b/rust/exo_pyo3_bindings/src/pylibp2p/multiaddr.rs new file mode 100644 index 00000000..4d398b53 --- /dev/null +++ b/rust/exo_pyo3_bindings/src/pylibp2p/multiaddr.rs @@ -0,0 +1,81 @@ +use crate::ext::ResultExt as _; +use libp2p::Multiaddr; +use pyo3::prelude::{PyBytesMethods as _, PyModule, PyModuleMethods as _}; +use pyo3::types::PyBytes; +use pyo3::{Bound, PyResult, Python, pyclass, pymethods}; +use pyo3_stub_gen::derive::{gen_stub_pyclass, gen_stub_pymethods}; +use std::str::FromStr as _; + +/// Representation of a Multiaddr. +#[gen_stub_pyclass] +#[pyclass(name = "Multiaddr", frozen)] +#[derive(Debug, Clone)] +#[repr(transparent)] +pub struct PyMultiaddr(pub Multiaddr); + +#[gen_stub_pymethods] +#[pymethods] +#[allow(clippy::needless_pass_by_value)] +impl PyMultiaddr { + /// Create a new, empty multiaddress. + #[staticmethod] + fn empty() -> Self { + Self(Multiaddr::empty()) + } + + /// Create a new, empty multiaddress with the given capacity. + #[staticmethod] + fn with_capacity(n: usize) -> Self { + Self(Multiaddr::with_capacity(n)) + } + + /// Parse a `Multiaddr` value from its byte slice representation. + #[staticmethod] + fn from_bytes(bytes: Bound<'_, PyBytes>) -> PyResult { + let bytes = Vec::from(bytes.as_bytes()); + Ok(Self(Multiaddr::try_from(bytes).pyerr()?)) + } + + /// Parse a `Multiaddr` value from its string representation. + #[staticmethod] + fn from_string(string: String) -> PyResult { + Ok(Self(Multiaddr::from_str(&string).pyerr()?)) + } + + /// Return the length in bytes of this multiaddress. + fn len(&self) -> usize { + self.0.len() + } + + /// Returns true if the length of this multiaddress is 0. + fn is_empty(&self) -> bool { + self.0.is_empty() + } + + /// Return a copy of this [`Multiaddr`]'s byte representation. + fn to_bytes<'py>(&self, py: Python<'py>) -> Bound<'py, PyBytes> { + let bytes = self.0.to_vec(); + PyBytes::new(py, &bytes) + } + + /// Convert a Multiaddr to a string. + fn to_string(&self) -> String { + self.0.to_string() + } + + #[gen_stub(skip)] + fn __repr__(&self) -> String { + format!("Multiaddr({})", self.0) + } + + #[gen_stub(skip)] + fn __str__(&self) -> String { + self.to_string() + } +} + +pub fn multiaddr_submodule(m: &Bound<'_, PyModule>) -> PyResult<()> { + m.add_class::()?; + + Ok(()) +} diff --git a/rust/exo_pyo3_bindings/tests/dummy.rs b/rust/exo_pyo3_bindings/tests/dummy.rs new file mode 100644 index 00000000..7d1ce0e4 --- /dev/null +++ b/rust/exo_pyo3_bindings/tests/dummy.rs @@ -0,0 +1,54 @@ +#[cfg(test)] +mod tests { + use core::mem::drop; + use core::option::Option::Some; + use core::time::Duration; + use tokio; + use tokio::sync::mpsc; + + #[tokio::test] + async fn test_drop_channel() { + struct Ping; + + let (tx, mut rx) = mpsc::channel::(10); + + let _ = tokio::spawn(async move { + println!("TASK: entered"); + + loop { + tokio::select! { + result = rx.recv() => { + match result { + Some(_) => { + println!("TASK: pinged"); + } + None => { + println!("TASK: closing channel"); + break; + } + } + } + _ = tokio::time::sleep(Duration::from_secs_f32(0.1)) => { + println!("TASK: heartbeat"); + } + } + } + + println!("TASK: exited"); + }); + + let tx2 = tx.clone(); + + tokio::time::sleep(Duration::from_secs_f32(0.11)).await; + + tx.send(Ping).await.expect("Should not fail"); + drop(tx); + + tokio::time::sleep(Duration::from_secs_f32(0.11)).await; + + tx2.send(Ping).await.expect("Should not fail"); + drop(tx2); + + tokio::time::sleep(Duration::from_secs_f32(0.11)).await; + } +} diff --git a/rust/exo_pyo3_bindings/tests/test_python.py b/rust/exo_pyo3_bindings/tests/test_python.py new file mode 100644 index 00000000..ce5a676f --- /dev/null +++ b/rust/exo_pyo3_bindings/tests/test_python.py @@ -0,0 +1,34 @@ +import asyncio + +import pytest +from exo_pyo3_bindings import Keypair, NetworkingHandle, NoPeersSubscribedToTopicError + + +@pytest.mark.asyncio +async def test_sleep_on_multiple_items() -> None: + print("PYTHON: starting handle") + h = NetworkingHandle(Keypair.generate_ed25519()) + + ct = asyncio.create_task(_await_cons(h)) + mt = asyncio.create_task(_await_msg(h)) + + # sleep for 4 ticks + for i in range(4): + await asyncio.sleep(1) + + try: + await h.gossipsub_publish("topic", b"somehting or other") + except NoPeersSubscribedToTopicError as e: + print("caught it", e) + + +async def _await_cons(h: NetworkingHandle): + while True: + c = await h.connection_update_recv() + print(f"PYTHON: connection update: {c}") + + +async def _await_msg(h: NetworkingHandle): + while True: + m = await h.gossipsub_recv() + print(f"PYTHON: message: {m}") diff --git a/rust/networking/Cargo.toml b/rust/networking/Cargo.toml new file mode 100644 index 00000000..47d61f41 --- /dev/null +++ b/rust/networking/Cargo.toml @@ -0,0 +1,44 @@ +[package] +name = "networking" +version = { workspace = true } +edition = { workspace = true } +publish = false + +[lib] +doctest = false +name = "networking" +path = "src/lib.rs" + +[lints] +workspace = true + +[dependencies] +# datastructures +either = { workspace = true } + +# macro dependencies +extend = { workspace = true } +delegate = { workspace = true } +impl-trait-for-tuples = { workspace = true } +derive_more = { workspace = true } + +# async +tokio = { workspace = true, features = ["full"] } +futures = { workspace = true } +futures-timer = { workspace = true } + +# utility dependencies +util = { workspace = true } +thiserror = { workspace = true } +#internment = { workspace = true } +#recursion = { workspace = true } +#generativity = { workspace = true } +#itertools = { workspace = true } +tracing-subscriber = { version = "0.3.19", features = ["default", "env-filter"] } +keccak-const = { workspace = true } + +# tracing/logging +log = { workspace = true } + +# networking +libp2p = { workspace = true, features = ["full"] } \ No newline at end of file diff --git a/rust/networking/examples/chatroom.rs b/rust/networking/examples/chatroom.rs new file mode 100644 index 00000000..3371b46d --- /dev/null +++ b/rust/networking/examples/chatroom.rs @@ -0,0 +1,74 @@ +use futures::stream::StreamExt as _; +use libp2p::{gossipsub, identity, swarm::SwarmEvent}; +use networking::{discovery, swarm}; +use tokio::{io, io::AsyncBufReadExt as _, select}; +use tracing_subscriber::EnvFilter; +use tracing_subscriber::filter::LevelFilter; + +#[tokio::main] +async fn main() { + let _ = tracing_subscriber::fmt() + .with_env_filter(EnvFilter::from_default_env().add_directive(LevelFilter::INFO.into())) + .try_init(); + + // Configure swarm + let mut swarm = + swarm::create_swarm(identity::Keypair::generate_ed25519()).expect("Swarm creation failed"); + + // Create a Gossipsub topic & subscribe + let topic = gossipsub::IdentTopic::new("test-net"); + swarm + .behaviour_mut() + .gossipsub + .subscribe(&topic) + .expect("Subscribing to topic failed"); + + // Read full lines from stdin + let mut stdin = io::BufReader::new(io::stdin()).lines(); + println!("Enter messages via STDIN and they will be sent to connected peers using Gossipsub"); + + // Kick it off + loop { + select! { + // on gossipsub outgoing + Ok(Some(line)) = stdin.next_line() => { + if let Err(e) = swarm + .behaviour_mut().gossipsub + .publish(topic.clone(), line.as_bytes()) { + println!("Publish error: {e:?}"); + } + } + event = swarm.select_next_some() => match event { + // on gossipsub incoming + SwarmEvent::Behaviour(swarm::BehaviourEvent::Gossipsub(gossipsub::Event::Message { + propagation_source: peer_id, + message_id: id, + message, + })) => println!( + "\n\nGot message: '{}' with id: {id} from peer: {peer_id}\n\n", + String::from_utf8_lossy(&message.data), + ), + + // on discovery + SwarmEvent::Behaviour(swarm::BehaviourEvent::Discovery(e)) => match e { + discovery::Event::ConnectionEstablished { + peer_id, connection_id, remote_ip, remote_tcp_port + } => { + println!("\n\nConnected to: {peer_id}; connection ID: {connection_id}; remote IP: {remote_ip}; remote TCP port: {remote_tcp_port}\n\n"); + } + discovery::Event::ConnectionClosed { + peer_id, connection_id, remote_ip, remote_tcp_port + } => { + eprintln!("\n\nDisconnected from: {peer_id}; connection ID: {connection_id}; remote IP: {remote_ip}; remote TCP port: {remote_tcp_port}\n\n"); + } + } + + // ignore outgoing errors: those are normal + e@SwarmEvent::OutgoingConnectionError { .. } => { log::debug!("Outgoing connection error: {e:?}"); } + + // otherwise log any other event + e => { log::info!("Other event {e:?}"); } + } + } + } +} diff --git a/rust/networking/examples/chatroom_manual.rs b/rust/networking/examples/chatroom_manual.rs new file mode 100644 index 00000000..5d92ac86 --- /dev/null +++ b/rust/networking/examples/chatroom_manual.rs @@ -0,0 +1,127 @@ +// Copyright 2018 Parity Technologies (UK) Ltd. +// +// Permission is hereby granted, free of charge, to any person obtaining a +// copy of this software and associated documentation files (the "Software"), +// to deal in the Software without restriction, including without limitation +// the rights to use, copy, modify, merge, publish, distribute, sublicense, +// and/or sell copies of the Software, and to permit persons to whom the +// Software is furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS +// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +use futures::stream::StreamExt; +use libp2p::{ + gossipsub, mdns, noise, + swarm::{NetworkBehaviour, SwarmEvent}, + tcp, yamux, +}; +use std::time::Duration; +use std::{error::Error, hash::Hash}; +use tokio::{io, io::AsyncBufReadExt, select}; +use tracing_subscriber::EnvFilter; + +// We create a custom network behaviour that combines Gossipsub and Mdns. +#[derive(NetworkBehaviour)] +struct MyBehaviour { + gossipsub: gossipsub::Behaviour, + mdns: mdns::tokio::Behaviour, +} + +#[tokio::main] +async fn main() -> Result<(), Box> { + let _ = tracing_subscriber::fmt() + .with_env_filter(EnvFilter::from_default_env()) + .try_init(); + + let mut swarm = libp2p::SwarmBuilder::with_new_identity() + .with_tokio() + .with_tcp( + tcp::Config::default(), + noise::Config::new, + yamux::Config::default, + )? + .with_behaviour(|key| { + // Set a custom gossipsub configuration + let gossipsub_config = gossipsub::ConfigBuilder::default() + .heartbeat_interval(Duration::from_secs(10)) + .validation_mode(gossipsub::ValidationMode::Strict) // This sets the kind of message validation. The default is Strict (enforce message signing) + .build() + .map_err(io::Error::other)?; // Temporary hack because `build` does not return a proper `std::error::Error`. + + // build a gossipsub network behaviour + let gossipsub = gossipsub::Behaviour::new( + gossipsub::MessageAuthenticity::Signed(key.clone()), + gossipsub_config, + )?; + + let mdns = + mdns::tokio::Behaviour::new(mdns::Config::default(), key.public().to_peer_id())?; + Ok(MyBehaviour { gossipsub, mdns }) + })? + .build(); + + println!("Running swarm with identity {}", swarm.local_peer_id()); + + // Create a Gossipsub topic + let topic = gossipsub::IdentTopic::new("test-net"); + // subscribes to our topic + swarm.behaviour_mut().gossipsub.subscribe(&topic)?; + + // Read full lines from stdin + let mut stdin = io::BufReader::new(io::stdin()).lines(); + + // Listen on all interfaces and whatever port the OS assigns + swarm.listen_on("/ip4/0.0.0.0/tcp/0".parse()?)?; + + println!("Enter messages via STDIN and they will be sent to connected peers using Gossipsub"); + + // Kick it off + loop { + select! { + Ok(Some(line)) = stdin.next_line() => { + if let Err(e) = swarm + .behaviour_mut().gossipsub + .publish(topic.clone(), line.as_bytes()) { + println!("Publish error: {e:?}"); + } + } + event = swarm.select_next_some() => match event { + SwarmEvent::Behaviour(MyBehaviourEvent::Mdns(mdns::Event::Discovered(list))) => { + for (peer_id, multiaddr) in list { + println!("mDNS discovered a new peer: {peer_id} on {multiaddr}"); + swarm.behaviour_mut().gossipsub.add_explicit_peer(&peer_id); + } + }, + SwarmEvent::Behaviour(MyBehaviourEvent::Mdns(mdns::Event::Expired(list))) => { + for (peer_id, multiaddr) in list { + println!("mDNS discover peer has expired: {peer_id} on {multiaddr}"); + swarm.behaviour_mut().gossipsub.remove_explicit_peer(&peer_id); + } + }, + SwarmEvent::Behaviour(MyBehaviourEvent::Gossipsub(gossipsub::Event::Message { + propagation_source: peer_id, + message_id: id, + message, + })) => println!( + "Got message: '{}' with id: {id} from peer: {peer_id}", + String::from_utf8_lossy(&message.data), + ), + SwarmEvent::NewListenAddr { address, .. } => { + println!("Local node is listening on {address}"); + } + e => { + println!("Other swarm event: {:?}", e); + } + } + } + } +} diff --git a/rust/networking/src/RESEARCH_NOTES.txt b/rust/networking/src/RESEARCH_NOTES.txt new file mode 100644 index 00000000..2beeca57 --- /dev/null +++ b/rust/networking/src/RESEARCH_NOTES.txt @@ -0,0 +1,44 @@ +https://github.com/ml-explore/mlx/commit/3fe98bacc7640d857acf3539f1d21b47a32e5609 +^raw sockets distributed -> `` -> https://newosxbook.com/code/xnu-3247.1.106/bsd/net/ndrv.h.auto.html +--> header file for a networking component found in the macOS kernel (XNU) that defines structures for network device driver registration, specifically the ndrv_demux_desc and ndrv_protocol_desc structures used for demultiplexing protocol data at the network interface level. It specifies how to describe protocol data, such as an Ethernet type or a SNAP header, and how to associate these descriptions with a specific protocol family to receive matching packets. +--> Used to bind an NDRV socket so that packets that match given protocol demux descriptions can be received. +--> An NDRV socket is a special kind of socket in the Darwin/macOS operating system's XNU kernel, used for low-level network packet manipulation and binding to specific protocols for packet processing. It allows user-space applications or drivers to directly write Layer 2 (L2) network packets or interact with the network stack at a lower level, often by binding to protocol descriptors like the ndrv_protocol_desc. This type of socket is used for functions such as capturing and injecting packets, especially in network infrastructure software like routers or for kernel-level network monitoring and security tools. +--> also called PF_NDRV sockets --> https://newosxbook.com/bonus/vol1ch16.html +----> they are conceptually similar to https://scapy.disruptivelabs.in/networking/socket-interface PF_RAW or PF_PACKET + +https://stackoverflow.com/questions/17169298/af-packet-on-osx +^AF_PACKET duplicates the packets as soon as it receives them from the physical layer (for incoming packets) or just before sending them out to the physical layer (for outgoing packets). -> this is on Linux only +^it doesn't exist on OS X so you can use /dev/bpfX (Berkeley Packet Filter) for sniffing + +https://www.unix.com/man_page/mojave/4/ip/ +^OS X manpages for IP + +https://developer.apple.com/documentation/kernel/implementing_drivers_system_extensions_and_kexts +^driver kit, system extensions & kexts for macOS + +---- + +To set up a Linux system to use a Thunderbolt connection as a network device, connect the two computers with a Thunderbolt cable, load the thunderbolt-net kernel module (usually automatic but modprobe is an option for manual loading), and then the operating system will create virtual Ethernet interfaces (e.g., thunderbolt0) for networking. You can then use standard tools like ifconfig or your desktop environment's network manager to configure these new interfaces for a link-local network. +--> https://gist.github.com/geosp/80fbd39e617b7d1d9421683df4ea224a +----> here is a guide on how to set up thunderbolt-ethernet on linux +----> I may be able to steal the thunderbolt-net code ideas to implement a kernel module for MacOS + +https://chatgpt.com/s/t_68af8e41a8548191993281a014f846a7 +^GPT discussion about making socket interface + +https://chatgpt.com/s/t_68afb798a85c8191973c02a0fa7a48a3 --> link-local address,,?? +https://chatgpt.com/s/t_68afb02987e08191b2b0044d3667ece2 +^GPT discussion about accessing TB on MacOS low level interactions + +-------------------------------- + +https://www.intel.com/content/www/us/en/support/articles/000098893/software.html +^Thunderbolt Share & Thunderbolt Networking Mode => intel's equivalent of thunderbolt bridge + + +--------------------------------- + +https://www.zerotier.com/blog/how-zerotier-eliminated-kernel-extensions-on-macos/ +-->fake ethernet devices on MacOS -> omg??? we can detect thunderbolt bridge, then bind to it, then re-expose it as fake ethernet?? +-->ps: https://chatgpt.com/s/t_68afb2b25fb881919526763fb5d7359c, AF/PF_NDRV are one and the same!!! +-->https://github.com/zerotier/ZeroTierOne/blob/dev/osdep/MacEthernetTapAgent.c \ No newline at end of file diff --git a/rust/networking/src/discovery.rs b/rust/networking/src/discovery.rs new file mode 100644 index 00000000..b9a4052c --- /dev/null +++ b/rust/networking/src/discovery.rs @@ -0,0 +1,383 @@ +use crate::ext::MultiaddrExt; +use crate::keep_alive; +use delegate::delegate; +use either::Either; +use futures::FutureExt; +use futures_timer::Delay; +use libp2p::core::transport::PortUse; +use libp2p::core::{ConnectedPoint, Endpoint}; +use libp2p::swarm::behaviour::ConnectionEstablished; +use libp2p::swarm::dial_opts::DialOpts; +use libp2p::swarm::{ + CloseConnection, ConnectionClosed, ConnectionDenied, ConnectionHandler, + ConnectionHandlerSelect, ConnectionId, FromSwarm, NetworkBehaviour, THandler, THandlerInEvent, + THandlerOutEvent, ToSwarm, dummy, +}; +use libp2p::{Multiaddr, PeerId, identity, mdns}; +use std::collections::{BTreeSet, HashMap}; +use std::convert::Infallible; +use std::io; +use std::net::IpAddr; +use std::task::{Context, Poll}; +use std::time::Duration; +use util::wakerdeque::WakerDeque; + +const RETRY_CONNECT_INTERVAL: Duration = Duration::from_secs(5); + +mod managed { + use libp2p::swarm::NetworkBehaviour; + use libp2p::{identity, mdns, ping}; + use std::io; + use std::time::Duration; + + const MDNS_RECORD_TTL: Duration = Duration::from_secs(2_500); + const MDNS_QUERY_INTERVAL: Duration = Duration::from_secs(1_500); + const PING_TIMEOUT: Duration = Duration::from_millis(2_500); + const PING_INTERVAL: Duration = Duration::from_millis(2_500); + + #[derive(NetworkBehaviour)] + pub struct Behaviour { + mdns: mdns::tokio::Behaviour, + ping: ping::Behaviour, + } + + impl Behaviour { + pub fn new(keypair: &identity::Keypair) -> io::Result { + Ok(Self { + mdns: mdns_behaviour(keypair)?, + ping: ping_behaviour(), + }) + } + } + + fn mdns_behaviour(keypair: &identity::Keypair) -> io::Result { + use mdns::{Config, tokio}; + + // mDNS config => enable IPv6 + let mdns_config = Config { + ttl: MDNS_RECORD_TTL, + query_interval: MDNS_QUERY_INTERVAL, + + // enable_ipv6: true, // TODO: for some reason, TCP+mDNS don't work well with ipv6?? figure out how to make work + ..Default::default() + }; + + let mdns_behaviour = tokio::Behaviour::new(mdns_config, keypair.public().to_peer_id()); + Ok(mdns_behaviour?) + } + + fn ping_behaviour() -> ping::Behaviour { + ping::Behaviour::new( + ping::Config::new() + .with_timeout(PING_TIMEOUT) + .with_interval(PING_INTERVAL), + ) + } +} + +/// Events for when a listening connection is truly established and truly closed. +#[derive(Debug, Clone)] +pub enum Event { + ConnectionEstablished { + peer_id: PeerId, + connection_id: ConnectionId, + remote_ip: IpAddr, + remote_tcp_port: u16, + }, + ConnectionClosed { + peer_id: PeerId, + connection_id: ConnectionId, + remote_ip: IpAddr, + remote_tcp_port: u16, + }, +} + +/// Discovery behavior that wraps mDNS to produce truly discovered durable peer-connections. +/// +/// The behaviour operates as such: +/// 1) All true (listening) connections/disconnections are tracked, emitting corresponding events +/// to the swarm. +/// 1) mDNS discovered/expired peers are tracked; discovered but not connected peers are dialed +/// immediately, and expired but connected peers are disconnected from immediately. +/// 2) Every fixed interval: discovered but not connected peers are dialed, and expired but +/// connected peers are disconnected from. +pub struct Behaviour { + // state-tracking for managed behaviors & mDNS-discovered peers + managed: managed::Behaviour, + mdns_discovered: HashMap>, + + retry_delay: Delay, // retry interval + + // pending events to emmit => waker-backed Deque to control polling + pending_events: WakerDeque>, +} + +impl Behaviour { + pub fn new(keypair: &identity::Keypair) -> io::Result { + Ok(Self { + managed: managed::Behaviour::new(keypair)?, + mdns_discovered: HashMap::new(), + retry_delay: Delay::new(RETRY_CONNECT_INTERVAL), + pending_events: WakerDeque::new(), + }) + } + + fn dial(&mut self, peer_id: PeerId, addr: Multiaddr) { + self.pending_events.push_back(ToSwarm::Dial { + opts: DialOpts::peer_id(peer_id).addresses(vec![addr]).build(), + }) + } + + fn close_connection(&mut self, peer_id: PeerId, connection: ConnectionId) { + // push front to make this IMMEDIATE + self.pending_events.push_front(ToSwarm::CloseConnection { + peer_id, + connection: CloseConnection::One(connection), + }) + } + + fn handle_mdns_discovered(&mut self, peers: Vec<(PeerId, Multiaddr)>) { + for (p, ma) in peers { + self.dial(p, ma.clone()); // always connect + + // get peer's multi-addresses or insert if missing + let Some(mas) = self.mdns_discovered.get_mut(&p) else { + self.mdns_discovered.insert(p, BTreeSet::from([ma])); + continue; + }; + + // multiaddress should never already be present - else something has gone wrong + let is_new_addr = mas.insert(ma); + assert!(is_new_addr, "cannot discover a discovered peer"); + } + } + + fn handle_mdns_expired(&mut self, peers: Vec<(PeerId, Multiaddr)>) { + for (p, ma) in peers { + // at this point, we *must* have the peer + let mas = self + .mdns_discovered + .get_mut(&p) + .expect("nonexistent peer cannot expire"); + + // at this point, we *must* have the multiaddress + let was_present = mas.remove(&ma); + assert!(was_present, "nonexistent multiaddress cannot expire"); + + // if empty, remove the peer-id entirely + if mas.is_empty() { + self.mdns_discovered.remove(&p); + } + } + } + + fn on_connection_established( + &mut self, + peer_id: PeerId, + connection_id: ConnectionId, + remote_ip: IpAddr, + remote_tcp_port: u16, + ) { + // send out connected event + self.pending_events + .push_back(ToSwarm::GenerateEvent(Event::ConnectionEstablished { + peer_id, + connection_id, + remote_ip, + remote_tcp_port, + })); + } + + fn on_connection_closed( + &mut self, + peer_id: PeerId, + connection_id: ConnectionId, + remote_ip: IpAddr, + remote_tcp_port: u16, + ) { + // send out disconnected event + self.pending_events + .push_back(ToSwarm::GenerateEvent(Event::ConnectionClosed { + peer_id, + connection_id, + remote_ip, + remote_tcp_port, + })); + } +} + +impl NetworkBehaviour for Behaviour { + type ConnectionHandler = + ConnectionHandlerSelect>; + type ToSwarm = Event; + + // simply delegate to underlying mDNS behaviour + + delegate! { + to self.managed { + fn handle_pending_inbound_connection(&mut self, connection_id: ConnectionId, local_addr: &Multiaddr, remote_addr: &Multiaddr) -> Result<(), ConnectionDenied>; + fn handle_pending_outbound_connection(&mut self, connection_id: ConnectionId, maybe_peer: Option, addresses: &[Multiaddr], effective_role: Endpoint) -> Result, ConnectionDenied>; + } + } + + fn handle_established_inbound_connection( + &mut self, + connection_id: ConnectionId, + peer: PeerId, + local_addr: &Multiaddr, + remote_addr: &Multiaddr, + ) -> Result, ConnectionDenied> { + Ok(ConnectionHandler::select( + dummy::ConnectionHandler, + self.managed.handle_established_inbound_connection( + connection_id, + peer, + local_addr, + remote_addr, + )?, + )) + } + + #[allow(clippy::needless_question_mark)] + fn handle_established_outbound_connection( + &mut self, + connection_id: ConnectionId, + peer: PeerId, + addr: &Multiaddr, + role_override: Endpoint, + port_use: PortUse, + ) -> Result, ConnectionDenied> { + Ok(ConnectionHandler::select( + dummy::ConnectionHandler, + self.managed.handle_established_outbound_connection( + connection_id, + peer, + addr, + role_override, + port_use, + )?, + )) + } + + fn on_connection_handler_event( + &mut self, + peer_id: PeerId, + connection_id: ConnectionId, + event: THandlerOutEvent, + ) { + match event { + Either::Left(ev) => libp2p::core::util::unreachable(ev), + Either::Right(ev) => { + self.managed + .on_connection_handler_event(peer_id, connection_id, ev) + } + } + } + + // hook into these methods to drive behavior + + fn on_swarm_event(&mut self, event: FromSwarm) { + self.managed.on_swarm_event(event); // let mDNS handle swarm events + + // handle swarm events to update internal state: + match event { + FromSwarm::ConnectionEstablished(ConnectionEstablished { + peer_id, + connection_id, + endpoint, + .. + }) => { + let remote_address = match endpoint { + ConnectedPoint::Dialer { address, .. } => address, + ConnectedPoint::Listener { send_back_addr, .. } => send_back_addr, + }; + + if let Some((ip, port)) = remote_address.try_to_tcp_addr() { + // handle connection established event which is filtered correctly + self.on_connection_established(peer_id, connection_id, ip, port) + } + } + FromSwarm::ConnectionClosed(ConnectionClosed { + peer_id, + connection_id, + endpoint, + .. + }) => { + let remote_address = match endpoint { + ConnectedPoint::Dialer { address, .. } => address, + ConnectedPoint::Listener { send_back_addr, .. } => send_back_addr, + }; + + if let Some((ip, port)) = remote_address.try_to_tcp_addr() { + // handle connection closed event which is filtered correctly + self.on_connection_closed(peer_id, connection_id, ip, port) + } + } + + // since we are running TCP/IP transport layer, we are assuming that + // no address changes can occur, hence encountering one is a fatal error + FromSwarm::AddressChange(a) => { + unreachable!("unhandlable: address change encountered: {:?}", a) + } + _ => {} + } + } + + fn poll(&mut self, cx: &mut Context) -> Poll>> { + // delegate to managed behaviors for any behaviors they need to perform + match self.managed.poll(cx) { + Poll::Ready(ToSwarm::GenerateEvent(e)) => { + match e { + // handle discovered and expired events from mDNS + managed::BehaviourEvent::Mdns(e) => match e.clone() { + mdns::Event::Discovered(peers) => { + self.handle_mdns_discovered(peers); + } + mdns::Event::Expired(peers) => { + self.handle_mdns_expired(peers); + } + }, + + // handle ping events => if error then disconnect + managed::BehaviourEvent::Ping(e) => { + if let Err(_) = e.result { + self.close_connection(e.peer, e.connection.clone()) + } + } + } + + // since we just consumed an event, we should immediately wake just in case + // there are more events to come where that came from + cx.waker().wake_by_ref(); + } + + // forward any other mDNS event to the swarm or its connection handler(s) + Poll::Ready(e) => { + return Poll::Ready( + e.map_out(|_| unreachable!("events returning to swarm already handled")) + .map_in(Either::Right), + ); + } + + Poll::Pending => {} + } + + // retry connecting to all mDNS peers periodically (fails safely if already connected) + if self.retry_delay.poll_unpin(cx).is_ready() { + for (p, mas) in self.mdns_discovered.clone() { + for ma in mas { + self.dial(p, ma) + } + } + self.retry_delay.reset(RETRY_CONNECT_INTERVAL) // reset timeout + } + + // send out any pending events from our own service + if let Some(e) = self.pending_events.pop_front(cx) { + return Poll::Ready(e.map_in(Either::Left)); + } + + // wait for pending events + Poll::Pending + } +} diff --git a/rust/networking/src/keep_alive.rs b/rust/networking/src/keep_alive.rs new file mode 100644 index 00000000..881b11d7 --- /dev/null +++ b/rust/networking/src/keep_alive.rs @@ -0,0 +1,44 @@ +use delegate::delegate; +use libp2p::swarm::handler::ConnectionEvent; +use libp2p::swarm::{ConnectionHandlerEvent, SubstreamProtocol, dummy, handler}; +use std::task::{Context, Poll}; + +/// An implementation of [`ConnectionHandler`] that doesn't handle any protocols, but it keeps +/// the connection alive. +#[derive(Clone)] +#[repr(transparent)] +pub struct ConnectionHandler(dummy::ConnectionHandler); + +impl ConnectionHandler { + pub fn new() -> Self { + ConnectionHandler(dummy::ConnectionHandler) + } +} + +impl handler::ConnectionHandler for ConnectionHandler { + // delegate types and implementation mostly to dummy handler + type FromBehaviour = ::FromBehaviour; + type ToBehaviour = ::ToBehaviour; + type InboundProtocol = + ::InboundProtocol; + type OutboundProtocol = + ::OutboundProtocol; + type InboundOpenInfo = + ::InboundOpenInfo; + type OutboundOpenInfo = + ::OutboundOpenInfo; + + delegate! { + to self.0 { + fn listen_protocol(&self) -> SubstreamProtocol; + fn poll(&mut self, cx: &mut Context<'_>) -> Poll>; + fn on_behaviour_event(&mut self, event: Self::FromBehaviour); + fn on_connection_event(&mut self, event: ConnectionEvent); + } + } + + // specifically override this to force connection to stay alive + fn connection_keep_alive(&self) -> bool { + true + } +} diff --git a/rust/networking/src/lib.rs b/rust/networking/src/lib.rs new file mode 100644 index 00000000..59b83817 --- /dev/null +++ b/rust/networking/src/lib.rs @@ -0,0 +1,64 @@ +//! TODO: crate documentation +//! +//! this is here as a placeholder documentation +//! +//! + +// enable Rust-unstable features for convenience +#![feature(trait_alias)] +// #![feature(stmt_expr_attributes)] +// #![feature(unboxed_closures)] +// #![feature(assert_matches)] +// #![feature(async_fn_in_dyn_trait)] +// #![feature(async_for_loop)] +// #![feature(auto_traits)] +// #![feature(negative_impls)] + +pub mod discovery; +pub mod keep_alive; +pub mod swarm; + +/// Namespace for all the type/trait aliases used by this crate. +pub(crate) mod alias { + use std::error::Error; + + pub type AnyError = Box; + pub type AnyResult = Result; +} + +/// Namespace for crate-wide extension traits/methods +pub(crate) mod ext { + use extend::ext; + use libp2p::Multiaddr; + use libp2p::multiaddr::Protocol; + use std::net::IpAddr; + + #[ext(pub, name = MultiaddrExt)] + impl Multiaddr { + /// If the multiaddress corresponds to a TCP address, extracts it + fn try_to_tcp_addr(&self) -> Option<(IpAddr, u16)> { + let mut ps = self.into_iter(); + let ip = if let Some(p) = ps.next() { + match p { + Protocol::Ip4(ip) => IpAddr::V4(ip), + Protocol::Ip6(ip) => IpAddr::V6(ip), + _ => return None, + } + } else { + return None; + }; + let Some(Protocol::Tcp(port)) = ps.next() else { + return None; + }; + Some((ip, port)) + } + } +} + +pub(crate) mod private { + #![allow(dead_code)] + + /// Sealed traits support + pub trait Sealed {} + impl Sealed for T {} +} diff --git a/rust/networking/src/swarm.rs b/rust/networking/src/swarm.rs new file mode 100644 index 00000000..a5c87af5 --- /dev/null +++ b/rust/networking/src/swarm.rs @@ -0,0 +1,145 @@ +use crate::alias; +use crate::swarm::transport::tcp_transport; +pub use behaviour::{Behaviour, BehaviourEvent}; +use libp2p::{SwarmBuilder, identity}; + +pub type Swarm = libp2p::Swarm; + +/// The current version of the network: this prevents devices running different versions of the +/// software from interacting with each other. +/// +/// TODO: right now this is a hardcoded constant; figure out what the versioning semantics should +/// even be, and how to inject the right version into this config/initialization. E.g. should +/// this be passed in as a parameter? What about rapidly changing versions in debug builds? +/// this is all VERY very hard to figure out and needs to be mulled over as a team. +pub const NETWORK_VERSION: &[u8] = b"v0.0.1"; +pub const OVERRIDE_VERSION_ENV_VAR: &str = "EXO_LIBP2P_NAMESPACE"; + +/// Create and configure a swarm which listens to all ports on OS +pub fn create_swarm(keypair: identity::Keypair) -> alias::AnyResult { + let mut swarm = SwarmBuilder::with_existing_identity(keypair) + .with_tokio() + .with_other_transport(tcp_transport)? + .with_behaviour(Behaviour::new)? + .build(); + + // Listen on all interfaces and whatever port the OS assigns + swarm.listen_on("/ip4/0.0.0.0/tcp/0".parse()?)?; + Ok(swarm) +} + +mod transport { + use crate::alias; + use crate::swarm::{NETWORK_VERSION, OVERRIDE_VERSION_ENV_VAR}; + use futures::{AsyncRead, AsyncWrite}; + use keccak_const::Sha3_256; + use libp2p::core::muxing; + use libp2p::core::transport::Boxed; + use libp2p::pnet::{PnetError, PnetOutput}; + use libp2p::{PeerId, Transport, identity, noise, pnet, yamux}; + use std::{env, sync::LazyLock}; + + /// Key used for networking's private network; parametrized on the [`NETWORK_VERSION`]. + /// See [`pnet_upgrade`] for more. + static PNET_PRESHARED_KEY: LazyLock<[u8; 32]> = LazyLock::new(|| { + let builder = Sha3_256::new().update(b"exo_discovery_network"); + + if let Ok(var) = env::var(OVERRIDE_VERSION_ENV_VAR) { + let bytes = var.into_bytes(); + builder.update(&bytes) + } else { + builder.update(NETWORK_VERSION) + } + .finalize() + }); + + /// Make the Swarm run on a private network, as to not clash with public libp2p nodes and + /// also different-versioned instances of this same network. + /// This is implemented as an additional "upgrade" ontop of existing [`libp2p::Transport`] layers. + async fn pnet_upgrade( + socket: TSocket, + _: impl Sized, + ) -> Result, PnetError> + where + TSocket: AsyncRead + AsyncWrite + Send + Unpin + 'static, + { + use pnet::{PnetConfig, PreSharedKey}; + PnetConfig::new(PreSharedKey::new(*PNET_PRESHARED_KEY)) + .handshake(socket) + .await + } + + /// TCP/IP transport layer configuration. + pub fn tcp_transport( + keypair: &identity::Keypair, + ) -> alias::AnyResult> { + use libp2p::{ + core::upgrade::Version, + tcp::{Config, tokio}, + }; + + // `TCP_NODELAY` enabled => avoid latency + let tcp_config = Config::default().nodelay(true); + + // V1 + lazy flushing => 0-RTT negotiation + let upgrade_version = Version::V1Lazy; + + // Noise is faster than TLS + we don't care much for security + let noise_config = noise::Config::new(keypair)?; + + // Use default Yamux config for multiplexing + let yamux_config = yamux::Config::default(); + + // Create new Tokio-driven TCP/IP transport layer + let base_transport = tokio::Transport::new(tcp_config) + .and_then(pnet_upgrade) + .upgrade(upgrade_version) + .authenticate(noise_config) + .multiplex(yamux_config); + + // Return boxed transport (to flatten complex type) + Ok(base_transport.boxed()) + } +} + +mod behaviour { + use crate::{alias, discovery}; + use libp2p::swarm::NetworkBehaviour; + use libp2p::{gossipsub, identity}; + use std::time::Duration; + + /// Behavior of the Swarm which composes all desired behaviors: + /// Right now its just [`discovery::Behaviour`] and [`gossipsub::Behaviour`]. + #[derive(NetworkBehaviour)] + pub struct Behaviour { + pub discovery: discovery::Behaviour, + pub gossipsub: gossipsub::Behaviour, + } + + impl Behaviour { + pub fn new(keypair: &identity::Keypair) -> alias::AnyResult { + Ok(Self { + discovery: discovery::Behaviour::new(keypair)?, + gossipsub: gossipsub_behaviour(keypair), + }) + } + } + + fn gossipsub_behaviour(keypair: &identity::Keypair) -> gossipsub::Behaviour { + use gossipsub::{ConfigBuilder, MessageAuthenticity, ValidationMode}; + + // build a gossipsub network behaviour + // => signed message authenticity + strict validation mode means the message-ID is + // automatically provided by gossipsub w/out needing to provide custom message-ID function + gossipsub::Behaviour::new( + MessageAuthenticity::Signed(keypair.clone()), + ConfigBuilder::default() + .publish_queue_duration(Duration::from_secs(15)) + .max_transmit_size(1024 * 1024) + .validation_mode(ValidationMode::Strict) + .build() + .expect("the configuration should always be valid"), + ) + .expect("creating gossipsub behavior should always work") + } +} diff --git a/rust/networking/tests/dummy.rs b/rust/networking/tests/dummy.rs new file mode 100644 index 00000000..ddaa8cc2 --- /dev/null +++ b/rust/networking/tests/dummy.rs @@ -0,0 +1,7 @@ +// maybe this will hold test in the future...?? + +#[cfg(test)] +mod tests { + #[test] + fn does_nothing() {} +} diff --git a/rust/rust-toolchain.toml b/rust/rust-toolchain.toml new file mode 100644 index 00000000..271800cb --- /dev/null +++ b/rust/rust-toolchain.toml @@ -0,0 +1,2 @@ +[toolchain] +channel = "nightly" \ No newline at end of file diff --git a/rust/system_custodian/Cargo.toml b/rust/system_custodian/Cargo.toml new file mode 100644 index 00000000..46e530b1 --- /dev/null +++ b/rust/system_custodian/Cargo.toml @@ -0,0 +1,47 @@ +[package] +name = "system_custodian" +version = { workspace = true } +edition = { workspace = true } +publish = false + +[lib] +doctest = false +name = "system_custodian" +path = "src/lib.rs" + +[[bin]] +path = "src/bin/main.rs" +name = "system_custodian" +doc = false + +[lints] +workspace = true + +[dependencies] +# datastructures +either = { workspace = true } + +# macro dependencies +extend = { workspace = true } +delegate = { workspace = true } +impl-trait-for-tuples = { workspace = true } +derive_more = { workspace = true } + +# async +tokio = { workspace = true, features = ["full"] } +futures = { workspace = true } +futures-timer = { workspace = true } + +# utility dependencies +util = { workspace = true } +thiserror = { workspace = true } +#internment = { workspace = true } +#recursion = { workspace = true } +#generativity = { workspace = true } +#itertools = { workspace = true } +tracing-subscriber = { version = "0.3.19", features = ["default", "env-filter"] } +keccak-const = { workspace = true } + +# tracing/logging +log = { workspace = true } + diff --git a/rust/system_custodian/src/bin/main.rs b/rust/system_custodian/src/bin/main.rs new file mode 100644 index 00000000..2345c633 --- /dev/null +++ b/rust/system_custodian/src/bin/main.rs @@ -0,0 +1,4 @@ +//! TODO: documentation +//! + +fn main() {} diff --git a/rust/system_custodian/src/lib.rs b/rust/system_custodian/src/lib.rs new file mode 100644 index 00000000..cf856239 --- /dev/null +++ b/rust/system_custodian/src/lib.rs @@ -0,0 +1,69 @@ +//! This crate defines the logic of, and ways to interact with, Exo's **_System Custodian_** daemon. +//! +//! The **_System Custodian_** daemon is supposed to be a long-living process that precedes the +//! launch of the Exo application, and responsible for ensuring the system (configuration, settings, +//! etc.) is in an appropriate state to facilitate the running of Exo application. +//! The **_System Custodian_** daemon shall expose a [D-Bus](https://www.freedesktop.org/wiki/Software/dbus/) +//! service which Exo application use to _control & query_ it. +//! +//! # Lifecycle +//! When the Exo application starts, it will _wake_ the **_System Custodian_** daemon for the +//! duration of its lifetime, and after it has terminated the daemon will go back to sleep. When +//! the daemon wakes up, it will configure the system into a state suitable for the Exo Application; +//! When the daemon goes to sleep, it will revert those changes as much as it can in case they were +//! destructive to the user's pre-existing configurations. +//! +//! # Responsibilities +//! TODO: these are purely on MacOS, but change to be more broad +//! The **_System Custodian_** daemon is responsible for using System Configuration framework to +//! 1. duplicate the current network set +//! 2. modify existing services to turn on IPv6 if not there +//! 3. remove any bridge services & add any missing services that AREN'T bridge +//! TODO: In the future: +//! 1. run a dummy AWDL service to [allow for macOS peer-to-peer wireless networking](https://yggdrasil-network.github.io/2019/08/19/awdl.html) +//! 2. toggle some GPU/memory configurations to speed up GPU (ask Alex what those configurations are) +//! 3. if we ever decide to provide our **own network interfaces** that abstract over some userland +//! logic, this would be the place to spin that up. +//! +//! Then it will watch the SCDynamicStore for: +//! 1. all __actual__ network interfaces -> collect information on them e.g. their BSD name, MAC +//! address, MTU, IPv6 addresses, etc. -> and set up watchers/notifiers to inform the DBus +//! interface of any changes +//! 2. watch for any __undesirable__ changes to configuration and revert it +//! +//! It should somehow (probably through system sockets and/or BSD interface) trigger IPv6 NDP on +//! each of the interfaces & also listen to/query for any changes on the OS routing cache?? +//! Basically emulate the `ping6 ff02::1%enX` and `ndp -an` commands BUT BETTER!!! +//! 1. all that info should coalesce back to the overall state colleted -> should be queryable +//! over D-Bus +//! TODO: +//! 1. we might potentially add to this step a handshake of some kind...? To ensure that we can +//! ACTUALLY communicate with that machine over that link over e.g. TCP, UDP, etc. Will the +//! handshake require to know Node ID? Will the handshake require heartbeats? Who knows... +//! 2. if we ever decide to write proprietary L2/L3 protocols for quicker communication, +//! e.g. [AF_NDRV](https://www.zerotier.com/blog/how-zerotier-eliminated-kernel-extensions-on-macos/) +//! for raw ethernet frame communication, or even a [custom thunderbolt PCIe driver](https://developer.apple.com/documentation/pcidriverkit/creating-custom-pcie-drivers-for-thunderbolt-devices), +//! then this would be the place to carry out discovery and propper handshakes with devices +//! on the other end of the link. +//! + +// enable Rust-unstable features for convenience +#![feature(trait_alias)] +#![feature(stmt_expr_attributes)] +#![feature(type_alias_impl_trait)] +#![feature(specialization)] +#![feature(unboxed_closures)] +#![feature(const_trait_impl)] +#![feature(fn_traits)] + +pub(crate) mod private { + // sealed traits support + pub trait Sealed {} + impl Sealed for T {} +} + +/// Namespace for all the type/trait aliases used by this crate. +pub(crate) mod alias {} + +/// Namespace for crate-wide extension traits/methods +pub(crate) mod ext {} diff --git a/rust/util/Cargo.toml b/rust/util/Cargo.toml new file mode 100644 index 00000000..aeae3534 --- /dev/null +++ b/rust/util/Cargo.toml @@ -0,0 +1,25 @@ +[package] +name = "util" +version = { workspace = true } +edition = { workspace = true } +publish = false + +[lib] +doctest = false +name = "util" +path = "src/lib.rs" + +[lints] +workspace = true + +[dependencies] +# macro dependencies +extend = { workspace = true } + +# utility dependencies +thiserror = { workspace = true } +once_cell = { workspace = true } +internment = { workspace = true } +derive_more = { workspace = true } +bon = { workspace = true } +recursion = { workspace = true } diff --git a/rust/util/src/lib.rs b/rust/util/src/lib.rs new file mode 100644 index 00000000..60e11f3a --- /dev/null +++ b/rust/util/src/lib.rs @@ -0,0 +1,53 @@ +//! TODO: crate documentation +//! +//! this is here as a placeholder documentation +//! +//! + +// enable Rust-unstable features for convenience +#![feature(trait_alias)] +#![feature(stmt_expr_attributes)] +#![feature(type_alias_impl_trait)] +#![feature(specialization)] +#![feature(unboxed_closures)] +#![feature(const_trait_impl)] +#![feature(fn_traits)] + +pub mod nonempty; +pub mod wakerdeque; + +pub(crate) mod private { + // sealed traits support + pub trait Sealed {} + impl Sealed for T {} +} + +/// Namespace for all the type/trait aliases used by this crate. +pub(crate) mod alias {} + +/// Namespace for crate-wide extension traits/methods +pub mod ext { + use extend::ext; + + #[ext(pub, name = BoxedSliceExt)] + impl Box<[T]> { + #[inline] + fn map(self, f: F) -> Box<[B]> + where + F: FnMut(T) -> B, + { + self.into_iter().map(f).collect() + } + } + + #[ext(pub, name = VecExt)] + impl Vec { + #[inline] + fn map(self, f: F) -> Vec + where + F: FnMut(T) -> B, + { + self.into_iter().map(f).collect() + } + } +} diff --git a/rust/util/src/nonempty.rs b/rust/util/src/nonempty.rs new file mode 100644 index 00000000..e9eb8620 --- /dev/null +++ b/rust/util/src/nonempty.rs @@ -0,0 +1,138 @@ +use std::slice::SliceIndex; +use std::{ops, slice}; +use thiserror::Error; + +#[derive(Error, Debug)] +#[error("Cannot create to `NonemptyArray` because the supplied slice is empty")] +pub struct EmptySliceError; + +/// A pointer to a non-empty fixed-size slice allocated on the heap. +#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] +#[repr(transparent)] +pub struct NonemptyArray(Box<[T]>); + +#[allow(clippy::arbitrary_source_item_ordering)] +impl NonemptyArray { + #[inline] + pub fn singleton(value: T) -> Self { + Self(Box::new([value])) + } + + #[allow(clippy::missing_errors_doc)] + #[inline] + pub fn try_from_boxed_slice>>( + boxed_slice: S, + ) -> Result { + let boxed_slice = boxed_slice.into(); + if boxed_slice.is_empty() { + Err(EmptySliceError) + } else { + Ok(Self(boxed_slice)) + } + } + + #[must_use] + #[inline] + pub fn into_boxed_slice(self) -> Box<[T]> { + self.0 + } + + #[must_use] + #[inline] + pub fn to_vec(&self) -> Vec + where + T: Clone, + { + self.0.to_vec() + } + + #[must_use] + #[inline] + pub const fn as_slice(&self) -> &[T] { + &self.0 + } + + #[allow(clippy::indexing_slicing)] + #[must_use] + #[inline] + pub fn first(&self) -> &T { + &self.0[0] + } + + #[allow(clippy::indexing_slicing, clippy::arithmetic_side_effects)] + #[must_use] + #[inline] + pub fn last(&self) -> &T { + &self.0[self.0.len() - 1] + } + + #[must_use] + #[inline] + pub fn get(&self, index: I) -> Option<&I::Output> + where + I: SliceIndex<[T]>, + { + self.0.get(index) + } + + #[allow(clippy::len_without_is_empty)] + #[must_use] + #[inline] + pub const fn len(&self) -> usize { + self.0.len() + } + + #[allow(clippy::iter_without_into_iter)] + #[inline] + pub fn iter(&self) -> slice::Iter<'_, T> { + self.0.iter() + } + + #[allow(clippy::iter_without_into_iter)] + #[inline] + pub fn iter_mut(&mut self) -> slice::IterMut<'_, T> { + self.0.iter_mut() + } + + #[inline] + #[must_use] + pub fn map U>(self, f: F) -> NonemptyArray { + NonemptyArray(self.0.into_iter().map(f).collect()) + } +} + +impl From> for Box<[T]> { + #[inline] + fn from(value: NonemptyArray) -> Self { + value.into_boxed_slice() + } +} + +impl ops::Index for NonemptyArray { + type Output = T; + + #[inline] + fn index(&self, index: usize) -> &Self::Output { + self.0.index(index) + } +} + +impl IntoIterator for NonemptyArray { + type Item = T; + type IntoIter = std::vec::IntoIter; + + #[inline] + fn into_iter(self) -> Self::IntoIter { + self.into_boxed_slice().into_vec().into_iter() + } +} + +impl<'a, T> IntoIterator for &'a NonemptyArray { + type Item = &'a T; + type IntoIter = slice::Iter<'a, T>; + + #[inline] + fn into_iter(self) -> Self::IntoIter { + self.iter() + } +} diff --git a/rust/util/src/wakerdeque.rs b/rust/util/src/wakerdeque.rs new file mode 100644 index 00000000..336c0347 --- /dev/null +++ b/rust/util/src/wakerdeque.rs @@ -0,0 +1,55 @@ +use std::collections::VecDeque; +use std::fmt::{Debug, Formatter}; +use std::task::{Context, Waker}; + +/// A wrapper around [`VecDeque`] which wakes (if it can) on any `push_*` methods, +/// and updates the internally stored waker by consuming [`Context`] on any `pop_*` methods. +pub struct WakerDeque { + waker: Option, + deque: VecDeque, +} + +impl Debug for WakerDeque { + fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result { + self.deque.fmt(f) + } +} + +impl WakerDeque { + pub fn new() -> Self { + Self { + waker: None, + deque: VecDeque::new(), + } + } + + fn update(&mut self, cx: &mut Context<'_>) { + self.waker = Some(cx.waker().clone()); + } + + fn wake(&mut self) { + let Some(ref mut w) = self.waker else { return }; + w.wake_by_ref(); + self.waker = None; + } + + pub fn pop_front(&mut self, cx: &mut Context<'_>) -> Option { + self.update(cx); + self.deque.pop_front() + } + + pub fn pop_back(&mut self, cx: &mut Context<'_>) -> Option { + self.update(cx); + self.deque.pop_back() + } + + pub fn push_front(&mut self, value: T) { + self.wake(); + self.deque.push_front(value); + } + + pub fn push_back(&mut self, value: T) { + self.wake(); + self.deque.push_back(value); + } +} diff --git a/src/exo/__init__.py b/src/exo/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/__main__.py b/src/exo/__main__.py new file mode 100644 index 00000000..6cfe06a5 --- /dev/null +++ b/src/exo/__main__.py @@ -0,0 +1,4 @@ +from exo.main import main + +if __name__ == "__main__": + main() diff --git a/src/exo/main.py b/src/exo/main.py new file mode 100644 index 00000000..b859d2ce --- /dev/null +++ b/src/exo/main.py @@ -0,0 +1,255 @@ +import argparse +import multiprocessing as mp +import signal +from dataclasses import dataclass, field +from typing import Self + +import anyio +from anyio.abc import TaskGroup +from loguru import logger +from pydantic import PositiveInt + +import exo.routing.topics as topics +from exo.master.api import API # TODO: should API be in master? +from exo.master.main import Master +from exo.routing.router import Router, get_node_id_keypair +from exo.shared.constants import EXO_LOG +from exo.shared.election import Election, ElectionResult +from exo.shared.logging import logger_cleanup, logger_setup +from exo.shared.types.common import NodeId, SessionId +from exo.utils.channels import Receiver, channel +from exo.utils.pydantic_ext import CamelCaseModel +from exo.worker.download.impl_shard_downloader import exo_shard_downloader +from exo.worker.main import Worker + + +# I marked this as a dataclass as I want trivial constructors. +@dataclass +class Node: + router: Router + worker: Worker + election: Election # Every node participates in election, as we do want a node to become master even if it isn't a master candidate if no master candidates are present. + election_result_receiver: Receiver[ElectionResult] + master: Master | None + api: API | None + + node_id: NodeId + _tg: TaskGroup = field(init=False, default_factory=anyio.create_task_group) + + @classmethod + async def create(cls, args: "Args") -> "Self": + keypair = get_node_id_keypair() + node_id = NodeId(keypair.to_peer_id().to_base58()) + session_id = SessionId(master_node_id=node_id, election_clock=0) + router = Router.create(keypair) + await router.register_topic(topics.GLOBAL_EVENTS) + await router.register_topic(topics.LOCAL_EVENTS) + await router.register_topic(topics.COMMANDS) + await router.register_topic(topics.ELECTION_MESSAGES) + await router.register_topic(topics.CONNECTION_MESSAGES) + + logger.info(f"Starting node {node_id}") + if args.spawn_api: + api = API( + node_id, + session_id, + port=args.api_port, + global_event_receiver=router.receiver(topics.GLOBAL_EVENTS), + command_sender=router.sender(topics.COMMANDS), + election_receiver=router.receiver(topics.ELECTION_MESSAGES), + ) + else: + api = None + + worker = Worker( + node_id, + session_id, + exo_shard_downloader(), + connection_message_receiver=router.receiver(topics.CONNECTION_MESSAGES), + global_event_receiver=router.receiver(topics.GLOBAL_EVENTS), + local_event_sender=router.sender(topics.LOCAL_EVENTS), + command_sender=router.sender(topics.COMMANDS), + ) + # We start every node with a master + master = Master( + node_id, + session_id, + global_event_sender=router.sender(topics.GLOBAL_EVENTS), + local_event_receiver=router.receiver(topics.LOCAL_EVENTS), + command_receiver=router.receiver(topics.COMMANDS), + tb_only=args.tb_only, + ) + + er_send, er_recv = channel[ElectionResult]() + election = Election( + node_id, + # If someone manages to assemble 1 MILLION devices into an exo cluster then. well done. good job champ. + seniority=1_000_000 if args.force_master else 0, + # nb: this DOES feedback right now. i have thoughts on how to address this, + # but ultimately it seems not worth the complexity + election_message_sender=router.sender(topics.ELECTION_MESSAGES), + election_message_receiver=router.receiver(topics.ELECTION_MESSAGES), + connection_message_receiver=router.receiver(topics.CONNECTION_MESSAGES), + command_receiver=router.receiver(topics.COMMANDS), + election_result_sender=er_send, + ) + + return cls(router, worker, election, er_recv, master, api, node_id) + + async def run(self): + async with self._tg as tg: + signal.signal(signal.SIGINT, lambda _, __: self.shutdown()) + tg.start_soon(self.router.run) + tg.start_soon(self.worker.run) + tg.start_soon(self.election.run) + if self.master: + tg.start_soon(self.master.run) + if self.api: + tg.start_soon(self.api.run) + tg.start_soon(self._elect_loop) + + def shutdown(self): + # if this is our second call to shutdown, just sys.exit + if self._tg.cancel_scope.cancel_called: + import sys + + sys.exit(1) + self._tg.cancel_scope.cancel() + + async def _elect_loop(self): + with self.election_result_receiver as results: + async for result in results: + # This function continues to have a lot of very specific entangled logic + # At least it's somewhat contained + + # I don't like this duplication, but it's manageable for now. + # TODO: This function needs refactoring generally + + # Ok: + # On new master: + # - Elect master locally if necessary + # - Shutdown and re-create the worker + # - Shut down and re-create the API + + if ( + result.session_id.master_node_id == self.node_id + and self.master is not None + ): + logger.info("Node elected Master") + elif ( + result.session_id.master_node_id == self.node_id + and self.master is None + ): + logger.info("Node elected Master - promoting self") + self.master = Master( + self.node_id, + result.session_id, + global_event_sender=self.router.sender(topics.GLOBAL_EVENTS), + local_event_receiver=self.router.receiver(topics.LOCAL_EVENTS), + command_receiver=self.router.receiver(topics.COMMANDS), + ) + self._tg.start_soon(self.master.run) + elif ( + result.session_id.master_node_id != self.node_id + and self.master is not None + ): + logger.info( + f"Node {result.session_id.master_node_id} elected master - demoting self" + ) + await self.master.shutdown() + self.master = None + else: + logger.info( + f"Node {result.session_id.master_node_id} elected master" + ) + if result.is_new_master: + await anyio.sleep(0) + if self.worker: + self.worker.shutdown() + # TODO: add profiling etc to resource monitor + self.worker = Worker( + self.node_id, + result.session_id, + exo_shard_downloader(), + connection_message_receiver=self.router.receiver( + topics.CONNECTION_MESSAGES + ), + global_event_receiver=self.router.receiver( + topics.GLOBAL_EVENTS + ), + local_event_sender=self.router.sender(topics.LOCAL_EVENTS), + command_sender=self.router.sender(topics.COMMANDS), + ) + self._tg.start_soon(self.worker.run) + if self.api: + self.api.reset(result.session_id, result.won_clock) + else: + if self.api: + self.api.unpause(result.won_clock) + + +def main(): + args = Args.parse() + + mp.set_start_method("spawn") + # TODO: Refactor the current verbosity system + logger_setup(EXO_LOG, args.verbosity) + logger.info("Starting EXO") + + node = anyio.run(Node.create, args) + anyio.run(node.run) + logger.info("EXO Shutdown complete") + logger_cleanup() + + +class Args(CamelCaseModel): + verbosity: int = 0 + force_master: bool = False + spawn_api: bool = False + api_port: PositiveInt = 8000 + tb_only: bool = False + + @classmethod + def parse(cls) -> Self: + parser = argparse.ArgumentParser(prog="EXO") + default_verbosity = 0 + parser.add_argument( + "-q", + "--quiet", + action="store_const", + const=-1, + dest="verbosity", + default=default_verbosity, + ) + parser.add_argument( + "-v", + "--verbose", + action="count", + dest="verbosity", + default=default_verbosity, + ) + parser.add_argument( + "-m", + "--force-master", + action="store_true", + dest="force_master", + ) + parser.add_argument( + "--no-api", + action="store_false", + dest="spawn_api", + ) + parser.add_argument( + "--api-port", + type=int, + dest="api_port", + default=8000, + ) + parser.add_argument( + "--tb-only", + action="store_true", + dest="tb_only", + ) + + args = parser.parse_args() + return cls(**vars(args)) # pyright: ignore[reportAny] - We are intentionally validating here, we can't do it statically diff --git a/src/exo/master/__init__.py b/src/exo/master/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/master/api.py b/src/exo/master/api.py new file mode 100644 index 00000000..ffbf3fde --- /dev/null +++ b/src/exo/master/api.py @@ -0,0 +1,502 @@ +import time +from collections.abc import AsyncGenerator +from typing import cast + +import anyio +from anyio import create_task_group +from anyio.abc import TaskGroup +from fastapi import FastAPI, HTTPException +from fastapi.middleware.cors import CORSMiddleware +from fastapi.responses import StreamingResponse +from fastapi.staticfiles import StaticFiles +from hypercorn.asyncio import serve # pyright: ignore[reportUnknownVariableType] +from hypercorn.config import Config +from hypercorn.typing import ASGIFramework +from loguru import logger + +from exo.master.placement import place_instance as get_instance_placements +from exo.shared.apply import apply +from exo.shared.election import ElectionMessage +from exo.shared.logging import InterceptLogger +from exo.shared.models.model_cards import MODEL_CARDS +from exo.shared.models.model_meta import get_model_meta +from exo.shared.types.api import ( + ChatCompletionMessage, + ChatCompletionResponse, + CreateInstanceParams, + CreateInstanceResponse, + DeleteInstanceResponse, + ModelList, + ModelListModel, + PlaceInstanceParams, + PlacementPreview, + PlacementPreviewResponse, + StreamingChoiceResponse, +) +from exo.shared.types.chunks import TokenChunk +from exo.shared.types.commands import ( + ChatCompletion, + Command, + CreateInstance, + DeleteInstance, + ForwarderCommand, + PlaceInstance, + TaskFinished, +) +from exo.shared.types.common import CommandId, NodeId, SessionId +from exo.shared.types.events import ChunkGenerated, Event, ForwarderEvent, IndexedEvent +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.state import State +from exo.shared.types.tasks import ChatCompletionTaskParams +from exo.shared.types.worker.instances import Instance, InstanceId, InstanceMeta +from exo.shared.types.worker.shards import Sharding +from exo.utils.banner import print_startup_banner +from exo.utils.channels import Receiver, Sender, channel +from exo.utils.dashboard_path import find_dashboard +from exo.utils.event_buffer import OrderedBuffer + +HIDE_THINKING = False + + +def chunk_to_response( + chunk: TokenChunk, command_id: CommandId +) -> ChatCompletionResponse: + return ChatCompletionResponse( + id=command_id, + created=int(time.time()), + model=chunk.model, + choices=[ + StreamingChoiceResponse( + index=0, + delta=ChatCompletionMessage(role="assistant", content=chunk.text), + finish_reason=chunk.finish_reason, + ) + ], + ) + + +async def resolve_model_meta(model_id: str) -> ModelMetadata: + if model_id in MODEL_CARDS: + model_card = MODEL_CARDS[model_id] + return model_card.metadata + else: + return await get_model_meta(model_id) + + +class API: + def __init__( + self, + node_id: NodeId, + session_id: SessionId, + *, + port: int = 8000, + # Ideally this would be a MasterForwarderEvent but type system says no :( + global_event_receiver: Receiver[ForwarderEvent], + command_sender: Sender[ForwarderCommand], + # This lets us pause the API if an election is running + election_receiver: Receiver[ElectionMessage], + ) -> None: + self.state = State() + self._event_log: list[Event] = [] + self.command_sender = command_sender + self.global_event_receiver = global_event_receiver + self.election_receiver = election_receiver + self.event_buffer: OrderedBuffer[Event] = OrderedBuffer[Event]() + self.node_id: NodeId = node_id + self.session_id: SessionId = session_id + self.last_completed_election: int = 0 + self.port = port + + self.paused: bool = False + self.paused_ev: anyio.Event = anyio.Event() + + self.app = FastAPI() + self._setup_cors() + self._setup_routes() + + self.app.mount( + "/", + StaticFiles( + directory=find_dashboard(), + html=True, + ), + name="dashboard", + ) + + self._chat_completion_queues: dict[CommandId, Sender[TokenChunk]] = {} + self._tg: TaskGroup | None = None + + def reset(self, new_session_id: SessionId, result_clock: int): + logger.info("Resetting API State") + self.state = State() + self.session_id = new_session_id + self.event_buffer = OrderedBuffer[Event]() + self._chat_completion_queues = {} + self.unpause(result_clock) + + def unpause(self, result_clock: int): + logger.info("Unpausing API") + self.last_completed_election = result_clock + self.paused = False + self.paused_ev.set() + self.paused_ev = anyio.Event() + + def _setup_cors(self) -> None: + self.app.add_middleware( + CORSMiddleware, + allow_origins=["*"], + allow_credentials=True, + allow_methods=["*"], + allow_headers=["*"], + ) + + def _setup_routes(self) -> None: + self.app.get("/node_id")(lambda: self.node_id) + self.app.post("/instance")(self.create_instance) + self.app.post("/place_instance")(self.place_instance) + self.app.get("/instance/placement")(self.get_placement) + self.app.get("/instance/previews")(self.get_placement_previews) + self.app.get("/instance/{instance_id}")(self.get_instance) + self.app.delete("/instance/{instance_id}")(self.delete_instance) + self.app.get("/models")(self.get_models) + self.app.get("/v1/models")(self.get_models) + self.app.post("/v1/chat/completions")(self.chat_completions) + self.app.get("/state")(lambda: self.state) + self.app.get("/events")(lambda: self._event_log) + + async def place_instance(self, payload: PlaceInstanceParams): + command = PlaceInstance( + model_meta=await resolve_model_meta(payload.model_id), + sharding=payload.sharding, + instance_meta=payload.instance_meta, + min_nodes=payload.min_nodes, + ) + await self._send(command) + + return CreateInstanceResponse( + message="Command received.", + command_id=command.command_id, + ) + + async def create_instance( + self, payload: CreateInstanceParams + ) -> CreateInstanceResponse: + command = CreateInstance(instance=payload.instance) + await self._send(command) + + return CreateInstanceResponse( + message="Command received.", + command_id=command.command_id, + ) + + async def get_placement( + self, + model_id: str, + sharding: Sharding = Sharding.Pipeline, + instance_meta: InstanceMeta = InstanceMeta.MlxRing, + min_nodes: int = 1, + ) -> Instance: + model_meta = await resolve_model_meta(model_id) + + try: + placements = get_instance_placements( + PlaceInstance( + model_meta=model_meta, + sharding=sharding, + instance_meta=instance_meta, + min_nodes=min_nodes, + ), + topology=self.state.topology, + current_instances=self.state.instances, + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + current_ids = set(self.state.instances.keys()) + new_ids = [ + instance_id for instance_id in placements if instance_id not in current_ids + ] + if len(new_ids) != 1: + raise HTTPException( + status_code=500, + detail="Expected exactly one new instance from placement", + ) + + return placements[new_ids[0]] + + async def get_placement_previews( + self, model_id: ModelId + ) -> PlacementPreviewResponse: + seen: set[tuple[ModelId, Sharding, InstanceMeta, int]] = set() + previews: list[PlacementPreview] = [] + if len(list(self.state.topology.list_nodes())) == 0: + return PlacementPreviewResponse(previews=[]) + + cards = [card for card in MODEL_CARDS.values() if card.short_id == model_id] + if not cards: + raise HTTPException(status_code=404, detail=f"Model {model_id} not found") + + instance_combinations: list[tuple[Sharding, InstanceMeta, int]] = [] + for sharding in (Sharding.Pipeline, Sharding.Tensor): + for instance_meta in (InstanceMeta.MlxRing, InstanceMeta.MlxJaccl): + instance_combinations.extend( + [ + (sharding, instance_meta, i) + for i in range( + 1, len(list(self.state.topology.list_nodes())) + 1 + ) + ] + ) + # TODO: PDD + # instance_combinations.append((Sharding.PrefillDecodeDisaggregation, InstanceMeta.MlxRing, 1)) + + for card in cards: + model_meta = card.metadata + for sharding, instance_meta, min_nodes in instance_combinations: + try: + placements = get_instance_placements( + PlaceInstance( + model_meta=model_meta, + sharding=sharding, + instance_meta=instance_meta, + min_nodes=min_nodes, + ), + topology=self.state.topology, + current_instances=self.state.instances, + ) + except ValueError as exc: + if (card.model_id, sharding, instance_meta, 0) not in seen: + previews.append( + PlacementPreview( + model_id=card.model_id, + sharding=sharding, + instance_meta=instance_meta, + instance=None, + error=str(exc), + ) + ) + seen.add((card.model_id, sharding, instance_meta, 0)) + continue + + current_ids = set(self.state.instances.keys()) + new_instances = [ + instance + for instance_id, instance in placements.items() + if instance_id not in current_ids + ] + + if len(new_instances) != 1: + if (card.model_id, sharding, instance_meta, 0) not in seen: + previews.append( + PlacementPreview( + model_id=card.model_id, + sharding=sharding, + instance_meta=instance_meta, + instance=None, + error="Expected exactly one new instance from placement", + ) + ) + seen.add((card.model_id, sharding, instance_meta, 0)) + continue + + instance = new_instances[0] + shard_assignments = instance.shard_assignments + node_ids = list(shard_assignments.node_to_runner.keys()) + + memory_delta_by_node: dict[str, int] = {} + if node_ids: + total_bytes = model_meta.storage_size.in_bytes + per_node = total_bytes // len(node_ids) + remainder = total_bytes % len(node_ids) + for index, node_id in enumerate(sorted(node_ids, key=str)): + extra = 1 if index < remainder else 0 + memory_delta_by_node[str(node_id)] = per_node + extra + + if ( + card.model_id, + sharding, + instance_meta, + len(node_ids), + ) not in seen: + previews.append( + PlacementPreview( + model_id=card.model_id, + sharding=sharding, + instance_meta=instance_meta, + instance=instance, + memory_delta_by_node=memory_delta_by_node or None, + error=None, + ) + ) + seen.add((card.model_id, sharding, instance_meta, len(node_ids))) + + return PlacementPreviewResponse(previews=previews) + + def get_instance(self, instance_id: InstanceId) -> Instance: + if instance_id not in self.state.instances: + raise HTTPException(status_code=404, detail="Instance not found") + return self.state.instances[instance_id] + + async def delete_instance(self, instance_id: InstanceId) -> DeleteInstanceResponse: + if instance_id not in self.state.instances: + raise HTTPException(status_code=404, detail="Instance not found") + + command = DeleteInstance( + instance_id=instance_id, + ) + await self._send(command) + return DeleteInstanceResponse( + message="Command received.", + command_id=command.command_id, + instance_id=instance_id, + ) + + async def _generate_chat_stream( + self, command_id: CommandId + ) -> AsyncGenerator[str, None]: + """Generate chat completion stream as JSON strings.""" + + try: + self._chat_completion_queues[command_id], recv = channel[TokenChunk]() + + is_thinking = False + with recv as token_chunks: + async for chunk in token_chunks: + if HIDE_THINKING: + if chunk.text == "": + is_thinking = True + if chunk.text == "": + is_thinking = False + chunk_response: ChatCompletionResponse = chunk_to_response( + chunk, command_id + ) + if not (is_thinking and HIDE_THINKING): + logger.debug(f"chunk_response: {chunk_response}") + yield f"data: {chunk_response.model_dump_json()}\n\n" + + if chunk.finish_reason is not None: + yield "data: [DONE]\n\n" + break + + except anyio.get_cancelled_exc_class(): + # TODO: TaskCancelled + """ + self.command_sender.send_nowait( + ForwarderCommand(origin=self.node_id, command=command) + ) + """ + raise + finally: + command = TaskFinished(finished_command_id=command_id) + await self._send(command) + del self._chat_completion_queues[command_id] + + async def _trigger_notify_user_to_download_model(self, model_id: str) -> None: + logger.warning( + "TODO: we should send a notification to the user to download the model" + ) + + async def chat_completions( + self, payload: ChatCompletionTaskParams + ) -> StreamingResponse: + """Handle chat completions with proper streaming response.""" + model_meta = await resolve_model_meta(payload.model) + payload.model = model_meta.model_id + + if not any( + instance.shard_assignments.model_id == payload.model + for instance in self.state.instances.values() + ): + await self._trigger_notify_user_to_download_model(payload.model) + raise HTTPException( + status_code=404, detail=f"No instance found for model {payload.model}" + ) + + command = ChatCompletion( + request_params=payload, + ) + await self._send(command) + return StreamingResponse( + self._generate_chat_stream(command.command_id), + media_type="text/event-stream", + ) + + def _calculate_total_available_memory(self) -> Memory: + """Calculate total available memory across all nodes in bytes.""" + total_available = Memory() + + for node in self.state.topology.list_nodes(): + if node.node_profile is not None: + total_available += node.node_profile.memory.ram_available + + return total_available + + async def get_models(self) -> ModelList: + """Returns list of available models.""" + return ModelList( + data=[ + ModelListModel( + id=card.short_id, + hugging_face_id=card.model_id, + name=card.name, + description=card.description, + tags=card.tags, + ) + for card in MODEL_CARDS.values() + ] + ) + + async def run(self): + cfg = Config() + cfg.bind = f"0.0.0.0:{self.port}" + # nb: shared.logging needs updating if any of this changes + cfg.accesslog = None + cfg.errorlog = "-" + cfg.logger_class = InterceptLogger + + async with create_task_group() as tg: + self._tg = tg + logger.info("Starting API") + tg.start_soon(self._applystate) + tg.start_soon(self._pause_on_new_election) + print_startup_banner(self.port) + await serve( + cast(ASGIFramework, self.app), + cfg, + shutdown_trigger=lambda: anyio.sleep_forever(), + ) + + self.command_sender.close() + self.global_event_receiver.close() + + async def _applystate(self): + with self.global_event_receiver as events: + async for f_event in events: + if f_event.origin != self.session_id.master_node_id: + continue + self.event_buffer.ingest(f_event.origin_idx, f_event.event) + for idx, event in self.event_buffer.drain_indexed(): + self._event_log.append(event) + self.state = apply(self.state, IndexedEvent(event=event, idx=idx)) + if ( + isinstance(event, ChunkGenerated) + and event.command_id in self._chat_completion_queues + ): + assert isinstance(event.chunk, TokenChunk) + await self._chat_completion_queues[event.command_id].send( + event.chunk + ) + + async def _pause_on_new_election(self): + with self.election_receiver as ems: + async for message in ems: + if message.clock > self.last_completed_election: + self.paused = True + + async def _send(self, command: Command): + while self.paused: + await self.paused_ev.wait() + await self.command_sender.send( + ForwarderCommand(origin=self.node_id, command=command) + ) diff --git a/src/exo/master/main.py b/src/exo/master/main.py new file mode 100644 index 00000000..55b72d7d --- /dev/null +++ b/src/exo/master/main.py @@ -0,0 +1,272 @@ +from datetime import datetime, timedelta, timezone + +import anyio +from anyio.abc import TaskGroup +from loguru import logger + +from exo.master.placement import ( + add_instance_to_placements, + delete_instance, + get_transition_events, + place_instance, +) +from exo.shared.apply import apply +from exo.shared.types.commands import ( + ChatCompletion, + CreateInstance, + DeleteInstance, + ForwarderCommand, + PlaceInstance, + RequestEventLog, + TaskFinished, + TestCommand, +) +from exo.shared.types.common import CommandId, NodeId, SessionId +from exo.shared.types.events import ( + Event, + ForwarderEvent, + IndexedEvent, + InstanceDeleted, + NodeTimedOut, + TaskCreated, + TaskDeleted, +) +from exo.shared.types.state import State +from exo.shared.types.tasks import ( + ChatCompletion as ChatCompletionTask, +) +from exo.shared.types.tasks import ( + TaskId, + TaskStatus, +) +from exo.shared.types.worker.instances import InstanceId +from exo.utils.channels import Receiver, Sender, channel +from exo.utils.event_buffer import MultiSourceBuffer + + +class Master: + def __init__( + self, + node_id: NodeId, + session_id: SessionId, + *, + command_receiver: Receiver[ForwarderCommand], + # Receiving indexed events from the forwarder to be applied to state + # Ideally these would be WorkerForwarderEvents but type system says no :( + local_event_receiver: Receiver[ForwarderEvent], + # Send events to the forwarder to be indexed (usually from command processing) + # Ideally these would be MasterForwarderEvents but type system says no :( + global_event_sender: Sender[ForwarderEvent], + tb_only: bool = False, + ): + self.state = State() + self._tg: TaskGroup = anyio.create_task_group() + self.node_id = node_id + self.session_id = session_id + self.command_task_mapping: dict[CommandId, TaskId] = {} + self.command_receiver = command_receiver + self.local_event_receiver = local_event_receiver + self.global_event_sender = global_event_sender + send, recv = channel[Event]() + self.event_sender: Sender[Event] = send + self._loopback_event_receiver: Receiver[Event] = recv + self._loopback_event_sender: Sender[ForwarderEvent] = ( + local_event_receiver.clone_sender() + ) + self._multi_buffer = MultiSourceBuffer[NodeId, Event]() + # TODO: not have this + self._event_log: list[Event] = [] + self.tb_only = tb_only + + async def run(self): + logger.info("Starting Master") + + async with self._tg as tg: + tg.start_soon(self._event_processor) + tg.start_soon(self._command_processor) + tg.start_soon(self._loopback_processor) + tg.start_soon(self._plan) + self.global_event_sender.close() + self.local_event_receiver.close() + self.command_receiver.close() + self._loopback_event_sender.close() + self._loopback_event_receiver.close() + + async def shutdown(self): + logger.info("Stopping Master") + self._tg.cancel_scope.cancel() + + async def _command_processor(self) -> None: + with self.command_receiver as commands: + async for forwarder_command in commands: + try: + logger.info(f"Executing command: {forwarder_command.command}") + generated_events: list[Event] = [] + command = forwarder_command.command + match command: + case TestCommand(): + pass + case ChatCompletion(): + instance_task_counts: dict[InstanceId, int] = {} + for instance in self.state.instances.values(): + if ( + instance.shard_assignments.model_id + == command.request_params.model + ): + task_count = sum( + 1 + for task in self.state.tasks.values() + if task.instance_id == instance.instance_id + ) + instance_task_counts[instance.instance_id] = ( + task_count + ) + + if not instance_task_counts: + raise ValueError( + f"No instance found for model {command.request_params.model}" + ) + + available_instance_ids = sorted( + instance_task_counts.keys(), + key=lambda instance_id: instance_task_counts[ + instance_id + ], + ) + + task_id = TaskId() + generated_events.append( + TaskCreated( + task_id=task_id, + task=ChatCompletionTask( + task_id=task_id, + command_id=command.command_id, + instance_id=available_instance_ids[0], + task_status=TaskStatus.Pending, + task_params=command.request_params, + ), + ) + ) + + self.command_task_mapping[command.command_id] = task_id + case DeleteInstance(): + placement = delete_instance(command, self.state.instances) + transition_events = get_transition_events( + self.state.instances, placement + ) + generated_events.extend(transition_events) + case PlaceInstance(): + placement = place_instance( + command, + self.state.topology, + self.state.instances, + ) + transition_events = get_transition_events( + self.state.instances, placement + ) + generated_events.extend(transition_events) + case CreateInstance(): + placement = add_instance_to_placements( + command, + self.state.topology, + self.state.instances, + ) + transition_events = get_transition_events( + self.state.instances, placement + ) + generated_events.extend(transition_events) + case TaskFinished(): + generated_events.append( + TaskDeleted( + task_id=self.command_task_mapping[ + command.finished_command_id + ] + ) + ) + if command.finished_command_id in self.command_task_mapping: + del self.command_task_mapping[ + command.finished_command_id + ] + case RequestEventLog(): + # We should just be able to send everything, since other buffers will ignore old messages + for i in range(command.since_idx, len(self._event_log)): + await self._send_event( + IndexedEvent(idx=i, event=self._event_log[i]) + ) + for event in generated_events: + await self.event_sender.send(event) + except ValueError as e: + logger.opt(exception=e).warning("Error in command processor") + + # These plan loops are the cracks showing in our event sourcing architecture - more things could be commands + async def _plan(self) -> None: + while True: + # kill broken instances + connected_node_ids = set( + [x.node_id for x in self.state.topology.list_nodes()] + ) + for instance_id, instance in self.state.instances.items(): + for node_id in instance.shard_assignments.node_to_runner: + if node_id not in connected_node_ids: + await self.event_sender.send( + InstanceDeleted(instance_id=instance_id) + ) + break + + # time out dead nodes + for node_id, time in self.state.last_seen.items(): + now = datetime.now(tz=timezone.utc) + if now - time > timedelta(seconds=30): + logger.info(f"Manually removing node {node_id} due to inactivity") + await self.event_sender.send(NodeTimedOut(node_id=node_id)) + + await anyio.sleep(10) + + async def _event_processor(self) -> None: + with self.local_event_receiver as local_events: + async for local_event in local_events: + # Discard all events not from our session + if local_event.session != self.session_id: + continue + self._multi_buffer.ingest( + local_event.origin_idx, + local_event.event, + local_event.origin, + ) + for event in self._multi_buffer.drain(): + logger.debug(f"Master indexing event: {str(event)[:100]}") + indexed = IndexedEvent(event=event, idx=len(self._event_log)) + self.state = apply(self.state, indexed) + + event._master_time_stamp = datetime.now(tz=timezone.utc) # pyright: ignore[reportPrivateUsage] + + self._event_log.append(event) + await self._send_event(indexed) + + async def _loopback_processor(self) -> None: + # this would ideally not be necessary. + # this is WAY less hacky than how I was working around this before + local_index = 0 + with self._loopback_event_receiver as events: + async for event in events: + await self._loopback_event_sender.send( + ForwarderEvent( + origin=NodeId(f"master_{self.node_id}"), + origin_idx=local_index, + session=self.session_id, + event=event, + ) + ) + local_index += 1 + + # This function is re-entrant, take care! + async def _send_event(self, event: IndexedEvent): + # Convenience method since this line is ugly + await self.global_event_sender.send( + ForwarderEvent( + origin=self.node_id, + origin_idx=event.idx, + session=self.session_id, + event=event.event, + ) + ) diff --git a/src/exo/master/placement.py b/src/exo/master/placement.py new file mode 100644 index 00000000..f3856f93 --- /dev/null +++ b/src/exo/master/placement.py @@ -0,0 +1,183 @@ +import random +from collections.abc import Mapping +from copy import deepcopy +from typing import Sequence + +from loguru import logger + +from exo.master.placement_utils import ( + filter_cycles_by_memory, + get_hosts_from_subgraph, + get_mlx_ibv_coordinators, + get_mlx_ibv_devices_matrix, + get_shard_assignments, + get_smallest_cycles, +) +from exo.shared.topology import Topology +from exo.shared.types.commands import ( + CreateInstance, + DeleteInstance, + PlaceInstance, +) +from exo.shared.types.common import Host +from exo.shared.types.events import Event, InstanceCreated, InstanceDeleted +from exo.shared.types.memory import Memory +from exo.shared.types.topology import NodeInfo +from exo.shared.types.worker.instances import ( + Instance, + InstanceId, + InstanceMeta, + MlxJacclInstance, + MlxRingInstance, +) + + +def random_ephemeral_port() -> int: + return random.randint(49152, 65535) + + +def add_instance_to_placements( + command: CreateInstance, + topology: Topology, + current_instances: Mapping[InstanceId, Instance], +) -> Mapping[InstanceId, Instance]: + # TODO: validate against topology + + return {**current_instances, command.instance.instance_id: command.instance} + + +def place_instance( + command: PlaceInstance, + topology: Topology, + current_instances: Mapping[InstanceId, Instance], +) -> dict[InstanceId, Instance]: + all_nodes = list(topology.list_nodes()) + + logger.info("finding cycles:") + cycles = topology.get_cycles() + singleton_cycles = [[node] for node in all_nodes] + candidate_cycles = list( + filter(lambda it: len(it) >= command.min_nodes, cycles + singleton_cycles) + ) + cycles_with_sufficient_memory = filter_cycles_by_memory( + candidate_cycles, command.model_meta.storage_size + ) + if not cycles_with_sufficient_memory: + raise ValueError("No cycles found with sufficient memory") + + smallest_cycles = get_smallest_cycles(cycles_with_sufficient_memory) + + smallest_tb_cycles = [ + cycle + for cycle in smallest_cycles + if topology.get_subgraph_from_nodes(cycle).is_thunderbolt_cycle(cycle) + ] + + if smallest_tb_cycles != []: + smallest_cycles = smallest_tb_cycles + + cycles_with_leaf_nodes: list[list[NodeInfo]] = [ + cycle + for cycle in smallest_cycles + if any(topology.node_is_leaf(node.node_id) for node in cycle) + ] + + selected_cycle = max( + cycles_with_leaf_nodes if cycles_with_leaf_nodes != [] else smallest_cycles, + key=lambda cycle: sum( + ( + node.node_profile.memory.ram_available + for node in cycle + if node.node_profile is not None + ), + start=Memory(), + ), + ) + + shard_assignments = get_shard_assignments( + command.model_meta, selected_cycle, command.sharding + ) + + cycle_digraph: Topology = topology.get_subgraph_from_nodes(selected_cycle) + + instance_id = InstanceId() + target_instances = dict(deepcopy(current_instances)) + + if len(selected_cycle) == 1: + logger.warning( + "You have likely selected ibv for a single node instance; falling back to MlxRing" + ) + + command.instance_meta = InstanceMeta.MlxRing + + # TODO: Single node instances + match command.instance_meta: + case InstanceMeta.MlxJaccl: + mlx_ibv_devices = get_mlx_ibv_devices_matrix( + selected_cycle, + cycle_digraph, + ) + mlx_ibv_coordinators = get_mlx_ibv_coordinators( + selected_cycle, + coordinator_port=random_ephemeral_port(), + cycle_digraph=cycle_digraph, + ) + target_instances[instance_id] = MlxJacclInstance( + instance_id=instance_id, + shard_assignments=shard_assignments, + ibv_devices=mlx_ibv_devices, + ibv_coordinators=mlx_ibv_coordinators, + ) + case InstanceMeta.MlxRing: + hosts: list[Host] = get_hosts_from_subgraph(cycle_digraph) + target_instances[instance_id] = MlxRingInstance( + instance_id=instance_id, + shard_assignments=shard_assignments, + hosts=[ + Host( + ip=host.ip, + port=random_ephemeral_port(), + ) + for host in hosts + ], + ) + + return target_instances + + +def delete_instance( + command: DeleteInstance, + current_instances: Mapping[InstanceId, Instance], +) -> dict[InstanceId, Instance]: + target_instances = dict(deepcopy(current_instances)) + if command.instance_id in target_instances: + del target_instances[command.instance_id] + return target_instances + raise ValueError(f"Instance {command.instance_id} not found") + + +def get_transition_events( + current_instances: Mapping[InstanceId, Instance], + target_instances: Mapping[InstanceId, Instance], +) -> Sequence[Event]: + events: list[Event] = [] + + # find instances to create + for instance_id, instance in target_instances.items(): + if instance_id not in current_instances: + events.append( + InstanceCreated( + instance=instance, + ) + ) + + # find instances to delete + for instance_id in current_instances: + if instance_id not in target_instances: + events.append( + InstanceDeleted( + instance_id=instance_id, + ) + ) + + return events diff --git a/src/exo/master/placement_utils.py b/src/exo/master/placement_utils.py new file mode 100644 index 00000000..24461b42 --- /dev/null +++ b/src/exo/master/placement_utils.py @@ -0,0 +1,299 @@ +from collections.abc import Generator +from typing import TypeGuard, cast + +from loguru import logger +from pydantic import BaseModel + +from exo.shared.topology import Topology +from exo.shared.types.common import Host, NodeId +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelMetadata +from exo.shared.types.profiling import NodePerformanceProfile +from exo.shared.types.topology import NodeInfo +from exo.shared.types.worker.runners import RunnerId, ShardAssignments +from exo.shared.types.worker.shards import ( + PipelineShardMetadata, + Sharding, + ShardMetadata, + TensorShardMetadata, +) + + +class NodeWithProfile(BaseModel): + node_id: NodeId + node_profile: NodePerformanceProfile + + +def narrow_all_nodes(nodes: list[NodeInfo]) -> TypeGuard[list[NodeWithProfile]]: + return all(node.node_profile is not None for node in nodes) + + +def filter_cycles_by_memory( + cycles: list[list[NodeInfo]], required_memory: Memory +) -> list[list[NodeInfo]]: + filtered_cycles: list[list[NodeInfo]] = [] + for cycle in cycles: + if not narrow_all_nodes(cycle): + continue + + total_mem = sum( + (node.node_profile.memory.ram_available for node in cycle), start=Memory() + ) + if total_mem >= required_memory: + filtered_cycles.append(cast(list[NodeInfo], cycle)) + return filtered_cycles + + +def get_smallest_cycles(cycles: list[list[NodeInfo]]) -> list[list[NodeInfo]]: + min_nodes = min(len(cycle) for cycle in cycles) + return [cycle for cycle in cycles if len(cycle) == min_nodes] + + +def get_shard_assignments_for_pipeline_parallel( + model_meta: ModelMetadata, + selected_cycle: list[NodeWithProfile], +): + cycle_memory = sum( + (node.node_profile.memory.ram_available for node in selected_cycle), + start=Memory(), + ) + total_layers = model_meta.n_layers + world_size = len(selected_cycle) + runner_to_shard: dict[RunnerId, ShardMetadata] = {} + node_to_runner: dict[NodeId, RunnerId] = {} + + layers_assigned = 0 + for i, node in enumerate(selected_cycle): + if i == len(selected_cycle) - 1: + node_layers = total_layers - layers_assigned + else: + node_layers = round( + total_layers + * ( + node.node_profile.memory.ram_available.in_bytes + / cycle_memory.in_bytes + ) + ) + node_layers = max(1, node_layers) + + runner_id = RunnerId() + + shard = PipelineShardMetadata( + model_meta=model_meta, + device_rank=i, + world_size=world_size, + start_layer=layers_assigned, + end_layer=layers_assigned + node_layers, + n_layers=total_layers, + ) + + runner_to_shard[runner_id] = shard + node_to_runner[node.node_id] = runner_id + layers_assigned += node_layers + + shard_assignments = ShardAssignments( + model_id=model_meta.model_id, + runner_to_shard=runner_to_shard, + node_to_runner=node_to_runner, + ) + + return shard_assignments + + +def get_shard_assignments_for_tensor_parallel( + model_meta: ModelMetadata, + selected_cycle: list[NodeWithProfile], +): + total_layers = model_meta.n_layers + world_size = len(selected_cycle) + runner_to_shard: dict[RunnerId, ShardMetadata] = {} + node_to_runner: dict[NodeId, RunnerId] = {} + + for i, node in enumerate(selected_cycle): + shard = TensorShardMetadata( + model_meta=model_meta, + device_rank=i, + world_size=world_size, + start_layer=0, + end_layer=total_layers, + n_layers=total_layers, + ) + + runner_id = RunnerId() + + runner_to_shard[runner_id] = shard + node_to_runner[node.node_id] = runner_id + + shard_assignments = ShardAssignments( + model_id=model_meta.model_id, + runner_to_shard=runner_to_shard, + node_to_runner=node_to_runner, + ) + + return shard_assignments + + +def get_shard_assignments( + model_meta: ModelMetadata, + selected_cycle: list[NodeInfo], + sharding: Sharding, +) -> ShardAssignments: + if not narrow_all_nodes(selected_cycle): + raise ValueError("All nodes must have profiles to create shard assignments") + match sharding: + case Sharding.Pipeline: + return get_shard_assignments_for_pipeline_parallel( + model_meta=model_meta, + selected_cycle=selected_cycle, + ) + case Sharding.Tensor: + return get_shard_assignments_for_tensor_parallel( + model_meta=model_meta, + selected_cycle=selected_cycle, + ) + + +def get_hosts_from_subgraph(cycle_digraph: Topology) -> list[Host]: + cycles = cycle_digraph.get_cycles() + expected_length = len(list(cycle_digraph.list_nodes())) + cycles = [cycle for cycle in cycles if len(cycle) == expected_length] + if not cycles: + if expected_length > 1: + logger.warning( + f"No cycles of length {expected_length} found even though chosen subgraph contained {expected_length} nodes" + ) + return [] + + get_thunderbolt = False + if cycle_digraph.is_thunderbolt_cycle(cycles[0]): + get_thunderbolt = True + + logger.info(f"Using thunderbolt cycle: {get_thunderbolt}") + + cycle = cycles[0] + hosts: list[Host] = [] + for i in range(len(cycle)): + current_node = cycle[i] + next_node = cycle[(i + 1) % len(cycle)] + + for connection in cycle_digraph.list_connections(): + if ( + connection.local_node_id == current_node.node_id + and connection.send_back_node_id == next_node.node_id + ): + if get_thunderbolt and not connection.is_thunderbolt(): + continue + assert connection.send_back_multiaddr is not None + host = Host( + ip=connection.send_back_multiaddr.ip_address, + port=connection.send_back_multiaddr.port, + ) + hosts.append(host) + break + + return hosts + + +def get_mlx_ibv_devices_matrix( + selected_cycle: list[NodeInfo], + cycle_digraph: Topology, +) -> list[list[str | None]]: + """Build connectivity matrix mapping device i to device j via RDMA interface names. + + The matrix element [i][j] contains the interface name on device i that connects + to device j, or None if no connection exists or no interface name is found. + Diagonal elements are always None. + """ + num_nodes = len(selected_cycle) + matrix: list[list[str | None]] = [ + [None for _ in range(num_nodes)] for _ in range(num_nodes) + ] + + for i, node_i in enumerate(selected_cycle): + for j, node_j in enumerate(selected_cycle): + if i == j: + continue + + # Find the IP J uses to talk to I + for connection_ip in _find_connection_ip(node_j, node_i, cycle_digraph): + # This is a local IP on I, which is attached to an interface: find that interface + if interface_name := _find_interface_name_for_ip(connection_ip, node_i): + matrix[i][j] = interface_name + logger.info( + f"Interface name for {connection_ip} on {node_i.node_id}: {interface_name}" + ) + break + else: + logger.warning( + f"Failed to find interface name between {node_i.node_id} and {node_j.node_id}" + ) + raise ValueError( + "Current ibv backend requires all-to-all rdma connections" + ) + + return matrix + + +def _find_connection_ip( + node_i: NodeInfo, + node_j: NodeInfo, + cycle_digraph: Topology, +) -> Generator[str]: + """Find all IP addresses that connect node i to node j.""" + for connection in cycle_digraph.list_connections(): + if ( + connection.local_node_id == node_i.node_id + and connection.send_back_node_id == node_j.node_id + ): + yield connection.send_back_multiaddr.ip_address + + +def _find_interface_name_for_ip( + ip_address: str, + node_info: NodeInfo, +) -> str | None: + if node_info.node_profile is None: + return None + + logger.info(f"Searching {node_info.node_id} for ip {ip_address}:") + for interface in node_info.node_profile.network_interfaces: + if interface.name not in ["en2", "en3", "en4", "en5", "en6", "en7"]: + continue + logger.info(f" | {interface.name}: {interface.ip_address}") + if interface.ip_address != ip_address: + continue + + logger.info("Found") + return f"rdma_{interface.name}" + + return None + + +def get_mlx_ibv_coordinators( + selected_cycle: list[NodeInfo], + coordinator_port: int, + cycle_digraph: Topology, +) -> dict[NodeId, str]: + """Get the coordinator addresses for MLX IBV (rank 0 device). + + Select an IP address that each node can reach for the rank 0 node. Returns + address in format "X.X.X.X:PORT" per node. + """ + rank_0_node = selected_cycle[0] + logger.info(f"Selecting coordinator from rank 0 node: {rank_0_node.node_id}") + + def get_ip_for_node(n: NodeInfo) -> str: + if n.node_id == rank_0_node.node_id: + return "0.0.0.0" + + for ip in _find_connection_ip(n, rank_0_node, cycle_digraph): + return ip + + logger.warning( + f"Failed to find directly connected ip between {n.node_id} and {rank_0_node.node_id}" + ) + raise ValueError("Current ibv backend requires all-to-all rdma connections") + + return { + n.node_id: f"{get_ip_for_node(n)}:{coordinator_port}" for n in selected_cycle + } diff --git a/src/exo/master/tests/conftest.py b/src/exo/master/tests/conftest.py new file mode 100644 index 00000000..8441cef8 --- /dev/null +++ b/src/exo/master/tests/conftest.py @@ -0,0 +1,67 @@ +from typing import Callable + +import pytest + +from exo.shared.types.common import NodeId +from exo.shared.types.multiaddr import Multiaddr +from exo.shared.types.profiling import ( + MemoryPerformanceProfile, + NodePerformanceProfile, + SystemPerformanceProfile, +) +from exo.shared.types.topology import Connection, ConnectionProfile, NodeInfo + + +@pytest.fixture +def create_node(): + def _create_node(memory: int, node_id: NodeId | None = None) -> NodeInfo: + if node_id is None: + node_id = NodeId() + return NodeInfo( + node_id=node_id, + node_profile=NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=MemoryPerformanceProfile.from_bytes( + ram_total=1000, + ram_available=memory, + swap_total=1000, + swap_available=1000, + ), + network_interfaces=[], + system=SystemPerformanceProfile(), + ), + ) + + return _create_node + + +# TODO: this is a hack to get the port for the send_back_multiaddr +@pytest.fixture +def create_connection() -> Callable[[NodeId, NodeId, int | None], Connection]: + port_counter = 1235 + ip_counter = 1 + + def _create_connection( + source_node_id: NodeId, sink_node_id: NodeId, send_back_port: int | None = None + ) -> Connection: + nonlocal port_counter + nonlocal ip_counter + # assign unique ips + ip_counter += 1 + if send_back_port is None: + send_back_port = port_counter + port_counter += 1 + return Connection( + local_node_id=source_node_id, + send_back_node_id=sink_node_id, + send_back_multiaddr=Multiaddr( + address=f"/ip4/169.254.0.{ip_counter}/tcp/{send_back_port}" + ), + connection_profile=ConnectionProfile( + throughput=1000, latency=1000, jitter=1000 + ), + ) + + return _create_connection diff --git a/src/exo/master/tests/test_master.py b/src/exo/master/tests/test_master.py new file mode 100644 index 00000000..c2111baf --- /dev/null +++ b/src/exo/master/tests/test_master.py @@ -0,0 +1,203 @@ +from datetime import datetime, timezone +from typing import Sequence + +import anyio +import pytest +from loguru import logger + +from exo.master.main import Master +from exo.routing.router import get_node_id_keypair +from exo.shared.types.api import ChatCompletionMessage, ChatCompletionTaskParams +from exo.shared.types.commands import ( + ChatCompletion, + CommandId, + ForwarderCommand, + PlaceInstance, +) +from exo.shared.types.common import NodeId, SessionId +from exo.shared.types.events import ( + ForwarderEvent, + IndexedEvent, + InstanceCreated, + NodePerformanceMeasured, + TaskCreated, +) +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.profiling import ( + MemoryPerformanceProfile, + NodePerformanceProfile, + SystemPerformanceProfile, +) +from exo.shared.types.tasks import ChatCompletion as ChatCompletionTask +from exo.shared.types.tasks import TaskStatus +from exo.shared.types.worker.instances import ( + InstanceMeta, + MlxRingInstance, + ShardAssignments, +) +from exo.shared.types.worker.shards import PipelineShardMetadata, Sharding +from exo.utils.channels import channel + + +@pytest.mark.asyncio +async def test_master(): + keypair = get_node_id_keypair() + node_id = NodeId(keypair.to_peer_id().to_base58()) + session_id = SessionId(master_node_id=node_id, election_clock=0) + + ge_sender, global_event_receiver = channel[ForwarderEvent]() + command_sender, co_receiver = channel[ForwarderCommand]() + local_event_sender, le_receiver = channel[ForwarderEvent]() + + all_events: list[IndexedEvent] = [] + + def _get_events() -> Sequence[IndexedEvent]: + orig_events = global_event_receiver.collect() + for e in orig_events: + all_events.append( + IndexedEvent( + event=e.event, + idx=len(all_events), # origin=e.origin, + ) + ) + return all_events + + master = Master( + node_id, + session_id, + global_event_sender=ge_sender, + local_event_receiver=le_receiver, + command_receiver=co_receiver, + tb_only=False, + ) + logger.info("run the master") + async with anyio.create_task_group() as tg: + tg.start_soon(master.run) + + sender_node_id = NodeId(f"{keypair.to_peer_id().to_base58()}_sender") + # inject a NodePerformanceProfile event + logger.info("inject a NodePerformanceProfile event") + await local_event_sender.send( + ForwarderEvent( + origin_idx=0, + origin=sender_node_id, + session=session_id, + event=( + NodePerformanceMeasured( + when=str(datetime.now(tz=timezone.utc)), + node_id=node_id, + node_profile=NodePerformanceProfile( + model_id="maccy", + chip_id="arm", + friendly_name="test", + memory=MemoryPerformanceProfile( + ram_total=Memory.from_bytes(678948 * 1024), + ram_available=Memory.from_bytes(678948 * 1024), + swap_total=Memory.from_bytes(0), + swap_available=Memory.from_bytes(0), + ), + network_interfaces=[], + system=SystemPerformanceProfile(), + ), + ) + ), + ) + ) + + # wait for initial topology event + logger.info("wait for initial topology event") + while len(list(master.state.topology.list_nodes())) == 0: + await anyio.sleep(0.001) + while len(master.state.node_profiles) == 0: + await anyio.sleep(0.001) + + logger.info("inject a CreateInstance Command") + await command_sender.send( + ForwarderCommand( + origin=node_id, + command=( + PlaceInstance( + command_id=CommandId(), + model_meta=ModelMetadata( + model_id=ModelId("llama-3.2-1b"), + pretty_name="Llama 3.2 1B", + n_layers=16, + storage_size=Memory.from_bytes(678948), + ), + sharding=Sharding.Pipeline, + instance_meta=InstanceMeta.MlxRing, + min_nodes=1, + ) + ), + ) + ) + logger.info("wait for an instance") + while len(master.state.instances.keys()) == 0: + await anyio.sleep(0.001) + logger.info("inject a ChatCompletion Command") + await command_sender.send( + ForwarderCommand( + origin=node_id, + command=( + ChatCompletion( + command_id=CommandId(), + request_params=ChatCompletionTaskParams( + model="llama-3.2-1b", + messages=[ + ChatCompletionMessage( + role="user", content="Hello, how are you?" + ) + ], + ), + ) + ), + ) + ) + while len(_get_events()) < 3: + await anyio.sleep(0.01) + + events = _get_events() + assert len(events) == 3 + assert events[0].idx == 0 + assert events[1].idx == 1 + assert events[2].idx == 2 + assert isinstance(events[0].event, NodePerformanceMeasured) + assert isinstance(events[1].event, InstanceCreated) + runner_id = list( + events[1].event.instance.shard_assignments.runner_to_shard.keys() + )[0] + assert events[1].event.instance == MlxRingInstance( + instance_id=events[1].event.instance.instance_id, + shard_assignments=ShardAssignments( + model_id=ModelId("llama-3.2-1b"), + runner_to_shard={ + (runner_id): PipelineShardMetadata( + start_layer=0, + end_layer=16, + n_layers=16, + model_meta=ModelMetadata( + model_id=ModelId("llama-3.2-1b"), + pretty_name="Llama 3.2 1B", + n_layers=16, + storage_size=Memory.from_bytes(678948), + ), + device_rank=0, + world_size=1, + ) + }, + node_to_runner={node_id: runner_id}, + ), + hosts=[], + ) + assert isinstance(events[2].event, TaskCreated) + assert events[2].event.task.task_status == TaskStatus.Pending + assert isinstance(events[2].event.task, ChatCompletionTask) + assert events[2].event.task.task_params == ChatCompletionTaskParams( + model="llama-3.2-1b", + messages=[ + ChatCompletionMessage(role="user", content="Hello, how are you?") + ], + ) + + await master.shutdown() diff --git a/src/exo/master/tests/test_placement.py b/src/exo/master/tests/test_placement.py new file mode 100644 index 00000000..c688e8ff --- /dev/null +++ b/src/exo/master/tests/test_placement.py @@ -0,0 +1,474 @@ +from typing import Callable + +import pytest +from loguru import logger + +from exo.master.placement import ( + get_transition_events, + place_instance, +) +from exo.shared.topology import Topology +from exo.shared.types.commands import PlaceInstance +from exo.shared.types.common import CommandId, NodeId +from exo.shared.types.events import InstanceCreated, InstanceDeleted +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.profiling import NetworkInterfaceInfo, NodePerformanceProfile +from exo.shared.types.topology import Connection, NodeInfo +from exo.shared.types.worker.instances import ( + Instance, + InstanceId, + InstanceMeta, + MlxJacclInstance, + MlxRingInstance, +) +from exo.shared.types.worker.runners import ShardAssignments +from exo.shared.types.worker.shards import Sharding + + +@pytest.fixture +def topology() -> Topology: + return Topology() + + +@pytest.fixture +def instance() -> Instance: + return MlxRingInstance( + instance_id=InstanceId(), + shard_assignments=ShardAssignments( + model_id=ModelId("test-model"), runner_to_shard={}, node_to_runner={} + ), + hosts=[], + ) + + +@pytest.fixture +def model_meta() -> ModelMetadata: + return ModelMetadata( + model_id=ModelId("test-model"), + storage_size=Memory.from_kb(1000), + pretty_name="Test Model", + n_layers=10, + ) + + +def place_instance_command(model_meta: ModelMetadata) -> PlaceInstance: + return PlaceInstance( + command_id=CommandId(), + model_meta=model_meta, + sharding=Sharding.Pipeline, + instance_meta=InstanceMeta.MlxRing, + min_nodes=1, + ) + + +@pytest.mark.parametrize( + "available_memory,total_layers,expected_layers", + [ + ((500, 500, 1000), 12, (3, 3, 6)), + ((500, 500, 500), 12, (4, 4, 4)), + ((312, 518, 1024), 12, (2, 3, 7)), + ], +) +def test_get_instance_placements_create_instance( + available_memory: tuple[int, int, int], + total_layers: int, + expected_layers: tuple[int, int, int], + topology: Topology, + model_meta: ModelMetadata, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + # arrange + model_meta.n_layers = total_layers + model_meta.storage_size.in_bytes = sum( + available_memory + ) # make it exactly fit across all nodes + + cic = place_instance_command(model_meta) + node_id_a = NodeId() + node_id_b = NodeId() + node_id_c = NodeId() + topology.add_node(create_node(available_memory[0], node_id_a)) + topology.add_node(create_node(available_memory[1], node_id_b)) + topology.add_node(create_node(available_memory[2], node_id_c)) + topology.add_connection(create_connection(node_id_a, node_id_b)) + topology.add_connection(create_connection(node_id_b, node_id_c)) + topology.add_connection(create_connection(node_id_c, node_id_a)) + + # act + placements = place_instance(cic, topology, {}) + + # assert + assert len(placements) == 1 + instance_id = list(placements.keys())[0] + instance = placements[instance_id] + assert instance.shard_assignments.model_id == model_meta.model_id + + runner_id_a = instance.shard_assignments.node_to_runner[node_id_a] + runner_id_b = instance.shard_assignments.node_to_runner[node_id_b] + runner_id_c = instance.shard_assignments.node_to_runner[node_id_c] + + shard_a = instance.shard_assignments.runner_to_shard[runner_id_a] + shard_b = instance.shard_assignments.runner_to_shard[runner_id_b] + shard_c = instance.shard_assignments.runner_to_shard[runner_id_c] + + assert shard_a.end_layer - shard_a.start_layer == expected_layers[0] + assert shard_b.end_layer - shard_b.start_layer == expected_layers[1] + assert shard_c.end_layer - shard_c.start_layer == expected_layers[2] + + shards = [shard_a, shard_b, shard_c] + shards_sorted = sorted(shards, key=lambda s: s.start_layer) + assert shards_sorted[0].start_layer == 0 + assert shards_sorted[-1].end_layer == total_layers + + +def test_get_instance_placements_one_node_exact_fit( + create_node: Callable[[int, NodeId | None], NodeInfo], +) -> None: + topology = Topology() + node_id = NodeId() + topology.add_node(create_node(1000 * 1024, node_id)) + cic = place_instance_command( + ModelMetadata( + model_id=ModelId("test-model"), + storage_size=Memory.from_kb(1000), + pretty_name="Test Model", + n_layers=10, + ), + ) + placements = place_instance(cic, topology, {}) + + assert len(placements) == 1 + instance_id = list(placements.keys())[0] + instance = placements[instance_id] + assert instance.shard_assignments.model_id == "test-model" + assert len(instance.shard_assignments.node_to_runner) == 1 + assert len(instance.shard_assignments.runner_to_shard) == 1 + assert len(instance.shard_assignments.runner_to_shard) == 1 + + +def test_get_instance_placements_one_node_fits_with_extra_memory( + create_node: Callable[[int, NodeId | None], NodeInfo], +) -> None: + topology = Topology() + node_id = NodeId() + topology.add_node(create_node(1001 * 1024, node_id)) + cic = place_instance_command( + ModelMetadata( + model_id=ModelId("test-model"), + storage_size=Memory.from_kb(1000), + pretty_name="Test Model", + n_layers=10, + ), + ) + placements = place_instance(cic, topology, {}) + + assert len(placements) == 1 + instance_id = list(placements.keys())[0] + instance = placements[instance_id] + assert instance.shard_assignments.model_id == "test-model" + assert len(instance.shard_assignments.node_to_runner) == 1 + assert len(instance.shard_assignments.runner_to_shard) == 1 + assert len(instance.shard_assignments.runner_to_shard) == 1 + + +def test_get_instance_placements_one_node_not_fit( + create_node: Callable[[int, NodeId | None], NodeInfo], +) -> None: + topology = Topology() + node_id = NodeId() + topology.add_node(create_node(1000 * 1024, node_id)) + cic = place_instance_command( + model_meta=ModelMetadata( + model_id=ModelId("test-model"), + storage_size=Memory.from_kb(1001), + pretty_name="Test Model", + n_layers=10, + ), + ) + + with pytest.raises(ValueError, match="No cycles found with sufficient memory"): + place_instance(cic, topology, {}) + + +def test_get_transition_events_no_change(instance: Instance): + # arrange + instance_id = InstanceId() + current_instances = {instance_id: instance} + target_instances = {instance_id: instance} + + # act + events = get_transition_events(current_instances, target_instances) + + # assert + assert len(events) == 0 + + +def test_get_transition_events_create_instance(instance: Instance): + # arrange + instance_id = InstanceId() + current_instances: dict[InstanceId, Instance] = {} + target_instances: dict[InstanceId, Instance] = {instance_id: instance} + + # act + events = get_transition_events(current_instances, target_instances) + + # assert + assert len(events) == 1 + assert isinstance(events[0], InstanceCreated) + + +def test_get_transition_events_delete_instance(instance: Instance): + # arrange + instance_id = InstanceId() + current_instances: dict[InstanceId, Instance] = {instance_id: instance} + target_instances: dict[InstanceId, Instance] = {} + + # act + events = get_transition_events(current_instances, target_instances) + + # assert + assert len(events) == 1 + assert isinstance(events[0], InstanceDeleted) + assert events[0].instance_id == instance_id + + +def test_placement_prioritizes_leaf_cycle_with_less_memory( + topology: Topology, + model_meta: ModelMetadata, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + # Arrange two 3-node cycles. The A-B-C cycle has a leaf node (only one outgoing + # neighbor per node). The D-E-F cycle has extra outgoing edges making its nodes + # non-leaves. Ensure both cycles have sufficient total memory, with the A-B-C + # cycle having LESS total memory than D-E-F. The algorithm should still choose + # the cycle that contains a leaf node. + + # Model requires more than any single node but fits within a 3-node cycle + model_meta.storage_size.in_bytes = 1500 + model_meta.n_layers = 12 + + # Create node ids + node_id_a = NodeId() + node_id_b = NodeId() + node_id_c = NodeId() + node_id_d = NodeId() + node_id_e = NodeId() + node_id_f = NodeId() + + # Extra sink nodes to make D/E/F non-leaf via additional outgoing edges + node_id_x = NodeId() + node_id_y = NodeId() + node_id_z = NodeId() + + # A-B-C cycle total memory = 1600 (< D-E-F total) + topology.add_node(create_node(400, node_id_a)) + topology.add_node(create_node(400, node_id_b)) + topology.add_node(create_node(800, node_id_c)) + + # D-E-F cycle total memory = 1800 (> A-B-C total) + topology.add_node(create_node(600, node_id_d)) + topology.add_node(create_node(600, node_id_e)) + topology.add_node(create_node(600, node_id_f)) + + # Extra nodes with tiny memory so they can't form singleton placements + topology.add_node(create_node(10, node_id_x)) + topology.add_node(create_node(10, node_id_y)) + topology.add_node(create_node(10, node_id_z)) + + # Build directed cycles + topology.add_connection(create_connection(node_id_a, node_id_b)) + topology.add_connection(create_connection(node_id_b, node_id_c)) + topology.add_connection(create_connection(node_id_c, node_id_a)) + + topology.add_connection(create_connection(node_id_d, node_id_e)) + topology.add_connection(create_connection(node_id_e, node_id_f)) + topology.add_connection(create_connection(node_id_f, node_id_d)) + + # Add extra outgoing edges from D/E/F so none of them are leaves + topology.add_connection(create_connection(node_id_d, node_id_x)) + topology.add_connection(create_connection(node_id_e, node_id_y)) + topology.add_connection(create_connection(node_id_f, node_id_z)) + + cic = place_instance_command( + model_meta=model_meta, + ) + + # Act + placements = place_instance(cic, topology, {}) + + # Assert the chosen cycle is A-B-C (contains at least one leaf node), even though + # D-E-F has more total memory. + assert len(placements) == 1 + instance_id = list(placements.keys())[0] + instance = placements[instance_id] + + assigned_nodes = set(instance.shard_assignments.node_to_runner.keys()) + expected_leaf_cycle_nodes = {node_id_a, node_id_b, node_id_c} + non_leaf_cycle_nodes = {node_id_d, node_id_e, node_id_f} + + assert expected_leaf_cycle_nodes.issubset(assigned_nodes) + assert assigned_nodes.isdisjoint(non_leaf_cycle_nodes) + + +def test_tensor_rdma_backend_connectivity_matrix( + topology: Topology, + model_meta: ModelMetadata, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + model_meta.n_layers = 12 + model_meta.storage_size.in_bytes = 1500 + + node_id_a = NodeId() + node_id_b = NodeId() + node_id_c = NodeId() + + node_a = create_node(500, node_id_a) + node_b = create_node(500, node_id_b) + node_c = create_node(500, node_id_c) + + ethernet_interface = NetworkInterfaceInfo( + name="en0", + ip_address="192.168.1.100", + ) + + assert node_a.node_profile is not None + assert node_b.node_profile is not None + assert node_c.node_profile is not None + + conn_a_b = create_connection(node_id_a, node_id_b) + conn_b_c = create_connection(node_id_b, node_id_c) + conn_c_a = create_connection(node_id_c, node_id_a) + + conn_b_a = create_connection(node_id_b, node_id_a) + conn_c_b = create_connection(node_id_c, node_id_b) + conn_a_c = create_connection(node_id_a, node_id_c) + + assert conn_a_b.send_back_multiaddr is not None + assert conn_b_c.send_back_multiaddr is not None + assert conn_c_a.send_back_multiaddr is not None + + assert conn_b_a.send_back_multiaddr is not None + assert conn_c_b.send_back_multiaddr is not None + assert conn_a_c.send_back_multiaddr is not None + + node_a.node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=node_a.node_profile.memory, + network_interfaces=[ + NetworkInterfaceInfo( + name="en3", + ip_address=conn_c_a.send_back_multiaddr.ip_address, + ), + NetworkInterfaceInfo( + name="en4", + ip_address=conn_b_a.send_back_multiaddr.ip_address, + ), + ethernet_interface, + ], + system=node_a.node_profile.system, + ) + node_b.node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=node_b.node_profile.memory, + network_interfaces=[ + NetworkInterfaceInfo( + name="en3", + ip_address=conn_c_b.send_back_multiaddr.ip_address, + ), + NetworkInterfaceInfo( + name="en4", + ip_address=conn_a_b.send_back_multiaddr.ip_address, + ), + ethernet_interface, + ], + system=node_b.node_profile.system, + ) + node_c.node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=node_c.node_profile.memory, + network_interfaces=[ + NetworkInterfaceInfo( + name="en3", + ip_address=conn_a_c.send_back_multiaddr.ip_address, + ), + NetworkInterfaceInfo( + name="en4", + ip_address=conn_b_c.send_back_multiaddr.ip_address, + ), + ethernet_interface, + ], + system=node_c.node_profile.system, + ) + + topology.add_node(node_a) + topology.add_node(node_b) + topology.add_node(node_c) + topology.add_connection(conn_a_b) + topology.add_connection(conn_b_c) + topology.add_connection(conn_c_a) + topology.add_connection(conn_b_a) + topology.add_connection(conn_c_b) + topology.add_connection(conn_a_c) + + cic = PlaceInstance( + sharding=Sharding.Tensor, + instance_meta=InstanceMeta.MlxJaccl, + command_id=CommandId(), + model_meta=model_meta, + min_nodes=1, + ) + + placements = place_instance(cic, topology, {}) + + assert len(placements) == 1 + instance_id = list(placements.keys())[0] + instance = placements[instance_id] + + assert isinstance(instance, MlxJacclInstance) + + assert instance.ibv_devices is not None + assert instance.ibv_coordinators is not None + + matrix = instance.ibv_devices + assert len(matrix) == 3 + + for i in range(3): + assert matrix[i][i] is None + + assigned_nodes = list(instance.shard_assignments.node_to_runner.keys()) + node_to_idx = {node_id: idx for idx, node_id in enumerate(assigned_nodes)} + + idx_a = node_to_idx[node_id_a] + idx_b = node_to_idx[node_id_b] + idx_c = node_to_idx[node_id_c] + + logger.info(matrix) + + assert matrix[idx_a][idx_b] == "rdma_en4" + assert matrix[idx_b][idx_c] == "rdma_en3" + assert matrix[idx_c][idx_a] == "rdma_en3" + + # Verify coordinators are set for all nodes + assert len(instance.ibv_coordinators) == 3 + for node_id in assigned_nodes: + assert node_id in instance.ibv_coordinators + coordinator = instance.ibv_coordinators[node_id] + assert ":" in coordinator + # Rank 0 node should use 0.0.0.0, others should use connection-specific IPs + if node_id == assigned_nodes[0]: + assert coordinator.startswith("0.0.0.0:") + else: + # Non-rank-0 nodes should have valid IP addresses (can be link-local) + ip_part = coordinator.split(":")[0] + # Just verify it's a valid IP format + assert len(ip_part.split(".")) == 4 diff --git a/src/exo/master/tests/test_placement_utils.py b/src/exo/master/tests/test_placement_utils.py new file mode 100644 index 00000000..ff6de72c --- /dev/null +++ b/src/exo/master/tests/test_placement_utils.py @@ -0,0 +1,397 @@ +from typing import Callable + +import pytest + +from exo.master.placement_utils import ( + filter_cycles_by_memory, + get_hosts_from_subgraph, + get_mlx_ibv_coordinators, + get_shard_assignments, + get_smallest_cycles, +) +from exo.shared.topology import Topology +from exo.shared.types.common import Host, NodeId +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.profiling import NetworkInterfaceInfo, NodePerformanceProfile +from exo.shared.types.topology import Connection, NodeInfo +from exo.shared.types.worker.shards import Sharding + + +@pytest.fixture +def topology() -> Topology: + topology = Topology() + return topology + + +def test_filter_cycles_by_memory( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + # arrange + node1_id = NodeId() + node2_id = NodeId() + + node1 = create_node(1000 * 1024, node1_id) + node2 = create_node(1000 * 1024, node2_id) + + topology.add_node(node1) + topology.add_node(node2) + + connection1 = create_connection(node1_id, node2_id) + connection2 = create_connection(node2_id, node1_id) + + topology.add_connection(connection1) + topology.add_connection(connection2) + + cycles = topology.get_cycles() + assert len(cycles) == 1 + assert len(cycles[0]) == 2 + + # act + filtered_cycles = filter_cycles_by_memory(cycles, Memory.from_bytes(1)) + + # assert + assert len(filtered_cycles) == 1 + assert len(filtered_cycles[0]) == 2 + assert set(n.node_id for n in filtered_cycles[0]) == {node1_id, node2_id} + + +def test_filter_cycles_by_insufficient_memory( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + # arrange + node1_id = NodeId() + node2_id = NodeId() + + node1 = create_node(1000 * 1024, node1_id) + node2 = create_node(1000 * 1024, node2_id) + + topology.add_node(node1) + topology.add_node(node2) + + connection1 = create_connection(node1_id, node2_id) + connection2 = create_connection(node2_id, node1_id) + + topology.add_connection(connection1) + topology.add_connection(connection2) + + # act + filtered_cycles = filter_cycles_by_memory( + topology.get_cycles(), Memory.from_kb(2001) + ) + + # assert + assert len(filtered_cycles) == 0 + + +def test_filter_multiple_cycles_by_memory( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + # arrange + node_a_id = NodeId() + node_b_id = NodeId() + node_c_id = NodeId() + + node_a = create_node(500 * 1024, node_a_id) + node_b = create_node(500 * 1024, node_b_id) + node_c = create_node(1000 * 1024, node_c_id) + + topology.add_node(node_a) + topology.add_node(node_b) + topology.add_node(node_c) + + topology.add_connection(create_connection(node_a_id, node_b_id)) + topology.add_connection(create_connection(node_b_id, node_a_id)) + + topology.add_connection(create_connection(node_a_id, node_c_id)) + topology.add_connection(create_connection(node_c_id, node_b_id)) + + cycles = topology.get_cycles() + + # act + filtered_cycles = filter_cycles_by_memory(cycles, Memory.from_kb(1500)) + + # assert + assert len(filtered_cycles) == 1 + assert len(filtered_cycles[0]) == 3 + assert set(n.node_id for n in filtered_cycles[0]) == { + node_a_id, + node_b_id, + node_c_id, + } + + +def test_get_smallest_cycles( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], +): + # arrange + node_a_id = NodeId() + node_b_id = NodeId() + node_c_id = NodeId() + + node_a = create_node(500 * 1024, node_a_id) + node_b = create_node(500 * 1024, node_b_id) + node_c = create_node(1000 * 1024, node_c_id) + + topology.add_node(node_a) + topology.add_node(node_b) + topology.add_node(node_c) + + topology.add_connection(create_connection(node_a_id, node_b_id)) + topology.add_connection(create_connection(node_b_id, node_c_id)) + topology.add_connection(create_connection(node_c_id, node_a_id)) + topology.add_connection(create_connection(node_b_id, node_a_id)) + + # act + smallest_cycles = get_smallest_cycles(topology.get_cycles()) + + # assert + assert len(smallest_cycles) == 1 + assert len(smallest_cycles[0]) == 2 + assert set(n.node_id for n in smallest_cycles[0]) == {node_a_id, node_b_id} + + +@pytest.mark.parametrize( + "available_memory,total_layers,expected_layers", + [ + ((500, 500, 1000), 12, (3, 3, 6)), + ((500, 500, 500), 12, (4, 4, 4)), + ((312, 518, 1024), 12, (2, 3, 7)), + ], +) +def test_get_shard_assignments( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId], Connection], + available_memory: tuple[int, int, int], + total_layers: int, + expected_layers: tuple[int, int, int], +): + # arrange + node_a_id = NodeId() + node_b_id = NodeId() + node_c_id = NodeId() + + node_a = create_node(available_memory[0] * 1024, node_a_id) + node_b = create_node(available_memory[1] * 1024, node_b_id) + node_c = create_node(available_memory[2] * 1024, node_c_id) + + topology.add_node(node_a) + topology.add_node(node_b) + topology.add_node(node_c) + + topology.add_connection(create_connection(node_a_id, node_b_id)) + topology.add_connection(create_connection(node_b_id, node_c_id)) + topology.add_connection(create_connection(node_c_id, node_a_id)) + topology.add_connection(create_connection(node_b_id, node_a_id)) + + model_meta = ModelMetadata( + model_id=ModelId("test-model"), + pretty_name="Test Model", + n_layers=total_layers, + storage_size=Memory.from_kb(1000), + ) + cycles = topology.get_cycles() + selected_cycle = cycles[0] + + # act + shard_assignments = get_shard_assignments( + model_meta, selected_cycle, Sharding.Pipeline + ) + + # assert + runner_id_a = shard_assignments.node_to_runner[node_a_id] + runner_id_b = shard_assignments.node_to_runner[node_b_id] + runner_id_c = shard_assignments.node_to_runner[node_c_id] + assert ( + shard_assignments.runner_to_shard[runner_id_c].end_layer + - shard_assignments.runner_to_shard[runner_id_c].start_layer + == expected_layers[2] + ) + assert ( + shard_assignments.runner_to_shard[runner_id_a].end_layer + - shard_assignments.runner_to_shard[runner_id_a].start_layer + == expected_layers[0] + ) + assert ( + shard_assignments.runner_to_shard[runner_id_b].end_layer + - shard_assignments.runner_to_shard[runner_id_b].start_layer + == expected_layers[1] + ) + + +def test_get_hosts_from_subgraph( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId, int | None], Connection], +): + # arrange + node_a_id = NodeId() + node_b_id = NodeId() + node_c_id = NodeId() + + node_a = create_node(500, node_a_id) + node_b = create_node(500, node_b_id) + node_c = create_node(1000, node_c_id) + + topology.add_node(node_a) + topology.add_node(node_b) + topology.add_node(node_c) + + topology.add_connection(create_connection(node_a_id, node_b_id, 5001)) + topology.add_connection(create_connection(node_b_id, node_c_id, 5002)) + topology.add_connection(create_connection(node_c_id, node_a_id, 5003)) + topology.add_connection(create_connection(node_b_id, node_a_id, 5004)) + + # act + hosts = get_hosts_from_subgraph(topology) + + # assert + assert len(hosts) == 3 + expected_hosts = [ + Host(ip=("169.254.0.2"), port=5001), + Host(ip=("169.254.0.3"), port=5002), + Host(ip=("169.254.0.4"), port=5003), + ] + for expected_host in expected_hosts: + assert expected_host in hosts + + +def test_get_mlx_ibv_coordinators( + topology: Topology, + create_node: Callable[[int, NodeId | None], NodeInfo], + create_connection: Callable[[NodeId, NodeId, int | None], Connection], +): + # arrange + node_a_id = NodeId() + node_b_id = NodeId() + node_c_id = NodeId() + + node_a = create_node(500 * 1024, node_a_id) + node_b = create_node(500 * 1024, node_b_id) + node_c = create_node(1000 * 1024, node_c_id) + + conn_a_b = create_connection(node_a_id, node_b_id, 5001) + conn_b_a = create_connection(node_b_id, node_a_id, 5002) + conn_b_c = create_connection(node_b_id, node_c_id, 5003) + conn_c_b = create_connection(node_c_id, node_b_id, 5004) + conn_c_a = create_connection(node_c_id, node_a_id, 5005) + conn_a_c = create_connection(node_a_id, node_c_id, 5006) + + # Update node profiles with network interfaces before adding to topology + assert node_a.node_profile is not None + assert node_b.node_profile is not None + assert node_c.node_profile is not None + + node_a.node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=node_a.node_profile.memory, + network_interfaces=[ + NetworkInterfaceInfo( + name="en3", + ip_address=conn_a_b.send_back_multiaddr.ip_address, + ), + NetworkInterfaceInfo( + name="en4", + ip_address=conn_a_c.send_back_multiaddr.ip_address, + ), + ], + system=node_a.node_profile.system, + ) + node_b.node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=node_b.node_profile.memory, + network_interfaces=[ + NetworkInterfaceInfo( + name="en3", + ip_address=conn_b_a.send_back_multiaddr.ip_address, + ), + NetworkInterfaceInfo( + name="en4", + ip_address=conn_b_c.send_back_multiaddr.ip_address, + ), + ], + system=node_b.node_profile.system, + ) + node_c.node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=node_c.node_profile.memory, + network_interfaces=[ + NetworkInterfaceInfo( + name="en3", + ip_address=conn_c_b.send_back_multiaddr.ip_address, + ), + NetworkInterfaceInfo( + name="en4", + ip_address=conn_c_a.send_back_multiaddr.ip_address, + ), + ], + system=node_c.node_profile.system, + ) + + topology.add_node(node_a) + topology.add_node(node_b) + topology.add_node(node_c) + + topology.add_connection(conn_a_b) + topology.add_connection(conn_b_a) + topology.add_connection(conn_b_c) + topology.add_connection(conn_c_b) + topology.add_connection(conn_c_a) + topology.add_connection(conn_a_c) + + cycle = [node_a, node_b, node_c] + + # act + coordinators = get_mlx_ibv_coordinators( + cycle, coordinator_port=5000, cycle_digraph=topology + ) + + # assert + assert len(coordinators) == 3 + assert node_a_id in coordinators + assert node_b_id in coordinators + assert node_c_id in coordinators + + # All coordinators should have IP:PORT format + for node_id, coordinator in coordinators.items(): + assert ":" in coordinator, ( + f"Coordinator for {node_id} should have ':' separator" + ) + + # Verify port is correct + for node_id, coordinator in coordinators.items(): + assert coordinator.endswith(":5000"), ( + f"Coordinator for {node_id} should use port 5000" + ) + + # Rank 0 (node_a) treats this as the listen socket so should listen on all + # IPs + assert coordinators[node_a_id].startswith("0.0.0.0:"), ( + "Rank 0 node should use localhost as coordinator" + ) + + # Non-rank-0 nodes should use the specific IP from their connection to rank 0 + # node_b uses the IP from conn_b_a (node_b -> node_a) + assert coordinators[node_b_id] == ( + f"{conn_b_a.send_back_multiaddr.ip_address}:5000" + ), "node_b should use the IP from conn_b_a" + + # node_c uses the IP from conn_c_a (node_c -> node_a) + assert coordinators[node_c_id] == ( + f"{conn_c_a.send_back_multiaddr.ip_address}:5000" + ), "node_c should use the IP from conn_c_a" diff --git a/src/exo/master/tests/test_topology.py b/src/exo/master/tests/test_topology.py new file mode 100644 index 00000000..d6afb339 --- /dev/null +++ b/src/exo/master/tests/test_topology.py @@ -0,0 +1,204 @@ +import pytest + +from exo.shared.topology import Topology +from exo.shared.types.multiaddr import Multiaddr +from exo.shared.types.profiling import ( + MemoryPerformanceProfile, + NodePerformanceProfile, + SystemPerformanceProfile, +) +from exo.shared.types.topology import Connection, ConnectionProfile, NodeId, NodeInfo + + +@pytest.fixture +def topology() -> Topology: + return Topology() + + +@pytest.fixture +def connection() -> Connection: + return Connection( + local_node_id=NodeId(), + send_back_node_id=NodeId(), + send_back_multiaddr=Multiaddr(address="/ip4/127.0.0.1/tcp/1235"), + connection_profile=ConnectionProfile( + throughput=1000, latency=1000, jitter=1000 + ), + ) + + +@pytest.fixture +def node_profile() -> NodePerformanceProfile: + memory_profile = MemoryPerformanceProfile.from_bytes( + ram_total=1000, ram_available=1000, swap_total=1000, swap_available=1000 + ) + system_profile = SystemPerformanceProfile() + return NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=memory_profile, + network_interfaces=[], + system=system_profile, + ) + + +@pytest.fixture +def connection_profile() -> ConnectionProfile: + return ConnectionProfile(throughput=1000, latency=1000, jitter=1000) + + +def test_add_node(topology: Topology, node_profile: NodePerformanceProfile): + # arrange + node_id = NodeId() + + # act + topology.add_node(NodeInfo(node_id=node_id, node_profile=node_profile)) + + # assert + data = topology.get_node_profile(node_id) + assert data == node_profile + + +def test_add_connection( + topology: Topology, node_profile: NodePerformanceProfile, connection: Connection +): + # arrange + topology.add_node( + NodeInfo(node_id=connection.local_node_id, node_profile=node_profile) + ) + topology.add_node( + NodeInfo(node_id=connection.send_back_node_id, node_profile=node_profile) + ) + topology.add_connection(connection) + + # act + data = topology.get_connection_profile(connection) + + # assert + assert data == connection.connection_profile + + +def test_update_node_profile( + topology: Topology, node_profile: NodePerformanceProfile, connection: Connection +): + # arrange + topology.add_node( + NodeInfo(node_id=connection.local_node_id, node_profile=node_profile) + ) + topology.add_node( + NodeInfo(node_id=connection.send_back_node_id, node_profile=node_profile) + ) + topology.add_connection(connection) + + new_node_profile = NodePerformanceProfile( + model_id="test", + chip_id="test", + friendly_name="test", + memory=MemoryPerformanceProfile.from_bytes( + ram_total=1000, ram_available=1000, swap_total=1000, swap_available=1000 + ), + network_interfaces=[], + system=SystemPerformanceProfile(), + ) + + # act + topology.update_node_profile( + connection.local_node_id, node_profile=new_node_profile + ) + + # assert + data = topology.get_node_profile(connection.local_node_id) + assert data == new_node_profile + + +def test_update_connection_profile( + topology: Topology, node_profile: NodePerformanceProfile, connection: Connection +): + # arrange + topology.add_node( + NodeInfo(node_id=connection.local_node_id, node_profile=node_profile) + ) + topology.add_node( + NodeInfo(node_id=connection.send_back_node_id, node_profile=node_profile) + ) + topology.add_connection(connection) + + new_connection_profile = ConnectionProfile( + throughput=2000, latency=2000, jitter=2000 + ) + connection = Connection( + local_node_id=connection.local_node_id, + send_back_node_id=connection.send_back_node_id, + send_back_multiaddr=connection.send_back_multiaddr, + connection_profile=new_connection_profile, + ) + + # act + topology.update_connection_profile(connection) + + # assert + data = topology.get_connection_profile(connection) + assert data == new_connection_profile + + +def test_remove_connection_still_connected( + topology: Topology, node_profile: NodePerformanceProfile, connection: Connection +): + # arrange + topology.add_node( + NodeInfo(node_id=connection.local_node_id, node_profile=node_profile) + ) + topology.add_node( + NodeInfo(node_id=connection.send_back_node_id, node_profile=node_profile) + ) + topology.add_connection(connection) + + # act + topology.remove_connection(connection) + + # assert + assert topology.get_connection_profile(connection) is None + + +def test_remove_node_still_connected( + topology: Topology, node_profile: NodePerformanceProfile, connection: Connection +): + # arrange + topology.add_node( + NodeInfo(node_id=connection.local_node_id, node_profile=node_profile) + ) + topology.add_node( + NodeInfo(node_id=connection.send_back_node_id, node_profile=node_profile) + ) + topology.add_connection(connection) + + # act + topology.remove_node(connection.local_node_id) + + # assert + assert topology.get_node_profile(connection.local_node_id) is None + + +def test_list_nodes( + topology: Topology, node_profile: NodePerformanceProfile, connection: Connection +): + # arrange + topology.add_node( + NodeInfo(node_id=connection.local_node_id, node_profile=node_profile) + ) + topology.add_node( + NodeInfo(node_id=connection.send_back_node_id, node_profile=node_profile) + ) + topology.add_connection(connection) + + # act + nodes = list(topology.list_nodes()) + + # assert + assert len(nodes) == 2 + assert all(isinstance(node, NodeInfo) for node in nodes) + assert {node.node_id for node in nodes} == { + connection.local_node_id, + connection.send_back_node_id, + } diff --git a/src/exo/routing/__init__.py b/src/exo/routing/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/routing/connection_message.py b/src/exo/routing/connection_message.py new file mode 100644 index 00000000..665483ac --- /dev/null +++ b/src/exo/routing/connection_message.py @@ -0,0 +1,37 @@ +from enum import Enum + +from exo_pyo3_bindings import ConnectionUpdate, ConnectionUpdateType + +from exo.shared.types.common import NodeId +from exo.utils.pydantic_ext import CamelCaseModel + +"""Serialisable types for Connection Updates/Messages""" + + +class ConnectionMessageType(Enum): + Connected = 0 + Disconnected = 1 + + @staticmethod + def from_update_type(update_type: ConnectionUpdateType): + match update_type: + case ConnectionUpdateType.Connected: + return ConnectionMessageType.Connected + case ConnectionUpdateType.Disconnected: + return ConnectionMessageType.Disconnected + + +class ConnectionMessage(CamelCaseModel): + node_id: NodeId + connection_type: ConnectionMessageType + remote_ipv4: str + remote_tcp_port: int + + @classmethod + def from_update(cls, update: ConnectionUpdate) -> "ConnectionMessage": + return cls( + node_id=NodeId(update.peer_id.to_base58()), + connection_type=ConnectionMessageType.from_update_type(update.update_type), + remote_ipv4=update.remote_ipv4, + remote_tcp_port=update.remote_tcp_port, + ) diff --git a/src/exo/routing/router.py b/src/exo/routing/router.py new file mode 100644 index 00000000..ac6073af --- /dev/null +++ b/src/exo/routing/router.py @@ -0,0 +1,240 @@ +from copy import copy +from itertools import count +from math import inf +from os import PathLike +from pathlib import Path +from typing import cast + +from anyio import ( + BrokenResourceError, + ClosedResourceError, + create_task_group, + sleep_forever, +) +from anyio.abc import TaskGroup +from exo_pyo3_bindings import ( + AllQueuesFullError, + Keypair, + NetworkingHandle, + NoPeersSubscribedToTopicError, +) +from filelock import FileLock +from loguru import logger + +from exo.shared.constants import EXO_NODE_ID_KEYPAIR +from exo.utils.channels import Receiver, Sender, channel +from exo.utils.pydantic_ext import CamelCaseModel + +from .connection_message import ConnectionMessage +from .topics import CONNECTION_MESSAGES, PublishPolicy, TypedTopic + + +# A significant current limitation of the TopicRouter is that it is not capable +# of preventing feedback, as it does not ask for a system id so cannot tell +# which message is coming/going to which system. +# This is currently only relevant for Election +class TopicRouter[T: CamelCaseModel]: + def __init__( + self, + topic: TypedTopic[T], + networking_sender: Sender[tuple[str, bytes]], + max_buffer_size: float = inf, + ): + self.topic: TypedTopic[T] = topic + self.senders: set[Sender[T]] = set() + send, recv = channel[T]() + self.receiver: Receiver[T] = recv + self._sender: Sender[T] = send + self.networking_sender: Sender[tuple[str, bytes]] = networking_sender + + async def run(self): + logger.debug(f"Topic Router {self.topic} ready to send") + with self.receiver as items: + async for item in items: + # Check if we should send to network + if ( + len(self.senders) == 0 + and self.topic.publish_policy is PublishPolicy.Minimal + ): + await self._send_out(item) + continue + if self.topic.publish_policy is PublishPolicy.Always: + await self._send_out(item) + # Then publish to all senders + await self.publish(item) + + async def shutdown(self): + logger.debug(f"Shutting down Topic Router {self.topic}") + # Close all the things! + for sender in self.senders: + sender.close() + self._sender.close() + self.receiver.close() + + async def publish(self, item: T): + """ + Publish item T on this topic to all senders. + NB: this sends to ALL receivers, potentially including receivers held by the object doing the sending. + You should handle your own output if you hold a sender + receiver pair. + """ + to_clear: set[Sender[T]] = set() + for sender in copy(self.senders): + try: + await sender.send(item) + except (ClosedResourceError, BrokenResourceError): + to_clear.add(sender) + self.senders -= to_clear + + async def publish_bytes(self, data: bytes): + await self.publish(self.topic.deserialize(data)) + + def new_sender(self) -> Sender[T]: + return self._sender.clone() + + async def _send_out(self, item: T): + logger.trace(f"TopicRouter {self.topic.topic} sending {item}") + await self.networking_sender.send( + (str(self.topic.topic), self.topic.serialize(item)) + ) + + +class Router: + @classmethod + def create(cls, identity: Keypair) -> "Router": + return cls(handle=NetworkingHandle(identity)) + + def __init__(self, handle: NetworkingHandle): + self.topic_routers: dict[str, TopicRouter[CamelCaseModel]] = {} + send, recv = channel[tuple[str, bytes]]() + self.networking_receiver: Receiver[tuple[str, bytes]] = recv + self._net: NetworkingHandle = handle + self._tmp_networking_sender: Sender[tuple[str, bytes]] | None = send + self._id_count = count() + self._tg: TaskGroup | None = None + + async def register_topic[T: CamelCaseModel](self, topic: TypedTopic[T]): + assert self._tg is None, "Attempted to register topic after setup time" + send = self._tmp_networking_sender + if send: + self._tmp_networking_sender = None + else: + send = self.networking_receiver.clone_sender() + router = TopicRouter[T](topic, send) + self.topic_routers[topic.topic] = cast(TopicRouter[CamelCaseModel], router) + await self._networking_subscribe(str(topic.topic)) + + def sender[T: CamelCaseModel](self, topic: TypedTopic[T]) -> Sender[T]: + router = self.topic_routers.get(topic.topic, None) + # There's gotta be a way to do this without THIS many asserts + assert router is not None + assert router.topic == topic + sender = cast(TopicRouter[T], router).new_sender() + return sender + + def receiver[T: CamelCaseModel](self, topic: TypedTopic[T]) -> Receiver[T]: + router = self.topic_routers.get(topic.topic, None) + # There's gotta be a way to do this without THIS many asserts + + assert router is not None + assert router.topic == topic + assert router.topic.model_type == topic.model_type + + send, recv = channel[T]() + router.senders.add(cast(Sender[CamelCaseModel], send)) + + return recv + + async def run(self): + logger.debug("Starting Router") + async with create_task_group() as tg: + self._tg = tg + for topic in self.topic_routers: + router = self.topic_routers[topic] + tg.start_soon(router.run) + tg.start_soon(self._networking_recv) + tg.start_soon(self._networking_recv_connection_messages) + tg.start_soon(self._networking_publish) + # Router only shuts down if you cancel it. + await sleep_forever() + for topic in self.topic_routers: + await self._networking_unsubscribe(str(topic)) + + async def shutdown(self): + logger.debug("Shutting down Router") + if not self._tg: + return + self._tg.cancel_scope.cancel() + + async def _networking_subscribe(self, topic: str): + logger.info(f"Subscribing to {topic}") + await self._net.gossipsub_subscribe(topic) + + async def _networking_unsubscribe(self, topic: str): + logger.info(f"Unsubscribing from {topic}") + await self._net.gossipsub_unsubscribe(topic) + + async def _networking_recv(self): + while True: + topic, data = await self._net.gossipsub_recv() + logger.trace(f"Received message on {topic} with payload {data}") + if topic not in self.topic_routers: + logger.warning(f"Received message on unknown or inactive topic {topic}") + continue + + router = self.topic_routers[topic] + await router.publish_bytes(data) + + async def _networking_recv_connection_messages(self): + while True: + update = await self._net.connection_update_recv() + message = ConnectionMessage.from_update(update) + logger.trace( + f"Received message on connection_messages with payload {message}" + ) + if CONNECTION_MESSAGES.topic in self.topic_routers: + router = self.topic_routers[CONNECTION_MESSAGES.topic] + assert router.topic.model_type == ConnectionMessage + router = cast(TopicRouter[ConnectionMessage], router) + await router.publish(message) + + async def _networking_publish(self): + with self.networking_receiver as networked_items: + async for topic, data in networked_items: + try: + logger.trace(f"Sending message on {topic} with payload {data}") + await self._net.gossipsub_publish(topic, data) + # As a hack, this also catches AllQueuesFull + # Need to fix that ASAP. + except (NoPeersSubscribedToTopicError, AllQueuesFullError): + pass + + +def get_node_id_keypair( + path: str | bytes | PathLike[str] | PathLike[bytes] = EXO_NODE_ID_KEYPAIR, +) -> Keypair: + """ + Obtains the :class:`Keypair` associated with this node-ID. + Obtain the :class:`PeerId` by from it. + """ + + def lock_path(path: str | bytes | PathLike[str] | PathLike[bytes]) -> Path: + return Path(str(path) + ".lock") + + # operate with cross-process lock to avoid race conditions + with FileLock(lock_path(path)): + with open(path, "a+b") as f: # opens in append-mode => starts at EOF + # if non-zero EOF, then file exists => use to get node-ID + if f.tell() != 0: + f.seek(0) # go to start & read protobuf-encoded bytes + protobuf_encoded = f.read() + + try: # if decoded successfully, save & return + return Keypair.from_protobuf_encoding(protobuf_encoded) + except ValueError as e: # on runtime error, assume corrupt file + logger.warning(f"Encountered error when trying to get keypair: {e}") + + # if no valid credentials, create new ones and persist + with open(path, "w+b") as f: + keypair = Keypair.generate_ed25519() + f.write(keypair.to_protobuf_encoding()) + return keypair diff --git a/src/exo/routing/tests/test_event_buffer.py b/src/exo/routing/tests/test_event_buffer.py new file mode 100644 index 00000000..215f53e2 --- /dev/null +++ b/src/exo/routing/tests/test_event_buffer.py @@ -0,0 +1,143 @@ +import pytest + +from exo.shared.types.events import Event, TestEvent +from exo.utils.event_buffer import OrderedBuffer + + +def make_indexed_event(idx: int) -> tuple[int, Event]: + """Factory function to create a unique ForwarderEvent for a given index.""" + return (idx, TestEvent()) + + +@pytest.fixture +def buffer() -> OrderedBuffer[Event]: + """Provides a clean instance of OrderedBuffer[Event] for each test.""" + return OrderedBuffer[Event]() + + +@pytest.mark.asyncio +async def test_initial_state(buffer: OrderedBuffer[Event]): + """Tests that a new buffer is empty and starts at index 1.""" + assert buffer.next_idx_to_release == 0 + assert not buffer.store + assert buffer.drain() == [] + + +@pytest.mark.asyncio +async def test_ingest_and_drain_sequential_events(buffer: OrderedBuffer[Event]): + """Tests ingesting and draining a simple, ordered sequence of events.""" + events = [make_indexed_event(0), make_indexed_event(1), make_indexed_event(2)] + [buffer.ingest(*ev) for ev in events] + + drained_events = buffer.drain_indexed() + assert drained_events == events + assert buffer.next_idx_to_release == 3 + assert not buffer.store + + +@pytest.mark.asyncio +async def test_ingest_out_of_order_events(buffer: OrderedBuffer[Event]): + """Tests that out-of-order events are buffered and drained in the correct sequence.""" + event1 = make_indexed_event(0) + event2 = make_indexed_event(1) + event3 = make_indexed_event(2) + + buffer.ingest(*event3) + buffer.ingest(*event1) + buffer.ingest(*event2) + + drained_events = buffer.drain_indexed() + assert drained_events == [event1, event2, event3] + assert buffer.next_idx_to_release == 3 + + +@pytest.mark.asyncio +async def test_drain_with_gap_in_sequence(buffer: OrderedBuffer[Event]): + """Tests that draining stops when there is a gap in the event indices.""" + event1 = make_indexed_event(0) + event3 = make_indexed_event(2) + + buffer.ingest(*event1) + buffer.ingest(*event3) + + drained_events = buffer.drain_indexed() + assert drained_events == [event1] + assert buffer.next_idx_to_release == 1 + + assert buffer.drain() == [] + assert 2 in buffer.store + + +@pytest.mark.asyncio +async def test_fill_gap_and_drain_remaining(buffer: OrderedBuffer[Event]): + """Tests that once a gap is filled, the rest of the sequence is drained.""" + event0 = make_indexed_event(0) + event2 = make_indexed_event(2) + buffer.ingest(*event0) + buffer.ingest(*event2) + + buffer.drain() + assert buffer.next_idx_to_release == 1 + + event1 = make_indexed_event(1) + buffer.ingest(*event1) + + drained_events = buffer.drain_indexed() + assert [e[0] for e in drained_events] == [1, 2] + assert buffer.next_idx_to_release == 3 + + +@pytest.mark.asyncio +async def test_ingest_drops_duplicate_indices(buffer: OrderedBuffer[Event]): + """Tests that if multiple events for the same index are ingested, the first one wins.""" + event2_first = make_indexed_event(1) + event2_second = (1, TestEvent()) + + buffer.ingest(*make_indexed_event(0)) + buffer.ingest(*event2_first) + + with pytest.raises(AssertionError): + buffer.ingest(*event2_second) # This duplicate should be ignored + + drained = buffer.drain_indexed() + assert len(drained) == 2 + + assert drained[1][1].event_id == event2_first[1].event_id + assert drained[1][1].event_id != event2_second[1].event_id + + +@pytest.mark.asyncio +async def test_ingest_drops_stale_events(buffer: OrderedBuffer[Event]): + """Tests that events with an index lower than next_idx_to_release are dropped.""" + buffer.ingest(*make_indexed_event(0)) + buffer.ingest(*make_indexed_event(1)) + buffer.drain() + + assert buffer.next_idx_to_release == 2 + + stale_event1 = make_indexed_event(0) + stale_event2 = make_indexed_event(1) + buffer.ingest(*stale_event1) + buffer.ingest(*stale_event2) + + assert not buffer.store + assert buffer.drain() == [] + + +@pytest.mark.asyncio +async def test_drain_and_ingest_with_new_sequence(buffer: OrderedBuffer[Event]): + """Tests reusing the buffer after it has been fully drained.""" + buffer.ingest(*make_indexed_event(0)) + buffer.ingest(*make_indexed_event(1)) + buffer.drain() + + assert buffer.next_idx_to_release == 2 + assert not buffer.store + + buffer.ingest(*make_indexed_event(4)) + buffer.ingest(*make_indexed_event(2)) + + drained = buffer.drain_indexed() + assert [e[0] for e in drained] == [2] + assert buffer.next_idx_to_release == 3 + assert 4 in buffer.store diff --git a/src/exo/routing/topics.py b/src/exo/routing/topics.py new file mode 100644 index 00000000..50f1c9af --- /dev/null +++ b/src/exo/routing/topics.py @@ -0,0 +1,47 @@ +from dataclasses import dataclass +from enum import Enum + +from exo.routing.connection_message import ConnectionMessage +from exo.shared.election import ElectionMessage +from exo.shared.types.commands import ForwarderCommand +from exo.shared.types.events import ( + ForwarderEvent, +) +from exo.utils.pydantic_ext import CamelCaseModel + + +class PublishPolicy(str, Enum): + Never = "Never" + """Never publish to the network - this is a local message""" + Minimal = "Minimal" + """Only publish when there is no local receiver for this type of message""" + Always = "Always" + """Always publish to the network""" + + +@dataclass # (frozen=True) +class TypedTopic[T: CamelCaseModel]: + topic: str + publish_policy: PublishPolicy + + model_type: type[ + T + ] # This can be worked around with evil type hacking, see https://stackoverflow.com/a/71720366 - I don't think it's necessary here. + + @staticmethod + def serialize(t: T) -> bytes: + return t.model_dump_json().encode("utf-8") + + def deserialize(self, b: bytes) -> T: + return self.model_type.model_validate_json(b.decode("utf-8")) + + +GLOBAL_EVENTS = TypedTopic("global_events", PublishPolicy.Always, ForwarderEvent) +LOCAL_EVENTS = TypedTopic("local_events", PublishPolicy.Always, ForwarderEvent) +COMMANDS = TypedTopic("commands", PublishPolicy.Always, ForwarderCommand) +ELECTION_MESSAGES = TypedTopic( + "election_messages", PublishPolicy.Always, ElectionMessage +) +CONNECTION_MESSAGES = TypedTopic( + "connection_messages", PublishPolicy.Never, ConnectionMessage +) diff --git a/src/exo/shared/__init__.py b/src/exo/shared/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/shared/apply.py b/src/exo/shared/apply.py new file mode 100644 index 00000000..9bb597cb --- /dev/null +++ b/src/exo/shared/apply.py @@ -0,0 +1,307 @@ +import copy +from collections.abc import Mapping, Sequence +from datetime import datetime + +from loguru import logger + +from exo.shared.types.common import NodeId +from exo.shared.types.events import ( + ChunkGenerated, + Event, + IndexedEvent, + InstanceCreated, + InstanceDeleted, + NodeCreated, + NodeDownloadProgress, + NodeMemoryMeasured, + NodePerformanceMeasured, + NodeTimedOut, + RunnerDeleted, + RunnerStatusUpdated, + TaskAcknowledged, + TaskCreated, + TaskDeleted, + TaskFailed, + TaskStatusUpdated, + TestEvent, + TopologyEdgeCreated, + TopologyEdgeDeleted, +) +from exo.shared.types.profiling import NodePerformanceProfile, SystemPerformanceProfile +from exo.shared.types.state import State +from exo.shared.types.tasks import Task, TaskId, TaskStatus +from exo.shared.types.topology import NodeInfo +from exo.shared.types.worker.downloads import DownloadProgress +from exo.shared.types.worker.instances import Instance, InstanceId +from exo.shared.types.worker.runners import RunnerId, RunnerStatus + + +def event_apply(event: Event, state: State) -> State: + """Apply an event to state.""" + match event: + case ( + TestEvent() | ChunkGenerated() | TaskAcknowledged() + ): # TaskAcknowledged should never be sent by a worker but i dont mind if it just gets ignored + return state + case InstanceCreated(): + return apply_instance_created(event, state) + case InstanceDeleted(): + return apply_instance_deleted(event, state) + case NodeCreated(): + return apply_topology_node_created(event, state) + case NodeTimedOut(): + return apply_node_timed_out(event, state) + case NodePerformanceMeasured(): + return apply_node_performance_measured(event, state) + case NodeDownloadProgress(): + return apply_node_download_progress(event, state) + case NodeMemoryMeasured(): + return apply_node_memory_measured(event, state) + case RunnerDeleted(): + return apply_runner_deleted(event, state) + case RunnerStatusUpdated(): + return apply_runner_status_updated(event, state) + case TaskCreated(): + return apply_task_created(event, state) + case TaskDeleted(): + return apply_task_deleted(event, state) + case TaskFailed(): + return apply_task_failed(event, state) + case TaskStatusUpdated(): + return apply_task_status_updated(event, state) + case TopologyEdgeCreated(): + return apply_topology_edge_created(event, state) + case TopologyEdgeDeleted(): + return apply_topology_edge_deleted(event, state) + + +def apply(state: State, event: IndexedEvent) -> State: + # Just to test that events are only applied in correct order + if state.last_event_applied_idx != event.idx - 1: + logger.warning( + f"Expected event {state.last_event_applied_idx + 1} but received {event.idx}" + ) + assert state.last_event_applied_idx == event.idx - 1 + new_state: State = event_apply(event.event, state) + return new_state.model_copy(update={"last_event_applied_idx": event.idx}) + + +def apply_node_download_progress(event: NodeDownloadProgress, state: State) -> State: + """ + Update or add a node download progress to state. + """ + dp = event.download_progress + node_id = dp.node_id + + current = list(state.downloads.get(node_id, ())) + + replaced = False + for i, existing_dp in enumerate(current): + if existing_dp.shard_metadata == dp.shard_metadata: + current[i] = dp + replaced = True + break + + if not replaced: + current.append(dp) + + new_downloads: Mapping[NodeId, Sequence[DownloadProgress]] = { + **state.downloads, + node_id: current, + } + return state.model_copy(update={"downloads": new_downloads}) + + +def apply_task_created(event: TaskCreated, state: State) -> State: + new_tasks: Mapping[TaskId, Task] = {**state.tasks, event.task_id: event.task} + return state.model_copy(update={"tasks": new_tasks}) + + +def apply_task_deleted(event: TaskDeleted, state: State) -> State: + new_tasks: Mapping[TaskId, Task] = { + tid: task for tid, task in state.tasks.items() if tid != event.task_id + } + return state.model_copy(update={"tasks": new_tasks}) + + +def apply_task_status_updated(event: TaskStatusUpdated, state: State) -> State: + if event.task_id not in state.tasks: + # maybe should raise + return state + + update: dict[str, TaskStatus | None] = { + "task_status": event.task_status, + } + if event.task_status != TaskStatus.Failed: + update["error_type"] = None + update["error_message"] = None + + updated_task = state.tasks[event.task_id].model_copy(update=update) + new_tasks: Mapping[TaskId, Task] = {**state.tasks, event.task_id: updated_task} + return state.model_copy(update={"tasks": new_tasks}) + + +def apply_task_failed(event: TaskFailed, state: State) -> State: + if event.task_id not in state.tasks: + # maybe should raise + return state + + updated_task = state.tasks[event.task_id].model_copy( + update={"error_type": event.error_type, "error_message": event.error_message} + ) + new_tasks: Mapping[TaskId, Task] = {**state.tasks, event.task_id: updated_task} + return state.model_copy(update={"tasks": new_tasks}) + + +def apply_instance_created(event: InstanceCreated, state: State) -> State: + instance = event.instance + new_instances: Mapping[InstanceId, Instance] = { + **state.instances, + instance.instance_id: instance, + } + return state.model_copy(update={"instances": new_instances}) + + +def apply_instance_deleted(event: InstanceDeleted, state: State) -> State: + new_instances: Mapping[InstanceId, Instance] = { + iid: inst for iid, inst in state.instances.items() if iid != event.instance_id + } + return state.model_copy(update={"instances": new_instances}) + + +def apply_runner_status_updated(event: RunnerStatusUpdated, state: State) -> State: + new_runners: Mapping[RunnerId, RunnerStatus] = { + **state.runners, + event.runner_id: event.runner_status, + } + return state.model_copy(update={"runners": new_runners}) + + +def apply_runner_deleted(event: RunnerDeleted, state: State) -> State: + assert event.runner_id in state.runners, ( + "RunnerDeleted before any RunnerStatusUpdated events" + ) + new_runners: Mapping[RunnerId, RunnerStatus] = { + rid: rs for rid, rs in state.runners.items() if rid != event.runner_id + } + return state.model_copy(update={"runners": new_runners}) + + +def apply_node_timed_out(event: NodeTimedOut, state: State) -> State: + topology = copy.copy(state.topology) + state.topology.remove_node(event.node_id) + node_profiles = { + key: value for key, value in state.node_profiles.items() if key != event.node_id + } + last_seen = { + key: value for key, value in state.last_seen.items() if key != event.node_id + } + return state.model_copy( + update={ + "topology": topology, + "node_profiles": node_profiles, + "last_seen": last_seen, + } + ) + + +def apply_node_performance_measured( + event: NodePerformanceMeasured, state: State +) -> State: + new_profiles: Mapping[NodeId, NodePerformanceProfile] = { + **state.node_profiles, + event.node_id: event.node_profile, + } + last_seen: Mapping[NodeId, datetime] = { + **state.last_seen, + event.node_id: datetime.fromisoformat(event.when), + } + state = state.model_copy(update={"node_profiles": new_profiles}) + topology = copy.copy(state.topology) + # TODO: NodeCreated + if not topology.contains_node(event.node_id): + topology.add_node(NodeInfo(node_id=event.node_id)) + topology.update_node_profile(event.node_id, event.node_profile) + return state.model_copy( + update={ + "node_profiles": new_profiles, + "topology": topology, + "last_seen": last_seen, + } + ) + + +def apply_node_memory_measured(event: NodeMemoryMeasured, state: State) -> State: + existing = state.node_profiles.get(event.node_id) + topology = copy.copy(state.topology) + + if existing is None: + created = NodePerformanceProfile( + model_id="unknown", + chip_id="unknown", + friendly_name="Unknown", + memory=event.memory, + network_interfaces=[], + system=SystemPerformanceProfile( + # TODO: flops_fp16=0.0, + gpu_usage=0.0, + temp=0.0, + sys_power=0.0, + pcpu_usage=0.0, + ecpu_usage=0.0, + ane_power=0.0, + ), + ) + created_profiles: Mapping[NodeId, NodePerformanceProfile] = { + **state.node_profiles, + event.node_id: created, + } + last_seen: Mapping[NodeId, datetime] = { + **state.last_seen, + event.node_id: datetime.fromisoformat(event.when), + } + if not topology.contains_node(event.node_id): + topology.add_node(NodeInfo(node_id=event.node_id)) + # TODO: NodeCreated + topology.update_node_profile(event.node_id, created) + return state.model_copy( + update={ + "node_profiles": created_profiles, + "topology": topology, + "last_seen": last_seen, + } + ) + + updated = existing.model_copy(update={"memory": event.memory}) + updated_profiles: Mapping[NodeId, NodePerformanceProfile] = { + **state.node_profiles, + event.node_id: updated, + } + # TODO: NodeCreated + if not topology.contains_node(event.node_id): + topology.add_node(NodeInfo(node_id=event.node_id)) + topology.update_node_profile(event.node_id, updated) + return state.model_copy( + update={"node_profiles": updated_profiles, "topology": topology} + ) + + +def apply_topology_node_created(event: NodeCreated, state: State) -> State: + topology = copy.copy(state.topology) + topology.add_node(NodeInfo(node_id=event.node_id)) + return state.model_copy(update={"topology": topology}) + + +def apply_topology_edge_created(event: TopologyEdgeCreated, state: State) -> State: + topology = copy.copy(state.topology) + topology.add_connection(event.edge) + return state.model_copy(update={"topology": topology}) + + +def apply_topology_edge_deleted(event: TopologyEdgeDeleted, state: State) -> State: + topology = copy.copy(state.topology) + if not topology.contains_connection(event.edge): + return state + topology.remove_connection(event.edge) + # TODO: Clean up removing the reverse connection + return state.model_copy(update={"topology": topology}) diff --git a/src/exo/shared/constants.py b/src/exo/shared/constants.py new file mode 100644 index 00000000..63ff8526 --- /dev/null +++ b/src/exo/shared/constants.py @@ -0,0 +1,35 @@ +import os +from pathlib import Path + +EXO_HOME_RELATIVE_PATH = os.environ.get("EXO_HOME", ".exo") +EXO_HOME = Path.home() / EXO_HOME_RELATIVE_PATH + +EXO_MODELS_DIR_ENV = os.environ.get("EXO_MODELS_DIR") +EXO_MODELS_DIR = Path(EXO_MODELS_DIR_ENV) if EXO_MODELS_DIR_ENV else EXO_HOME / "models" + +EXO_GLOBAL_EVENT_DB = EXO_HOME / "global_events.db" +EXO_WORKER_EVENT_DB = EXO_HOME / "worker_events.db" +EXO_MASTER_STATE = EXO_HOME / "master_state.json" +EXO_WORKER_STATE = EXO_HOME / "worker_state.json" +EXO_MASTER_LOG = EXO_HOME / "master.log" +EXO_WORKER_LOG = EXO_HOME / "worker.log" +EXO_LOG = EXO_HOME / "exo.log" +EXO_TEST_LOG = EXO_HOME / "exo_test.log" + +EXO_NODE_ID_KEYPAIR = EXO_HOME / "node_id.keypair" + +EXO_WORKER_KEYRING_FILE = EXO_HOME / "worker_keyring" +EXO_MASTER_KEYRING_FILE = EXO_HOME / "master_keyring" + +EXO_IPC_DIR = EXO_HOME / "ipc" + +# libp2p topics for event forwarding +LIBP2P_LOCAL_EVENTS_TOPIC = "worker_events" +LIBP2P_GLOBAL_EVENTS_TOPIC = "global_events" +LIBP2P_ELECTION_MESSAGES_TOPIC = "election_message" +LIBP2P_COMMANDS_TOPIC = "commands" + +# lower bounds define timeouts for flops and memory bandwidth - these are the values for the M1 chip. +LB_TFLOPS = 2.3 +LB_MEMBW_GBPS = 68 +LB_DISK_GBPS = 1.5 diff --git a/src/exo/shared/election.py b/src/exo/shared/election.py new file mode 100644 index 00000000..9d030d5e --- /dev/null +++ b/src/exo/shared/election.py @@ -0,0 +1,273 @@ +from typing import Self + +import anyio +from anyio import ( + CancelScope, + Event, + create_task_group, + get_cancelled_exc_class, +) +from anyio.abc import TaskGroup +from loguru import logger + +from exo.routing.connection_message import ConnectionMessage +from exo.shared.types.commands import ForwarderCommand +from exo.shared.types.common import NodeId, SessionId +from exo.utils.channels import Receiver, Sender +from exo.utils.pydantic_ext import CamelCaseModel + +DEFAULT_ELECTION_TIMEOUT = 3.0 + + +class ElectionMessage(CamelCaseModel): + clock: int + seniority: int + proposed_session: SessionId + commands_seen: int + + # Could eventually include a list of neighbour nodes for centrality + def __lt__(self, other: Self) -> bool: + if self.clock != other.clock: + return self.clock < other.clock + if self.seniority != other.seniority: + return self.seniority < other.seniority + elif self.commands_seen != other.commands_seen: + return self.commands_seen < other.commands_seen + else: + return ( + self.proposed_session.master_node_id + < other.proposed_session.master_node_id + ) + + +class ElectionResult(CamelCaseModel): + session_id: SessionId + won_clock: int + is_new_master: bool + + +class Election: + def __init__( + self, + node_id: NodeId, + *, + election_message_receiver: Receiver[ElectionMessage], + election_message_sender: Sender[ElectionMessage], + election_result_sender: Sender[ElectionResult], + connection_message_receiver: Receiver[ConnectionMessage], + command_receiver: Receiver[ForwarderCommand], + is_candidate: bool = True, + seniority: int = 0, + ): + # If we aren't a candidate, simply don't increment seniority. + # For reference: This node can be elected master if all nodes are not master candidates + # Any master candidate will automatically win out over this node. + self.seniority = seniority if is_candidate else -1 + self.clock = 0 + self.node_id = node_id + self.commands_seen = 0 + # Every node spawns as master + self.current_session: SessionId = SessionId( + master_node_id=node_id, election_clock=0 + ) + + # Senders/Receivers + self._em_sender = election_message_sender + self._em_receiver = election_message_receiver + self._er_sender = election_result_sender + self._cm_receiver = connection_message_receiver + self._co_receiver = command_receiver + + # Campaign state + self._candidates: list[ElectionMessage] = [] + self._campaign_cancel_scope: CancelScope | None = None + self._campaign_done: Event | None = None + self._tg: TaskGroup | None = None + + async def run(self): + logger.info("Starting Election") + async with create_task_group() as tg: + self._tg = tg + tg.start_soon(self._election_receiver) + tg.start_soon(self._connection_receiver) + tg.start_soon(self._command_counter) + + # And start an election immediately, that instantly resolves + candidates: list[ElectionMessage] = [] + logger.debug("Starting initial campaign") + self._candidates = candidates + await self._campaign(candidates, campaign_timeout=0.0) + logger.debug("Initial campaign finished") + + # Cancel and wait for the last election to end + if self._campaign_cancel_scope is not None: + logger.debug("Cancelling campaign") + self._campaign_cancel_scope.cancel() + if self._campaign_done is not None: + logger.debug("Waiting for campaign to finish") + await self._campaign_done.wait() + logger.debug("Campaign cancelled and finished") + logger.info("Election finished") + + async def elect(self, em: ElectionMessage) -> None: + logger.debug(f"Electing: {em}") + is_new_master = em.proposed_session != self.current_session + self.current_session = em.proposed_session + logger.debug(f"Current session: {self.current_session}") + await self._er_sender.send( + ElectionResult( + won_clock=em.clock, + session_id=em.proposed_session, + is_new_master=is_new_master, + ) + ) + + async def shutdown(self) -> None: + if not self._tg: + logger.warning( + "Attempted to shutdown election service that was not running" + ) + return + self._tg.cancel_scope.cancel() + + async def _election_receiver(self) -> None: + with self._em_receiver as election_messages: + async for message in election_messages: + logger.debug(f"Election message received: {message}") + if message.proposed_session.master_node_id == self.node_id: + logger.debug("Dropping message from ourselves") + # Drop messages from us (See exo.routing.router) + continue + # If a new round is starting, we participate + if message.clock > self.clock: + self.clock = message.clock + logger.debug(f"New clock: {self.clock}") + assert self._tg is not None + logger.debug("Starting new campaign") + candidates: list[ElectionMessage] = [message] + logger.debug(f"Candidates: {candidates}") + logger.debug(f"Current candidates: {self._candidates}") + self._candidates = candidates + logger.debug(f"New candidates: {self._candidates}") + logger.debug("Starting new campaign") + self._tg.start_soon( + self._campaign, candidates, DEFAULT_ELECTION_TIMEOUT + ) + logger.debug("Campaign started") + continue + # Dismiss old messages + if message.clock < self.clock: + logger.debug(f"Dropping old message: {message}") + continue + logger.debug(f"Election added candidate {message}") + # Now we are processing this rounds messages - including the message that triggered this round. + self._candidates.append(message) + + async def _connection_receiver(self) -> None: + with self._cm_receiver as connection_messages: + async for first in connection_messages: + # Delay after connection message for time to symmetrically setup + await anyio.sleep(0.2) + rest = connection_messages.collect() + + logger.debug( + f"Connection messages received: {first} followed by {rest}" + ) + logger.debug(f"Current clock: {self.clock}") + # These messages are strictly peer to peer + self.clock += 1 + logger.debug(f"New clock: {self.clock}") + assert self._tg is not None + candidates: list[ElectionMessage] = [] + self._candidates = candidates + logger.debug("Starting new campaign") + self._tg.start_soon( + self._campaign, candidates, DEFAULT_ELECTION_TIMEOUT + ) + logger.debug("Campaign started") + logger.debug("Connection message added") + + async def _command_counter(self) -> None: + with self._co_receiver as commands: + async for _command in commands: + self.commands_seen += 1 + + async def _campaign( + self, candidates: list[ElectionMessage], campaign_timeout: float + ) -> None: + clock = self.clock + + # Kill the old campaign + if self._campaign_cancel_scope: + logger.info("Cancelling other campaign") + self._campaign_cancel_scope.cancel() + if self._campaign_done: + logger.info("Waiting for other campaign to finish") + await self._campaign_done.wait() + + done = Event() + self._campaign_done = done + scope = CancelScope() + self._campaign_cancel_scope = scope + + try: + with scope: + logger.debug(f"Election {clock} started") + + status = self._election_status(clock) + candidates.append(status) + await self._em_sender.send(status) + + logger.debug(f"Sleeping for {campaign_timeout} seconds") + await anyio.sleep(campaign_timeout) + # minor hack - rebroadcast status in case anyone has missed it. + await self._em_sender.send(status) + logger.debug("Woke up from sleep") + # add an anyio checkpoint - anyio.lowlevel.chekpoint() or checkpoint_if_cancelled() is preferred, but wasn't typechecking last I checked + await anyio.sleep(0) + + # Election finished! + elected = max(candidates) + logger.debug(f"Election queue {candidates}") + logger.debug(f"Elected: {elected}") + if ( + self.node_id == elected.proposed_session.master_node_id + and self.seniority >= 0 + ): + logger.debug( + f"Node is a candidate and seniority is {self.seniority}" + ) + self.seniority = max(self.seniority, len(candidates)) + logger.debug(f"New seniority: {self.seniority}") + else: + logger.debug( + f"Node is not a candidate or seniority is not {self.seniority}" + ) + logger.debug( + f"Election finished, new SessionId({elected.proposed_session}) with queue {candidates}" + ) + logger.debug("Sending election result") + await self.elect(elected) + logger.debug("Election result sent") + except get_cancelled_exc_class(): + logger.debug(f"Election {clock} cancelled") + finally: + logger.debug(f"Election {clock} finally") + if self._campaign_cancel_scope is scope: + self._campaign_cancel_scope = None + logger.debug("Setting done event") + done.set() + logger.debug("Done event set") + + def _election_status(self, clock: int | None = None) -> ElectionMessage: + c = self.clock if clock is None else clock + return ElectionMessage( + proposed_session=( + self.current_session + if self.current_session.master_node_id == self.node_id + else SessionId(master_node_id=self.node_id, election_clock=c) + ), + clock=c, + seniority=self.seniority, + commands_seen=self.commands_seen, + ) diff --git a/src/exo/shared/logging.py b/src/exo/shared/logging.py new file mode 100644 index 00000000..75040cfd --- /dev/null +++ b/src/exo/shared/logging.py @@ -0,0 +1,92 @@ +import logging +import sys +from pathlib import Path + +from hypercorn import Config +from hypercorn.logging import Logger as HypercornLogger +from loguru import logger + + +class InterceptLogger(HypercornLogger): + def __init__(self, config: Config): + super().__init__(config) + assert self.error_logger + # TODO: Decide if we want to provide access logs + # assert self.access_logger + # self.access_logger.handlers = [_InterceptHandler()] + self.error_logger.handlers = [_InterceptHandler()] + + +class _InterceptHandler(logging.Handler): + def emit(self, record: logging.LogRecord): + try: + level = logger.level(record.levelname).name + except ValueError: + level = record.levelno + + logger.opt(depth=3, exception=record.exc_info).log(level, record.getMessage()) + + +def logger_setup(log_file: Path | None, verbosity: int = 0): + """Set up logging for this process - formatting, file handles, verbosity and output""" + logger.remove() + + # replace all stdlib loggers with _InterceptHandlers that log to loguru + logging.basicConfig(handlers=[_InterceptHandler()], level=0) + + if verbosity == 0: + logger.add( + sys.__stderr__, # type: ignore + format="[ {time:hh:mm:ss.SSSSA} | {level: <8}] {message}", + level="INFO", + colorize=True, + enqueue=True, + ) + else: + logger.add( + sys.__stderr__, # type: ignore + format="[ {time:HH:mm:ss.SSS} | {level: <8} | {name}:{function}:{line} ] {message}", + level="DEBUG", + colorize=True, + enqueue=True, + ) + if log_file: + logger.add( + log_file, + format="[ {time:YYYY-MM-DD HH:mm:ss.SSS} | {level: <8} | {name}:{function}:{line} ] {message}", + level="INFO", + colorize=False, + enqueue=True, + rotation="1 week", + ) + + +def logger_cleanup(): + """Flush all queues before shutting down so any in-flight logs are written to disk""" + logger.complete() + + +""" --- TODO: Capture MLX Log output: +import contextlib +import sys +from loguru import logger + +class StreamToLogger: + + def __init__(self, level="INFO"): + self._level = level + + def write(self, buffer): + for line in buffer.rstrip().splitlines(): + logger.opt(depth=1).log(self._level, line.rstrip()) + + def flush(self): + pass + +logger.remove() +logger.add(sys.__stdout__) + +stream = StreamToLogger() +with contextlib.redirect_stdout(stream): + print("Standard output is sent to added handlers.") +""" diff --git a/src/exo/shared/models/model_cards.py b/src/exo/shared/models/model_cards.py new file mode 100644 index 00000000..17f00e4c --- /dev/null +++ b/src/exo/shared/models/model_cards.py @@ -0,0 +1,451 @@ +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.utils.pydantic_ext import CamelCaseModel + + +class ModelCard(CamelCaseModel): + short_id: str + model_id: ModelId + name: str + description: str + tags: list[str] + metadata: ModelMetadata + + +MODEL_CARDS: dict[str, ModelCard] = { + # deepseek v3 + # "deepseek-v3-0324:4bit": ModelCard( + # short_id="deepseek-v3-0324:4bit", + # model_id="mlx-community/DeepSeek-V3-0324-4bit", + # name="DeepSeek V3 0324 (4-bit)", + # description="""DeepSeek V3 is a large language model trained on the DeepSeek V3 dataset.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/DeepSeek-V3-0324-4bit"), + # pretty_name="DeepSeek V3 0324 (4-bit)", + # storage_size=Memory.from_kb(409706307), + # n_layers=61, + # ), + # ), + # "deepseek-v3-0324": ModelCard( + # short_id="deepseek-v3-0324", + # model_id="mlx-community/DeepSeek-v3-0324-8bit", + # name="DeepSeek V3 0324 (8-bit)", + # description="""DeepSeek V3 is a large language model trained on the DeepSeek V3 dataset.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/DeepSeek-v3-0324-8bit"), + # pretty_name="DeepSeek V3 0324 (8-bit)", + # storage_size=Memory.from_kb(754706307), + # n_layers=61, + # ), + # ), + "deepseek-v3.1-4bit": ModelCard( + short_id="deepseek-v3.1-4bit", + model_id=ModelId("mlx-community/DeepSeek-V3.1-4bit"), + name="DeepSeek V3.1 (4-bit)", + description="""DeepSeek V3.1 is a large language model trained on the DeepSeek V3.1 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/DeepSeek-V3.1-4bit"), + pretty_name="DeepSeek V3.1 (4-bit)", + storage_size=Memory.from_gb(378), + n_layers=61, + ), + ), + "deepseek-v3.1-8bit": ModelCard( + short_id="deepseek-v3.1-8bit", + model_id=ModelId("mlx-community/DeepSeek-V3.1-8bit"), + name="DeepSeek V3.1 (8-bit)", + description="""DeepSeek V3.1 is a large language model trained on the DeepSeek V3.1 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/DeepSeek-V3.1-8bit"), + pretty_name="DeepSeek V3.1 (8-bit)", + storage_size=Memory.from_gb(713), + n_layers=61, + ), + ), + # "deepseek-v3.2": ModelCard( + # short_id="deepseek-v3.2", + # model_id=ModelId("mlx-community/DeepSeek-V3.2-8bit"), + # name="DeepSeek V3.2 (8-bit)", + # description="""DeepSeek V3.2 is a large language model trained on the DeepSeek V3.2 dataset.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/DeepSeek-V3.2-8bit"), + # pretty_name="DeepSeek V3.2 (8-bit)", + # storage_size=Memory.from_kb(754706307), + # n_layers=61, + # hidden_size=7168, + # ), + # ), + # "deepseek-v3.2-4bit": ModelCard( + # short_id="deepseek-v3.2-4bit", + # model_id=ModelId("mlx-community/DeepSeek-V3.2-4bit"), + # name="DeepSeek V3.2 (4-bit)", + # description="""DeepSeek V3.2 is a large language model trained on the DeepSeek V3.2 dataset.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/DeepSeek-V3.2-4bit"), + # pretty_name="DeepSeek V3.2 (4-bit)", + # storage_size=Memory.from_kb(754706307 // 2), # TODO !!!!! + # n_layers=61, + # hidden_size=7168, + # ), + # ), + # deepseek r1 + # "deepseek-r1-0528-4bit": ModelCard( + # short_id="deepseek-r1-0528-4bit", + # model_id="mlx-community/DeepSeek-R1-0528-4bit", + # name="DeepSeek-R1-0528 (4-bit)", + # description="""DeepSeek R1 is a large language model trained on the DeepSeek R1 dataset.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/DeepSeek-R1-0528-4bit"), + # pretty_name="DeepSeek R1 671B (4-bit)", + # storage_size=Memory.from_kb(409706307), + # n_layers=61, + # hidden_size=7168, + # ), + # ), + # "deepseek-r1-0528": ModelCard( + # short_id="deepseek-r1-0528", + # model_id="mlx-community/DeepSeek-R1-0528-8bit", + # name="DeepSeek-R1-0528 (8-bit)", + # description="""DeepSeek R1 is a large language model trained on the DeepSeek R1 dataset.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/DeepSeek-R1-0528-8bit"), + # pretty_name="DeepSeek R1 671B (8-bit)", + # storage_size=Memory.from_bytes(754998771712), + # n_layers=61, + # . hidden_size=7168, + # ), + # ), + # kimi k2 + "kimi-k2-instruct-4bit": ModelCard( + short_id="kimi-k2-instruct-4bit", + model_id=ModelId("mlx-community/Kimi-K2-Instruct-4bit"), + name="Kimi K2 Instruct (4-bit)", + description="""Kimi K2 is a large language model trained on the Kimi K2 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Kimi-K2-Instruct-4bit"), + pretty_name="Kimi K2 Instruct (4-bit)", + storage_size=Memory.from_gb(578), + n_layers=61, + ), + ), + "kimi-k2-thinking": ModelCard( + short_id="kimi-k2-thinking", + model_id=ModelId("mlx-community/Kimi-K2-Thinking"), + name="Kimi K2 Thinking (4-bit)", + description="""Kimi K2 Thinking is the latest, most capable version of open-source thinking model.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Kimi-K2-Thinking"), + pretty_name="Kimi K2 Thinking (4-bit)", + storage_size=Memory.from_gb(658), + n_layers=61, + ), + ), + # llama-3.1 + "llama-3.1-8b": ModelCard( + short_id="llama-3.1-8b", + model_id=ModelId("mlx-community/Meta-Llama-3.1-8B-Instruct-4bit"), + name="Llama 3.1 8B (4-bit)", + description="""Llama 3.1 is a large language model trained on the Llama 3.1 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Meta-Llama-3.1-8B-Instruct-4bit"), + pretty_name="Llama 3.1 8B (4-bit)", + storage_size=Memory.from_mb(4423), + n_layers=32, + ), + ), + "llama-3.1-70b": ModelCard( + short_id="llama-3.1-70b", + model_id=ModelId("mlx-community/Meta-Llama-3.1-70B-Instruct-4bit"), + name="Llama 3.1 70B (4-bit)", + description="""Llama 3.1 is a large language model trained on the Llama 3.1 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Meta-Llama-3.1-70B-Instruct-4bit"), + pretty_name="Llama 3.1 70B (4-bit)", + storage_size=Memory.from_mb(38769), + n_layers=80, + ), + ), + # llama-3.2 + "llama-3.2-1b": ModelCard( + short_id="llama-3.2-1b", + model_id=ModelId("mlx-community/Llama-3.2-1B-Instruct-4bit"), + name="Llama 3.2 1B (4-bit)", + description="""Llama 3.2 is a large language model trained on the Llama 3.2 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Llama-3.2-1B-Instruct-4bit"), + pretty_name="Llama 3.2 1B (4-bit)", + storage_size=Memory.from_mb(696), + n_layers=16, + ), + ), + "llama-3.2-3b": ModelCard( + short_id="llama-3.2-3b", + model_id=ModelId("mlx-community/Llama-3.2-3B-Instruct-4bit"), + name="Llama 3.2 3B (4-bit)", + description="""Llama 3.2 is a large language model trained on the Llama 3.2 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Llama-3.2-3B-Instruct-4bit"), + pretty_name="Llama 3.2 3B (4-bit)", + storage_size=Memory.from_mb(1777), + n_layers=28, + ), + ), + "llama-3.2-3b-8bit": ModelCard( + short_id="llama-3.2-3b-8bit", + model_id=ModelId("mlx-community/Llama-3.2-3B-Instruct-8bit"), + name="Llama 3.2 3B (8-bit)", + description="""Llama 3.2 is a large language model trained on the Llama 3.2 dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Llama-3.2-3B-Instruct-8bit"), + pretty_name="Llama 3.2 3B (8-bit)", + storage_size=Memory.from_mb(3339), + n_layers=28, + ), + ), + # llama-3.3 + "llama-3.3-70b": ModelCard( + short_id="llama-3.3-70b", + model_id=ModelId("mlx-community/Llama-3.3-70B-Instruct-4bit"), + name="Llama 3.3 70B (4-bit)", + description="""The Meta Llama 3.3 multilingual large language model (LLM) is an instruction tuned generative model in 70B (text in/text out)""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Llama-3.3-70B-Instruct-4bit"), + pretty_name="Llama 3.3 70B", + storage_size=Memory.from_mb(38769), + n_layers=80, + ), + ), + "llama-3.3-70b-8bit": ModelCard( + short_id="llama-3.3-70b-8bit", + model_id=ModelId("mlx-community/Llama-3.3-70B-Instruct-8bit"), + name="Llama 3.3 70B (8-bit)", + description="""The Meta Llama 3.3 multilingual large language model (LLM) is an instruction tuned generative model in 70B (text in/text out)""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Llama-3.3-70B-Instruct-8bit"), + pretty_name="Llama 3.3 70B (8-bit)", + storage_size=Memory.from_mb(73242), + n_layers=80, + ), + ), + "llama-3.3-70b-fp16": ModelCard( + short_id="llama-3.3-70b-fp16", + model_id=ModelId("mlx-community/llama-3.3-70b-instruct-fp16"), + name="Llama 3.3 70B (FP16)", + description="""The Meta Llama 3.3 multilingual large language model (LLM) is an instruction tuned generative model in 70B (text in/text out)""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/llama-3.3-70b-instruct-fp16"), + pretty_name="Llama 3.3 70B (FP16)", + storage_size=Memory.from_mb(137695), + n_layers=80, + ), + ), + # phi-3 + "phi-3-mini": ModelCard( + short_id="phi-3-mini", + model_id=ModelId("mlx-community/Phi-3-mini-128k-instruct-4bit"), + name="Phi 3 Mini 128k (4-bit)", + description="""Phi 3 Mini is a large language model trained on the Phi 3 Mini dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Phi-3-mini-128k-instruct-4bit"), + pretty_name="Phi 3 Mini 128k (4-bit)", + storage_size=Memory.from_mb(2099), + n_layers=32, + ), + ), + # qwen3 + "qwen3-0.6b": ModelCard( + short_id="qwen3-0.6b", + model_id=ModelId("mlx-community/Qwen3-0.6B-4bit"), + name="Qwen3 0.6B (4-bit)", + description="""Qwen3 0.6B is a large language model trained on the Qwen3 0.6B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-0.6B-4bit"), + pretty_name="Qwen3 0.6B (4-bit)", + storage_size=Memory.from_mb(327), + n_layers=28, + ), + ), + "qwen3-0.6b-8bit": ModelCard( + short_id="qwen3-0.6b-8bit", + model_id=ModelId("mlx-community/Qwen3-0.6B-8bit"), + name="Qwen3 0.6B (8-bit)", + description="""Qwen3 0.6B is a large language model trained on the Qwen3 0.6B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-0.6B-8bit"), + pretty_name="Qwen3 0.6B (8-bit)", + storage_size=Memory.from_mb(666), + n_layers=28, + ), + ), + "qwen3-30b": ModelCard( + short_id="qwen3-30b", + model_id=ModelId("mlx-community/Qwen3-30B-A3B-4bit"), + name="Qwen3 30B A3B (4-bit)", + description="""Qwen3 30B is a large language model trained on the Qwen3 30B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-30B-A3B-4bit"), + pretty_name="Qwen3 30B A3B (4-bit)", + storage_size=Memory.from_mb(16797), + n_layers=48, + ), + ), + "qwen3-30b-8bit": ModelCard( + short_id="qwen3-30b-8bit", + model_id=ModelId("mlx-community/Qwen3-30B-A3B-8bit"), + name="Qwen3 30B A3B (8-bit)", + description="""Qwen3 30B is a large language model trained on the Qwen3 30B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-30B-A3B-8bit"), + pretty_name="Qwen3 30B A3B (8-bit)", + storage_size=Memory.from_mb(31738), + n_layers=48, + ), + ), + "qwen3-235b-a22b-4bit": ModelCard( + short_id="qwen3-235b-a22b-4bit", + model_id=ModelId("mlx-community/Qwen3-235B-A22B-Instruct-2507-4bit"), + name="Qwen3 235B A22B (4-bit)", + description="""Qwen3 235B (Active 22B) is a large language model trained on the Qwen3 235B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-235B-A22B-Instruct-2507-4bit"), + pretty_name="Qwen3 235B A22B (4-bit)", + storage_size=Memory.from_gb(132), + n_layers=94, + ), + ), + "qwen3-235b-a22b-8bit": ModelCard( + short_id="qwen3-235b-a22b-8bit", + model_id=ModelId("mlx-community/Qwen3-235B-A22B-Instruct-2507-8bit"), + name="Qwen3 235B A22B (8-bit)", + description="""Qwen3 235B (Active 22B) is a large language model trained on the Qwen3 235B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-235B-A22B-Instruct-2507-8bit"), + pretty_name="Qwen3 235B A22B (8-bit)", + storage_size=Memory.from_gb(250), + n_layers=94, + ), + ), + "qwen3-coder-480b-a35b-4bit": ModelCard( + short_id="qwen3-coder-480b-a35b-4bit", + model_id=ModelId("mlx-community/Qwen3-Coder-480B-A35B-Instruct-4bit"), + name="Qwen3 Coder 480B A35B (4-bit)", + description="""Qwen3 Coder 480B (Active 35B) is a large language model trained on the Qwen3 Coder 480B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-Coder-480B-A35B-Instruct-4bit"), + pretty_name="Qwen3 Coder 480B A35B (4-bit)", + storage_size=Memory.from_gb(270), + n_layers=62, + ), + ), + "qwen3-coder-480b-a35b-8bit": ModelCard( + short_id="qwen3-coder-480b-a35b-8bit", + model_id=ModelId("mlx-community/Qwen3-Coder-480B-A35B-Instruct-8bit"), + name="Qwen3 Coder 480B A35B (8-bit)", + description="""Qwen3 Coder 480B (Active 35B) is a large language model trained on the Qwen3 Coder 480B dataset.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/Qwen3-Coder-480B-A35B-Instruct-8bit"), + pretty_name="Qwen3 Coder 480B A35B (8-bit)", + storage_size=Memory.from_gb(540), + n_layers=62, + ), + ), + # granite + "granite-3.3-2b": ModelCard( + short_id="granite-3.3-2b", + model_id=ModelId("mlx-community/granite-3.3-2b-instruct-fp16"), + name="Granite 3.3 2B (FP16)", + description="""Granite-3.3-2B-Instruct is a 2-billion parameter 128K context length language model fine-tuned for improved reasoning and instruction-following capabilities.""", + tags=[], + metadata=ModelMetadata( + model_id=ModelId("mlx-community/granite-3.3-2b-instruct-fp16"), + pretty_name="Granite 3.3 2B (FP16)", + storage_size=Memory.from_mb(4951), + n_layers=40, + ), + ), + # "granite-3.3-8b": ModelCard( + # short_id="granite-3.3-8b", + # model_id=ModelId("mlx-community/granite-3.3-8b-instruct-fp16"), + # name="Granite 3.3 8B", + # description="""Granite-3.3-8B-Instruct is a 8-billion parameter 128K context length language model fine-tuned for improved reasoning and instruction-following capabilities.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/granite-3.3-8b-instruct-fp16"), + # pretty_name="Granite 3.3 8B", + # storage_size=Memory.from_kb(15958720), + # n_layers=40, + # ), + # ), + # smol-lm + # "smol-lm-135m": ModelCard( + # short_id="smol-lm-135m", + # model_id="mlx-community/SmolLM-135M-4bit", + # name="Smol LM 135M", + # description="""SmolLM is a series of state-of-the-art small language models available in three sizes: 135M, 360M, and 1.7B parameters. """, + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/SmolLM-135M-4bit"), + # pretty_name="Smol LM 135M", + # storage_size=Memory.from_kb(73940), + # n_layers=30, + # ), + # ), + # gpt-oss + # "gpt-oss-120b-MXFP4-Q8": ModelCard( + # short_id="gpt-oss-120b-MXFP4-Q8", + # model_id=ModelId("mlx-community/gpt-oss-120b-MXFP4-Q8"), + # name="GPT-OSS 120B (MXFP4-Q8, MLX)", + # description="""OpenAI's GPT-OSS 120B is a 117B-parameter Mixture-of-Experts model designed for high-reasoning and general-purpose use; this variant is a 4-bit MLX conversion for Apple Silicon.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/gpt-oss-120b-MXFP4-Q8"), + # pretty_name="GPT-OSS 120B (MXFP4-Q8, MLX)", + # storage_size=Memory.from_kb(68_996_301), + # n_layers=36, + # hidden_size=2880, + # supports_tensor=True, + # ), + # ), + # "gpt-oss-20b-4bit": ModelCard( + # short_id="gpt-oss-20b-4bit", + # model_id=ModelId("mlx-community/gpt-oss-20b-MXFP4-Q4"), + # name="GPT-OSS 20B (MXFP4-Q4, MLX)", + # description="""OpenAI's GPT-OSS 20B is a medium-sized MoE model for lower-latency and local or specialized use cases; this MLX variant uses MXFP4 4-bit quantization.""", + # tags=[], + # metadata=ModelMetadata( + # model_id=ModelId("mlx-community/gpt-oss-20b-MXFP4-Q4"), + # pretty_name="GPT-OSS 20B (MXFP4-Q4, MLX)", + # storage_size=Memory.from_kb(11_744_051), + # n_layers=24, + # hidden_size=2880, + # supports_tensor=True, + # ), + # ), +} diff --git a/src/exo/shared/models/model_meta.py b/src/exo/shared/models/model_meta.py new file mode 100644 index 00000000..24da284c --- /dev/null +++ b/src/exo/shared/models/model_meta.py @@ -0,0 +1,115 @@ +from typing import Annotated + +import aiofiles +import aiofiles.os as aios +from huggingface_hub import model_info +from loguru import logger +from pydantic import BaseModel, Field + +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.worker.download.download_utils import ( + ModelSafetensorsIndex, + download_file_with_retry, + ensure_models_dir, +) + + +class ConfigData(BaseModel): + model_config = {"extra": "ignore"} # Allow unknown fields + + # Common field names for number of layers across different architectures + num_hidden_layers: Annotated[int, Field(ge=0)] | None = None + num_layers: Annotated[int, Field(ge=0)] | None = None + n_layer: Annotated[int, Field(ge=0)] | None = None + n_layers: Annotated[int, Field(ge=0)] | None = None # Sometimes used + num_decoder_layers: Annotated[int, Field(ge=0)] | None = None # Transformer models + decoder_layers: Annotated[int, Field(ge=0)] | None = None # Some architectures + + @property + def layer_count(self) -> int: + # Check common field names for layer count + layer_fields = [ + self.num_hidden_layers, + self.num_layers, + self.n_layer, + self.n_layers, + self.num_decoder_layers, + self.decoder_layers, + ] + + for layer_count in layer_fields: + if layer_count is not None: + return layer_count + + raise ValueError( + f"No layer count found in config.json: {self.model_dump_json()}" + ) + + +async def get_config_data(model_id: str) -> ConfigData: + """Downloads and parses config.json for a model.""" + target_dir = (await ensure_models_dir()) / str(model_id).replace("/", "--") + await aios.makedirs(target_dir, exist_ok=True) + config_path = await download_file_with_retry( + model_id, + "main", + "config.json", + target_dir, + lambda curr_bytes, total_bytes, is_renamed: logger.info( + f"Downloading config.json for {model_id}: {curr_bytes}/{total_bytes} ({is_renamed=})" + ), + ) + async with aiofiles.open(config_path, "r") as f: + return ConfigData.model_validate_json(await f.read()) + + +async def get_safetensors_size(model_id: str) -> Memory: + """Gets model size from safetensors index or falls back to HF API.""" + target_dir = (await ensure_models_dir()) / str(model_id).replace("/", "--") + await aios.makedirs(target_dir, exist_ok=True) + index_path = await download_file_with_retry( + model_id, + "main", + "model.safetensors.index.json", + target_dir, + lambda curr_bytes, total_bytes, is_renamed: logger.info( + f"Downloading model.safetensors.index.json for {model_id}: {curr_bytes}/{total_bytes} ({is_renamed=})" + ), + ) + async with aiofiles.open(index_path, "r") as f: + index_data = ModelSafetensorsIndex.model_validate_json(await f.read()) + + metadata = index_data.metadata + if metadata is not None: + return Memory.from_bytes(metadata.total_size) + + info = model_info(model_id) + if info.safetensors is None: + raise ValueError(f"No safetensors info found for {model_id}") + return Memory.from_bytes(info.safetensors.total) + + +_model_meta_cache: dict[str, ModelMetadata] = {} + + +async def get_model_meta(model_id: str) -> ModelMetadata: + if model_id in _model_meta_cache: + return _model_meta_cache[model_id] + model_meta = await _get_model_meta(model_id) + _model_meta_cache[model_id] = model_meta + return model_meta + + +async def _get_model_meta(model_id: str) -> ModelMetadata: + """Fetches storage size and number of layers for a Hugging Face model, returns Pydantic ModelMeta.""" + config_data = await get_config_data(model_id) + num_layers = config_data.layer_count + mem_size_bytes = await get_safetensors_size(model_id) + + return ModelMetadata( + model_id=ModelId(model_id), + pretty_name=model_id, + storage_size=mem_size_bytes, + n_layers=num_layers, + ) diff --git a/src/exo/shared/tests/__init__.py b/src/exo/shared/tests/__init__.py new file mode 100644 index 00000000..09c36e8f --- /dev/null +++ b/src/exo/shared/tests/__init__.py @@ -0,0 +1 @@ +# Test package for shared utilities diff --git a/src/exo/shared/tests/conftest.py b/src/exo/shared/tests/conftest.py new file mode 100644 index 00000000..1a6092f1 --- /dev/null +++ b/src/exo/shared/tests/conftest.py @@ -0,0 +1,58 @@ +"""Pytest configuration and shared fixtures for shared package tests.""" + +import asyncio +from typing import Generator + +import pytest +from _pytest.logging import LogCaptureFixture +from loguru import logger + +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.worker.shards import PipelineShardMetadata, ShardMetadata + + +@pytest.fixture(scope="session") +def event_loop() -> Generator[asyncio.AbstractEventLoop, None, None]: + """Create an event loop for the test session.""" + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + yield loop + loop.close() + + +@pytest.fixture(autouse=True) +def reset_event_loop(): + """Reset the event loop for each test to ensure clean state.""" + # This ensures each test gets a fresh event loop state + + +def get_pipeline_shard_metadata( + model_id: ModelId, device_rank: int, world_size: int = 1 +) -> ShardMetadata: + return PipelineShardMetadata( + model_meta=ModelMetadata( + model_id=model_id, + pretty_name=str(model_id), + storage_size=Memory.from_mb(100000), + n_layers=32, + ), + device_rank=device_rank, + world_size=world_size, + start_layer=0, + end_layer=32, + n_layers=32, + ) + + +@pytest.fixture +def caplog(caplog: LogCaptureFixture): + handler_id = logger.add( + caplog.handler, + format="{message}", + level=0, + filter=lambda record: record["level"].no >= caplog.handler.level, + enqueue=True, # Set to 'True' if your test is spawning child processes. + ) + yield caplog + logger.remove(handler_id) diff --git a/src/exo/shared/tests/test_apply/test_apply_node_download.py b/src/exo/shared/tests/test_apply/test_apply_node_download.py new file mode 100644 index 00000000..4745c7a0 --- /dev/null +++ b/src/exo/shared/tests/test_apply/test_apply_node_download.py @@ -0,0 +1,45 @@ +from exo.shared.apply import apply_node_download_progress +from exo.shared.tests.conftest import get_pipeline_shard_metadata +from exo.shared.types.common import NodeId +from exo.shared.types.events import NodeDownloadProgress +from exo.shared.types.state import State +from exo.shared.types.worker.downloads import DownloadCompleted +from exo.worker.tests.constants import MODEL_A_ID, MODEL_B_ID + + +def test_apply_node_download_progress(): + state = State() + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + event = DownloadCompleted( + node_id=NodeId("node-1"), + shard_metadata=shard1, + ) + + new_state = apply_node_download_progress( + NodeDownloadProgress(download_progress=event), state + ) + + assert new_state == State(downloads={NodeId("node-1"): [event]}) + + +def test_apply_two_node_download_progress(): + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard2 = get_pipeline_shard_metadata(MODEL_B_ID, device_rank=0, world_size=2) + event1 = DownloadCompleted( + node_id=NodeId("node-1"), + shard_metadata=shard1, + ) + event2 = DownloadCompleted( + node_id=NodeId("node-1"), + shard_metadata=shard2, + ) + state = State(downloads={NodeId("node-1"): [event1]}) + + new_state = apply_node_download_progress( + NodeDownloadProgress(download_progress=event2), state + ) + + # TODO: This test is failing. We should support the following: + # 1. Downloading multiple models concurrently on the same node (one per runner is fine). + # 2. Downloading a model, it completes, then downloading a different model on the same node. + assert new_state == State(downloads={NodeId("node-1"): [event1, event2]}) diff --git a/src/exo/shared/tests/test_election.py b/src/exo/shared/tests/test_election.py new file mode 100644 index 00000000..49550601 --- /dev/null +++ b/src/exo/shared/tests/test_election.py @@ -0,0 +1,411 @@ +import pytest +from anyio import create_task_group, fail_after, move_on_after + +from exo.routing.connection_message import ConnectionMessage, ConnectionMessageType +from exo.shared.election import Election, ElectionMessage, ElectionResult +from exo.shared.types.commands import ForwarderCommand, TestCommand +from exo.shared.types.common import NodeId, SessionId +from exo.utils.channels import channel + +# ======= # +# Helpers # +# ======= # + + +def em( + clock: int, + seniority: int, + node_id: str, + commands_seen: int = 0, + election_clock: int | None = None, +) -> ElectionMessage: + """ + Helper to build ElectionMessages for a given proposer node. + + The new API carries a proposed SessionId (master_node_id + election_clock). + By default we use the same value for election_clock as the 'clock' of the round. + """ + return ElectionMessage( + clock=clock, + seniority=seniority, + proposed_session=SessionId( + master_node_id=NodeId(node_id), + election_clock=clock if election_clock is None else election_clock, + ), + commands_seen=commands_seen, + ) + + +# ======================================= # +# TESTS # +# ======================================= # + + +@pytest.fixture(autouse=True) +def fast_election_timeout(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setattr("exo.shared.election.DEFAULT_ELECTION_TIMEOUT", 0.1) + + +@pytest.mark.anyio +async def test_single_round_broadcasts_and_updates_seniority_on_self_win() -> None: + """ + Start a round by injecting an ElectionMessage with higher clock. + With only our node effectively 'winning', we should broadcast once and update seniority. + """ + # Outbound election messages from the Election (we'll observe these) + em_out_tx, em_out_rx = channel[ElectionMessage]() + # Inbound election messages to the Election (we'll inject these) + em_in_tx, em_in_rx = channel[ElectionMessage]() + # Election results produced by the Election (we'll observe these) + er_tx, er_rx = channel[ElectionResult]() + # Connection messages + cm_tx, cm_rx = channel[ConnectionMessage]() + # Commands + co_tx, co_rx = channel[ForwarderCommand]() + + election = Election( + node_id=NodeId("B"), + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + # Trigger new round at clock=1 (peer announces it) + await em_in_tx.send(em(clock=1, seniority=0, node_id="A")) + + # Expect our broadcast back to the peer side for this round only + while True: + got = await em_out_rx.receive() + if got.clock == 1 and got.proposed_session.master_node_id == NodeId( + "B" + ): + break + + # Wait for the round to finish and produce an ElectionResult + result = await er_rx.receive() + assert result.session_id.master_node_id == NodeId("B") + # We spawned as master; electing ourselves again is not "new master". + assert result.is_new_master is False + + # Close inbound streams to end the receivers (and run()) + em_in_tx.close() + cm_tx.close() + co_tx.close() + + # We should have updated seniority to 2 (A + B). + assert election.seniority == 2 + + +@pytest.mark.anyio +async def test_peer_with_higher_seniority_wins_and_we_switch_master() -> None: + """ + If a peer with clearly higher seniority participates in the round, they should win. + We should broadcast our status exactly once for this round, then switch master. + """ + em_out_tx, em_out_rx = channel[ElectionMessage]() + em_in_tx, em_in_rx = channel[ElectionMessage]() + er_tx, er_rx = channel[ElectionResult]() + cm_tx, cm_rx = channel[ConnectionMessage]() + co_tx, co_rx = channel[ForwarderCommand]() + + election = Election( + node_id=NodeId("ME"), + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + + # Start round with peer's message (higher seniority) + await em_in_tx.send(em(clock=1, seniority=10, node_id="PEER")) + + # We should still broadcast our status exactly once for this round + while True: + got = await em_out_rx.receive() + if got.clock == 1: + assert got.seniority == 0 + break + + # After the timeout, election result for clock=1 should report the peer as master + # (Skip any earlier result from the boot campaign at clock=0 by filtering on election_clock) + while True: + result = await er_rx.receive() + if result.session_id.election_clock == 1: + break + + assert result.session_id.master_node_id == NodeId("PEER") + assert result.is_new_master is True + + em_in_tx.close() + cm_tx.close() + co_tx.close() + + # We lost → seniority unchanged + assert election.seniority == 0 + + +@pytest.mark.anyio +async def test_ignores_older_messages() -> None: + """ + Messages with a lower clock than the current round are ignored by the receiver. + Expect exactly one broadcast for the higher clock round. + """ + em_out_tx, em_out_rx = channel[ElectionMessage]() + em_in_tx, em_in_rx = channel[ElectionMessage]() + er_tx, _er_rx = channel[ElectionResult]() + cm_tx, cm_rx = channel[ConnectionMessage]() + co_tx, co_rx = channel[ForwarderCommand]() + + election = Election( + node_id=NodeId("ME"), + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + + # Newer round arrives first -> triggers campaign at clock=2 + await em_in_tx.send(em(clock=2, seniority=0, node_id="A")) + while True: + first = await em_out_rx.receive() + if first.clock == 2: + break + + # Older message (clock=1) must be ignored (no second broadcast) + await em_in_tx.send(em(clock=1, seniority=999, node_id="B")) + + got_second = False + with move_on_after(0.05): + _ = await em_out_rx.receive() + got_second = True + assert not got_second, "Should not receive a broadcast for an older round" + + em_in_tx.close() + cm_tx.close() + co_tx.close() + + # Not asserting on the result; focus is on ignore behavior. + + +@pytest.mark.anyio +async def test_two_rounds_emit_two_broadcasts_and_increment_clock() -> None: + """ + Two successive rounds → two broadcasts. Second round triggered by a higher-clock message. + """ + em_out_tx, em_out_rx = channel[ElectionMessage]() + em_in_tx, em_in_rx = channel[ElectionMessage]() + er_tx, _er_rx = channel[ElectionResult]() + cm_tx, cm_rx = channel[ConnectionMessage]() + co_tx, co_rx = channel[ForwarderCommand]() + + election = Election( + node_id=NodeId("ME"), + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + + # Round 1 at clock=1 + await em_in_tx.send(em(clock=1, seniority=0, node_id="X")) + while True: + m1 = await em_out_rx.receive() + if m1.clock == 1: + break + + # Round 2 at clock=2 + await em_in_tx.send(em(clock=2, seniority=0, node_id="Y")) + while True: + m2 = await em_out_rx.receive() + if m2.clock == 2: + break + + em_in_tx.close() + cm_tx.close() + co_tx.close() + + # Not asserting on who won; just that both rounds were broadcast. + + +@pytest.mark.anyio +async def test_promotion_new_seniority_counts_participants() -> None: + """ + When we win against two peers in the same round, our seniority becomes + max(existing, number_of_candidates). With existing=0: expect 3 (us + A + B). + """ + em_out_tx, em_out_rx = channel[ElectionMessage]() + em_in_tx, em_in_rx = channel[ElectionMessage]() + er_tx, er_rx = channel[ElectionResult]() + cm_tx, cm_rx = channel[ConnectionMessage]() + co_tx, co_rx = channel[ForwarderCommand]() + + election = Election( + node_id=NodeId("ME"), + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + + # Start round at clock=7 with two peer participants + await em_in_tx.send(em(clock=7, seniority=0, node_id="A")) + await em_in_tx.send(em(clock=7, seniority=0, node_id="B")) + + # We should see exactly one broadcast from us for this round + while True: + got = await em_out_rx.receive() + if got.clock == 7 and got.proposed_session.master_node_id == NodeId( + "ME" + ): + break + + # Wait for the election to finish so seniority updates + _ = await er_rx.receive() + + em_in_tx.close() + cm_tx.close() + co_tx.close() + + # We + A + B = 3 → new seniority expected to be 3 + assert election.seniority == 3 + + +@pytest.mark.anyio +async def test_connection_message_triggers_new_round_broadcast() -> None: + """ + A connection message increments the clock and starts a new campaign. + We should observe a broadcast at the incremented clock. + """ + em_out_tx, em_out_rx = channel[ElectionMessage]() + em_in_tx, em_in_rx = channel[ElectionMessage]() + er_tx, _er_rx = channel[ElectionResult]() + cm_tx, cm_rx = channel[ConnectionMessage]() + co_tx, co_rx = channel[ForwarderCommand]() + + election = Election( + node_id=NodeId("ME"), + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + + # Send any connection message object; we close quickly to cancel before result creation + await cm_tx.send( + ConnectionMessage( + node_id=NodeId(), + connection_type=ConnectionMessageType.Connected, + remote_ipv4="", + remote_tcp_port=0, + ) + ) + + # Expect a broadcast for the new round at clock=1 + while True: + got = await em_out_rx.receive() + if got.clock == 1 and got.proposed_session.master_node_id == NodeId( + "ME" + ): + break + + # Close promptly to avoid waiting for campaign completion + em_in_tx.close() + cm_tx.close() + co_tx.close() + + # After cancellation (before election finishes), no seniority changes asserted here. + + +@pytest.mark.anyio +async def test_tie_breaker_prefers_node_with_more_commands_seen() -> None: + """ + With equal seniority, the node that has seen more commands should win the election. + We increase our local 'commands_seen' by sending TestCommand()s before triggering the round. + """ + em_out_tx, em_out_rx = channel[ElectionMessage]() + em_in_tx, em_in_rx = channel[ElectionMessage]() + er_tx, er_rx = channel[ElectionResult]() + cm_tx, cm_rx = channel[ConnectionMessage]() + co_tx, co_rx = channel[ForwarderCommand]() + + me = NodeId("ME") + + election = Election( + node_id=me, + election_message_receiver=em_in_rx, + election_message_sender=em_out_tx, + election_result_sender=er_tx, + connection_message_receiver=cm_rx, + command_receiver=co_rx, + is_candidate=True, + seniority=0, + ) + + async with create_task_group() as tg: + with fail_after(2): + tg.start_soon(election.run) + + # Pump local commands so our commands_seen is high before the round starts + for _ in range(50): + await co_tx.send( + ForwarderCommand(origin=NodeId("SOMEONE"), command=TestCommand()) + ) + + # Trigger a round at clock=1 with a peer of equal seniority but fewer commands + await em_in_tx.send( + em(clock=1, seniority=0, node_id="PEER", commands_seen=5) + ) + + # Observe our broadcast for this round (to ensure we've joined the round) + while True: + got = await em_out_rx.receive() + if got.clock == 1 and got.proposed_session.master_node_id == me: + # We don't assert exact count, just that we've participated this round. + break + + # The elected result for clock=1 should be us due to higher commands_seen + while True: + result = await er_rx.receive() + if result.session_id.master_node_id == me: + assert result.session_id.election_clock in (0, 1) + break + + em_in_tx.close() + cm_tx.close() + co_tx.close() diff --git a/src/exo/shared/tests/test_node_id_persistence.py b/src/exo/shared/tests/test_node_id_persistence.py new file mode 100644 index 00000000..8b241aa5 --- /dev/null +++ b/src/exo/shared/tests/test_node_id_persistence.py @@ -0,0 +1,92 @@ +import contextlib +import multiprocessing +import os +from multiprocessing import Event, Queue, Semaphore +from multiprocessing.process import BaseProcess +from multiprocessing.queues import Queue as QueueT +from multiprocessing.synchronize import Event as EventT +from multiprocessing.synchronize import Semaphore as SemaphoreT + +from loguru import logger +from pytest import LogCaptureFixture + +from exo.routing.router import get_node_id_keypair +from exo.shared.constants import EXO_NODE_ID_KEYPAIR + +NUM_CONCURRENT_PROCS = 10 + + +def _get_keypair_concurrent_subprocess_task( + sem: SemaphoreT, ev: EventT, queue: QueueT[bytes] +) -> None: + # synchronise with parent process + sem.release() + # wait to be told to begin simultaneous read + ev.wait() + queue.put(get_node_id_keypair().to_protobuf_encoding()) + + +def _get_keypair_concurrent(num_procs: int) -> bytes: + assert num_procs > 0 + + sem = Semaphore(0) + ev = Event() + queue: QueueT[bytes] = Queue(maxsize=num_procs) + + # make parent process wait for all subprocesses to start + logger.info(f"PARENT: Starting {num_procs} subprocesses") + ps: list[BaseProcess] = [] + for _ in range(num_procs): + p = multiprocessing.get_context("fork").Process( + target=_get_keypair_concurrent_subprocess_task, args=(sem, ev, queue) + ) + ps.append(p) + p.start() + for _ in range(num_procs): + sem.acquire() + + # start all the sub processes simultaneously + logger.info("PARENT: Beginning read") + ev.set() + + # wait until all subprocesses are done & read results + for p in ps: + p.join() + + # check that the input/output order match, and that + # all subprocesses end up reading the same file + logger.info("PARENT: Checking consistency") + keypair: bytes | None = None + qsize = 0 # cannot use Queue.qsize due to MacOS incompatibility :( + while not queue.empty(): + qsize += 1 + temp_keypair = queue.get() + if keypair is None: + keypair = temp_keypair + else: + assert keypair == temp_keypair + assert num_procs == qsize + return keypair # pyright: ignore[reportReturnType] + + +def _delete_if_exists(p: str | bytes | os.PathLike[str] | os.PathLike[bytes]): + with contextlib.suppress(OSError): + os.remove(p) + + +def test_node_id_fetching(caplog: LogCaptureFixture): + reps = 10 + + # delete current file and write a new one + _delete_if_exists(EXO_NODE_ID_KEYPAIR) + kp = _get_keypair_concurrent(NUM_CONCURRENT_PROCS) + + with caplog.at_level(101): # supress logs + # make sure that continuous fetches return the same value + for _ in range(reps): + assert kp == _get_keypair_concurrent(NUM_CONCURRENT_PROCS) + + # make sure that after deleting, we are not fetching the same value + _delete_if_exists(EXO_NODE_ID_KEYPAIR) + for _ in range(reps): + assert kp != _get_keypair_concurrent(NUM_CONCURRENT_PROCS) diff --git a/src/exo/shared/tests/test_state_serialization.py b/src/exo/shared/tests/test_state_serialization.py new file mode 100644 index 00000000..5935d444 --- /dev/null +++ b/src/exo/shared/tests/test_state_serialization.py @@ -0,0 +1,27 @@ +from exo.shared.types.common import NodeId +from exo.shared.types.multiaddr import Multiaddr +from exo.shared.types.state import State +from exo.shared.types.topology import Connection + + +def test_state_serialization_roundtrip() -> None: + """Verify that State → JSON → State round-trip preserves topology.""" + + # --- build a simple state ------------------------------------------------ + node_a = NodeId("node-a") + node_b = NodeId("node-b") + + connection = Connection( + local_node_id=node_a, + send_back_node_id=node_b, + send_back_multiaddr=Multiaddr(address="/ip4/127.0.0.1/tcp/10001"), + ) + + state = State() + state.topology.add_connection(connection) + + json_repr = state.model_dump_json() + restored_state = State.model_validate_json(json_repr) + + assert state.topology.to_snapshot() == restored_state.topology.to_snapshot() + assert restored_state.model_dump_json() == json_repr diff --git a/src/exo/shared/topology.py b/src/exo/shared/topology.py new file mode 100644 index 00000000..46419d72 --- /dev/null +++ b/src/exo/shared/topology.py @@ -0,0 +1,212 @@ +import contextlib +from typing import Iterable + +import rustworkx as rx +from pydantic import BaseModel, ConfigDict + +from exo.shared.types.common import NodeId +from exo.shared.types.profiling import ConnectionProfile, NodePerformanceProfile +from exo.shared.types.topology import Connection, NodeInfo + + +class TopologySnapshot(BaseModel): + nodes: list[NodeInfo] + connections: list[Connection] + + model_config = ConfigDict(frozen=True, extra="forbid", strict=True) + + +class Topology: + def __init__(self) -> None: + self._graph: rx.PyDiGraph[NodeInfo, Connection] = rx.PyDiGraph() + self._node_id_to_rx_id_map: dict[NodeId, int] = dict() + self._rx_id_to_node_id_map: dict[int, NodeId] = dict() + self._edge_id_to_rx_id_map: dict[Connection, int] = dict() + + def to_snapshot(self) -> TopologySnapshot: + return TopologySnapshot( + nodes=list(self.list_nodes()), + connections=list(self.list_connections()), + ) + + @classmethod + def from_snapshot(cls, snapshot: TopologySnapshot) -> "Topology": + topology = cls() + + for node in snapshot.nodes: + with contextlib.suppress(ValueError): + topology.add_node(node) + + for connection in snapshot.connections: + topology.add_connection(connection) + + return topology + + def add_node(self, node: NodeInfo) -> None: + if node.node_id in self._node_id_to_rx_id_map: + return + rx_id = self._graph.add_node(node) + self._node_id_to_rx_id_map[node.node_id] = rx_id + self._rx_id_to_node_id_map[rx_id] = node.node_id + + def node_is_leaf(self, node_id: NodeId) -> bool: + return ( + node_id in self._node_id_to_rx_id_map + and len(self._graph.neighbors(self._node_id_to_rx_id_map[node_id])) == 1 + ) + + def neighbours(self, node_id: NodeId) -> list[NodeId]: + return [ + self._rx_id_to_node_id_map[rx_id] + for rx_id in self._graph.neighbors(self._node_id_to_rx_id_map[node_id]) + ] + + def out_edges(self, node_id: NodeId) -> list[tuple[NodeId, Connection]]: + if node_id not in self._node_id_to_rx_id_map: + return [] + return [ + (self._rx_id_to_node_id_map[nid], conn) + for _, nid, conn in self._graph.out_edges( + self._node_id_to_rx_id_map[node_id] + ) + ] + + def contains_node(self, node_id: NodeId) -> bool: + return node_id in self._node_id_to_rx_id_map + + def contains_connection(self, connection: Connection) -> bool: + return connection in self._edge_id_to_rx_id_map + + def add_connection( + self, + connection: Connection, + ) -> None: + if connection.local_node_id not in self._node_id_to_rx_id_map: + self.add_node(NodeInfo(node_id=connection.local_node_id)) + if connection.send_back_node_id not in self._node_id_to_rx_id_map: + self.add_node(NodeInfo(node_id=connection.send_back_node_id)) + + if connection in self._edge_id_to_rx_id_map: + return + + src_id = self._node_id_to_rx_id_map[connection.local_node_id] + sink_id = self._node_id_to_rx_id_map[connection.send_back_node_id] + + rx_id = self._graph.add_edge(src_id, sink_id, connection) + self._edge_id_to_rx_id_map[connection] = rx_id + + def list_nodes(self) -> Iterable[NodeInfo]: + return (self._graph[i] for i in self._graph.node_indices()) + + def list_connections(self) -> Iterable[Connection]: + return (connection for _, _, connection in self._graph.weighted_edge_list()) + + def get_node_profile(self, node_id: NodeId) -> NodePerformanceProfile | None: + try: + rx_idx = self._node_id_to_rx_id_map[node_id] + return self._graph.get_node_data(rx_idx).node_profile + except KeyError: + return None + + def update_node_profile( + self, node_id: NodeId, node_profile: NodePerformanceProfile + ) -> None: + rx_idx = self._node_id_to_rx_id_map[node_id] + self._graph[rx_idx].node_profile = node_profile + + def update_connection_profile(self, connection: Connection) -> None: + rx_idx = self._edge_id_to_rx_id_map[connection] + self._graph.update_edge_by_index(rx_idx, connection) + + def get_connection_profile( + self, connection: Connection + ) -> ConnectionProfile | None: + try: + rx_idx = self._edge_id_to_rx_id_map[connection] + return self._graph.get_edge_data_by_index(rx_idx).connection_profile + except KeyError: + return None + + def remove_node(self, node_id: NodeId) -> None: + if node_id not in self._node_id_to_rx_id_map: + return + + for connection in self.list_connections(): + if ( + connection.local_node_id == node_id + or connection.send_back_node_id == node_id + ): + self.remove_connection(connection) + + rx_idx = self._node_id_to_rx_id_map[node_id] + self._graph.remove_node(rx_idx) + + del self._node_id_to_rx_id_map[node_id] + del self._rx_id_to_node_id_map[rx_idx] + + def remove_connection(self, connection: Connection) -> None: + if connection not in self._edge_id_to_rx_id_map: + return + rx_idx = self._edge_id_to_rx_id_map[connection] + self._graph.remove_edge_from_index(rx_idx) + del self._edge_id_to_rx_id_map[connection] + + def get_cycles(self) -> list[list[NodeInfo]]: + cycle_idxs = rx.simple_cycles(self._graph) + cycles: list[list[NodeInfo]] = [] + for cycle_idx in cycle_idxs: + cycle = [self._graph[idx] for idx in cycle_idx] + cycles.append(cycle) + + return cycles + + def get_cycles_tb(self) -> list[list[NodeInfo]]: + tb_edges = [ + (u, v, conn) + for u, v, conn in self._graph.weighted_edge_list() + if conn.is_thunderbolt() + ] + + tb_graph: rx.PyDiGraph[NodeInfo, Connection] = rx.PyDiGraph() + tb_graph.add_nodes_from(self._graph.nodes()) + + for u, v, conn in tb_edges: + tb_graph.add_edge(u, v, conn) + + cycle_idxs = rx.simple_cycles(tb_graph) + cycles: list[list[NodeInfo]] = [] + for cycle_idx in cycle_idxs: + cycle = [tb_graph[idx] for idx in cycle_idx] + cycles.append(cycle) + + return cycles + + def get_subgraph_from_nodes(self, nodes: list[NodeInfo]) -> "Topology": + node_idxs = [node.node_id for node in nodes] + rx_idxs = [self._node_id_to_rx_id_map[idx] for idx in node_idxs] + topology = Topology() + for rx_idx in rx_idxs: + topology.add_node(self._graph[rx_idx]) + for connection in self.list_connections(): + if ( + connection.local_node_id in node_idxs + and connection.send_back_node_id in node_idxs + ): + topology.add_connection(connection) + return topology + + def is_thunderbolt_cycle(self, cycle: list[NodeInfo]) -> bool: + node_idxs = [node.node_id for node in cycle] + rx_idxs = [self._node_id_to_rx_id_map[idx] for idx in node_idxs] + for rid in rx_idxs: + for neighbor_rid in self._graph.neighbors(rid): + if neighbor_rid not in rx_idxs: + continue + has_tb = False + for edge in self._graph.get_all_edge_data(rid, neighbor_rid): + if edge.is_thunderbolt(): + has_tb = True + break + if not has_tb: + return False + return True diff --git a/src/exo/shared/types/__init__.py b/src/exo/shared/types/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/shared/types/api.py b/src/exo/shared/types/api.py new file mode 100644 index 00000000..30b01e3e --- /dev/null +++ b/src/exo/shared/types/api.py @@ -0,0 +1,182 @@ +import time +from typing import Any, Literal + +from pydantic import BaseModel, Field, field_validator +from pydantic_core import PydanticUseDefault + +from exo.shared.types.common import CommandId +from exo.shared.types.models import ModelId +from exo.shared.types.worker.instances import Instance, InstanceId, InstanceMeta +from exo.shared.types.worker.shards import Sharding + +FinishReason = Literal[ + "stop", "length", "tool_calls", "content_filter", "function_call" +] + + +class ModelListModel(BaseModel): + id: str + object: str = "model" + created: int = Field(default_factory=lambda: int(time.time())) + owned_by: str = "exo" + # openwebui fields + hugging_face_id: str = Field(default="") + name: str = Field(default="") + description: str = Field(default="") + context_length: int = Field(default=0) + tags: list[str] = Field(default=[]) + storage_size_megabytes: int = Field(default=0) + supports_tensor: bool = Field(default=False) + + +class ModelList(BaseModel): + object: Literal["list"] = "list" + data: list[ModelListModel] + + +class ChatCompletionMessageText(BaseModel): + type: Literal["text"] = "text" + text: str + + +class ChatCompletionMessage(BaseModel): + role: Literal["system", "user", "assistant", "developer", "tool", "function"] + content: ( + str | ChatCompletionMessageText | list[ChatCompletionMessageText] | None + ) = None + thinking: str | None = None # Added for GPT-OSS harmony format support + name: str | None = None + tool_calls: list[dict[str, Any]] | None = None + tool_call_id: str | None = None + function_call: dict[str, Any] | None = None + + +class TopLogprobItem(BaseModel): + token: str + logprob: float + bytes: list[int] | None = None + + +class LogprobsContentItem(BaseModel): + token: str + logprob: float + bytes: list[int] | None = None + top_logprobs: list[TopLogprobItem] + + +class Logprobs(BaseModel): + content: list[LogprobsContentItem] | None = None + + +class PromptTokensDetails(BaseModel): + cached_tokens: int = 0 + audio_tokens: int = 0 + + +class CompletionTokensDetails(BaseModel): + reasoning_tokens: int = 0 + audio_tokens: int = 0 + accepted_prediction_tokens: int = 0 + rejected_prediction_tokens: int = 0 + + +class Usage(BaseModel): + prompt_tokens: int + completion_tokens: int + total_tokens: int + prompt_tokens_details: PromptTokensDetails | None = None + completion_tokens_details: CompletionTokensDetails | None = None + + +class StreamingChoiceResponse(BaseModel): + index: int + delta: ChatCompletionMessage + logprobs: Logprobs | None = None + finish_reason: FinishReason | None = None + usage: Usage | None = None + + +class ChatCompletionChoice(BaseModel): + index: int + message: ChatCompletionMessage + logprobs: Logprobs | None = None + finish_reason: FinishReason | None = None + + +class ChatCompletionResponse(BaseModel): + id: str + object: Literal["chat.completion"] = "chat.completion" + created: int + model: str + choices: list[ChatCompletionChoice | StreamingChoiceResponse] + usage: Usage | None = None + service_tier: str | None = None + + +class ChatCompletionTaskParams(BaseModel): + model: str + frequency_penalty: float | None = None + messages: list[ChatCompletionMessage] + logit_bias: dict[str, int] | None = None + logprobs: bool | None = None + top_logprobs: int | None = None + max_tokens: int | None = None + n: int | None = None + presence_penalty: float | None = None + response_format: dict[str, Any] | None = None + seed: int | None = None + stop: str | list[str] | None = None + stream: bool = False + temperature: float | None = None + top_p: float | None = None + tools: list[dict[str, Any]] | None = None + tool_choice: str | dict[str, Any] | None = None + parallel_tool_calls: bool | None = None + user: str | None = None + + +class PlaceInstanceParams(BaseModel): + model_id: str + sharding: Sharding = Sharding.Pipeline + instance_meta: InstanceMeta = InstanceMeta.MlxRing + min_nodes: int = 1 + + @field_validator("sharding", "instance_meta", mode="plain") + @classmethod + def use_default(cls, v: object): + if not v or not isinstance(v, (Sharding, InstanceMeta)): + raise PydanticUseDefault() + return v + + +class CreateInstanceParams(BaseModel): + instance: Instance + + +class PlacementPreview(BaseModel): + model_id: ModelId + sharding: Sharding + instance_meta: InstanceMeta + instance: Instance | None = None + # Keys are NodeId strings, values are additional bytes that would be used on that node + memory_delta_by_node: dict[str, int] | None = None + error: str | None = None + + +class PlacementPreviewResponse(BaseModel): + previews: list[PlacementPreview] + + +class DeleteInstanceTaskParams(BaseModel): + instance_id: str + + +class CreateInstanceResponse(BaseModel): + message: str + command_id: CommandId + + +class DeleteInstanceResponse(BaseModel): + message: str + command_id: CommandId + instance_id: InstanceId diff --git a/src/exo/shared/types/chunks.py b/src/exo/shared/types/chunks.py new file mode 100644 index 00000000..ac90d20c --- /dev/null +++ b/src/exo/shared/types/chunks.py @@ -0,0 +1,29 @@ +from enum import Enum + +from exo.utils.pydantic_ext import TaggedModel + +from .api import FinishReason +from .models import ModelId + + +class ChunkType(str, Enum): + Token = "Token" + Image = "Image" + + +class BaseChunk(TaggedModel): + idx: int + model: ModelId + + +class TokenChunk(BaseChunk): + text: str + token_id: int + finish_reason: FinishReason | None = None + + +class ImageChunk(BaseChunk): + data: bytes + + +GenerationChunk = TokenChunk | ImageChunk diff --git a/src/exo/shared/types/commands.py b/src/exo/shared/types/commands.py new file mode 100644 index 00000000..5d8a5026 --- /dev/null +++ b/src/exo/shared/types/commands.py @@ -0,0 +1,59 @@ +from pydantic import Field + +from exo.shared.types.api import ChatCompletionTaskParams +from exo.shared.types.common import CommandId, NodeId +from exo.shared.types.models import ModelMetadata +from exo.shared.types.worker.instances import Instance, InstanceId, InstanceMeta +from exo.shared.types.worker.shards import Sharding +from exo.utils.pydantic_ext import CamelCaseModel, TaggedModel + + +class BaseCommand(TaggedModel): + command_id: CommandId = Field(default_factory=CommandId) + + +class TestCommand(BaseCommand): + __test__ = False + + +class ChatCompletion(BaseCommand): + request_params: ChatCompletionTaskParams + + +class PlaceInstance(BaseCommand): + model_meta: ModelMetadata + sharding: Sharding + instance_meta: InstanceMeta + min_nodes: int + + +class CreateInstance(BaseCommand): + instance: Instance + + +class DeleteInstance(BaseCommand): + instance_id: InstanceId + + +class TaskFinished(BaseCommand): + finished_command_id: CommandId + + +class RequestEventLog(BaseCommand): + since_idx: int + + +Command = ( + TestCommand + | RequestEventLog + | ChatCompletion + | PlaceInstance + | CreateInstance + | DeleteInstance + | TaskFinished +) + + +class ForwarderCommand(CamelCaseModel): + origin: NodeId + command: Command diff --git a/src/exo/shared/types/common.py b/src/exo/shared/types/common.py new file mode 100644 index 00000000..42b682dc --- /dev/null +++ b/src/exo/shared/types/common.py @@ -0,0 +1,47 @@ +from typing import Self +from uuid import uuid4 + +from pydantic import GetCoreSchemaHandler, field_validator +from pydantic_core import core_schema + +from exo.utils.pydantic_ext import CamelCaseModel + + +class Id(str): + def __new__(cls, value: str | None = None) -> Self: + return super().__new__(cls, value or str(uuid4())) + + @classmethod + def __get_pydantic_core_schema__( + cls, _source: type, handler: GetCoreSchemaHandler + ) -> core_schema.CoreSchema: + # Just use a plain string schema + return core_schema.str_schema() + + +class NodeId(Id): + pass + + +class SessionId(CamelCaseModel): + master_node_id: NodeId + election_clock: int + + +class CommandId(Id): + pass + + +class Host(CamelCaseModel): + ip: str + port: int + + def __str__(self) -> str: + return f"{self.ip}:{self.port}" + + @field_validator("port") + @classmethod + def check_port(cls, v: int) -> int: + if not (0 <= v <= 65535): + raise ValueError("Port must be between 0 and 65535") + return v diff --git a/src/exo/shared/types/events.py b/src/exo/shared/types/events.py new file mode 100644 index 00000000..29b750ef --- /dev/null +++ b/src/exo/shared/types/events.py @@ -0,0 +1,152 @@ +from datetime import datetime + +from pydantic import Field + +from exo.shared.topology import Connection, NodePerformanceProfile +from exo.shared.types.chunks import GenerationChunk +from exo.shared.types.common import CommandId, Id, NodeId, SessionId +from exo.shared.types.profiling import MemoryPerformanceProfile +from exo.shared.types.tasks import Task, TaskId, TaskStatus +from exo.shared.types.worker.downloads import DownloadProgress +from exo.shared.types.worker.instances import Instance, InstanceId +from exo.shared.types.worker.runners import RunnerId, RunnerStatus +from exo.utils.pydantic_ext import CamelCaseModel, TaggedModel + + +class EventId(Id): + """ + Newtype around `ID` + """ + + +class BaseEvent(TaggedModel): + event_id: EventId = Field(default_factory=EventId) + # Internal, for debugging. Please don't rely on this field for anything! + _master_time_stamp: None | datetime = None + + +class TestEvent(BaseEvent): + __test__ = False + + +class TaskCreated(BaseEvent): + task_id: TaskId + task: Task + + +class TaskAcknowledged(BaseEvent): + task_id: TaskId + + +class TaskDeleted(BaseEvent): + task_id: TaskId + + +class TaskStatusUpdated(BaseEvent): + task_id: TaskId + task_status: TaskStatus + + +class TaskFailed(BaseEvent): + task_id: TaskId + error_type: str + error_message: str + + +class InstanceCreated(BaseEvent): + instance: Instance + + def __eq__(self, other: object) -> bool: + if isinstance(other, InstanceCreated): + return self.instance == other.instance and self.event_id == other.event_id + + return False + + +class InstanceDeleted(BaseEvent): + instance_id: InstanceId + + +class RunnerStatusUpdated(BaseEvent): + runner_id: RunnerId + runner_status: RunnerStatus + + +class RunnerDeleted(BaseEvent): + runner_id: RunnerId + + +# TODO +class NodeCreated(BaseEvent): + node_id: NodeId + + +class NodeTimedOut(BaseEvent): + node_id: NodeId + + +class NodePerformanceMeasured(BaseEvent): + node_id: NodeId + when: str # this is a manually cast datetime overrode by the master when the event is indexed, rather than the local time on the device + node_profile: NodePerformanceProfile + + +class NodeMemoryMeasured(BaseEvent): + node_id: NodeId + when: str # this is a manually cast datetime overrode by the master when the event is indexed, rather than the local time on the device + memory: MemoryPerformanceProfile + + +class NodeDownloadProgress(BaseEvent): + download_progress: DownloadProgress + + +class ChunkGenerated(BaseEvent): + command_id: CommandId + chunk: GenerationChunk + + +class TopologyEdgeCreated(BaseEvent): + edge: Connection + + +class TopologyEdgeDeleted(BaseEvent): + edge: Connection + + +Event = ( + TestEvent + | TaskCreated + | TaskStatusUpdated + | TaskFailed + | TaskDeleted + | TaskAcknowledged + | InstanceCreated + | InstanceDeleted + | RunnerStatusUpdated + | RunnerDeleted + | NodeCreated + | NodeTimedOut + | NodePerformanceMeasured + | NodeMemoryMeasured + | NodeDownloadProgress + | ChunkGenerated + | TopologyEdgeCreated + | TopologyEdgeDeleted +) + + +class IndexedEvent(CamelCaseModel): + """An event indexed by the master, with a globally unique index""" + + idx: int = Field(ge=0) + event: Event + + +class ForwarderEvent(CamelCaseModel): + """An event the forwarder will serialize and send over the network""" + + origin_idx: int = Field(ge=0) + origin: NodeId + session: SessionId + event: Event diff --git a/src/exo/shared/types/memory.py b/src/exo/shared/types/memory.py new file mode 100644 index 00000000..b97fb345 --- /dev/null +++ b/src/exo/shared/types/memory.py @@ -0,0 +1,73 @@ +from math import ceil +from typing import Self + +from exo.utils.pydantic_ext import CamelCaseModel + + +class Memory(CamelCaseModel): + in_bytes: int = 0 + + @classmethod + def from_bytes(cls, val: int) -> Self: + """Construct a new Memory object from a number of bytes""" + return cls(in_bytes=val) + + @property + def in_kb(self) -> int: + """The approximate kilobytes this memory represents, rounded up. Setting this property rounds to the nearest byte.""" + return ceil(self.in_bytes / 1024) + + @in_kb.setter + def in_kb(self, val: int): + """Set this memorys value in kilobytes.""" + self.in_bytes = val * 1024 + + @classmethod + def from_kb(cls, val: int) -> Self: + """Construct a new Memory object from a number of kilobytes""" + return cls(in_bytes=val * 1024) + + @classmethod + def from_float_kb(cls, val: float) -> Self: + """Construct a new Memory object from a number of kilobytes, rounding where appropriate""" + return cls(in_bytes=round(val * 1024)) + + @property + def in_mb(self) -> float: + """The approximate megabytes this memory represents. Setting this property rounds to the nearest byte.""" + return self.in_bytes / (1024**2) + + @in_mb.setter + def in_mb(self, val: float): + """Set the megabytes for this memory, rounded to the nearest byte.""" + self.in_bytes = round(val * (1024**2)) + + @classmethod + def from_mb(cls, val: float) -> Self: + """Construct a new Memory object from a number of megabytes""" + return cls(in_bytes=round(val * (1024**2))) + + @classmethod + def from_gb(cls, val: float) -> Self: + """Construct a new Memory object from a number of megabytes""" + return cls(in_bytes=round(val * (1024**3))) + + @property + def in_gb(self) -> float: + """The approximate gigabytes this memory represents.""" + return self.in_bytes / (1024**3) + + def __add__(self, other: "Memory") -> "Memory": + return Memory.from_bytes(self.in_bytes + other.in_bytes) + + def __lt__(self, other: Self) -> bool: + return self.in_bytes < other.in_bytes + + def __le__(self, other: Self) -> bool: + return self.in_bytes <= other.in_bytes + + def __gt__(self, other: Self) -> bool: + return self.in_bytes > other.in_bytes + + def __ge__(self, other: Self) -> bool: + return self.in_bytes >= other.in_bytes diff --git a/src/exo/shared/types/models.py b/src/exo/shared/types/models.py new file mode 100644 index 00000000..b029fba0 --- /dev/null +++ b/src/exo/shared/types/models.py @@ -0,0 +1,16 @@ +from pydantic import PositiveInt + +from exo.shared.types.common import Id +from exo.shared.types.memory import Memory +from exo.utils.pydantic_ext import CamelCaseModel + + +class ModelId(Id): + pass + + +class ModelMetadata(CamelCaseModel): + model_id: ModelId + pretty_name: str + storage_size: Memory + n_layers: PositiveInt diff --git a/src/exo/shared/types/multiaddr.py b/src/exo/shared/types/multiaddr.py new file mode 100644 index 00000000..769e920d --- /dev/null +++ b/src/exo/shared/types/multiaddr.py @@ -0,0 +1,68 @@ +import re +from typing import ClassVar + +from pydantic import BaseModel, computed_field, field_validator + + +class Multiaddr(BaseModel): + address: str + + PATTERNS: ClassVar[list[str]] = [ + r"^/ip4/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(/tcp/(\d{1,5}))?(/p2p/[A-Za-z0-9]+)?$", + r"^/ip6/([0-9a-fA-F:]+)(/tcp/(\d{1,5}))?(/p2p/[A-Za-z0-9]+)?$", + r"^/dns[46]?/([a-zA-Z0-9.-]+)(/tcp/(\d{1,5}))?(/p2p/[A-Za-z0-9]+)?$", + ] + + @field_validator("address") + @classmethod + def validate_format(cls, v: str) -> str: + if not any(re.match(pattern, v) for pattern in cls.PATTERNS): + raise ValueError( + f"Invalid multiaddr format: {v}. " + "Expected format like /ip4/127.0.0.1/tcp/4001 or /dns/example.com/tcp/443" + ) + return v + + @computed_field + @property + def address_type(self) -> str: + for pattern in self.PATTERNS: + if re.match(pattern, self.address): + return pattern.split("/")[1] + raise ValueError(f"Invalid multiaddr format: {self.address}") + + @property + def ipv6_address(self) -> str: + match = re.match(r"^/ip6/([0-9a-fA-F:]+)", self.address) + if not match: + raise ValueError( + f"Invalid multiaddr format: {self.address}. Expected format like /ip6/::1/tcp/4001" + ) + return match.group(1) + + @property + def ipv4_address(self) -> str: + match = re.match(r"^/ip4/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", self.address) + if not match: + raise ValueError( + f"Invalid multiaddr format: {self.address}. Expected format like /ip4/127.0.0.1/tcp/4001" + ) + return match.group(1) + + @computed_field + @property + def ip_address(self) -> str: + return self.ipv4_address if self.address_type == "ip4" else self.ipv6_address + + @computed_field + @property + def port(self) -> int: + match = re.search(r"/tcp/(\d{1,5})", self.address) + if not match: + raise ValueError( + f"Invalid multiaddr format: {self.address}. Expected format like /ip4/127.0.0.1/tcp/4001" + ) + return int(match.group(1)) + + def __str__(self) -> str: + return self.address diff --git a/src/exo/shared/types/profiling.py b/src/exo/shared/types/profiling.py new file mode 100644 index 00000000..5ed6e0d4 --- /dev/null +++ b/src/exo/shared/types/profiling.py @@ -0,0 +1,67 @@ +from typing import Self + +import psutil + +from exo.shared.types.memory import Memory +from exo.utils.pydantic_ext import CamelCaseModel + + +class MemoryPerformanceProfile(CamelCaseModel): + ram_total: Memory + ram_available: Memory + swap_total: Memory + swap_available: Memory + + @classmethod + def from_bytes( + cls, *, ram_total: int, ram_available: int, swap_total: int, swap_available: int + ) -> Self: + return cls( + ram_total=Memory.from_bytes(ram_total), + ram_available=Memory.from_bytes(ram_available), + swap_total=Memory.from_bytes(swap_total), + swap_available=Memory.from_bytes(swap_available), + ) + + @classmethod + def from_psutil(cls, *, override_memory: int | None) -> Self: + vm = psutil.virtual_memory() + sm = psutil.swap_memory() + + return cls.from_bytes( + ram_total=vm.total, + ram_available=vm.available if override_memory is None else override_memory, + swap_total=sm.total, + swap_available=sm.free, + ) + + +class SystemPerformanceProfile(CamelCaseModel): + # TODO: flops_fp16: float + + gpu_usage: float = 0.0 + temp: float = 0.0 + sys_power: float = 0.0 + pcpu_usage: float = 0.0 + ecpu_usage: float = 0.0 + ane_power: float = 0.0 + + +class NetworkInterfaceInfo(CamelCaseModel): + name: str + ip_address: str + + +class NodePerformanceProfile(CamelCaseModel): + model_id: str + chip_id: str + friendly_name: str + memory: MemoryPerformanceProfile + network_interfaces: list[NetworkInterfaceInfo] = [] + system: SystemPerformanceProfile + + +class ConnectionProfile(CamelCaseModel): + throughput: float + latency: float + jitter: float diff --git a/src/exo/shared/types/state.py b/src/exo/shared/types/state.py new file mode 100644 index 00000000..58b14d2e --- /dev/null +++ b/src/exo/shared/types/state.py @@ -0,0 +1,63 @@ +from collections.abc import Mapping, Sequence +from datetime import datetime +from typing import Any, cast + +from pydantic import ConfigDict, Field, field_serializer, field_validator +from pydantic.alias_generators import to_camel + +from exo.shared.topology import Topology, TopologySnapshot +from exo.shared.types.common import NodeId +from exo.shared.types.profiling import NodePerformanceProfile +from exo.shared.types.tasks import Task, TaskId +from exo.shared.types.worker.downloads import DownloadProgress +from exo.shared.types.worker.instances import Instance, InstanceId +from exo.shared.types.worker.runners import RunnerId, RunnerStatus +from exo.utils.pydantic_ext import CamelCaseModel + + +class State(CamelCaseModel): + """Global system state. + + The :class:`Topology` instance is encoded/decoded via an immutable + :class:`~shared.topology.TopologySnapshot` to ensure compatibility with + standard JSON serialisation. + """ + + model_config = ConfigDict( + alias_generator=to_camel, + validate_by_name=True, + extra="forbid", + # I want to reenable this ASAP, but it's causing an issue with TaskStatus + strict=True, + arbitrary_types_allowed=True, + ) + instances: Mapping[InstanceId, Instance] = {} + runners: Mapping[RunnerId, RunnerStatus] = {} + downloads: Mapping[NodeId, Sequence[DownloadProgress]] = {} + tasks: Mapping[TaskId, Task] = {} + node_profiles: Mapping[NodeId, NodePerformanceProfile] = {} + last_seen: Mapping[NodeId, datetime] = {} + topology: Topology = Field(default_factory=Topology) + last_event_applied_idx: int = Field(default=-1, ge=-1) + + @field_serializer("topology", mode="plain") + def _encode_topology(self, value: Topology) -> TopologySnapshot: + return value.to_snapshot() + + @field_validator("topology", mode="before") + @classmethod + def _deserialize_topology(cls, value: object) -> Topology: # noqa: D401 – Pydantic validator signature + """Convert an incoming *value* into a :class:`Topology` instance. + + Accepts either an already constructed :class:`Topology` or a mapping + representing :class:`~shared.topology.TopologySnapshot`. + """ + + if isinstance(value, Topology): + return value + + if isinstance(value, Mapping): # likely a snapshot-dict coming from JSON + snapshot = TopologySnapshot(**cast(dict[str, Any], value)) # type: ignore[arg-type] + return Topology.from_snapshot(snapshot) + + raise TypeError("Invalid representation for Topology field in State") diff --git a/src/exo/shared/types/tasks.py b/src/exo/shared/types/tasks.py new file mode 100644 index 00000000..4951bc4a --- /dev/null +++ b/src/exo/shared/types/tasks.py @@ -0,0 +1,61 @@ +from enum import Enum + +from pydantic import Field + +from exo.shared.types.api import ChatCompletionTaskParams +from exo.shared.types.common import CommandId, Id +from exo.shared.types.worker.instances import BoundInstance, InstanceId +from exo.shared.types.worker.runners import RunnerId +from exo.shared.types.worker.shards import ShardMetadata +from exo.utils.pydantic_ext import TaggedModel + + +class TaskId(Id): + pass + + +class TaskStatus(str, Enum): + Pending = "Pending" + Running = "Running" + Complete = "Complete" + TimedOut = "TimedOut" + Failed = "Failed" + + +class BaseTask(TaggedModel): + task_id: TaskId = Field(default_factory=TaskId) + task_status: TaskStatus = Field(default=TaskStatus.Pending) + instance_id: InstanceId + + +class CreateRunner(BaseTask): # emitted by Worker + bound_instance: BoundInstance + + +class DownloadModel(BaseTask): # emitted by Worker + shard_metadata: ShardMetadata + + +class LoadModel(BaseTask): # emitted by Worker + pass + + +class StartWarmup(BaseTask): # emitted by Worker + pass + + +class ChatCompletion(BaseTask): # emitted by Master + command_id: CommandId + task_params: ChatCompletionTaskParams + + error_type: str | None = Field(default=None) + error_message: str | None = Field(default=None) + + +class Shutdown(BaseTask): # emitted by Worker + runner_id: RunnerId + + +Task = ( + CreateRunner | DownloadModel | LoadModel | StartWarmup | ChatCompletion | Shutdown +) diff --git a/src/exo/shared/types/topology.py b/src/exo/shared/types/topology.py new file mode 100644 index 00000000..0df83510 --- /dev/null +++ b/src/exo/shared/types/topology.py @@ -0,0 +1,37 @@ +from exo.shared.types.common import NodeId +from exo.shared.types.multiaddr import Multiaddr +from exo.shared.types.profiling import ConnectionProfile, NodePerformanceProfile +from exo.utils.pydantic_ext import CamelCaseModel + + +class NodeInfo(CamelCaseModel): + node_id: NodeId + node_profile: NodePerformanceProfile | None = None + + +class Connection(CamelCaseModel): + local_node_id: NodeId + send_back_node_id: NodeId + send_back_multiaddr: Multiaddr + connection_profile: ConnectionProfile | None = None + + def __hash__(self) -> int: + return hash( + ( + self.local_node_id, + self.send_back_node_id, + self.send_back_multiaddr.address, + ) + ) + + def __eq__(self, other: object) -> bool: + if not isinstance(other, Connection): + raise ValueError("Cannot compare Connection with non-Connection") + return ( + self.local_node_id == other.local_node_id + and self.send_back_node_id == other.send_back_node_id + and self.send_back_multiaddr == other.send_back_multiaddr + ) + + def is_thunderbolt(self) -> bool: + return str(self.send_back_multiaddr.ipv4_address).startswith("169.254") diff --git a/src/exo/shared/types/worker/downloads.py b/src/exo/shared/types/worker/downloads.py new file mode 100644 index 00000000..73255f62 --- /dev/null +++ b/src/exo/shared/types/worker/downloads.py @@ -0,0 +1,44 @@ +from exo.shared.types.common import NodeId +from exo.shared.types.memory import Memory +from exo.shared.types.worker.shards import ShardMetadata +from exo.utils.pydantic_ext import CamelCaseModel, TaggedModel + + +class DownloadProgressData(CamelCaseModel): + total_bytes: Memory + downloaded_bytes: Memory + downloaded_bytes_this_session: Memory + + completed_files: int + total_files: int + + speed: float + eta_ms: int + + files: dict[str, "DownloadProgressData"] + + +class BaseDownloadProgress(TaggedModel): + node_id: NodeId + shard_metadata: ShardMetadata + + +class DownloadPending(BaseDownloadProgress): + pass + + +class DownloadCompleted(BaseDownloadProgress): + pass + + +class DownloadFailed(BaseDownloadProgress): + error_message: str + + +class DownloadOngoing(BaseDownloadProgress): + download_progress: DownloadProgressData + + +DownloadProgress = ( + DownloadPending | DownloadCompleted | DownloadFailed | DownloadOngoing +) diff --git a/src/exo/shared/types/worker/instances.py b/src/exo/shared/types/worker/instances.py new file mode 100644 index 00000000..ea8e7887 --- /dev/null +++ b/src/exo/shared/types/worker/instances.py @@ -0,0 +1,58 @@ +from enum import Enum + +from pydantic import model_validator + +from exo.shared.types.common import Host, Id, NodeId +from exo.shared.types.worker.runners import RunnerId, ShardAssignments, ShardMetadata +from exo.utils.pydantic_ext import CamelCaseModel, TaggedModel + + +class InstanceId(Id): + pass + + +class InstanceMeta(str, Enum): + MlxRing = "MlxRing" + MlxJaccl = "MlxJaccl" + + +class BaseInstance(TaggedModel): + instance_id: InstanceId + shard_assignments: ShardAssignments + + def shard(self, runner_id: RunnerId) -> ShardMetadata | None: + return self.shard_assignments.runner_to_shard.get(runner_id, None) + + +class MlxRingInstance(BaseInstance): + hosts: list[Host] + + +class MlxJacclInstance(BaseInstance): + ibv_devices: list[list[str | None]] + ibv_coordinators: dict[NodeId, str] + + +# TODO: Single node instance +Instance = MlxRingInstance | MlxJacclInstance + + +class BoundInstance(CamelCaseModel): + instance: Instance + bound_runner_id: RunnerId + bound_node_id: NodeId + + @property + def bound_shard(self) -> ShardMetadata: + shard = self.instance.shard(self.bound_runner_id) + assert shard is not None + return shard + + @model_validator(mode="after") + def validate_shard_exists(self) -> "BoundInstance": + assert ( + self.bound_runner_id in self.instance.shard_assignments.runner_to_shard + ), ( + "Bound Instance must be constructed with a runner_id that is in the instances assigned shards" + ) + return self diff --git a/src/exo/shared/types/worker/resource_monitor.py b/src/exo/shared/types/worker/resource_monitor.py new file mode 100644 index 00000000..b351963c --- /dev/null +++ b/src/exo/shared/types/worker/resource_monitor.py @@ -0,0 +1,43 @@ +import asyncio +from abc import ABC, abstractmethod +from collections.abc import Coroutine +from typing import Callable + +from exo.shared.types.profiling import ( + MemoryPerformanceProfile, + SystemPerformanceProfile, +) + + +class ResourceCollector(ABC): + @abstractmethod + async def collect(self) -> SystemPerformanceProfile | MemoryPerformanceProfile: ... + + +class SystemResourceCollector(ResourceCollector): + async def collect(self) -> SystemPerformanceProfile: ... + + +class MemoryResourceCollector(ResourceCollector): + async def collect(self) -> MemoryPerformanceProfile: ... + + +class ResourceMonitor: + data_collectors: list[ResourceCollector] + effect_handlers: set[ + Callable[[SystemPerformanceProfile | MemoryPerformanceProfile], None] + ] + + async def _collect( + self, + ) -> list[SystemPerformanceProfile | MemoryPerformanceProfile]: + tasks: list[ + Coroutine[None, None, SystemPerformanceProfile | MemoryPerformanceProfile] + ] = [collector.collect() for collector in self.data_collectors] + return await asyncio.gather(*tasks) + + async def collect(self) -> None: + profiles = await self._collect() + for profile in profiles: + for effect_handler in self.effect_handlers: + effect_handler(profile) diff --git a/src/exo/shared/types/worker/runner_response.py b/src/exo/shared/types/worker/runner_response.py new file mode 100644 index 00000000..8c2d3754 --- /dev/null +++ b/src/exo/shared/types/worker/runner_response.py @@ -0,0 +1,21 @@ +from exo.shared.types.api import FinishReason +from exo.utils.pydantic_ext import TaggedModel + + +class BaseRunnerResponse(TaggedModel): + pass + + +class TokenizedResponse(BaseRunnerResponse): + prompt_tokens: int + + +class GenerationResponse(BaseRunnerResponse): + text: str + token: int + # logprobs: list[float] | None = None # too big. we can change to be top-k + finish_reason: FinishReason | None = None + + +class FinishedResponse(BaseRunnerResponse): + pass diff --git a/src/exo/shared/types/worker/runners.py b/src/exo/shared/types/worker/runners.py new file mode 100644 index 00000000..5cceb83b --- /dev/null +++ b/src/exo/shared/types/worker/runners.py @@ -0,0 +1,80 @@ +from collections.abc import Mapping + +from pydantic import model_validator + +from exo.shared.types.common import Id, NodeId +from exo.shared.types.models import ModelId +from exo.shared.types.worker.shards import ShardMetadata +from exo.utils.pydantic_ext import CamelCaseModel, TaggedModel + + +class RunnerId(Id): + pass + + +class RunnerError(Exception): + pass + + +class BaseRunnerStatus(TaggedModel): + def is_running(self): + return isinstance(self, RunnerRunning) + + +class RunnerWaitingForModel(BaseRunnerStatus): + pass + + +class RunnerLoading(BaseRunnerStatus): + pass + + +class RunnerLoaded(BaseRunnerStatus): + pass + + +class RunnerWarmingUp(BaseRunnerStatus): + pass + + +class RunnerReady(BaseRunnerStatus): + pass + + +class RunnerRunning(BaseRunnerStatus): + pass + + +class RunnerShutdown(BaseRunnerStatus): + pass + + +class RunnerFailed(BaseRunnerStatus): + error_message: str | None = None + + +RunnerStatus = ( + RunnerWaitingForModel + | RunnerLoading + | RunnerLoaded + | RunnerWarmingUp + | RunnerReady + | RunnerRunning + | RunnerShutdown + | RunnerFailed +) + + +class ShardAssignments(CamelCaseModel): + model_id: ModelId + runner_to_shard: Mapping[RunnerId, ShardMetadata] + node_to_runner: Mapping[NodeId, RunnerId] + + @model_validator(mode="after") + def validate_runners_exist(self) -> "ShardAssignments": + for runner_id in self.node_to_runner.values(): + if runner_id not in self.runner_to_shard: + raise ValueError( + f"Runner {runner_id} in node_to_runner does not exist in runner_to_shard" + ) + return self diff --git a/src/exo/shared/types/worker/shards.py b/src/exo/shared/types/worker/shards.py new file mode 100644 index 00000000..e8e86730 --- /dev/null +++ b/src/exo/shared/types/worker/shards.py @@ -0,0 +1,67 @@ +from enum import Enum + +from pydantic import Field + +from exo.shared.types.models import ModelMetadata +from exo.utils.pydantic_ext import TaggedModel + + +class Sharding(str, Enum): + Tensor = "Tensor" + Pipeline = "Pipeline" + + +class BaseShardMetadata(TaggedModel): + """ + Defines a specific shard of the model that is ready to be run on a device. + Replaces previous `Shard` object. + """ + + model_meta: ModelMetadata + device_rank: int + world_size: int + + # Error handling; equivalent to monkey-patch, but we can't monkey-patch runner.py + # This is kinda annoying because it allocates memory in the ShardMetadata object. Can be rethought after Shanghai. + immediate_exception: bool = False + should_timeout: float | None = None + + start_layer: int = Field(ge=0) + end_layer: int = Field(ge=0) + n_layers: int = Field(ge=0) + + @property + def is_first_layer(self) -> bool: + return self.start_layer == 0 + + @property + def is_last_layer(self) -> bool: + return self.end_layer == self.n_layers + + def __hash__(self) -> int: + return hash( + ( + self.model_meta.model_id, + self.start_layer, + self.end_layer, + self.n_layers, + self.device_rank, + self.world_size, + ) + ) + + +class PipelineShardMetadata(BaseShardMetadata): + """ + Pipeline parallelism shard meta. + + Layers are represented as a half-open interval [start_layer, end_layer), + where start_layer is inclusive and end_layer is exclusive. + """ + + +class TensorShardMetadata(BaseShardMetadata): + pass + + +ShardMetadata = PipelineShardMetadata | TensorShardMetadata diff --git a/src/exo/utils/__init__.py b/src/exo/utils/__init__.py new file mode 100644 index 00000000..53679125 --- /dev/null +++ b/src/exo/utils/__init__.py @@ -0,0 +1,16 @@ +from typing import Any, Type + +from .phantom import PhantomData + + +def ensure_type[T](obj: Any, expected_type: Type[T]) -> T: # type: ignore + if not isinstance(obj, expected_type): + raise TypeError(f"Expected {expected_type}, got {type(obj)}") # type: ignore + return obj + + +def todo[T]( + msg: str = "This code has not been implemented yet.", + _phantom: PhantomData[T] = None, +) -> T: + raise NotImplementedError(msg) diff --git a/src/exo/utils/banner.py b/src/exo/utils/banner.py new file mode 100644 index 00000000..eb6d7b08 --- /dev/null +++ b/src/exo/utils/banner.py @@ -0,0 +1,30 @@ +def print_startup_banner(port: int) -> None: + """Print a prominent startup banner with API endpoint information.""" + dashboard_url = f"http://localhost:{port}" + banner = f""" +╔═══════════════════════════════════════════════════════════════════════╗ +║ ║ +║ ███████╗██╗ ██╗ ██████╗ ║ +║ ██╔════╝╚██╗██╔╝██╔═══██╗ ║ +║ █████╗ ╚███╔╝ ██║ ██║ ║ +║ ██╔══╝ ██╔██╗ ██║ ██║ ║ +║ ███████╗██╔╝ ██╗╚██████╔╝ ║ +║ ╚══════╝╚═╝ ╚═╝ ╚═════╝ ║ +║ ║ +║ Distributed AI Inference Cluster ║ +║ ║ +╚═══════════════════════════════════════════════════════════════════════╝ + +╔═══════════════════════════════════════════════════════════════════════╗ +║ ║ +║ 🌐 Dashboard & API Ready ║ +║ ║ +║ {dashboard_url}{" " * (69 - len(dashboard_url))}║ +║ ║ +║ Click the URL above to open the dashboard in your browser ║ +║ ║ +╚═══════════════════════════════════════════════════════════════════════╝ + +""" + + print(banner) diff --git a/src/exo/utils/channels.py b/src/exo/utils/channels.py new file mode 100644 index 00000000..3db08d6b --- /dev/null +++ b/src/exo/utils/channels.py @@ -0,0 +1,296 @@ +import multiprocessing as mp +from dataclasses import dataclass, field +from math import inf +from multiprocessing.synchronize import Event +from queue import Empty, Full +from types import TracebackType +from typing import Self + +from anyio import ( + CapacityLimiter, + ClosedResourceError, + EndOfStream, + WouldBlock, + to_thread, +) +from anyio.streams.memory import ( + MemoryObjectReceiveStream as AnyioReceiver, +) +from anyio.streams.memory import ( + MemoryObjectSendStream as AnyioSender, +) +from anyio.streams.memory import ( + MemoryObjectStreamState as AnyioState, +) + + +class Sender[T](AnyioSender[T]): + def clone(self) -> "Sender[T]": + if self._closed: + raise ClosedResourceError + return Sender(_state=self._state) + + def clone_receiver(self) -> "Receiver[T]": + """Constructs a Receiver using a Senders shared state - similar to calling Receiver.clone() without needing the receiver""" + if self._closed: + raise ClosedResourceError + return Receiver(_state=self._state) + + +class Receiver[T](AnyioReceiver[T]): + def clone(self) -> "Receiver[T]": + if self._closed: + raise ClosedResourceError + return Receiver(_state=self._state) + + def clone_sender(self) -> Sender[T]: + """Constructs a Sender using a Receivers shared state - similar to calling Sender.clone() without needing the sender""" + if self._closed: + raise ClosedResourceError + return Sender(_state=self._state) + + def collect(self) -> list[T]: + """Collect all currently available items from this receiver""" + out: list[T] = [] + while True: + try: + item = self.receive_nowait() + out.append(item) + except WouldBlock: + break + return out + + async def receive_at_least(self, n: int) -> list[T]: + out: list[T] = [] + out.append(await self.receive()) + out.extend(self.collect()) + while len(out) < n: + out.append(await self.receive()) + out.extend(self.collect()) + return out + + def __enter__(self) -> Self: + return self + + +class _MpEndOfStream: + pass + + +class MpState[T]: + def __init__(self, max_buffer_size: float): + if max_buffer_size == inf: + max_buffer_size = 0 + assert isinstance(max_buffer_size, int), ( + "State should only ever be constructed with an integer or math.inf size." + ) + + self.max_buffer_size: float = max_buffer_size + self.buffer: mp.Queue[T | _MpEndOfStream] = mp.Queue(max_buffer_size) + self.closed: Event = mp.Event() + + def __getstate__(self): + d = self.__dict__.copy() + d.pop("__orig_class__", None) + return d + + +@dataclass(eq=False) +class MpSender[T]: + """ + An interprocess channel, mimicing the Anyio structure. + It should be noted that none of the clone methods are implemented for simplicity, for now. + """ + + _state: MpState[T] = field() + + def send_nowait(self, item: T) -> None: + if self._state.closed.is_set(): + raise ClosedResourceError + try: + self._state.buffer.put(item, block=False) + except Full: + raise WouldBlock from None + except ValueError as e: + print("Unreachable code path - let me know!") + raise ClosedResourceError from e + + def send(self, item: T) -> None: + if self._state.closed.is_set(): + raise ClosedResourceError + try: + self.send_nowait(item) + except WouldBlock: + # put anyway, blocking + self._state.buffer.put(item, block=True) + + async def send_async(self, item: T) -> None: + await to_thread.run_sync(self.send, item, limiter=CapacityLimiter(1)) + + def close(self) -> None: + if not self._state.closed.is_set(): + self._state.closed.set() + self._state.buffer.put(_MpEndOfStream()) + self._state.buffer.close() + + # == unique to Mp channels == + def join(self) -> None: + """Ensure any queued messages are resolved before continuing""" + assert self._state.closed.is_set(), ( + "Mp channels must be closed before being joined" + ) + self._state.buffer.join_thread() + + # == context manager support == + def __enter__(self) -> Self: + return self + + def __exit__( + self, + exc_type: type[BaseException] | None, + exc_val: BaseException | None, + exc_tb: TracebackType | None, + ) -> None: + self.close() + + def __getstate__(self): + d = self.__dict__.copy() + d.pop("__orig_class__", None) + return d + + +@dataclass(eq=False) +class MpReceiver[T]: + """ + An interprocess channel, mimicing the Anyio structure. + It should be noted that none of the clone methods are implemented for simplicity, for now. + """ + + _state: MpState[T] = field() + + def receive_nowait(self) -> T: + if self._state.closed.is_set(): + raise ClosedResourceError + + try: + item = self._state.buffer.get(block=False) + if isinstance(item, _MpEndOfStream): + self.close() + raise EndOfStream + return item + except Empty: + raise WouldBlock from None + except ValueError as e: + print("Unreachable code path - let me know!") + raise ClosedResourceError from e + + def receive(self) -> T: + try: + return self.receive_nowait() + except WouldBlock: + item = self._state.buffer.get() + if isinstance(item, _MpEndOfStream): + self.close() + raise EndOfStream from None + return item + + # nb: this function will not cancel particularly well + async def receive_async(self) -> T: + return await to_thread.run_sync(self.receive, limiter=CapacityLimiter(1)) + + def close(self) -> None: + if not self._state.closed.is_set(): + self._state.closed.set() + self._state.buffer.close() + + # == unique to Mp channels == + def join(self) -> None: + """Block until all enqueued messages are drained off our side of the buffer""" + assert self._state.closed.is_set(), ( + "Mp channels must be closed before being joined" + ) + self._state.buffer.join_thread() + + # == iterator support == + def __iter__(self) -> Self: + return self + + def __next__(self) -> T: + try: + return self.receive() + except EndOfStream: + raise StopIteration from None + + # == async iterator support == + def __aiter__(self) -> Self: + return self + + async def __anext__(self) -> T: + try: + return await self.receive_async() + except EndOfStream: + raise StopAsyncIteration from None + + # == context manager support == + def __enter__(self) -> Self: + return self + + def __exit__( + self, + exc_type: type[BaseException] | None, + exc_val: BaseException | None, + exc_tb: TracebackType | None, + ) -> None: + self.close() + + def collect(self) -> list[T]: + """Collect all currently available items from this receiver""" + out: list[T] = [] + while True: + try: + item = self.receive_nowait() + out.append(item) + except WouldBlock: + break + return out + + def receive_at_least(self, n: int) -> list[T]: + out: list[T] = [] + out.append(self.receive()) + out.extend(self.collect()) + while len(out) < n: + out.append(self.receive()) + out.extend(self.collect()) + return out + + def __getstate__(self): + d = self.__dict__.copy() + d.pop("__orig_class__", None) + return d + + +class channel[T]: # noqa: N801 + """Create a pair of asynchronous channels for communicating within the same process""" + + def __new__(cls, max_buffer_size: float = inf) -> tuple[Sender[T], Receiver[T]]: + if max_buffer_size != inf and not isinstance(max_buffer_size, int): + raise ValueError("max_buffer_size must be either an integer or math.inf") + state = AnyioState[T](max_buffer_size) + return Sender(_state=state), Receiver(_state=state) + + +class mp_channel[T]: # noqa: N801 + """Create a pair of synchronous channels for interprocess communication""" + + # max buffer size uses math.inf to represent an unbounded queue, and 0 to represent a yet unimplemented "unbuffered" queue. + def __new__(cls, max_buffer_size: float = inf) -> tuple[MpSender[T], MpReceiver[T]]: + if ( + max_buffer_size == 0 + or max_buffer_size != inf + and not isinstance(max_buffer_size, int) + ): + raise ValueError( + "max_buffer_size must be either an integer or math.inf. 0-sized buffers are not supported by multiprocessing" + ) + state = MpState[T](max_buffer_size) + return MpSender(_state=state), MpReceiver(_state=state) diff --git a/src/exo/utils/dashboard_path.py b/src/exo/utils/dashboard_path.py new file mode 100644 index 00000000..b9e6990c --- /dev/null +++ b/src/exo/utils/dashboard_path.py @@ -0,0 +1,45 @@ +import os +import sys +from pathlib import Path +from typing import cast + + +def find_dashboard() -> Path: + dashboard = ( + _find_dashboard_in_env() + or _find_dashboard_in_repo() + or _find_dashboard_in_bundle() + ) + if not dashboard: + raise FileNotFoundError( + "Unable to locate dashboard assets. Export DASHBOARD_DIR or rebuild the binary." + ) + return dashboard + + +def _find_dashboard_in_env() -> Path | None: + env = os.environ.get("DASHBOARD_DIR") + if not env: + return None + resolved_env = Path(env).expanduser().resolve() + + return resolved_env + + +def _find_dashboard_in_repo() -> Path | None: + current_module = Path(__file__).resolve() + for parent in current_module.parents: + build = parent / "dashboard" / "build" + if build.is_dir() and (build / "index.html").exists(): + return build + return None + + +def _find_dashboard_in_bundle() -> Path | None: + frozen_root = cast(str | None, getattr(sys, "_MEIPASS", None)) + if frozen_root is None: + return None + candidate = Path(frozen_root) / "dashboard" + if candidate.is_dir(): + return candidate + return None diff --git a/src/exo/utils/event_buffer.py b/src/exo/utils/event_buffer.py new file mode 100644 index 00000000..8fcf5fa2 --- /dev/null +++ b/src/exo/utils/event_buffer.py @@ -0,0 +1,70 @@ +from loguru import logger + + +class OrderedBuffer[T]: + """ + A buffer that resequences events to ensure their ordering is preserved. + Currently this buffer doesn't raise any errors if an event is lost + This buffer is NOT thread safe, and is designed to only be polled from one + source at a time. + """ + + def __init__(self): + self.store: dict[int, T] = {} + self.next_idx_to_release: int = 0 + + def ingest(self, idx: int, t: T): + """Ingest a sequence into the buffer""" + logger.trace(f"Ingested event {t}") + if idx < self.next_idx_to_release: + return + if idx in self.store: + assert self.store[idx] == t, ( + "Received different messages with identical indices, probable race condition" + ) + return + self.store[idx] = t + + def drain(self) -> list[T]: + """Drain all available events from the buffer""" + ret: list[T] = [] + while self.next_idx_to_release in self.store: + idx = self.next_idx_to_release + event = self.store.pop(idx) + ret.append(event) + self.next_idx_to_release += 1 + logger.trace(f"Releasing event {ret}") + return ret + + def drain_indexed(self) -> list[tuple[int, T]]: + """Drain all available events from the buffer""" + ret: list[tuple[int, T]] = [] + while self.next_idx_to_release in self.store: + idx = self.next_idx_to_release + event = self.store.pop(idx) + ret.append((idx, event)) + self.next_idx_to_release += 1 + logger.trace(f"Releasing event {ret}") + return ret + + +class MultiSourceBuffer[SourceId, T]: + """ + A buffer that resequences events to ensure their ordering is preserved. + Tracks events with multiple sources + """ + + def __init__(self): + self.stores: dict[SourceId, OrderedBuffer[T]] = {} + + def ingest(self, idx: int, t: T, source: SourceId): + if source not in self.stores: + self.stores[source] = OrderedBuffer() + buffer = self.stores[source] + buffer.ingest(idx, t) + + def drain(self) -> list[T]: + ret: list[T] = [] + for store in self.stores.values(): + ret.extend(store.drain()) + return ret diff --git a/src/exo/utils/fs.py b/src/exo/utils/fs.py new file mode 100644 index 00000000..5419bde9 --- /dev/null +++ b/src/exo/utils/fs.py @@ -0,0 +1,32 @@ +import contextlib +import os +import pathlib +import tempfile +from typing import LiteralString + +type StrPath = str | os.PathLike[str] +type BytesPath = bytes | os.PathLike[bytes] +type StrOrBytesPath = str | bytes | os.PathLike[str] | os.PathLike[bytes] + + +def delete_if_exists(filename: StrOrBytesPath) -> None: + with contextlib.suppress(FileNotFoundError): + os.remove(filename) + + +def ensure_parent_directory_exists(filename: StrPath) -> None: + """ + Ensure the directory containing the file exists (create it if necessary). + """ + pathlib.Path(filename).parent.mkdir(parents=True, exist_ok=True) + + +def ensure_directory_exists(dirname: StrPath) -> None: + """ + Ensure the directory exists (create it if necessary). + """ + pathlib.Path(dirname).mkdir(parents=True, exist_ok=True) + + +def make_temp_path(name: LiteralString) -> str: + return os.path.join(tempfile.mkdtemp(), name) diff --git a/src/exo/utils/phantom.py b/src/exo/utils/phantom.py new file mode 100644 index 00000000..72e33442 --- /dev/null +++ b/src/exo/utils/phantom.py @@ -0,0 +1,11 @@ +class _PhantomData[*T]: + """ + Internal machinery of the phantom data - it stores nothing. + """ + + +type PhantomData[*T] = _PhantomData[*T] | None +""" +Allows you to use generics in functions without storing anything of that generic type. +Just use `None` and you'll be fine +""" diff --git a/src/exo/utils/pydantic_ext.py b/src/exo/utils/pydantic_ext.py new file mode 100644 index 00000000..1c459b2d --- /dev/null +++ b/src/exo/utils/pydantic_ext.py @@ -0,0 +1,42 @@ +# pyright: reportAny=false, reportUnknownArgumentType=false, reportUnknownVariableType=false + +from typing import Any, Self + +from pydantic import BaseModel, ConfigDict, model_serializer, model_validator +from pydantic.alias_generators import to_camel +from pydantic_core.core_schema import ( + SerializerFunctionWrapHandler, + ValidatorFunctionWrapHandler, +) + + +class CamelCaseModel(BaseModel): + """ + A model whose fields are aliased to camel-case from snake-case. + """ + + model_config = ConfigDict( + alias_generator=to_camel, + validate_by_name=True, + extra="forbid", + # I want to reenable this ASAP, but it's causing an issue with TaskStatus + strict=True, + ) + + +class TaggedModel(CamelCaseModel): + @model_serializer(mode="wrap") + def _serialize(self, handler: SerializerFunctionWrapHandler): + inner = handler(self) + return {self.__class__.__name__: inner} + + @model_validator(mode="wrap") + @classmethod + def _validate(cls, v: Any, handler: ValidatorFunctionWrapHandler) -> Self: + if isinstance(v, dict) and len(v) == 1 and cls.__name__ in v: + return handler(v[cls.__name__]) + + return handler(v) + + def __str__(self) -> str: + return f"{self.__class__.__name__}({super().__str__()})" diff --git a/src/exo/utils/reactive.py b/src/exo/utils/reactive.py new file mode 100644 index 00000000..14c021d2 --- /dev/null +++ b/src/exo/utils/reactive.py @@ -0,0 +1,32 @@ +""" +Utilities for reactive variables + +""" + +from typing import Protocol + + +class OnChange[T](Protocol): + def __call__(self, old_value: T, new_value: T) -> None: ... + + +class Reactive[T]: + def __init__(self, initial_value: T, on_change: OnChange[T]): + self._value = initial_value + self._on_change = on_change + + @property + def value(self): + return self._value + + @value.setter + def value(self, new_value: T): + old_value = self._value + self._value = new_value + + # don't notify when not changed + if old_value == new_value: + return + + # notify of changes + self._on_change(old_value=old_value, new_value=new_value) diff --git a/src/exo/utils/tests/test_tagged.py b/src/exo/utils/tests/test_tagged.py new file mode 100644 index 00000000..6d417ed9 --- /dev/null +++ b/src/exo/utils/tests/test_tagged.py @@ -0,0 +1,250 @@ +import anyio +import pytest +from pydantic import BaseModel, TypeAdapter, ValidationError + +from exo.utils.pydantic_ext import TaggedModel + + +def test_plain_union_prefers_first_member_when_shapes_are_identical(): + class Foo1(BaseModel): + x: int + + class Foo2(BaseModel): + x: int + + # Base Pydantic behavior: ambiguous dict goes to the first union member + ta = TypeAdapter[Foo1 | Foo2](Foo1 | Foo2) + out = ta.validate_python({"x": 1}) + assert isinstance(out, Foo1), ( + "Base Pydantic should pick the first union member for identical shapes" + ) + + +def test_tagged_union_serializes_and_deserializes_two_identical_shapes_correctly(): + class Foo1(TaggedModel): + x: int + + class Foo2(TaggedModel): + x: int + + t1 = Foo1(x=1) + assert t1.model_dump() == {"Foo1": {"x": 1}} + + t2 = Foo2(x=2) + assert t2.model_dump() == {"Foo2": {"x": 2}} + + # ---- deserialize (TypeAdapter -> model_validator(before)) ---- + ta = TypeAdapter[Foo1 | Foo2](Foo1 | Foo2) + + out1 = ta.validate_python({"Foo1": {"x": 10}}) + assert isinstance(out1, Foo1) and out1.x == 10 + + out2 = ta.validate_python({"Foo2": {"x": 20}}) + assert isinstance(out2, Foo2) and out2.x == 20 + + +def test_tagged_union_rejects_unknown_tag(): + class Foo1(TaggedModel): + x: int + + class Foo2(TaggedModel): + x: int + + ta = TypeAdapter[Foo1 | Foo2](Foo1 | Foo2) + with pytest.raises(ValidationError): + ta.validate_python({"NotARealTag": {"x": 0}}) + + +def test_two_tagged_classes_with_different_shapes_are_independent_and_not_cross_deserializable(): + class A1(TaggedModel): + x: int + + class A2(TaggedModel): + name: str + + class B1(TaggedModel): + name: str + + class B2(TaggedModel): + active: bool + + a_payload = A1(x=123).model_dump() + b_payload = B1(name="neo").model_dump() + + assert a_payload == {"A1": {"x": 123}} + assert b_payload == {"B1": {"name": "neo"}} + + ta_a = TypeAdapter[A1 | A2](A1 | A2) + ta_b = TypeAdapter[B1 | B2](B1 | B2) + + with pytest.raises(ValidationError): + ta_a.validate_python(b_payload) + + with pytest.raises(ValidationError): + ta_b.validate_python(a_payload) + + +class Inner(TaggedModel): + x: int + + +class Outer(TaggedModel): + inner: Inner + + +class Wrapper(TaggedModel): + outer: Outer + label: str + + +class Container(TaggedModel): + items: list[Inner] + nested: Wrapper + + +def test_single_level_tagging(): + inner = Inner(x=10) + dumped = inner.model_dump() + assert dumped == {"Inner": {"x": 10}} + + restored = Inner.model_validate(dumped) + assert isinstance(restored, Inner) + assert restored.x == 10 + + +def test_nested_externally_tagged_union_serializes_recursively(): + outer = Outer(inner=Inner(x=42)) + dumped = outer.model_dump() + + assert dumped == {"Outer": {"inner": {"Inner": {"x": 42}}}} + + restored = Outer.model_validate(dumped) + assert isinstance(restored.inner, Inner) + assert restored.inner.x == 42 + + +def test_two_level_nested_tagging(): + outer = Outer(inner=Inner(x=123)) + dumped = outer.model_dump() + assert dumped == {"Outer": {"inner": {"Inner": {"x": 123}}}} + + restored = Outer.model_validate(dumped) + assert isinstance(restored.inner, Inner) + assert restored.inner.x == 123 + + +def test_three_level_nested_tagging(): + wrapper = Wrapper(label="deep", outer=Outer(inner=Inner(x=7))) + dumped = wrapper.model_dump() + # 3-level structure, each with exactly one tag + assert dumped == { + "Wrapper": { + "label": "deep", + "outer": {"Outer": {"inner": {"Inner": {"x": 7}}}}, + } + } + + restored = Wrapper.model_validate(dumped) + assert isinstance(restored.outer.inner, Inner) + assert restored.outer.inner.x == 7 + assert restored.label == "deep" + + +def test_lists_and_mixed_nested_structures(): + container = Container( + items=[Inner(x=1), Inner(x=2)], + nested=Wrapper(label="mix", outer=Outer(inner=Inner(x=9))), + ) + dumped = container.model_dump() + + assert dumped == { + "Container": { + "items": [ + {"Inner": {"x": 1}}, + {"Inner": {"x": 2}}, + ], + "nested": { + "Wrapper": { + "label": "mix", + "outer": {"Outer": {"inner": {"Inner": {"x": 9}}}}, + } + }, + } + } + + restored = Container.model_validate(dumped) + assert isinstance(restored.nested.outer.inner, Inner) + assert [i.x for i in restored.items] == [1, 2] + + +def test_no_double_tagging_on_repeated_calls(): + """Ensure multiple model_dump calls don't stack tags.""" + inner = Inner(x=11) + dumped1 = inner.model_dump() + dumped2 = inner.model_dump() + assert dumped1 == dumped2 == {"Inner": {"x": 11}} + + outer = Outer(inner=inner) + d1 = outer.model_dump() + d2 = outer.model_dump() + assert d1 == d2 == {"Outer": {"inner": {"Inner": {"x": 11}}}} + + +class L3A(TaggedModel): + x: int + + +class L3B(TaggedModel): + x: int + + +class L3C(TaggedModel): + x: int + + +L3 = L3A | L3B | L3C + + +class L2A(TaggedModel): + child: L3 + + +class L2B(TaggedModel): + child: L3 + + +class L2C(TaggedModel): + child: L3 + + +L2 = L2A | L2B | L2C + + +class L1A(TaggedModel): + child: L2 + + +class L1B(TaggedModel): + child: L2 + + +class L1C(TaggedModel): + child: L2 + + +L1 = L1A | L1B | L1C + + +@pytest.mark.anyio +async def test_tagged_union_is_fast(): + # payload along the "C" path (worst case for DFS if branches are tried A->B->C) + payload = {"L1C": {"child": {"L2C": {"child": {"L3C": {"x": 123}}}}}} + + with anyio.fail_after(0.1): + out = TypeAdapter(L1).validate_python(payload) # type: ignore + + # Sanity check the result + assert out.__class__.__name__ == "L1C" # type: ignore + assert out.child.__class__.__name__ == "L2C" # type: ignore + assert out.child.child.__class__.__name__ == "L3C" # type: ignore + assert out.child.child.x == 123 # type: ignore diff --git a/src/exo/utils/tests/testing_mp.py b/src/exo/utils/tests/testing_mp.py new file mode 100644 index 00000000..62eddf0c --- /dev/null +++ b/src/exo/utils/tests/testing_mp.py @@ -0,0 +1,41 @@ +import multiprocessing as mp +import time + +import pytest +from anyio import fail_after +from loguru import logger + +from exo.utils.channels import MpReceiver, MpSender, mp_channel + + +def foo(recv: MpReceiver[str]): + expected = ["hi", "hi 2", "bye"] + with recv as r: + for item in r: + assert item == expected.pop(0) + + +def bar(send: MpSender[str]): + logger.warning("hi") + send.send("hi") + time.sleep(0.1) + logger.warning("hi 2") + send.send("hi 2") + time.sleep(0.1) + logger.warning("bye") + send.send("bye") + time.sleep(0.1) + send.close() + + +# not async, just want the fail_after +@pytest.mark.anyio +async def test_channel_setup(): + with fail_after(0.5): + s, r = mp_channel[str]() + p1 = mp.Process(target=foo, args=(r,)) + p2 = mp.Process(target=bar, args=(s,)) + p1.start() + p2.start() + p1.join() + p2.join() diff --git a/src/exo/worker/__init__.py b/src/exo/worker/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/download/download_utils.py b/src/exo/worker/download/download_utils.py new file mode 100644 index 00000000..51addfbc --- /dev/null +++ b/src/exo/worker/download/download_utils.py @@ -0,0 +1,663 @@ +import asyncio +import hashlib +import os +import shutil +import time +import traceback +from datetime import timedelta +from pathlib import Path +from typing import Callable, Literal +from urllib.parse import urljoin + +import aiofiles +import aiofiles.os as aios +import aiohttp +from loguru import logger +from pydantic import ( + BaseModel, + ConfigDict, + DirectoryPath, + Field, + PositiveInt, + TypeAdapter, +) + +from exo.shared.constants import EXO_HOME, EXO_MODELS_DIR +from exo.shared.types.memory import Memory +from exo.shared.types.worker.downloads import DownloadProgressData +from exo.shared.types.worker.shards import ShardMetadata +from exo.worker.download.huggingface_utils import ( + filter_repo_objects, + get_allow_patterns, + get_auth_headers, + get_hf_endpoint, +) + + +class ModelSafetensorsIndexMetadata(BaseModel): + total_size: PositiveInt + + +class ModelSafetensorsIndex(BaseModel): + metadata: ModelSafetensorsIndexMetadata | None + weight_map: dict[str, str] + + +class FileListEntry(BaseModel): + type: Literal["file", "directory"] + path: str + size: int | None = None + + +class RepoFileDownloadProgress(BaseModel): + repo_id: str + repo_revision: str + file_path: str + downloaded: Memory + downloaded_this_session: Memory + total: Memory + speed: float + eta: timedelta + status: Literal["not_started", "in_progress", "complete"] + start_time: float + + model_config = ConfigDict(frozen=True) + + +class RepoDownloadProgress(BaseModel): + repo_id: str + repo_revision: str + shard: ShardMetadata + completed_files: int + total_files: int + downloaded_bytes: Memory + downloaded_bytes_this_session: Memory + total_bytes: Memory + overall_speed: float + overall_eta: timedelta + status: Literal["not_started", "in_progress", "complete"] + file_progress: dict[str, RepoFileDownloadProgress] = Field(default_factory=dict) + + model_config = ConfigDict(frozen=True) + + +def trim_etag(etag: str) -> str: + if (etag[0] == '"' and etag[-1] == '"') or (etag[0] == "'" and etag[-1] == "'"): + return etag[1:-1] + return etag + + +def map_repo_file_download_progress_to_download_progress_data( + repo_file_download_progress: RepoFileDownloadProgress, +) -> DownloadProgressData: + return DownloadProgressData( + downloaded_bytes=repo_file_download_progress.downloaded, + downloaded_bytes_this_session=repo_file_download_progress.downloaded_this_session, + total_bytes=repo_file_download_progress.total, + completed_files=1 if repo_file_download_progress.status == "complete" else 0, + total_files=1, + speed=repo_file_download_progress.speed, + eta_ms=int(repo_file_download_progress.eta.total_seconds() * 1000), + files={}, + ) + + +def map_repo_download_progress_to_download_progress_data( + repo_download_progress: RepoDownloadProgress, +) -> DownloadProgressData: + return DownloadProgressData( + total_bytes=repo_download_progress.total_bytes, + downloaded_bytes=repo_download_progress.downloaded_bytes, + downloaded_bytes_this_session=repo_download_progress.downloaded_bytes_this_session, + completed_files=repo_download_progress.completed_files, + total_files=repo_download_progress.total_files, + speed=repo_download_progress.overall_speed, + eta_ms=int(repo_download_progress.overall_eta.total_seconds() * 1000), + files={ + file_path: map_repo_file_download_progress_to_download_progress_data( + file_progress + ) + for file_path, file_progress in repo_download_progress.file_progress.items() + }, + ) + + +def build_model_path(model_id: str) -> DirectoryPath: + return EXO_MODELS_DIR / model_id.replace("/", "--") + + +async def resolve_model_path_for_repo(repo_id: str) -> Path: + return (await ensure_models_dir()) / repo_id.replace("/", "--") + + +async def ensure_exo_home() -> Path: + await aios.makedirs(EXO_HOME, exist_ok=True) + return EXO_HOME + + +async def has_exo_home_read_access() -> bool: + try: + return await aios.access(EXO_HOME, os.R_OK) + except OSError: + return False + + +async def has_exo_home_write_access() -> bool: + try: + return await aios.access(EXO_HOME, os.W_OK) + except OSError: + return False + + +async def ensure_models_dir() -> Path: + await aios.makedirs(EXO_MODELS_DIR, exist_ok=True) + return EXO_MODELS_DIR + + +async def delete_model(repo_id: str) -> bool: + model_dir = await ensure_models_dir() / repo_id.replace("/", "--") + if not await aios.path.exists(model_dir): + return False + await asyncio.to_thread(shutil.rmtree, model_dir, ignore_errors=False) + return True + + +async def seed_models(seed_dir: str | Path): + """Move model in resources folder of app to .cache/huggingface/hub""" + source_dir = Path(seed_dir) + dest_dir = await ensure_models_dir() + for path in source_dir.iterdir(): + if path.is_dir() and path.name.startswith("models--"): + dest_path = dest_dir / path.name + if await aios.path.exists(dest_path): + logger.info("Skipping moving model to .cache directory") + else: + try: + await aios.rename(str(path), str(dest_path)) + except Exception: + logger.error(f"Error seeding model {path} to {dest_path}") + logger.error(traceback.format_exc()) + + +async def fetch_file_list_with_cache( + repo_id: str, revision: str = "main", recursive: bool = False +) -> list[FileListEntry]: + target_dir = ( + (await ensure_models_dir()) / "caches" / str(repo_id).replace("/", "--") + ) + await aios.makedirs(target_dir, exist_ok=True) + cache_file = ( + target_dir / f"{repo_id.replace('/', '--')}--{revision}--file_list.json" + ) + if await aios.path.exists(cache_file): + async with aiofiles.open(cache_file, "r") as f: + return TypeAdapter(list[FileListEntry]).validate_json(await f.read()) + file_list = await fetch_file_list_with_retry(repo_id, revision, recursive=recursive) + await aios.makedirs(cache_file.parent, exist_ok=True) + async with aiofiles.open(cache_file, "w") as f: + await f.write(TypeAdapter(list[FileListEntry]).dump_json(file_list).decode()) + return file_list + + +async def fetch_file_list_with_retry( + repo_id: str, revision: str = "main", path: str = "", recursive: bool = False +) -> list[FileListEntry]: + n_attempts = 30 + for attempt in range(n_attempts): + try: + return await _fetch_file_list(repo_id, revision, path, recursive) + except Exception as e: + if attempt == n_attempts - 1: + raise e + await asyncio.sleep(min(8, 0.1 * float(2.0 ** int(attempt)))) + raise Exception( + f"Failed to fetch file list for {repo_id=} {revision=} {path=} {recursive=}" + ) + + +async def _fetch_file_list( + repo_id: str, revision: str = "main", path: str = "", recursive: bool = False +) -> list[FileListEntry]: + api_url = f"{get_hf_endpoint()}/api/models/{repo_id}/tree/{revision}" + url = f"{api_url}/{path}" if path else api_url + + headers = await get_download_headers() + async with ( + create_http_session(timeout_profile="short") as session, + session.get(url, headers=headers) as response, + ): + if response.status == 200: + data_json = await response.text() + data = TypeAdapter(list[FileListEntry]).validate_json(data_json) + files: list[FileListEntry] = [] + for item in data: + if item.type == "file": + files.append(FileListEntry.model_validate(item)) + elif item.type == "directory" and recursive: + subfiles = await _fetch_file_list( + repo_id, revision, item.path, recursive + ) + files.extend(subfiles) + return files + else: + raise Exception(f"Failed to fetch file list: {response.status}") + + +async def get_download_headers() -> dict[str, str]: + return {**(await get_auth_headers()), "Accept-Encoding": "identity"} + + +def create_http_session( + auto_decompress: bool = False, + timeout_profile: Literal["short", "long"] = "long", +) -> aiohttp.ClientSession: + if timeout_profile == "short": + total_timeout = 30 + connect_timeout = 10 + sock_read_timeout = 30 + sock_connect_timeout = 10 + else: + total_timeout = 1800 + connect_timeout = 60 + sock_read_timeout = 1800 + sock_connect_timeout = 60 + + return aiohttp.ClientSession( + auto_decompress=auto_decompress, + timeout=aiohttp.ClientTimeout( + total=total_timeout, + connect=connect_timeout, + sock_read=sock_read_timeout, + sock_connect=sock_connect_timeout, + ), + ) + + +async def calc_hash(path: Path, hash_type: Literal["sha1", "sha256"] = "sha1") -> str: + hasher = hashlib.sha1() if hash_type == "sha1" else hashlib.sha256() + if hash_type == "sha1": + header = f"blob {(await aios.stat(path)).st_size}\0".encode() + hasher.update(header) + async with aiofiles.open(path, "rb") as f: + while chunk := await f.read(8 * 1024 * 1024): + hasher.update(chunk) + return hasher.hexdigest() + + +async def file_meta( + repo_id: str, revision: str, path: str, redirected_location: str | None = None +) -> tuple[int, str]: + url = ( + urljoin(f"{get_hf_endpoint()}/{repo_id}/resolve/{revision}/", path) + if redirected_location is None + else f"{get_hf_endpoint()}{redirected_location}" + ) + headers = await get_download_headers() + async with ( + create_http_session(timeout_profile="short") as session, + session.head(url, headers=headers) as r, + ): + if r.status == 307: + # On redirect, only trust Hugging Face's x-linked-* headers. + x_linked_size = r.headers.get("x-linked-size") + x_linked_etag = r.headers.get("x-linked-etag") + if x_linked_size and x_linked_etag: + content_length = int(x_linked_size) + etag = trim_etag(x_linked_etag) + return content_length, etag + # Otherwise, follow the redirect to get authoritative size/hash + redirected_location = r.headers.get("location") + return await file_meta(repo_id, revision, path, redirected_location) + content_length = int( + r.headers.get("x-linked-size") or r.headers.get("content-length") or 0 + ) + etag = r.headers.get("x-linked-etag") or r.headers.get("etag") + assert content_length > 0, f"No content length for {url}" + assert etag is not None, f"No remote hash for {url}" + etag = trim_etag(etag) + return content_length, etag + + +async def download_file_with_retry( + repo_id: str, + revision: str, + path: str, + target_dir: Path, + on_progress: Callable[[int, int, bool], None] = lambda _, __, ___: None, +) -> Path: + n_attempts = 30 + for attempt in range(n_attempts): + try: + return await _download_file( + repo_id, revision, path, target_dir, on_progress + ) + except Exception as e: + if isinstance(e, FileNotFoundError) or attempt == n_attempts - 1: + raise e + logger.error( + f"Download error on attempt {attempt}/{n_attempts} for {repo_id=} {revision=} {path=} {target_dir=}" + ) + logger.error(traceback.format_exc()) + await asyncio.sleep(min(8, 0.1 * (2.0**attempt))) + raise Exception( + f"Failed to download file {repo_id=} {revision=} {path=} {target_dir=}" + ) + + +async def _download_file( + repo_id: str, + revision: str, + path: str, + target_dir: Path, + on_progress: Callable[[int, int, bool], None] = lambda _, __, ___: None, +) -> Path: + if await aios.path.exists(target_dir / path): + return target_dir / path + await aios.makedirs((target_dir / path).parent, exist_ok=True) + length, etag = await file_meta(repo_id, revision, path) + remote_hash = etag[:-5] if etag.endswith("-gzip") else etag + partial_path = target_dir / f"{path}.partial" + resume_byte_pos = ( + (await aios.stat(partial_path)).st_size + if (await aios.path.exists(partial_path)) + else None + ) + if resume_byte_pos != length: + url = urljoin(f"{get_hf_endpoint()}/{repo_id}/resolve/{revision}/", path) + headers = await get_download_headers() + if resume_byte_pos: + headers["Range"] = f"bytes={resume_byte_pos}-" + n_read = resume_byte_pos or 0 + async with ( + create_http_session(timeout_profile="long") as session, + session.get(url, headers=headers) as r, + ): + if r.status == 404: + raise FileNotFoundError(f"File not found: {url}") + assert r.status in [200, 206], ( + f"Failed to download {path} from {url}: {r.status}" + ) + async with aiofiles.open( + partial_path, "ab" if resume_byte_pos else "wb" + ) as f: + while chunk := await r.content.read(8 * 1024 * 1024): + n_read = n_read + (await f.write(chunk)) + on_progress(n_read, length, False) + + final_hash = await calc_hash( + partial_path, hash_type="sha256" if len(remote_hash) == 64 else "sha1" + ) + integrity = final_hash == remote_hash + if not integrity: + try: + await aios.remove(partial_path) + except Exception as e: + logger.error(f"Error removing partial file {partial_path}: {e}") + raise Exception( + f"Downloaded file {target_dir / path} has hash {final_hash} but remote hash is {remote_hash}" + ) + await aios.rename(partial_path, target_dir / path) + on_progress(length, length, True) + return target_dir / path + + +def calculate_repo_progress( + shard: ShardMetadata, + repo_id: str, + revision: str, + file_progress: dict[str, RepoFileDownloadProgress], + all_start_time: float, +) -> RepoDownloadProgress: + all_total_bytes = sum((p.total.in_bytes for p in file_progress.values()), 0) + all_downloaded_bytes = sum( + (p.downloaded.in_bytes for p in file_progress.values()), 0 + ) + all_downloaded_bytes_this_session = sum( + (p.downloaded_this_session.in_bytes for p in file_progress.values()), 0 + ) + elapsed_time = time.time() - all_start_time + all_speed = ( + all_downloaded_bytes_this_session / elapsed_time if elapsed_time > 0 else 0 + ) + all_eta = ( + timedelta(seconds=(all_total_bytes - all_downloaded_bytes) / all_speed) + if all_speed > 0 + else timedelta(seconds=0) + ) + status = ( + "complete" + if all(p.status == "complete" for p in file_progress.values()) + else "in_progress" + if any(p.status == "in_progress" for p in file_progress.values()) + else "not_started" + ) + return RepoDownloadProgress( + repo_id=repo_id, + repo_revision=revision, + shard=shard, + completed_files=len( + [p for p in file_progress.values() if p.downloaded == p.total] + ), + total_files=len(file_progress), + downloaded_bytes=Memory.from_bytes(all_downloaded_bytes), + downloaded_bytes_this_session=Memory.from_bytes( + all_downloaded_bytes_this_session + ), + total_bytes=Memory.from_bytes(all_total_bytes), + overall_speed=all_speed, + overall_eta=all_eta, + status=status, + file_progress=file_progress, + ) + + +async def get_weight_map(repo_id: str, revision: str = "main") -> dict[str, str]: + target_dir = (await ensure_models_dir()) / str(repo_id).replace("/", "--") + await aios.makedirs(target_dir, exist_ok=True) + index_file = await download_file_with_retry( + repo_id, revision, "model.safetensors.index.json", target_dir + ) + async with aiofiles.open(index_file, "r") as f: + index_data = ModelSafetensorsIndex.model_validate_json(await f.read()) + return index_data.weight_map + + +async def resolve_allow_patterns(shard: ShardMetadata) -> list[str]: + try: + weight_map = await get_weight_map(str(shard.model_meta.model_id)) + return get_allow_patterns(weight_map, shard) + except Exception: + logger.error(f"Error getting weight map for {shard.model_meta.model_id=}") + logger.error(traceback.format_exc()) + return ["*"] + + +async def get_downloaded_size(path: Path) -> int: + partial_path = path.with_suffix(path.suffix + ".partial") + if await aios.path.exists(path): + return (await aios.stat(path)).st_size + if await aios.path.exists(partial_path): + return (await aios.stat(partial_path)).st_size + return 0 + + +async def download_progress_for_local_path( + repo_id: str, shard: ShardMetadata, local_path: Path +) -> RepoDownloadProgress: + file_progress: dict[str, RepoFileDownloadProgress] = {} + total_files = 0 + total_bytes = 0 + + if await aios.path.isdir(local_path): + for root, _, files in os.walk(local_path): + for f in files: + if f.endswith((".safetensors", ".bin", ".pt", ".gguf", ".json")): + file_path = Path(root) / f + size = (await aios.stat(file_path)).st_size + rel_path = str(file_path.relative_to(local_path)) + file_progress[rel_path] = RepoFileDownloadProgress( + repo_id=repo_id, + repo_revision="local", + file_path=rel_path, + downloaded=Memory.from_bytes(size), + downloaded_this_session=Memory.from_bytes(0), + total=Memory.from_bytes(size), + speed=0, + eta=timedelta(0), + status="complete", + start_time=time.time(), + ) + total_files += 1 + total_bytes += size + else: + raise ValueError(f"Local path {local_path} is not a directory") + + return RepoDownloadProgress( + repo_id=repo_id, + repo_revision="local", + shard=shard, + completed_files=total_files, + total_files=total_files, + downloaded_bytes=Memory.from_bytes(total_bytes), + downloaded_bytes_this_session=Memory.from_bytes(0), + total_bytes=Memory.from_bytes(total_bytes), + overall_speed=0, + overall_eta=timedelta(0), + status="complete", + file_progress=file_progress, + ) + + +async def download_shard( + shard: ShardMetadata, + on_progress: Callable[[ShardMetadata, RepoDownloadProgress], None], + max_parallel_downloads: int = 8, + skip_download: bool = False, + allow_patterns: list[str] | None = None, +) -> tuple[Path, RepoDownloadProgress]: + if not skip_download: + logger.info(f"Downloading {shard.model_meta.model_id=}") + + # Handle local paths + if await aios.path.exists(str(shard.model_meta.model_id)): + logger.info(f"Using local model path {shard.model_meta.model_id}") + local_path = Path(str(shard.model_meta.model_id)) + return local_path, await download_progress_for_local_path( + str(shard.model_meta.model_id), shard, local_path + ) + + revision = "main" + target_dir = await ensure_models_dir() / str(shard.model_meta.model_id).replace( + "/", "--" + ) + if not skip_download: + await aios.makedirs(target_dir, exist_ok=True) + + if not allow_patterns: + allow_patterns = await resolve_allow_patterns(shard) + + logger.info(f"Downloading {shard.model_meta.model_id=} with {allow_patterns=}") + + all_start_time = time.time() + # TODO: currently not recursive. Some models might require subdirectories - thus this will need to be changed. + # Update: <- This does not seem to be the case. Yay? + file_list = await fetch_file_list_with_cache( + str(shard.model_meta.model_id), revision, recursive=True + ) + filtered_file_list = list( + filter_repo_objects( + file_list, allow_patterns=allow_patterns, key=lambda x: x.path + ) + ) + file_progress: dict[str, RepoFileDownloadProgress] = {} + + def on_progress_wrapper( + file: FileListEntry, curr_bytes: int, total_bytes: int, is_renamed: bool + ): + start_time = ( + file_progress[file.path].start_time + if file.path in file_progress + else time.time() + ) + downloaded_this_session = ( + file_progress[file.path].downloaded_this_session.in_bytes + + (curr_bytes - file_progress[file.path].downloaded.in_bytes) + if file.path in file_progress + else curr_bytes + ) + speed = ( + downloaded_this_session / (time.time() - start_time) + if time.time() - start_time > 0 + else 0 + ) + eta = ( + timedelta(seconds=(total_bytes - curr_bytes) / speed) + if speed > 0 + else timedelta(seconds=0) + ) + file_progress[file.path] = RepoFileDownloadProgress( + repo_id=str(shard.model_meta.model_id), + repo_revision=revision, + file_path=file.path, + downloaded=Memory.from_bytes(curr_bytes), + downloaded_this_session=Memory.from_bytes(downloaded_this_session), + total=Memory.from_bytes(total_bytes), + speed=speed, + eta=eta, + status="complete" + if curr_bytes == total_bytes and is_renamed + else "in_progress", + start_time=start_time, + ) + on_progress( + shard, + calculate_repo_progress( + shard, + str(shard.model_meta.model_id), + revision, + file_progress, + all_start_time, + ), + ) + + for file in filtered_file_list: + downloaded_bytes = await get_downloaded_size(target_dir / file.path) + file_progress[file.path] = RepoFileDownloadProgress( + repo_id=str(shard.model_meta.model_id), + repo_revision=revision, + file_path=file.path, + downloaded=Memory.from_bytes(downloaded_bytes), + downloaded_this_session=Memory.from_bytes(0), + total=Memory.from_bytes(file.size or 0), + speed=0, + eta=timedelta(0), + status="complete" if downloaded_bytes == file.size else "not_started", + start_time=time.time(), + ) + + semaphore = asyncio.Semaphore(max_parallel_downloads) + + async def download_with_semaphore(file: FileListEntry): + async with semaphore: + await download_file_with_retry( + str(shard.model_meta.model_id), + revision, + file.path, + target_dir, + lambda curr_bytes, total_bytes, is_renamed: on_progress_wrapper( + file, curr_bytes, total_bytes, is_renamed + ), + ) + + if not skip_download: + await asyncio.gather( + *[download_with_semaphore(file) for file in filtered_file_list] + ) + final_repo_progress = calculate_repo_progress( + shard, str(shard.model_meta.model_id), revision, file_progress, all_start_time + ) + on_progress(shard, final_repo_progress) + if gguf := next((f for f in filtered_file_list if f.path.endswith(".gguf")), None): + return target_dir / gguf.path, final_repo_progress + else: + return target_dir, final_repo_progress diff --git a/src/exo/worker/download/huggingface_utils.py b/src/exo/worker/download/huggingface_utils.py new file mode 100644 index 00000000..f83e5a55 --- /dev/null +++ b/src/exo/worker/download/huggingface_utils.py @@ -0,0 +1,117 @@ +import os +from fnmatch import fnmatch +from pathlib import Path +from typing import Callable, Generator, Iterable + +import aiofiles +import aiofiles.os as aios +from loguru import logger + +from exo.shared.types.worker.shards import ShardMetadata + + +def filter_repo_objects[T]( + items: Iterable[T], + *, + allow_patterns: list[str] | str | None = None, + ignore_patterns: list[str] | str | None = None, + key: Callable[[T], str] | None = None, +) -> Generator[T, None, None]: + if isinstance(allow_patterns, str): + allow_patterns = [allow_patterns] + if isinstance(ignore_patterns, str): + ignore_patterns = [ignore_patterns] + if allow_patterns is not None: + allow_patterns = [_add_wildcard_to_directories(p) for p in allow_patterns] + if ignore_patterns is not None: + ignore_patterns = [_add_wildcard_to_directories(p) for p in ignore_patterns] + + if key is None: + + def _identity(item: T) -> str: + if isinstance(item, str): + return item + if isinstance(item, Path): + return str(item) + raise ValueError( + f"Please provide `key` argument in `filter_repo_objects`: `{item}` is not a string." + ) + + key = _identity + + for item in items: + path = key(item) + if allow_patterns is not None and not any( + fnmatch(path, r) for r in allow_patterns + ): + continue + if ignore_patterns is not None and any( + fnmatch(path, r) for r in ignore_patterns + ): + continue + yield item + + +def _add_wildcard_to_directories(pattern: str) -> str: + if pattern[-1] == "/": + return pattern + "*" + return pattern + + +def get_hf_endpoint() -> str: + return os.environ.get("HF_ENDPOINT", "https://huggingface.co") + + +def get_hf_home() -> Path: + """Get the Hugging Face home directory.""" + return Path(os.environ.get("HF_HOME", Path.home() / ".cache" / "huggingface")) + + +async def get_hf_token() -> str | None: + """Retrieve the Hugging Face token from the user's HF_HOME directory.""" + token_path = get_hf_home() / "token" + if await aios.path.exists(token_path): + async with aiofiles.open(token_path, "r") as f: + return (await f.read()).strip() + return None + + +async def get_auth_headers() -> dict[str, str]: + """Get authentication headers if a token is available.""" + token = await get_hf_token() + if token: + return {"Authorization": f"Bearer {token}"} + return {} + + +def extract_layer_num(tensor_name: str) -> int | None: + # This is a simple example and might need to be adjusted based on the actual naming convention + parts = tensor_name.split(".") + for part in parts: + if part.isdigit(): + return int(part) + return None + + +def get_allow_patterns(weight_map: dict[str, str], shard: ShardMetadata) -> list[str]: + default_patterns = set( + ["*.json", "*.py", "tokenizer.model", "*.tiktoken", "*.txt", "*.jinja"] + ) + shard_specific_patterns: set[str] = set() + if weight_map: + for tensor_name, filename in weight_map.items(): + layer_num = extract_layer_num(tensor_name) + if ( + layer_num is not None + and shard.start_layer <= layer_num <= shard.end_layer + ): + shard_specific_patterns.add(filename) + layer_independent_files = set( + [v for k, v in weight_map.items() if extract_layer_num(k) is None] + ) + shard_specific_patterns.update(layer_independent_files) + logger.debug(f"get_allow_patterns {shard=} {layer_independent_files=}") + else: + shard_specific_patterns = set(["*.safetensors"]) + logger.info(f"get_allow_patterns {shard=} {shard_specific_patterns=}") + return list(default_patterns | shard_specific_patterns) diff --git a/src/exo/worker/download/impl_shard_downloader.py b/src/exo/worker/download/impl_shard_downloader.py new file mode 100644 index 00000000..46f55ff9 --- /dev/null +++ b/src/exo/worker/download/impl_shard_downloader.py @@ -0,0 +1,174 @@ +import asyncio +from pathlib import Path +from typing import AsyncIterator, Callable + +from exo.shared.models.model_cards import MODEL_CARDS +from exo.shared.models.model_meta import get_model_meta +from exo.shared.types.worker.shards import ( + PipelineShardMetadata, + ShardMetadata, +) +from exo.worker.download.download_utils import RepoDownloadProgress, download_shard +from exo.worker.download.shard_downloader import ShardDownloader + + +def exo_shard_downloader(max_parallel_downloads: int = 8) -> ShardDownloader: + return SingletonShardDownloader( + CachedShardDownloader(ResumableShardDownloader(max_parallel_downloads)) + ) + + +async def build_base_shard(model_id: str) -> ShardMetadata: + model_meta = await get_model_meta(model_id) + return PipelineShardMetadata( + model_meta=model_meta, + device_rank=0, + world_size=1, + start_layer=0, + end_layer=model_meta.n_layers, + n_layers=model_meta.n_layers, + ) + + +async def build_full_shard(model_id: str) -> PipelineShardMetadata: + base_shard = await build_base_shard(model_id) + return PipelineShardMetadata( + model_meta=base_shard.model_meta, + device_rank=base_shard.device_rank, + world_size=base_shard.world_size, + start_layer=base_shard.start_layer, + end_layer=base_shard.n_layers, + n_layers=base_shard.n_layers, + ) + + +class SingletonShardDownloader(ShardDownloader): + def __init__(self, shard_downloader: ShardDownloader): + self.shard_downloader = shard_downloader + self.active_downloads: dict[ShardMetadata, asyncio.Task[Path]] = {} + + def on_progress( + self, callback: Callable[[ShardMetadata, RepoDownloadProgress], None] + ) -> None: + self.shard_downloader.on_progress(callback) + + async def ensure_shard( + self, shard: ShardMetadata, config_only: bool = False + ) -> Path: + if shard not in self.active_downloads: + self.active_downloads[shard] = asyncio.create_task( + self.shard_downloader.ensure_shard(shard, config_only) + ) + try: + return await self.active_downloads[shard] + finally: + if shard in self.active_downloads and self.active_downloads[shard].done(): + del self.active_downloads[shard] + + async def get_shard_download_status( + self, + ) -> AsyncIterator[tuple[Path, RepoDownloadProgress]]: + async for path, status in self.shard_downloader.get_shard_download_status(): + yield path, status + + async def get_shard_download_status_for_shard( + self, shard: ShardMetadata + ) -> RepoDownloadProgress: + return await self.shard_downloader.get_shard_download_status_for_shard(shard) + + +class CachedShardDownloader(ShardDownloader): + def __init__(self, shard_downloader: ShardDownloader): + self.shard_downloader = shard_downloader + self.cache: dict[tuple[str, ShardMetadata], Path] = {} + + def on_progress( + self, callback: Callable[[ShardMetadata, RepoDownloadProgress], None] + ) -> None: + self.shard_downloader.on_progress(callback) + + async def ensure_shard( + self, shard: ShardMetadata, config_only: bool = False + ) -> Path: + if (shard.model_meta.model_id, shard) in self.cache: + return self.cache[(shard.model_meta.model_id, shard)] + + target_dir = await self.shard_downloader.ensure_shard(shard, config_only) + self.cache[(shard.model_meta.model_id, shard)] = target_dir + return target_dir + + async def get_shard_download_status( + self, + ) -> AsyncIterator[tuple[Path, RepoDownloadProgress]]: + async for path, status in self.shard_downloader.get_shard_download_status(): + yield path, status + + async def get_shard_download_status_for_shard( + self, shard: ShardMetadata + ) -> RepoDownloadProgress: + return await self.shard_downloader.get_shard_download_status_for_shard(shard) + + +class ResumableShardDownloader(ShardDownloader): + def __init__(self, max_parallel_downloads: int = 8): + self.max_parallel_downloads = max_parallel_downloads + self.on_progress_callbacks: list[ + Callable[[ShardMetadata, RepoDownloadProgress], None] + ] = [] + + def on_progress_wrapper( + self, shard: ShardMetadata, progress: RepoDownloadProgress + ) -> None: + for callback in self.on_progress_callbacks: + callback(shard, progress) + + def on_progress( + self, callback: Callable[[ShardMetadata, RepoDownloadProgress], None] + ) -> None: + self.on_progress_callbacks.append(callback) + + async def ensure_shard( + self, shard: ShardMetadata, config_only: bool = False + ) -> Path: + allow_patterns = ["config.json"] if config_only else None + + target_dir, _ = await download_shard( + shard, + self.on_progress_wrapper, + max_parallel_downloads=self.max_parallel_downloads, + allow_patterns=allow_patterns, + ) + return target_dir + + async def get_shard_download_status( + self, + ) -> AsyncIterator[tuple[Path, RepoDownloadProgress]]: + async def _status_for_model( + model_id: str, + ) -> tuple[Path, RepoDownloadProgress]: + """Helper coroutine that builds the shard for a model and gets its download status.""" + shard = await build_full_shard(model_id) + return await download_shard( + shard, self.on_progress_wrapper, skip_download=True + ) + + # Kick off download status coroutines concurrently + tasks = [ + asyncio.create_task(_status_for_model(model_card.model_id)) + for model_card in MODEL_CARDS.values() + ] + + for task in asyncio.as_completed(tasks): + try: + yield await task + # TODO: except Exception + except Exception as e: + print("Error downloading shard:", e) + + async def get_shard_download_status_for_shard( + self, shard: ShardMetadata + ) -> RepoDownloadProgress: + _, progress = await download_shard( + shard, self.on_progress_wrapper, skip_download=True + ) + return progress diff --git a/src/exo/worker/download/shard_downloader.py b/src/exo/worker/download/shard_downloader.py new file mode 100644 index 00000000..a41b3eeb --- /dev/null +++ b/src/exo/worker/download/shard_downloader.py @@ -0,0 +1,139 @@ +from abc import ABC, abstractmethod +from datetime import timedelta +from pathlib import Path +from typing import AsyncIterator, Callable + +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.worker.shards import ( + PipelineShardMetadata, + ShardMetadata, +) +from exo.worker.download.download_utils import RepoDownloadProgress + + +# TODO: the PipelineShardMetadata getting reinstantiated is a bit messy. Shoudl this be a classmethod? +class ShardDownloader(ABC): + @abstractmethod + async def ensure_shard( + self, shard: ShardMetadata, config_only: bool = False + ) -> Path: + """ + Ensures that the shard is downloaded. + Does not allow multiple overlapping downloads at once. + If you try to download a Shard which overlaps a Shard that is already being downloaded, + the download will be cancelled and a new download will start. + + Args: + shard (Shard): The shard to download. + """ + + @abstractmethod + def on_progress( + self, callback: Callable[[ShardMetadata, RepoDownloadProgress], None] + ) -> None: + pass + + @abstractmethod + async def get_shard_download_status( + self, + ) -> AsyncIterator[tuple[Path, RepoDownloadProgress]]: + """Get the download status of shards. + + Yields: + tuple[Path, RepoDownloadProgress]: The path and progress of a shard download. + """ + yield ( + Path("/tmp/noop_shard"), + RepoDownloadProgress( + repo_id="noop", + repo_revision="noop", + shard=PipelineShardMetadata( + model_meta=ModelMetadata( + model_id=ModelId("noop"), + pretty_name="noope", + storage_size=Memory.from_bytes(0), + n_layers=1, + ), + device_rank=0, + world_size=1, + start_layer=0, + end_layer=1, + n_layers=1, + ), + completed_files=0, + total_files=0, + downloaded_bytes=Memory.from_bytes(0), + downloaded_bytes_this_session=Memory.from_bytes(0), + total_bytes=Memory.from_bytes(0), + overall_speed=0, + overall_eta=timedelta(seconds=0), + status="complete", + ), + ) + + @abstractmethod + async def get_shard_download_status_for_shard( + self, shard: ShardMetadata + ) -> RepoDownloadProgress: ... + + +class NoopShardDownloader(ShardDownloader): + async def ensure_shard( + self, shard: ShardMetadata, config_only: bool = False + ) -> Path: + return Path("/tmp/noop_shard") + + def on_progress( + self, callback: Callable[[ShardMetadata, RepoDownloadProgress], None] + ) -> None: + pass + + async def get_shard_download_status( + self, + ) -> AsyncIterator[tuple[Path, RepoDownloadProgress]]: + yield ( + Path("/tmp/noop_shard"), + RepoDownloadProgress( + repo_id="noop", + repo_revision="noop", + shard=PipelineShardMetadata( + model_meta=ModelMetadata( + model_id=ModelId("noop"), + pretty_name="noope", + storage_size=Memory.from_bytes(0), + n_layers=1, + ), + device_rank=0, + world_size=1, + start_layer=0, + end_layer=1, + n_layers=1, + ), + completed_files=0, + total_files=0, + downloaded_bytes=Memory.from_bytes(0), + downloaded_bytes_this_session=Memory.from_bytes(0), + total_bytes=Memory.from_bytes(0), + overall_speed=0, + overall_eta=timedelta(seconds=0), + status="complete", + ), + ) + + async def get_shard_download_status_for_shard( + self, shard: ShardMetadata + ) -> RepoDownloadProgress: + return RepoDownloadProgress( + repo_id="noop", + repo_revision="noop", + shard=shard, + completed_files=0, + total_files=0, + downloaded_bytes=Memory.from_bytes(0), + downloaded_bytes_this_session=Memory.from_bytes(0), + total_bytes=Memory.from_bytes(0), + overall_speed=0, + overall_eta=timedelta(seconds=0), + status="complete", + ) diff --git a/src/exo/worker/engines/__init__.py b/src/exo/worker/engines/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/engines/mlx/__init__.py b/src/exo/worker/engines/mlx/__init__.py new file mode 100644 index 00000000..d6f0b6b3 --- /dev/null +++ b/src/exo/worker/engines/mlx/__init__.py @@ -0,0 +1,43 @@ +from typing import Any + +import mlx.core as mx +import mlx.nn as nn +from mlx_lm.models.cache import KVCache + +# These are wrapper functions to fix the fact that mlx is not strongly typed in the same way that EXO is. +# For example - MLX has no guarantee of the interface that nn.Module will expose. But we need a guarantee that it has a __call__() function + + +class Model(nn.Module): + layers: list[nn.Module] + + def __call__( + self, + x: mx.array, + cache: list[KVCache] | None, + input_embeddings: mx.array | None = None, + ) -> mx.array: ... + + +class Detokenizer: + def reset(self) -> None: ... + def add_token(self, token: int) -> None: ... + def finalize(self) -> None: ... + + @property + def last_segment(self) -> str: ... + + +class TokenizerWrapper: + bos_token: str | None + eos_token_ids: list[int] + detokenizer: Detokenizer + + def encode(self, text: str, add_special_tokens: bool = True) -> list[int]: ... + + def apply_chat_template( + self, + messages_dicts: list[dict[str, Any]], + tokenize: bool = False, + add_generation_prompt: bool = True, + ) -> str: ... diff --git a/src/exo/worker/engines/mlx/auto_parallel.py b/src/exo/worker/engines/mlx/auto_parallel.py new file mode 100644 index 00000000..1a6542f1 --- /dev/null +++ b/src/exo/worker/engines/mlx/auto_parallel.py @@ -0,0 +1,383 @@ +from abc import ABC, abstractmethod +from functools import partial +from inspect import signature +from typing import TYPE_CHECKING, Callable, Protocol, cast + +import mlx.core as mx +import mlx.nn as nn +from mlx.nn.layers.distributed import ( + shard_inplace, + shard_linear, + sum_gradients, +) +from mlx_lm.models.cache import ( + _BaseCache, # pyright: ignore[reportPrivateUsage] +) +from mlx_lm.models.deepseek_v3 import DeepseekV3MLP +from mlx_lm.models.deepseek_v3 import Model as DeepseekV3Model +from mlx_lm.models.llama import Model as LlamaModel +from mlx_lm.models.qwen3_moe import Model as Qwen3MoeModel +from mlx_lm.models.qwen3_moe import Qwen3MoeSparseMoeBlock + +from exo.shared.types.worker.shards import ( + PipelineShardMetadata, +) + + +class _LayerCallable(Protocol): + """Structural type that any compatible layer must satisfy. + + We require a single positional input of type ``mx.array`` and an + ``mx.array`` output, while permitting arbitrary *args / **kwargs so this + protocol matches the vast majority of `mlx.nn.Module` subclasses. + """ + + def __call__(self, x: mx.array, *args: object, **kwargs: object) -> mx.array: ... + + +class CustomMlxLayer(nn.Module): + """Base class for replacing an MLX layer with a custom implementation.""" + + def __init__(self, original_layer: _LayerCallable): + super().__init__() + # Set twice to avoid __setattr__ recursion + object.__setattr__(self, "_original_layer", original_layer) + self.original_layer: _LayerCallable = original_layer + + # Calls __getattr__ for any attributes not found on nn.Module (e.g. use_sliding) + if not TYPE_CHECKING: + + def __getattr__(self, name): + try: + return super().__getattr__(name) + except AttributeError: + original_layer = object.__getattribute__(self, "_original_layer") + return object.__getattribute__(original_layer, name) + + +class PipelineFirstLayer(CustomMlxLayer): + def __init__( + self, + original_layer: _LayerCallable, + r: int, + group: mx.distributed.Group, + ): + super().__init__(original_layer) + self.r: int = r + self.group = group + + def __call__(self, x: mx.array, *args: object, **kwargs: object) -> mx.array: + if self.r != 0: + x = mx.distributed.recv_like(x, (self.r - 1), group=self.group) + return self.original_layer(x, *args, **kwargs) + + +class PipelineLastLayer(CustomMlxLayer): + def __init__( + self, + original_layer: _LayerCallable, + r: int, + s: int, + group: mx.distributed.Group, + ): + super().__init__(original_layer) + self.r: int = r + self.s: int = s + self.group = group + self.original_layer_signature = signature(self.original_layer.__call__) + + def __call__(self, x: mx.array, *args: object, **kwargs: object) -> mx.array: + cache = self.original_layer_signature.bind_partial( + x, *args, **kwargs + ).arguments.get("cache", None) + + assert cache is None or issubclass(type(cache), _BaseCache) # type: ignore + + output: mx.array = self.original_layer(x, *args, **kwargs) + + if self.r != self.s - 1: + output = mx.distributed.send( + output, (self.r + 1) % self.s, group=self.group + ) + if cache is not None: + # This change happened upstream - check out mlx github somewhere?? + cache.keys = mx.depends(cache.keys, output) # type: ignore[reportUnknownMemberType] + + output = mx.distributed.all_gather(output, group=self.group)[-output.shape[0] :] + return output + + +def _inner_model(model: nn.Module) -> nn.Module: + inner = getattr(model, "model", None) + if isinstance(inner, nn.Module): + return inner + + inner = getattr(model, "transformer", None) + if isinstance(inner, nn.Module): + return inner + + raise ValueError("Model must either have a 'model' or 'transformer' attribute") + + +def _get_layers(inner_model_instance: nn.Module) -> list[_LayerCallable]: + # Handle both model.layers and model.h cases + layers: list[_LayerCallable] + if hasattr(inner_model_instance, "layers"): + layers = cast(list[_LayerCallable], inner_model_instance.layers) + elif hasattr(inner_model_instance, "h"): + layers = cast(list[_LayerCallable], inner_model_instance.h) + else: + raise ValueError("Model must have either a 'layers' or 'h' attribute") + + return layers + + +def _set_layers(model: nn.Module, layers: list[_LayerCallable]) -> None: + inner_model_instance = _inner_model(model) + if hasattr(inner_model_instance, "layers"): + inner_model_instance.layers = layers + + # Update DeepSeek V3 specific parameters when layers are shrunk + if isinstance(model, DeepseekV3Model) and hasattr( + inner_model_instance, "num_layers" + ): + inner_model_instance.start_idx = 0 + inner_model_instance.end_idx = len(layers) + inner_model_instance.num_layers = len(layers) + elif hasattr(inner_model_instance, "h"): + inner_model_instance.h = layers + else: + raise ValueError("Model must have either a 'layers' or 'h' attribute") + + +def pipeline_auto_parallel( + model: nn.Module, + group: mx.distributed.Group, + model_shard_meta: PipelineShardMetadata, +) -> nn.Module: + """ + Automatically parallelize a model across multiple devices. + Args: + model: The model to parallelize (must have a 'layers' or 'h' property) + model_shard_meta: The metadata for the model shard + Returns: + The parallelized model + """ + inner_model_instance: nn.Module = _inner_model(model) + + # Handle both model.layers and model.h cases + layers: list[_LayerCallable] = _get_layers(inner_model_instance) + + start_layer, end_layer = model_shard_meta.start_layer, model_shard_meta.end_layer + device_rank, world_size = model_shard_meta.device_rank, model_shard_meta.world_size + + layers = layers[start_layer:end_layer] + layers[0] = PipelineFirstLayer(layers[0], device_rank, group=group) + layers[-1] = PipelineLastLayer( + layers[-1], + device_rank, + world_size, + group=group, + ) + + _set_layers(model, layers) + + assert isinstance(layers, list), ( + "Expected a list of layers after auto-parallel initialisation" + ) + + return model + + +def tensor_auto_parallel( + model: nn.Module, + group: mx.distributed.Group, +) -> nn.Module: + all_to_sharded_linear = partial( + shard_linear, + sharding="all-to-sharded", + group=group, + ) + sharded_to_all_linear = partial( + shard_linear, + sharding="sharded-to-all", + group=group, + ) + + all_to_sharded_linear_in_place = partial( + shard_inplace, + sharding="all-to-sharded", + group=group, + ) + sharded_to_all_linear_in_place = partial( + shard_inplace, + sharding="sharded-to-all", + group=group, + ) + + if isinstance(model, LlamaModel): + tensor_parallel_sharding_strategy = LlamaShardingStrategy( + group, + all_to_sharded_linear, + sharded_to_all_linear, + all_to_sharded_linear_in_place, + sharded_to_all_linear_in_place, + ) + elif isinstance(model, DeepseekV3Model): + tensor_parallel_sharding_strategy = DeepSeekShardingStrategy( + group, + all_to_sharded_linear, + sharded_to_all_linear, + all_to_sharded_linear_in_place, + sharded_to_all_linear_in_place, + ) + elif isinstance(model, Qwen3MoeModel): + tensor_parallel_sharding_strategy = QwenShardingStrategy( + group, + all_to_sharded_linear, + sharded_to_all_linear, + all_to_sharded_linear_in_place, + sharded_to_all_linear_in_place, + ) + else: + raise ValueError(f"Unsupported model type: {type(model)}") + + return tensor_parallel_sharding_strategy.shard_model(model) + + +class TensorParallelShardingStrategy(ABC): + def __init__( + self, + group: mx.distributed.Group, + all_to_sharded_linear: Callable[..., nn.Linear], + sharded_to_all_linear: Callable[..., nn.Linear], + all_to_sharded_linear_in_place: Callable[..., None], + sharded_to_all_linear_in_place: Callable[..., None], + ): + self.all_to_sharded_linear = all_to_sharded_linear + self.sharded_to_all_linear = sharded_to_all_linear + self.all_to_sharded_linear_in_place = all_to_sharded_linear_in_place + self.sharded_to_all_linear_in_place = sharded_to_all_linear_in_place + self.group = group + self.N = group.size() + + @abstractmethod + def shard_model(self, model: nn.Module) -> nn.Module: ... + + +class LlamaShardingStrategy(TensorParallelShardingStrategy): + def shard_model(self, model: nn.Module) -> nn.Module: + model = cast(LlamaModel, model) + for layer in model.layers: + layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj) + layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj) + layer.self_attn.v_proj = self.all_to_sharded_linear(layer.self_attn.v_proj) + layer.self_attn.o_proj = self.sharded_to_all_linear(layer.self_attn.o_proj) + layer.self_attn.n_heads //= self.N + if layer.self_attn.n_kv_heads is not None: + layer.self_attn.n_kv_heads //= self.N + + layer.mlp.gate_proj = self.all_to_sharded_linear(layer.mlp.gate_proj) + layer.mlp.down_proj = self.sharded_to_all_linear(layer.mlp.down_proj) + layer.mlp.up_proj = self.all_to_sharded_linear(layer.mlp.up_proj) + + return model + + +class DeepSeekShardingStrategy(TensorParallelShardingStrategy): + def shard_model(self, model: nn.Module) -> nn.Module: + model = cast(DeepseekV3Model, model) + for layer in model.layers: + # Shard the self attention + if layer.self_attn.q_lora_rank is None: # pyright: ignore[reportUnnecessaryComparison] + # Unfortunately, q_lora_rank can be None despite typing hints. + layer.self_attn.q_proj = self.all_to_sharded_linear( + layer.self_attn.q_proj + ) + else: + layer.self_attn.q_b_proj = self.all_to_sharded_linear( + layer.self_attn.q_b_proj + ) + layer.self_attn.kv_b_proj = self.all_to_sharded_linear( + layer.self_attn.kv_b_proj + ) + layer.self_attn.o_proj = self.sharded_to_all_linear(layer.self_attn.o_proj) + layer.self_attn.num_heads //= self.N + + # Shard the MLP + if isinstance(layer.mlp, DeepseekV3MLP): + layer.mlp.gate_proj = self.all_to_sharded_linear(layer.mlp.gate_proj) + layer.mlp.down_proj = self.sharded_to_all_linear(layer.mlp.down_proj) + layer.mlp.up_proj = self.all_to_sharded_linear(layer.mlp.up_proj) + + # Shard the MoE. Shard in place since the MoE should be responsible + # for aggregating the results. + else: + self.all_to_sharded_linear_in_place(layer.mlp.shared_experts.gate_proj) + self.sharded_to_all_linear_in_place(layer.mlp.shared_experts.down_proj) + self.all_to_sharded_linear_in_place(layer.mlp.shared_experts.up_proj) + self.all_to_sharded_linear_in_place(layer.mlp.switch_mlp.gate_proj) + self.sharded_to_all_linear_in_place(layer.mlp.switch_mlp.down_proj) + self.all_to_sharded_linear_in_place(layer.mlp.switch_mlp.up_proj) + layer.mlp = ShardedDeepseekV3MoE(layer.mlp) # type: ignore + layer.mlp.sharding_group = self.group + + return model + + +class ShardedDeepseekV3MoE(CustomMlxLayer): + def __init__(self, layer: _LayerCallable): + super().__init__(layer) + self.sharding_group: mx.distributed.Group | None = None + + def __call__(self, x: mx.array) -> mx.array: + if self.sharding_group is not None: + x = sum_gradients(self.sharding_group)(x) + y = self.original_layer.__call__(x) + if self.sharding_group is not None: + y = mx.distributed.all_sum(y, group=self.sharding_group) + return y + + +class QwenShardingStrategy(TensorParallelShardingStrategy): + def shard_model(self, model: nn.Module) -> nn.Module: + model = cast(Qwen3MoeModel, model) + for layer in model.layers: + # Shard the self attention + layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj) + layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj) + layer.self_attn.v_proj = self.all_to_sharded_linear(layer.self_attn.v_proj) + layer.self_attn.o_proj = self.sharded_to_all_linear(layer.self_attn.o_proj) + layer.self_attn.n_heads //= self.N + layer.self_attn.n_kv_heads //= self.N + + # Shard the MoE. Shard in place since the MoE should be responsible + # for aggregating the results. + if isinstance(layer.mlp, Qwen3MoeSparseMoeBlock): + self.all_to_sharded_linear_in_place(layer.mlp.switch_mlp.gate_proj) + self.sharded_to_all_linear_in_place(layer.mlp.switch_mlp.down_proj) + self.all_to_sharded_linear_in_place(layer.mlp.switch_mlp.up_proj) + layer.mlp = ShardedQwenMoE(layer.mlp) # type: ignore + layer.mlp.sharding_group = self.group + + # Shard the MLP + else: + layer.mlp.gate_proj = self.all_to_sharded_linear(layer.mlp.gate_proj) + layer.mlp.down_proj = self.sharded_to_all_linear(layer.mlp.down_proj) + layer.mlp.up_proj = self.all_to_sharded_linear(layer.mlp.up_proj) + + return model + + +class ShardedQwenMoE(CustomMlxLayer): + def __init__(self, layer: _LayerCallable): + super().__init__(layer) + self.sharding_group: mx.distributed.Group | None = None + + def __call__(self, x: mx.array) -> mx.array: + if self.sharding_group is not None: + x = sum_gradients(self.sharding_group)(x) + y = self.original_layer.__call__(x) + if self.sharding_group is not None: + y = mx.distributed.all_sum(y, group=self.sharding_group) + return y diff --git a/src/exo/worker/engines/mlx/cache.py b/src/exo/worker/engines/mlx/cache.py new file mode 100644 index 00000000..8a7f828b --- /dev/null +++ b/src/exo/worker/engines/mlx/cache.py @@ -0,0 +1,104 @@ +# type: ignore +# TODO: Fix this file, including types! +from copy import deepcopy +from typing import Callable + +import mlx.core as mx +from mlx_lm import stream_generate +from mlx_lm.models.cache import _BaseCache, trim_prompt_cache +from mlx_lm.tokenizer_utils import TokenizerWrapper + +from exo.worker.engines.mlx import Model +from exo.worker.engines.mlx.constants import KEEP_KV_SIZE, KV_BITS, KV_GROUP_SIZE +from exo.worker.engines.mlx.utils_mlx import make_kv_cache + + +class KVPrefixCache: + def __init__(self): + # Only one prefix cache per runner. + self.prompts: list[mx.array] = [] # mx array of tokens (ints) + self.caches: list[list[_BaseCache]] = [] + + def add_kv_cache( + self, tokenizer: TokenizerWrapper, prompt: str, cache: list[_BaseCache] + ): + tokenized_prompt = self.encode_prompt(tokenizer, prompt) + self.prompts.append(tokenized_prompt) + self.caches.append(deepcopy(cache)) + + def get_kv_cache( + self, + model: Model, + tokenizer: TokenizerWrapper, + sampler: Callable[[mx.array], mx.array], + prompt: str, + ) -> list[_BaseCache]: + tokenized_prompt = self.encode_prompt(tokenizer, prompt) + max_length = len(tokenized_prompt) + + best_snapshot_index, best_snapshot_length = None, 0 + + for i, cached_prompt in enumerate(self.prompts): + length = _get_prefix_length(tokenized_prompt, cached_prompt) + + if length == max_length: + return self.caches[i] + + if length > best_snapshot_length: + best_snapshot_index, best_snapshot_length = i, length + + if best_snapshot_index is not None: + prompt_cache = deepcopy(self.caches[best_snapshot_index]) + trim_prompt_cache(prompt_cache, max_length - best_snapshot_length) + tokenized_prompt = tokenized_prompt[best_snapshot_index:] + + else: + prompt_cache = make_kv_cache( + model, + # max_kv_size=MAX_KV_SIZE, + # keep=KEEP_KV_SIZE + ) + + prefill(model, tokenizer, sampler, tokenized_prompt, prompt_cache) + + return prompt_cache + + def encode_prompt(self, tokenizer: TokenizerWrapper, prompt: str) -> mx.array: + add_special_tokens = tokenizer.bos_token is None or not prompt.startswith( + tokenizer.bos_token + ) + tokenized_prompt = tokenizer.encode( + prompt, add_special_tokens=add_special_tokens + ) + return mx.array(tokenized_prompt) + + +def _get_prefix_length(prompt: mx.array, cached_prompt: mx.array) -> int: + n = min(int(prompt.shape[0]), int(cached_prompt.shape[0]), KEEP_KV_SIZE) + if n == 0: + return 0 + + equal = (prompt[:n] == cached_prompt[:n]).astype(mx.int32) + prefix_mask = mx.cumprod(equal) # stays 1 until first mismatch, then 0 forever + return int(mx.sum(prefix_mask).item()) + + +def prefill( + model: Model, + tokenizer: TokenizerWrapper, + sampler: Callable[[mx.array], mx.array], + prompt: mx.array, + cache: list[_BaseCache], +) -> None: + for _ in stream_generate( + model=model, + tokenizer=tokenizer, + prompt=prompt, + max_tokens=0, + sampler=sampler, + prompt_cache=cache, + prefill_step_size=2048, + kv_group_size=KV_GROUP_SIZE, + kv_bits=KV_BITS, + ): + pass diff --git a/src/exo/worker/engines/mlx/constants.py b/src/exo/worker/engines/mlx/constants.py new file mode 100644 index 00000000..9b5db542 --- /dev/null +++ b/src/exo/worker/engines/mlx/constants.py @@ -0,0 +1,16 @@ +# TODO: Do we want so many constants? +# I think we want a lot of these as parameters? + +KV_GROUP_SIZE: int | None = 32 +KV_BITS: int | None = None +ATTENTION_KV_BITS: int | None = 4 +MAX_TOKENS: int = 8192 +MAX_KV_SIZE: int | None = 3200 +KEEP_KV_SIZE: int | None = 1600 +QUANTIZE_MODEL_MODE: str | None = "affine" +CACHE_GROUP_SIZE: int = 64 +KV_CACHE_BITS: int | None = 8 +TEMPERATURE: float = 1.0 + +# TODO: We should really make this opt-in, but Kimi requires trust_remote_code=True +TRUST_REMOTE_CODE: bool = True diff --git a/src/exo/worker/engines/mlx/generator/__init__.py b/src/exo/worker/engines/mlx/generator/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/engines/mlx/generator/generate.py b/src/exo/worker/engines/mlx/generator/generate.py new file mode 100644 index 00000000..9d90da06 --- /dev/null +++ b/src/exo/worker/engines/mlx/generator/generate.py @@ -0,0 +1,133 @@ +from typing import Any, Callable, Generator, cast, get_args + +import mlx.core as mx +from mlx_lm import stream_generate +from mlx_lm.models.cache import KVCache +from mlx_lm.tokenizer_utils import TokenizerWrapper + +# from exo.engines.mlx.cache import KVPrefixCache +from exo.shared.types.api import ChatCompletionMessage, FinishReason +from exo.shared.types.tasks import ChatCompletionTaskParams +from exo.shared.types.worker.runner_response import ( + GenerationResponse, +) +from exo.worker.engines.mlx import Model +from exo.worker.engines.mlx.constants import KV_BITS, KV_GROUP_SIZE, MAX_TOKENS +from exo.worker.engines.mlx.utils_mlx import ( + apply_chat_template, + make_kv_cache, + mx_barrier, +) +from exo.worker.runner.bootstrap import logger + +generation_stream = mx.new_stream(mx.default_device()) + + +def maybe_quantize_kv_cache( + prompt_cache: list[KVCache | Any], + quantized_kv_start: int, + kv_group_size: int, + kv_bits: int | None, +) -> None: + if kv_bits is None: + return + for e, c in enumerate(prompt_cache): + if ( + hasattr(c, "to_quantized") and c.offset >= quantized_kv_start # type: ignore + ): + prompt_cache[e] = c.to_quantized(group_size=kv_group_size, bits=kv_bits) + + +def warmup_inference( + model: Model, + tokenizer: TokenizerWrapper, + sampler: Callable[[mx.array], mx.array], +) -> int: + content = "Prompt to warm up the inference engine. Repeat this." + + warmup_prompt = apply_chat_template( + tokenizer=tokenizer, + chat_task_data=ChatCompletionTaskParams( + model="", + messages=[ + ChatCompletionMessage( + role="user", + content=content, + ) + ], + ), + ) + + tokens_generated = 0 + + cache = make_kv_cache( + model=model, + ) + + logger.info("Generating warmup tokens") + for _r in stream_generate( + model=model, + tokenizer=tokenizer, + prompt=warmup_prompt, + max_tokens=50, + sampler=sampler, + prompt_cache=cache, + prefill_step_size=65536, + kv_group_size=KV_GROUP_SIZE, + kv_bits=KV_BITS, + ): + logger.info("Generated warmup token: " + str(_r.text)) + tokens_generated += 1 + + logger.info("Generated ALL warmup tokens") + mx_barrier() + + return tokens_generated + + +def mlx_generate( + model: Model, + tokenizer: TokenizerWrapper, + sampler: Callable[[mx.array], mx.array], + task: ChatCompletionTaskParams, +) -> Generator[GenerationResponse]: + # Currently we support chat-completion tasks only. + logger.info(f"task_params: {task}") + + prompt = apply_chat_template( + tokenizer=tokenizer, + chat_task_data=task, + ) + + caches = make_kv_cache(model=model) + + max_tokens = task.max_tokens or MAX_TOKENS + for out in stream_generate( + model=model, + tokenizer=tokenizer, + prompt=prompt, + max_tokens=max_tokens, + sampler=sampler, + prompt_cache=caches, + prefill_step_size=65536, + kv_group_size=KV_GROUP_SIZE, + kv_bits=KV_BITS, + ): + logger.info(out.text) + if out.finish_reason is not None and out.finish_reason not in get_args( + FinishReason + ): + # We don't throw here as this failure case is really not all that bad + # Just log the error and move on + logger.warning( + f"Model generated unexpected finish_reason: {out.finish_reason}" + ) + + yield GenerationResponse( + text=out.text, + token=out.token, + finish_reason=cast(FinishReason | None, out.finish_reason), + ) + + if out.finish_reason is not None: + break diff --git a/src/exo/worker/engines/mlx/utils_mlx.py b/src/exo/worker/engines/mlx/utils_mlx.py new file mode 100644 index 00000000..3606b90b --- /dev/null +++ b/src/exo/worker/engines/mlx/utils_mlx.py @@ -0,0 +1,392 @@ +import json +import os +import resource +import time +from pathlib import Path +from typing import Any, Callable, cast + +from mlx_lm.models.cache import KVCache, QuantizedKVCache, RotatingKVCache +from mlx_lm.models.deepseek_v3 import DeepseekV3Model +from mlx_lm.sample_utils import make_sampler +from mlx_lm.tokenizer_utils import TokenizerWrapper + +from exo.worker.engines.mlx.constants import ( + CACHE_GROUP_SIZE, + KV_CACHE_BITS, + TEMPERATURE, + TRUST_REMOTE_CODE, +) + +try: + from mlx_lm.tokenizer_utils import load_tokenizer +except ImportError: + from mlx_lm.tokenizer_utils import load as load_tokenizer # type: ignore +import mlx.core as mx +import mlx.nn as nn +from mlx_lm.utils import load_model +from pydantic import RootModel + +from exo.shared.types.api import ChatCompletionMessageText +from exo.shared.types.common import Host +from exo.shared.types.memory import Memory +from exo.shared.types.tasks import ChatCompletionTaskParams +from exo.shared.types.worker.instances import ( + BoundInstance, + MlxJacclInstance, + MlxRingInstance, +) +from exo.shared.types.worker.shards import ( + PipelineShardMetadata, + ShardMetadata, + TensorShardMetadata, +) +from exo.worker.download.download_utils import build_model_path +from exo.worker.engines.mlx import Model +from exo.worker.engines.mlx.auto_parallel import ( + pipeline_auto_parallel, + tensor_auto_parallel, +) +from exo.worker.runner.bootstrap import logger + +# Needed for 8 bit model +resource.setrlimit(resource.RLIMIT_NOFILE, (2048, 4096)) + + +# TODO: Test this +# ALSO https://github.com/exo-explore/exo/pull/233#discussion_r2549683673 +def get_weights_size(model_shard_meta: ShardMetadata) -> Memory: + return Memory.from_float_kb( + (model_shard_meta.end_layer - model_shard_meta.start_layer) + / model_shard_meta.n_layers + * model_shard_meta.model_meta.storage_size.in_kb + / ( + 1 + if isinstance(model_shard_meta, PipelineShardMetadata) + else model_shard_meta.world_size + ) + ) + + +def mx_barrier(group: mx.distributed.Group | None = None): + mx.eval( + mx.distributed.all_sum( + mx.array(1.0), + stream=mx.default_stream(mx.Device(mx.cpu)), + group=group, + ) + ) + + +def broadcast_from_zero(value: int, group: mx.distributed.Group | None = None): + if group is None: + return value + + if group.rank() == 0: + a = mx.array([value], dtype=mx.int32) + else: + a = mx.array([0], dtype=mx.int32) + + m = mx.distributed.all_sum(a, stream=mx.Device(mx.DeviceType.cpu), group=group) + mx.eval(m) + return int(m.item()) + + +class HostList(RootModel[list[str]]): + @classmethod + def from_hosts(cls, hosts: list[Host]) -> "HostList": + return cls(root=[str(host) for host in hosts]) + + +def mlx_distributed_init( + bound_instance: BoundInstance, +) -> mx.distributed.Group: + """ + Initialize the MLX distributed (runs in thread pool). + + Either hosts or mlx_ibv_devices must be provided: + - hosts: traditional host-based connectivity using MLX_HOSTFILE + - mlx_ibv_devices: RDMA connectivity matrix using MLX_IBV_DEVICES + - mlx_ibv_coordinator: coordinator address (IP:PORT) for RDMA setup + - strict: if True, raise an error if the distributed backend is not available + """ + rank = bound_instance.bound_shard.device_rank + logger.info(f"Starting initialization for rank {rank}") + + # TODO: singleton instances + match bound_instance.instance: + case MlxRingInstance(hosts=hosts): + hostfile = f"./hosts_{rank}.json" + hosts_json = HostList.from_hosts(hosts).model_dump_json() + + with open(hostfile, "w") as f: + _ = f.write(hosts_json) + + logger.info(f"rank {rank} hostfile: {hostfile} hosts: {hosts_json}") + + os.environ["MLX_HOSTFILE"] = hostfile + os.environ["MLX_RANK"] = str(rank) + os.environ["MLX_RING_VERBOSE"] = "1" + group = mx.distributed.init(backend="ring", strict=True) + + case MlxJacclInstance( + ibv_devices=ibv_devices, ibv_coordinators=ibv_coordinators + ): + # Use RDMA connectivity matrix + devices_file = f"./hosts_{rank}.json" + ibv_devices_json = json.dumps(ibv_devices) + + with open(devices_file, "w") as f: + _ = f.write(ibv_devices_json) + + ibv_coordinator = ibv_coordinators[bound_instance.bound_node_id] + + logger.info(f"rank {rank} MLX_IBV_DEVICES: {ibv_devices_json}") + logger.info(f"rank {rank} MLX_IBV_COORDINATOR: {ibv_coordinator}") + os.environ["MLX_IBV_DEVICES"] = devices_file + os.environ["MLX_RANK"] = str(rank) + os.environ["MLX_IBV_COORDINATOR"] = ibv_coordinator + group = mx.distributed.init(backend="jaccl", strict=True) + + logger.info(f"Rank {rank} mlx distributed initialization complete") + + return group + + +def initialize_mlx( + bound_instance: BoundInstance, +) -> tuple[Model, TokenizerWrapper, Callable[[mx.array], mx.array]]: + """ + Initialize the MLX model, tokenizer, and sampler. Runs in the MLX thread. + """ + mx.random.seed(42) + + set_wired_limit_for_model(get_weights_size(bound_instance.bound_shard)) + + sampler: Callable[[mx.array], mx.array] = make_sampler(temp=TEMPERATURE) + logger.info("Created a sampler") + + if len(bound_instance.instance.shard_assignments.node_to_runner) <= 1: + logger.info(f"Single device used for {bound_instance.instance}") + model_path = build_model_path(bound_instance.bound_shard.model_meta.model_id) + start_time = time.perf_counter() + model, _ = load_model(model_path, strict=True) + end_time = time.perf_counter() + logger.info(f"Time taken to load model: {(end_time - start_time):.2f}s") + if hasattr(model, "model") and isinstance(model.model, DeepseekV3Model): # type: ignore + pass + # model, config = quantize_model( + # model, config, group_size=KV_GROUP_SIZE, bits=ATTENTION_KV_BITS, quant_predicate=quant_predicate, mode=QUANTIZE_MODEL_MODE + # ) + + tokenizer = get_tokenizer(model_path, bound_instance.bound_shard) + + else: + logger.info("Starting distributed init") + group = mlx_distributed_init(bound_instance) + + start_time = time.perf_counter() + model, tokenizer = shard_and_load(bound_instance.bound_shard, group=group) + end_time = time.perf_counter() + logger.info( + f"Time taken to shard and load model: {(end_time - start_time):.2f}s" + ) + + set_wired_limit_for_model(get_weights_size(bound_instance.bound_shard)) + + logger.debug(model) + + return cast(Model, model), tokenizer, sampler + + +def shard_and_load( + shard_metadata: ShardMetadata, + group: mx.distributed.Group, +) -> tuple[nn.Module, TokenizerWrapper]: + model_path = build_model_path(shard_metadata.model_meta.model_id) + + model, _ = load_model(model_path, lazy=True, strict=False) + logger.debug(model) + if hasattr(model, "model") and isinstance(model.model, DeepseekV3Model): # type: ignore + pass + # TODO: See if we should quantize the model. + # def is_attention_layer(path: str) -> bool: + # path = path.lower() + + # return "self_attn" in path and "layernorm" not in path + + # def quant_predicate(path: str, module: nn.Module): + # if not isinstance(module, nn.Linear): + # return False + + # return is_attention_layer(path) + # model, config = quantize_model( + # model, config, group_size=KV_GROUP_SIZE, bits=ATTENTION_KV_BITS, quant_predicate=quant_predicate, mode=QUANTIZE_MODEL_MODE + # ) + + assert isinstance(model, nn.Module) + + tokenizer = get_tokenizer(model_path, shard_metadata) + + logger.info(f"Group size: {group.size()}, group rank: {group.rank()}") + + match shard_metadata: + case TensorShardMetadata(): + logger.info(f"loading model from {model_path} with tensor parallelism") + model = tensor_auto_parallel(model, group) + case PipelineShardMetadata(): + logger.info(f"loading model from {model_path} with pipeline parallelism") + model = pipeline_auto_parallel(model, group, shard_metadata) + + mx.eval(model.parameters()) + + # TODO: Do we need this? + mx.eval(model) + + logger.debug("SHARDED") + logger.debug(model) + + # Synchronize processes before generation to avoid timeout + mx_barrier(group) + + return model, tokenizer + + +def get_tokenizer(model_path: Path, shard_metadata: ShardMetadata): + tokenizer = cast( + TokenizerWrapper, + load_tokenizer( + model_path, + tokenizer_config_extra={"trust_remote_code": TRUST_REMOTE_CODE}, + # TODO: HACK for Kimi K2 wrong eos token id + eos_token_ids=[163586] + if "kimi-k2" in shard_metadata.model_meta.model_id.lower() + else None, + ), + ) + assert isinstance(tokenizer, TokenizerWrapper) + + return tokenizer + + +def apply_chat_template( + tokenizer: TokenizerWrapper, + chat_task_data: ChatCompletionTaskParams, +) -> str: + # Now we can properly access the messages + messages = chat_task_data.messages + + formatted_messages: list[dict[str, Any]] = [] + for _, message in enumerate(messages): + if isinstance(message.content, ChatCompletionMessageText): + message.content = message.content.text + if isinstance(message.content, list): + if len(message.content) != 1: + logger.warning("Received malformed prompt") + continue + + message.content = message.content[0].text + if message.content is None and message.thinking is None: + continue + + # Null values are not valid when applying templates in tokenizer + formatted_messages.append( + {k: v for k, v in message.model_dump().items() if v is not None} # type: ignore + ) + + prompt: str = tokenizer.apply_chat_template( # type: ignore + formatted_messages, + tokenize=False, + add_generation_prompt=True, + ) + + return prompt # type: ignore + + +class NullKVCache(KVCache): + """ + A KVCache that pretends to exist but holds zero tokens. + It satisfies .state/.meta_state and never allocates real keys/values. + """ + + def __init__(self, dtype: mx.Dtype = mx.float16): + super().__init__() + # zero-length K/V so shapes/dtypes are defined but empty + self.keys = mx.zeros((1, 1, 0, 1), dtype=dtype) + self.values = mx.zeros((1, 1, 0, 1), dtype=dtype) + self.offset = 0 + + @property + def state(self) -> tuple[mx.array, mx.array]: + # matches what mx.save_safetensors / mx.eval expect + return self.keys, self.values + + @state.setter + def state(self, v: tuple[mx.array, mx.array]) -> None: + raise NotImplementedError("We should not be setting a NullKVCache.") + + +def make_kv_cache( + model: Model, max_kv_size: int | None = None, keep: int = 0 +) -> list[KVCache | RotatingKVCache | QuantizedKVCache]: + assert hasattr(model, "layers") + + if max_kv_size is None: + if KV_CACHE_BITS is None: + logger.info("Using default KV cache") + return [KVCache() for _ in model.layers] + else: + logger.info("Using quantized KV cache") + return [ + QuantizedKVCache(group_size=CACHE_GROUP_SIZE, bits=KV_CACHE_BITS) + for _ in model.layers + ] + else: + logger.info(f"Using rotating KV cache with {max_kv_size=} with {keep=}") + return [RotatingKVCache(max_size=max_kv_size, keep=keep) for _ in model.layers] + + +def mlx_force_oom(size: int = 40000) -> None: + """ + Force an Out-Of-Memory (OOM) error in MLX by performing large tensor operations. + """ + mx.set_default_device(mx.gpu) + a = mx.random.uniform(shape=(size, size), dtype=mx.float32) + b = mx.random.uniform(shape=(size, size), dtype=mx.float32) + mx.eval(a, b) + c = mx.matmul(a, b) + d = mx.matmul(a, c) + e = mx.matmul(b, c) + f = mx.sigmoid(d + e) + mx.eval(f) + + +def set_wired_limit_for_model(model_size: Memory): + """ + A context manager to temporarily change the wired limit. + + Note, the wired limit should not be changed during an async eval. If an + async eval could be running pass in the streams to synchronize with prior + to exiting the context manager. + """ + if not mx.metal.is_available(): + return + + model_bytes = model_size.in_bytes + max_rec_size = int(mx.metal.device_info()["max_recommended_working_set_size"]) + if model_bytes > 0.9 * max_rec_size: + model_mb = model_bytes // 2**20 + max_rec_mb = max_rec_size // 2**20 + logger.warning( + f"Generating with a model that requires {model_mb} MB " + f"which is close to the maximum recommended size of {max_rec_mb} " + "MB. This can be slow. See the documentation for possible work-arounds: " + "https://github.com/ml-explore/mlx-lm/tree/main#large-models" + ) + kv_bytes = int(0.02 * model_bytes) + target_cache = int(1.10 * (model_bytes + kv_bytes)) + target_cache = min(target_cache, max_rec_size) + mx.set_cache_limit(target_cache) + mx.set_wired_limit(max_rec_size) + logger.info( + f"Wired limit set to {max_rec_size}. Cache limit set to {target_cache}." + ) diff --git a/src/exo/worker/main.py b/src/exo/worker/main.py new file mode 100644 index 00000000..a5c049dc --- /dev/null +++ b/src/exo/worker/main.py @@ -0,0 +1,441 @@ +from datetime import datetime, timezone +from random import random + +import anyio +from anyio import CancelScope, create_task_group, current_time, fail_after +from anyio.abc import TaskGroup +from loguru import logger + +from exo.routing.connection_message import ConnectionMessage, ConnectionMessageType +from exo.shared.apply import apply +from exo.shared.types.commands import ForwarderCommand, RequestEventLog +from exo.shared.types.common import NodeId, SessionId +from exo.shared.types.events import ( + Event, + EventId, + ForwarderEvent, + IndexedEvent, + NodeDownloadProgress, + NodeMemoryMeasured, + NodePerformanceMeasured, + TaskCreated, + TaskStatusUpdated, + TopologyEdgeCreated, + TopologyEdgeDeleted, +) +from exo.shared.types.multiaddr import Multiaddr +from exo.shared.types.profiling import MemoryPerformanceProfile, NodePerformanceProfile +from exo.shared.types.state import State +from exo.shared.types.tasks import ( + CreateRunner, + DownloadModel, + Shutdown, + Task, + TaskStatus, +) +from exo.shared.types.topology import Connection +from exo.shared.types.worker.downloads import ( + DownloadCompleted, + DownloadOngoing, + DownloadPending, + DownloadProgress, +) +from exo.shared.types.worker.runners import RunnerId +from exo.shared.types.worker.shards import ShardMetadata +from exo.utils.channels import Receiver, Sender, channel +from exo.utils.event_buffer import OrderedBuffer +from exo.worker.download.download_utils import ( + map_repo_download_progress_to_download_progress_data, +) +from exo.worker.download.shard_downloader import RepoDownloadProgress, ShardDownloader +from exo.worker.plan import plan +from exo.worker.runner.runner_supervisor import RunnerSupervisor +from exo.worker.utils import start_polling_memory_metrics, start_polling_node_metrics +from exo.worker.utils.net_profile import check_reachable + + +class Worker: + def __init__( + self, + node_id: NodeId, + session_id: SessionId, + shard_downloader: ShardDownloader, + *, + connection_message_receiver: Receiver[ConnectionMessage], + global_event_receiver: Receiver[ForwarderEvent], + local_event_sender: Sender[ForwarderEvent], + # This is for requesting updates. It doesn't need to be a general command sender right now, + # but I think it's the correct way to be thinking about commands + command_sender: Sender[ForwarderCommand], + ): + self.node_id: NodeId = node_id + self.session_id: SessionId = session_id + + self.shard_downloader: ShardDownloader = shard_downloader + self._pending_downloads: dict[RunnerId, ShardMetadata] = {} + + self.global_event_receiver = global_event_receiver + self.local_event_sender = local_event_sender + self.local_event_index = 0 + self.command_sender = command_sender + self.connection_message_receiver = connection_message_receiver + self.event_buffer = OrderedBuffer[Event]() + self.out_for_delivery: dict[EventId, ForwarderEvent] = {} + + self.state: State = State() + self.download_status: dict[ShardMetadata, DownloadProgress] = {} + self.runners: dict[RunnerId, RunnerSupervisor] = {} + self._tg: TaskGroup | None = None + + self._nack_cancel_scope: CancelScope | None = None + self._nack_attempts: int = 0 + self._nack_base_seconds: float = 0.5 + self._nack_cap_seconds: float = 10.0 + + self.event_sender, self.event_receiver = channel[Event]() + + async def run(self): + logger.info("Starting Worker") + + # TODO: CLEANUP HEADER + async def resource_monitor_callback( + node_performance_profile: NodePerformanceProfile, + ) -> None: + await self.event_sender.send( + NodePerformanceMeasured( + node_id=self.node_id, + node_profile=node_performance_profile, + when=str(datetime.now(tz=timezone.utc)), + ), + ) + + async def memory_monitor_callback( + memory_profile: MemoryPerformanceProfile, + ) -> None: + await self.event_sender.send( + NodeMemoryMeasured( + node_id=self.node_id, + memory=memory_profile, + when=str(datetime.now(tz=timezone.utc)), + ) + ) + + # END CLEANUP + + async with create_task_group() as tg: + self._tg = tg + tg.start_soon(self.plan_step) + tg.start_soon(start_polling_node_metrics, resource_monitor_callback) + + tg.start_soon(start_polling_memory_metrics, memory_monitor_callback) + tg.start_soon(self._connection_message_event_writer) + tg.start_soon(self._resend_out_for_delivery) + tg.start_soon(self._event_applier) + tg.start_soon(self._forward_events) + tg.start_soon(self._poll_connection_updates) + + # Actual shutdown code - waits for all tasks to complete before executing. + self.local_event_sender.close() + self.command_sender.close() + for runner in self.runners.values(): + runner.shutdown() + + async def _event_applier(self): + with self.global_event_receiver as events: + async for f_event in events: + if f_event.origin != self.session_id.master_node_id: + continue + self.event_buffer.ingest(f_event.origin_idx, f_event.event) + event_id = f_event.event.event_id + if event_id in self.out_for_delivery: + del self.out_for_delivery[event_id] + + # 2. for each event, apply it to the state + indexed_events = self.event_buffer.drain_indexed() + if indexed_events: + self._nack_attempts = 0 + + if not indexed_events and ( + self._nack_cancel_scope is None + or self._nack_cancel_scope.cancel_called + ): + assert self._tg + # Request the next index. + self._tg.start_soon( + self._nack_request, self.state.last_event_applied_idx + 1 + ) + continue + elif indexed_events and self._nack_cancel_scope: + self._nack_cancel_scope.cancel() + + for idx, event in indexed_events: + self.state = apply(self.state, IndexedEvent(idx=idx, event=event)) + + async def plan_step(self): + while True: + await anyio.sleep(0.1) + # 3. based on the updated state, we plan & execute an operation. + task: Task | None = plan( + self.node_id, + self.runners, + self.download_status, + self.state.downloads, + self.state.instances, + self.state.runners, + self.state.tasks, + ) + if task is None: + continue + logger.info(f"Worker plan: {task.__class__.__name__}") + assert task.task_status + await self.event_sender.send(TaskCreated(task_id=task.task_id, task=task)) + + # lets not kill the worker if a runner is unresponsive + match task: + case CreateRunner(): + self._create_supervisor(task) + await self.event_sender.send( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.Complete + ) + ) + case DownloadModel(shard_metadata=shard): + if shard not in self.download_status: + progress = DownloadPending( + shard_metadata=shard, node_id=self.node_id + ) + self.download_status[shard] = progress + await self.event_sender.send( + NodeDownloadProgress(download_progress=progress) + ) + initial_progress = ( + await self.shard_downloader.get_shard_download_status_for_shard( + shard + ) + ) + if initial_progress.status == "complete": + progress = DownloadCompleted( + shard_metadata=shard, node_id=self.node_id + ) + self.download_status[shard] = progress + await self.event_sender.send( + NodeDownloadProgress(download_progress=progress) + ) + await self.event_sender.send( + TaskStatusUpdated( + task_id=task.task_id, + task_status=TaskStatus.Complete, + ) + ) + else: + self.event_sender.send_nowait( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.Running + ) + ) + self._handle_shard_download_process(task, initial_progress) + case Shutdown(runner_id=runner_id): + try: + with fail_after(3): + await self.runners.pop(runner_id).start_task(task) + except TimeoutError: + await self.event_sender.send( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.TimedOut + ) + ) + case task: + await self.runners[self._task_to_runner_id(task)].start_task(task) + + def shutdown(self): + if self._tg: + self._tg.cancel_scope.cancel() + + def _task_to_runner_id(self, task: Task): + instance = self.state.instances[task.instance_id] + return instance.shard_assignments.node_to_runner[self.node_id] + + async def _connection_message_event_writer(self): + with self.connection_message_receiver as connection_messages: + async for msg in connection_messages: + await self.event_sender.send( + self._convert_connection_message_to_event(msg) + ) + + def _convert_connection_message_to_event(self, msg: ConnectionMessage): + match msg.connection_type: + case ConnectionMessageType.Connected: + return TopologyEdgeCreated( + edge=Connection( + local_node_id=self.node_id, + send_back_node_id=msg.node_id, + send_back_multiaddr=Multiaddr( + address=f"/ip4/{msg.remote_ipv4}/tcp/{msg.remote_tcp_port}" + ), + ) + ) + + case ConnectionMessageType.Disconnected: + return TopologyEdgeDeleted( + edge=Connection( + local_node_id=self.node_id, + send_back_node_id=msg.node_id, + send_back_multiaddr=Multiaddr( + address=f"/ip4/{msg.remote_ipv4}/tcp/{msg.remote_tcp_port}" + ), + ) + ) + + async def _nack_request(self, since_idx: int) -> None: + # We request all events after (and including) the missing index. + # This function is started whenever we receive an event that is out of sequence. + # It is cancelled as soon as we receiver an event that is in sequence. + + if since_idx < 0: + logger.warning(f"Negative value encountered for nack request {since_idx=}") + since_idx = 0 + + with CancelScope() as scope: + self._nack_cancel_scope = scope + delay: float = self._nack_base_seconds * (2.0**self._nack_attempts) + delay = min(self._nack_cap_seconds, delay) + self._nack_attempts += 1 + try: + await anyio.sleep(delay) + logger.info( + f"Nack attempt {self._nack_attempts}: Requesting Event Log from {since_idx}" + ) + await self.command_sender.send( + ForwarderCommand( + origin=self.node_id, + command=RequestEventLog(since_idx=since_idx), + ) + ) + finally: + if self._nack_cancel_scope is scope: + self._nack_cancel_scope = None + + async def _resend_out_for_delivery(self) -> None: + # This can also be massively tightened, we should check events are at least a certain age before resending. + # Exponential backoff would also certainly help here. + while True: + await anyio.sleep(1 + random()) + for event in self.out_for_delivery.copy().values(): + await self.local_event_sender.send(event) + + ## Op Executors + + def _create_supervisor(self, task: CreateRunner) -> RunnerSupervisor: + """Creates and stores a new AssignedRunner with initial downloading status.""" + runner = RunnerSupervisor.create( + bound_instance=task.bound_instance, + event_sender=self.event_sender.clone(), + ) + self.runners[task.bound_instance.bound_runner_id] = runner + assert self._tg + self._tg.start_soon(runner.run) + return runner + + def _handle_shard_download_process( + self, + task: DownloadModel, + initial_progress: RepoDownloadProgress, + ): + """Manages the shard download process with progress tracking.""" + status = DownloadOngoing( + node_id=self.node_id, + shard_metadata=task.shard_metadata, + download_progress=map_repo_download_progress_to_download_progress_data( + initial_progress + ), + ) + self.download_status[task.shard_metadata] = status + self.event_sender.send_nowait(NodeDownloadProgress(download_progress=status)) + + last_progress_time = 0.0 + throttle_interval_secs = 1.0 + + # TODO: i hate callbacks + def download_progress_callback( + shard: ShardMetadata, progress: RepoDownloadProgress + ) -> None: + nonlocal self + nonlocal last_progress_time + if progress.status == "complete": + status = DownloadCompleted(shard_metadata=shard, node_id=self.node_id) + self.download_status[shard] = status + # Footgun! + self.event_sender.send_nowait( + NodeDownloadProgress(download_progress=status) + ) + self.event_sender.send_nowait( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.Complete + ) + ) + elif ( + progress.status == "in_progress" + and current_time() - last_progress_time > throttle_interval_secs + ): + status = DownloadOngoing( + node_id=self.node_id, + shard_metadata=shard, + download_progress=map_repo_download_progress_to_download_progress_data( + progress + ), + ) + self.download_status[shard] = status + self.event_sender.send_nowait( + NodeDownloadProgress(download_progress=status) + ) + last_progress_time = current_time() + + self.shard_downloader.on_progress(download_progress_callback) + assert self._tg + self._tg.start_soon(self.shard_downloader.ensure_shard, task.shard_metadata) + + async def _forward_events(self) -> None: + with self.event_receiver as events: + async for event in events: + fe = ForwarderEvent( + origin_idx=self.local_event_index, + origin=self.node_id, + session=self.session_id, + event=event, + ) + logger.debug( + f"Worker published event {self.local_event_index}: {str(event)[:100]}" + ) + self.local_event_index += 1 + await self.local_event_sender.send(fe) + self.out_for_delivery[event.event_id] = fe + + async def _poll_connection_updates(self): + while True: + # TODO: EdgeDeleted + edges = set(self.state.topology.list_connections()) + conns = await check_reachable(self.state.topology) + for nid in conns: + for ip in conns[nid]: + edge = Connection( + local_node_id=self.node_id, + send_back_node_id=nid, + # nonsense multiaddr + send_back_multiaddr=Multiaddr(address=f"/ip4/{ip}/tcp/8000") + if "." in ip + # nonsense multiaddr + else Multiaddr(address=f"/ip6/{ip}/tcp/8000"), + ) + if edge not in edges: + logger.debug(f"ping discovered {edge=}") + await self.event_sender.send(TopologyEdgeCreated(edge=edge)) + + for nid, conn in self.state.topology.out_edges(self.node_id): + if ( + nid not in conns + or conn.send_back_multiaddr.ip_address not in conns.get(nid, set()) + ): + logger.debug(f"ping failed to discover {conn=}") + await self.event_sender.send(TopologyEdgeDeleted(edge=conn)) + + await anyio.sleep(10) diff --git a/src/exo/worker/plan.py b/src/exo/worker/plan.py new file mode 100644 index 00000000..01106d24 --- /dev/null +++ b/src/exo/worker/plan.py @@ -0,0 +1,228 @@ +# pyright: reportUnusedImport = false + +from collections.abc import Mapping, Sequence + +from exo.shared.types.common import NodeId +from exo.shared.types.tasks import ( + ChatCompletion, + CreateRunner, + DownloadModel, + LoadModel, + Shutdown, + StartWarmup, + Task, + TaskId, + TaskStatus, +) +from exo.shared.types.worker.downloads import DownloadCompleted, DownloadProgress +from exo.shared.types.worker.instances import BoundInstance, Instance, InstanceId +from exo.shared.types.worker.runners import ( + RunnerFailed, + RunnerId, + RunnerLoaded, + RunnerLoading, + RunnerReady, + RunnerRunning, + RunnerStatus, + RunnerWaitingForModel, + RunnerWarmingUp, +) +from exo.shared.types.worker.shards import ShardMetadata +from exo.worker.runner.runner_supervisor import RunnerSupervisor + + +def plan( + node_id: NodeId, + # Runners is expected to be FRESH and so should not come from state + runners: Mapping[RunnerId, RunnerSupervisor], + # DL_status is expected to be FRESH and so should not come from state + download_status: Mapping[ShardMetadata, DownloadProgress], + # gdls is not expected to be fresh + global_download_status: Mapping[NodeId, Sequence[DownloadProgress]], + instances: Mapping[InstanceId, Instance], + all_runners: Mapping[RunnerId, RunnerStatus], # all global + tasks: Mapping[TaskId, Task], +) -> Task | None: + # Python short circuiting OR logic should evaluate these sequentially. + return ( + _kill_runner(runners, all_runners, instances) + or _create_runner(node_id, runners, instances) + or _model_needs_download(runners, download_status) + or _load_model(runners, all_runners, global_download_status) + or _ready_to_warmup(runners, all_runners) + or _pending_tasks(runners, tasks, all_runners) + ) + + +def _kill_runner( + runners: Mapping[RunnerId, RunnerSupervisor], + all_runners: Mapping[RunnerId, RunnerStatus], + instances: Mapping[InstanceId, Instance], +) -> Shutdown | None: + for runner in runners.values(): + runner_id = runner.bound_instance.bound_runner_id + if (instance_id := runner.bound_instance.instance.instance_id) not in instances: + return Shutdown(instance_id=instance_id, runner_id=runner_id) + + for ( + global_runner_id + ) in runner.bound_instance.instance.shard_assignments.node_to_runner.values(): + if runner_id == global_runner_id: + continue + + if isinstance(all_runners.get(global_runner_id, None), RunnerFailed): + return Shutdown( + instance_id=instance_id, + runner_id=runner_id, + ) + + +def _create_runner( + node_id: NodeId, + runners: Mapping[RunnerId, RunnerSupervisor], + instances: Mapping[InstanceId, Instance], +) -> CreateRunner | None: + for instance in instances.values(): + runner_id = instance.shard_assignments.node_to_runner.get(node_id, None) + if runner_id is None: + continue + + if runner_id in runners: + continue + + shard = instance.shard(runner_id) + assert shard is not None + + return CreateRunner( + instance_id=instance.instance_id, + bound_instance=BoundInstance( + instance=instance, bound_runner_id=runner_id, bound_node_id=node_id + ), + ) + + +def _model_needs_download( + runners: Mapping[RunnerId, RunnerSupervisor], + download_status: Mapping[ShardMetadata, DownloadProgress], +) -> DownloadModel | None: + for runner in runners.values(): + if ( + isinstance(runner.status, RunnerWaitingForModel) + and runner.bound_instance.bound_shard not in download_status + ): + # We don't invalidate download_status randomly in case a file gets deleted on disk + return DownloadModel( + instance_id=runner.bound_instance.instance.instance_id, + shard_metadata=runner.bound_instance.bound_shard, + ) + + +""" --- TODO! +def _init_backend( + runners: Mapping[RunnerId, RunnerSupervisor], + all_runners: Mapping[RunnerId, RunnerStatus], +) -> LoadModel | None: + for runner in runner.values() + pass +""" + + +def _load_model( + runners: Mapping[RunnerId, RunnerSupervisor], + all_runners: Mapping[RunnerId, RunnerStatus], + global_download_status: Mapping[NodeId, Sequence[DownloadProgress]], +) -> LoadModel | None: + for runner in runners.values(): + instance = runner.bound_instance.instance + shard_assignments = instance.shard_assignments + + all_downloads_complete_local = all( + nid in global_download_status + and any( + isinstance(dp, DownloadCompleted) + and dp.shard_metadata == shard_assignments.runner_to_shard[rid] + for dp in global_download_status[nid] + ) + for nid, rid in shard_assignments.node_to_runner.items() + ) + + runner_is_waiting = isinstance(runner.status, RunnerWaitingForModel) + + all_runners_expecting_model = all( + isinstance( + all_runners.get(global_runner_id), + (RunnerWaitingForModel, RunnerLoading, RunnerLoaded), + ) + for global_runner_id in shard_assignments.runner_to_shard + ) + + if ( + all_downloads_complete_local + and runner_is_waiting + and all_runners_expecting_model + ): + return LoadModel(instance_id=instance.instance_id) + + return None + + +def _ready_to_warmup( + runners: Mapping[RunnerId, RunnerSupervisor], + all_runners: Mapping[RunnerId, RunnerStatus], +) -> StartWarmup | None: + for runner in runners.values(): + instance = runner.bound_instance.instance + shard_assignments = instance.shard_assignments + shard = runner.bound_instance.bound_shard + device_rank = shard.device_rank + runner_id = runner.bound_instance.bound_runner_id + world_size = shard.world_size + + is_runner_loaded = isinstance(runner.status, RunnerLoaded) + + assert device_rank < world_size + assert device_rank >= 0 + + # Rank != n-1 + accepting_ranks_ready = device_rank != world_size - 1 and all( + isinstance( + all_runners.get(global_runner_id, None), + (RunnerLoaded, RunnerWarmingUp), + ) + for global_runner_id in shard_assignments.runner_to_shard + ) + + # Rank = n-1 + connecting_rank_ready = device_rank == world_size - 1 and all( + isinstance(all_runners.get(global_runner_id, None), RunnerWarmingUp) + for global_runner_id in shard_assignments.runner_to_shard + if global_runner_id != runner_id + ) + + if is_runner_loaded and (accepting_ranks_ready or connecting_rank_ready): + return StartWarmup(instance_id=instance.instance_id) + + return None + + +def _pending_tasks( + runners: Mapping[RunnerId, RunnerSupervisor], + tasks: Mapping[TaskId, Task], + all_runners: Mapping[RunnerId, RunnerStatus], +) -> Task | None: + for task in tasks.values(): + # for now, just forward chat completions + if not isinstance(task, ChatCompletion): + continue + if task.task_status not in (TaskStatus.Pending, TaskStatus.Running): + continue + + for runner in runners.values(): + if task.instance_id != runner.bound_instance.instance.instance_id: + continue + + if isinstance(runner.status, RunnerReady) and all( + isinstance(all_runners[global_runner_id], (RunnerReady, RunnerRunning)) + for global_runner_id in runner.bound_instance.instance.shard_assignments.runner_to_shard + ): + return task diff --git a/src/exo/worker/runner/bootstrap.py b/src/exo/worker/runner/bootstrap.py new file mode 100644 index 00000000..24d30cb8 --- /dev/null +++ b/src/exo/worker/runner/bootstrap.py @@ -0,0 +1,35 @@ +import os + +import loguru + +from exo.shared.types.events import Event +from exo.shared.types.tasks import Task +from exo.shared.types.worker.instances import BoundInstance, MlxJacclInstance +from exo.utils.channels import MpReceiver, MpSender + +logger: "loguru.Logger" + + +if os.getenv("EXO_TESTS") == "1": + logger = loguru.logger + + +def entrypoint( + bound_instance: BoundInstance, + event_sender: MpSender[Event], + task_receiver: MpReceiver[Task], + _logger: "loguru.Logger", +) -> None: + if ( + isinstance(bound_instance.instance, MlxJacclInstance) + and len(bound_instance.instance.ibv_devices) >= 2 + ): + os.environ["MLX_METAL_FAST_SYNCH"] = "1" + + global logger + logger = _logger + + # Import main after setting global logger - this lets us just import logger from this module + from exo.worker.runner.runner import main + + main(bound_instance, event_sender, task_receiver) diff --git a/src/exo/worker/runner/runner.py b/src/exo/worker/runner/runner.py new file mode 100644 index 00000000..c44783a3 --- /dev/null +++ b/src/exo/worker/runner/runner.py @@ -0,0 +1,241 @@ +import time + +from exo.shared.types.api import ChatCompletionMessageText +from exo.shared.types.chunks import TokenChunk +from exo.shared.types.events import ( + ChunkGenerated, + Event, + RunnerStatusUpdated, + TaskAcknowledged, + TaskStatusUpdated, +) +from exo.shared.types.tasks import ( + ChatCompletion, + LoadModel, + Shutdown, + StartWarmup, + Task, + TaskStatus, +) +from exo.shared.types.worker.instances import BoundInstance +from exo.shared.types.worker.runner_response import ( + GenerationResponse, +) +from exo.shared.types.worker.runners import ( + RunnerFailed, + RunnerLoaded, + RunnerLoading, + RunnerReady, + RunnerRunning, + RunnerShutdown, + RunnerStatus, + RunnerWaitingForModel, + RunnerWarmingUp, +) +from exo.utils.channels import ClosedResourceError, MpReceiver, MpSender +from exo.worker.engines.mlx.generator.generate import mlx_generate, warmup_inference +from exo.worker.engines.mlx.utils_mlx import ( + initialize_mlx, + mlx_force_oom, +) +from exo.worker.runner.bootstrap import logger + + +def main( + bound_instance: BoundInstance, + event_sender: MpSender[Event], + task_receiver: MpReceiver[Task], +): + instance, runner_id, shard_metadata = ( + bound_instance.instance, + bound_instance.bound_runner_id, + bound_instance.bound_shard, + ) + try: + logger.info("hello from the runner") + if getattr(shard_metadata, "immediate_exception", False): + raise Exception("Fake exception - runner failed to spin up.") + if timeout := getattr(shard_metadata, "should_timeout", 0): + time.sleep(timeout) + + setup_start_time = time.time() + + model = None + tokenizer = None + sampler = None + + current_status: RunnerStatus = RunnerWaitingForModel() + logger.info("runner waiting for model") + event_sender.send( + RunnerStatusUpdated(runner_id=runner_id, runner_status=current_status) + ) + with task_receiver as tasks: + for task in tasks: + event_sender.send( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.Running + ) + ) + event_sender.send(TaskAcknowledged(task_id=task.task_id)) + match task: + case LoadModel() if isinstance( + current_status, (RunnerWaitingForModel, RunnerFailed) + ): + current_status = RunnerLoading() + logger.info("runner loading") + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, runner_status=current_status + ) + ) + + model, tokenizer, sampler = initialize_mlx(bound_instance) + + current_status = RunnerLoaded() + logger.info("runner loaded") + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, runner_status=current_status + ) + ) + case StartWarmup() if isinstance(current_status, RunnerLoaded): + assert model + assert tokenizer + assert sampler + current_status = RunnerWarmingUp() + logger.info("runner warming up") + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, runner_status=current_status + ) + ) + + logger.info(f"warming up inference for instance: {instance}") + toks = warmup_inference( + model=model, + tokenizer=tokenizer, + sampler=sampler, + # kv_prefix_cache=kv_prefix_cache, # supply for warmup-time prefix caching + ) + logger.info(f"warmed up by generating {toks} tokens") + logger.info( + f"runner initialized in {time.time() - setup_start_time} seconds" + ) + current_status = RunnerReady() + logger.info("runner ready") + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, runner_status=RunnerReady() + ) + ) + case ChatCompletion( + task_params=task_params, command_id=command_id + ) if isinstance(current_status, RunnerReady): + assert model + assert tokenizer + assert sampler + logger.info(f"received chat request: {str(task)[:500]}") + current_status = RunnerRunning() + logger.info("runner running") + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, runner_status=current_status + ) + ) + assert task_params.messages[0].content is not None + _check_for_debug_prompts(task_params.messages[0].content) + + # Generate responses using the actual MLX generation + for response in mlx_generate( + model=model, + tokenizer=tokenizer, + sampler=sampler, + task=task_params, + ): + match response: + case GenerationResponse(): + if shard_metadata.device_rank == 0: + event_sender.send( + ChunkGenerated( + command_id=command_id, + chunk=TokenChunk( + idx=response.token, + model=shard_metadata.model_meta.model_id, + text=response.text, + token_id=response.token, + finish_reason=response.finish_reason, + ), + ) + ) + # case TokenizedResponse(): + # TODO: something here ig + + current_status = RunnerReady() + logger.info("runner ready") + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, runner_status=RunnerReady() + ) + ) + case Shutdown(): + logger.info("runner shutting down") + event_sender.send( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.Complete + ) + ) + break + case _: + raise ValueError("Received task outside of state machine") + event_sender.send( + TaskStatusUpdated( + task_id=task.task_id, task_status=TaskStatus.Complete + ) + ) + event_sender.send( + RunnerStatusUpdated(runner_id=runner_id, runner_status=RunnerShutdown()) + ) + except ClosedResourceError: + logger.warning("runner communication closed unexpectedly") + except Exception as e: + logger.opt(exception=e).warning( + f"Runner {runner_id} crashed with critical exception {e}" + ) + event_sender.send( + RunnerStatusUpdated( + runner_id=runner_id, + runner_status=RunnerFailed(error_message=str(e)), + ) + ) + finally: + event_sender.close() + task_receiver.close() + event_sender.join() + task_receiver.join() + logger.info("bye from the runner") + + +EXO_RUNNER_MUST_FAIL = "EXO RUNNER MUST FAIL" +EXO_RUNNER_MUST_OOM = "EXO RUNNER MUST OOM" +EXO_RUNNER_MUST_TIMEOUT = "EXO RUNNER MUST TIMEOUT" + + +def _check_for_debug_prompts( + prompt: str | ChatCompletionMessageText | list[ChatCompletionMessageText], +): + if isinstance(prompt, list): + if len(prompt) == 0: + logger.debug("Empty message prompt received in debug prompt") + return + prompt = prompt[0] + + if isinstance(prompt, ChatCompletionMessageText): + prompt = prompt.text + + if EXO_RUNNER_MUST_FAIL in prompt: + logger.info("raising exception") + raise Exception("Artificial runner exception - for testing purposes only.") + if EXO_RUNNER_MUST_OOM in prompt: + mlx_force_oom() + if EXO_RUNNER_MUST_TIMEOUT in prompt: + time.sleep(100) diff --git a/src/exo/worker/runner/runner_supervisor.py b/src/exo/worker/runner/runner_supervisor.py new file mode 100644 index 00000000..9f84b588 --- /dev/null +++ b/src/exo/worker/runner/runner_supervisor.py @@ -0,0 +1,180 @@ +import contextlib +import signal +from dataclasses import dataclass, field +from multiprocessing import Process +from typing import Self + +import anyio +from anyio import ( + BrokenResourceError, + ClosedResourceError, + create_task_group, + to_thread, +) +from anyio.abc import TaskGroup +from loguru import logger + +from exo.shared.types.events import Event, RunnerStatusUpdated, TaskAcknowledged +from exo.shared.types.tasks import Task, TaskId +from exo.shared.types.worker.instances import BoundInstance +from exo.shared.types.worker.runners import ( + RunnerFailed, + RunnerStatus, + RunnerWaitingForModel, +) +from exo.shared.types.worker.shards import ShardMetadata +from exo.utils.channels import MpReceiver, MpSender, Sender, mp_channel +from exo.worker.runner.bootstrap import entrypoint + +PREFILL_TIMEOUT_SECONDS = 60 +DECODE_TIMEOUT_SECONDS = 5 + + +@dataclass(eq=False) +class RunnerSupervisor: + shard_metadata: ShardMetadata + bound_instance: BoundInstance + runner_process: Process + initialize_timeout: float + _ev_recv: MpReceiver[Event] + _task_sender: MpSender[Task] + _event_sender: Sender[Event] + # err_path: str + _tg: TaskGroup | None = field(default=None, init=False) + status: RunnerStatus = field(default_factory=RunnerWaitingForModel, init=False) + pending: dict[TaskId, anyio.Event] = field(default_factory=dict, init=False) + + @classmethod + def create( + cls, + *, + bound_instance: BoundInstance, + event_sender: Sender[Event], + initialize_timeout: float = 400, + ) -> Self: + ev_send, ev_recv = mp_channel[Event]() + # A task is kind of a runner command + task_sender, task_recv = mp_channel[Task]() + + runner_process = Process( + target=entrypoint, + args=( + bound_instance, + ev_send, + task_recv, + logger, + ), + daemon=True, + ) + + shard_metadata = bound_instance.bound_shard + + self = cls( + bound_instance=bound_instance, + shard_metadata=shard_metadata, + runner_process=runner_process, + initialize_timeout=initialize_timeout, + _ev_recv=ev_recv, + _task_sender=task_sender, + _event_sender=event_sender, + # err_path=err_path, + ) + + return self + + async def run(self): + self.runner_process.start() + async with create_task_group() as tg: + self._tg = tg + tg.start_soon(self._forward_events) + + self._ev_recv.close() + self._task_sender.close() + self._event_sender.close() + await to_thread.run_sync(self.runner_process.join, 30) + if not self.runner_process.is_alive(): + return + + # This is overkill but it's not technically bad, just unnecessary. + logger.warning("Runner process didn't shutdown succesfully, terminating") + self.runner_process.terminate() + await to_thread.run_sync(self.runner_process.join, 5) + if not self.runner_process.is_alive(): + return + + logger.critical("Runner process didn't respond to SIGTERM, killing") + self.runner_process.kill() + + await to_thread.run_sync(self.runner_process.join, 5) + if not self.runner_process.is_alive(): + return + + logger.critical( + "Runner process didn't respond to SIGKILL. System resources may have leaked" + ) + + def shutdown(self): + assert self._tg + self._tg.cancel_scope.cancel() + + async def start_task(self, task: Task): + logger.info(f"Starting task {task}") + event = anyio.Event() + self.pending[task.task_id] = event + try: + self._task_sender.send(task) + except ClosedResourceError: + logger.warning(f"Task {task} dropped, runner closed communication.") + return + await event.wait() + logger.info(f"Finished task {task}") + + async def _forward_events(self): + with self._ev_recv as events: + try: + async for event in events: + if isinstance(event, RunnerStatusUpdated): + self.status = event.runner_status + if isinstance(event, TaskAcknowledged): + self.pending.pop(event.task_id).set() + continue + await self._event_sender.send(event) + except (ClosedResourceError, BrokenResourceError) as e: + await self._check_runner(e) + for tid in self.pending: + self.pending[tid].set() + + def __del__(self) -> None: + if self.runner_process.is_alive(): + logger.warning("RunnerSupervisor was not stopped cleanly.") + with contextlib.suppress(ValueError): + self.runner_process.kill() + + async def _check_runner(self, e: Exception) -> None: + logger.info("Checking runner's status") + if self.runner_process.is_alive(): + logger.info("Runner was found to be alive, attempting to join process") + await to_thread.run_sync(self.runner_process.join, 1) + rc = self.runner_process.exitcode + logger.info(f"RunnerSupervisor exited with exit code {rc}") + if rc == 0: + return + + if isinstance(rc, int) and rc < 0: + sig = -rc + try: + cause = f"signal={sig} ({signal.strsignal(sig)})" + except Exception: + cause = f"signal={sig}" + else: + cause = f"exitcode={rc}" + + logger.opt(exception=e).error(f"Runner terminated ({cause})") + + await self._event_sender.send( + RunnerStatusUpdated( + runner_id=self.bound_instance.bound_runner_id, + runner_status=RunnerFailed(error_message=f"Terminated ({cause})"), + ) + ) + self.shutdown() diff --git a/src/exo/worker/tests/TODO.tests b/src/exo/worker/tests/TODO.tests new file mode 100644 index 00000000..de72268b --- /dev/null +++ b/src/exo/worker/tests/TODO.tests @@ -0,0 +1,57 @@ +Unit Tests +1. Test worker plans as expected + - State transitions are correct + - Unexpected states throw + +2. Test runner + - Stays loaded + - Unloads under end condition + - Accepts tasks + - Returns ChunkGenerated events + +3. Test mlx engine + - Autoparallel on n of the same nodes returns tensors with 1/n size + - mx.barrier forces computation + - Distributed init returns expected configuration + - initialize_mlx sets wired limit + - shard_and_load returns expected model + - Quantization returns quantized layers + + 4. Download + - hits the correct endpoint + - normalizes tags correctly + - updates download progress + + 5. Serialization/Deserialization of tagged models + + + + + +Integration tests: +1. Test model inference is "sensible" (per-configuration) + - Non-empty response + - Sensible inference speed + - Answers are non-gibberish for many seeds (What is the capital of France? -> "Paris" in answer.) + - Answer is the same for particular seed + +2. Test that node count does not affect inference result (per-configuration) + - Llama on 1 node, and on 2 nodes returns the same result, given temperature 0 and set seed. + - Do for all configurations (Ring/Jaccl, Pipeline/Tensor) + +3. Test supervisor catches exceptions gracefully + - Timeouts + - OOM + - MLX error + +4. distributed init memory requirements are as expected + +5. MLX + - KVCache size is same length as prompt tokens + - Prefix cache (once implemented) + +6. Spin up creates a runner or goes to failed status + + +Regression tests: +1. Per-configuration baseline performance - no 20% drop in performance (device, node count, model, strategy, backend) diff --git a/src/exo/worker/tests/__init__.py b/src/exo/worker/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/tests/constants.py b/src/exo/worker/tests/constants.py new file mode 100644 index 00000000..787f2ff7 --- /dev/null +++ b/src/exo/worker/tests/constants.py @@ -0,0 +1,26 @@ +from typing import Final + +from exo.shared.types.common import CommandId, NodeId +from exo.shared.types.models import ModelId +from exo.shared.types.tasks import TaskId +from exo.shared.types.worker.instances import InstanceId, RunnerId + +MASTER_NODE_ID = NodeId("ffffffff-aaaa-4aaa-8aaa-aaaaaaaaaaaa") + +NODE_A: Final[NodeId] = NodeId("aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa") +NODE_B: Final[NodeId] = NodeId("bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb") + +RUNNER_1_ID: Final[RunnerId] = RunnerId("11111111-1111-4111-8111-111111111111") +RUNNER_2_ID: Final[RunnerId] = RunnerId("33333333-3333-4333-8333-333333333333") + +INSTANCE_1_ID: Final[InstanceId] = InstanceId("22222222-2222-4222-8222-222222222222") +INSTANCE_2_ID: Final[InstanceId] = InstanceId("44444444-4444-4444-8444-444444444444") + +MODEL_A_ID: Final[ModelId] = ModelId("mlx-community/Llama-3.2-1B-Instruct-4bit") +MODEL_B_ID: Final[ModelId] = ModelId("mlx-community/TinyLlama-1.1B-Chat-v1.0") + +TASK_1_ID: Final[TaskId] = TaskId("55555555-5555-4555-8555-555555555555") +TASK_2_ID: Final[TaskId] = TaskId("66666666-6666-4666-8666-666666666666") + +COMMAND_1_ID: Final[CommandId] = CommandId("77777777-7777-4777-8777-777777777777") +COMMAND_2_ID: Final[CommandId] = CommandId("88888888-8888-4888-8888-888888888888") diff --git a/src/exo/worker/tests/unittests/__init__.py b/src/exo/worker/tests/unittests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/tests/unittests/conftest.py b/src/exo/worker/tests/unittests/conftest.py new file mode 100644 index 00000000..48fc387a --- /dev/null +++ b/src/exo/worker/tests/unittests/conftest.py @@ -0,0 +1,71 @@ +from dataclasses import dataclass + +from exo.shared.types.common import NodeId +from exo.shared.types.memory import Memory +from exo.shared.types.models import ModelId, ModelMetadata +from exo.shared.types.tasks import BaseTask +from exo.shared.types.worker.instances import ( + BoundInstance, + Instance, + InstanceId, + MlxRingInstance, +) +from exo.shared.types.worker.runners import RunnerId, RunnerStatus, ShardAssignments +from exo.shared.types.worker.shards import PipelineShardMetadata, ShardMetadata + + +@dataclass(frozen=True) +class FakeRunnerSupervisor: + bound_instance: BoundInstance + status: RunnerStatus + + +class OtherTask(BaseTask): + pass + + +# TODO: Is this actually better than using Mock/Fake dataclasses? +# e.g. commit d01cd292344df15759070966826a6c027945792b +def get_pipeline_shard_metadata( + model_id: ModelId, device_rank: int, world_size: int = 1 +) -> ShardMetadata: + return PipelineShardMetadata( + model_meta=ModelMetadata( + model_id=model_id, + pretty_name=str(model_id), + storage_size=Memory.from_mb(100000), + n_layers=32, + ), + device_rank=device_rank, + world_size=world_size, + start_layer=0, + end_layer=32, + n_layers=32, + ) + + +def get_shard_assignments( + model_id: ModelId, + node_to_runner: dict[NodeId, RunnerId], + runner_to_shard: dict[RunnerId, ShardMetadata], +) -> ShardAssignments: + return ShardAssignments( + model_id=model_id, + node_to_runner=node_to_runner, + runner_to_shard=runner_to_shard, + ) + + +def get_mlx_ring_instance( + instance_id: InstanceId, + model_id: ModelId, + node_to_runner: dict[NodeId, RunnerId], + runner_to_shard: dict[RunnerId, ShardMetadata], +) -> Instance: + return MlxRingInstance( + instance_id=instance_id, + shard_assignments=get_shard_assignments( + model_id, node_to_runner, runner_to_shard + ), + hosts=[], + ) diff --git a/src/exo/worker/tests/unittests/test_download/__init__.py b/src/exo/worker/tests/unittests/test_download/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/tests/unittests/test_mlx/__init__.py b/src/exo/worker/tests/unittests/test_mlx/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/tests/unittests/test_plan/__init__.py b/src/exo/worker/tests/unittests/test_plan/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/tests/unittests/test_plan/test_download_and_loading.py b/src/exo/worker/tests/unittests/test_plan/test_download_and_loading.py new file mode 100644 index 00000000..5d6e4e2c --- /dev/null +++ b/src/exo/worker/tests/unittests/test_plan/test_download_and_loading.py @@ -0,0 +1,215 @@ +import exo.worker.plan as plan_mod +from exo.shared.types.common import NodeId +from exo.shared.types.tasks import LoadModel +from exo.shared.types.worker.downloads import DownloadCompleted, DownloadProgress +from exo.shared.types.worker.instances import BoundInstance +from exo.shared.types.worker.runners import ( + RunnerWaitingForModel, +) +from exo.shared.types.worker.shards import ShardMetadata +from exo.worker.tests.constants import ( + INSTANCE_1_ID, + MODEL_A_ID, + NODE_A, + NODE_B, + RUNNER_1_ID, + RUNNER_2_ID, +) +from exo.worker.tests.unittests.conftest import ( + FakeRunnerSupervisor, + get_mlx_ring_instance, + get_pipeline_shard_metadata, +) + + +def test_plan_requests_download_when_waiting_and_shard_not_downloaded(): + """ + When a runner is waiting for a model and its shard is not in the + local download_status map, plan() should emit DownloadModel. + """ + + shard = get_pipeline_shard_metadata(model_id=MODEL_A_ID, device_rank=0) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID}, + runner_to_shard={RUNNER_1_ID: shard}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerWaitingForModel() + ) + + runners = {RUNNER_1_ID: runner} + instances = {INSTANCE_1_ID: instance} + all_runners = {RUNNER_1_ID: RunnerWaitingForModel()} + + # No entry for this shard -> should trigger DownloadModel + download_status: dict[ShardMetadata, DownloadProgress] = {} + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status=download_status, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert isinstance(result, plan_mod.DownloadModel) + assert result.instance_id == INSTANCE_1_ID + assert result.shard_metadata == shard + + +def test_plan_loads_model_when_all_shards_downloaded_and_waiting(): + """ + When all shards for an instance are DownloadCompleted (globally) and + all runners are in waiting/loading/loaded states, plan() should emit + LoadModel once. + """ + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard2 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard1, RUNNER_2_ID: shard2}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerWaitingForModel() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + + all_runners = { + RUNNER_1_ID: RunnerWaitingForModel(), + RUNNER_2_ID: RunnerWaitingForModel(), + } + + # Local node has already marked its shard as downloaded (not actually used by _load_model) + local_download_status = { + shard1: DownloadCompleted(shard_metadata=shard1, node_id=NODE_A) # type: ignore[reportUnhashable] + } + + # Global view has completed downloads for both nodes + global_download_status = { + NODE_A: [DownloadCompleted(shard_metadata=shard1, node_id=NODE_A)], + NODE_B: [DownloadCompleted(shard_metadata=shard2, node_id=NODE_B)], + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status=local_download_status, + global_download_status=global_download_status, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert isinstance(result, LoadModel) + assert result.instance_id == INSTANCE_1_ID + + +def test_plan_does_not_request_download_when_shard_already_downloaded(): + """ + If the local shard already has a DownloadCompleted entry, plan() + should not re-emit DownloadModel while global state is still catching up. + """ + shard = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID}, + runner_to_shard={RUNNER_1_ID: shard}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerWaitingForModel() + ) + + runners = {RUNNER_1_ID: runner} + instances = {INSTANCE_1_ID: instance} + all_runners = {RUNNER_1_ID: RunnerWaitingForModel()} + + # Local status claims the shard is downloaded already + local_download_status = { + shard: DownloadCompleted(shard_metadata=shard, node_id=NODE_A) # type: ignore[reportUnhashable] + } + + # Global view hasn't caught up yet (no completed shards recorded for NODE_A) + global_download_status: dict[NodeId, list[DownloadProgress]] = { + NODE_A: [], + NODE_B: [], + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status=local_download_status, + global_download_status=global_download_status, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None + + +def test_plan_does_not_load_model_until_all_shards_downloaded_globally(): + """ + LoadModel should not be emitted while some shards are still missing from + the global_download_status. + """ + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard2 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard1, RUNNER_2_ID: shard2}, + ) + + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerWaitingForModel() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerWaitingForModel(), + RUNNER_2_ID: RunnerWaitingForModel(), + } + + # Only NODE_A's shard is recorded as downloaded globally + local_download_status = { + shard1: DownloadCompleted(shard_metadata=shard1, node_id=NODE_A) # type: ignore[reportUnhashable] + } + global_download_status = { + NODE_A: [DownloadCompleted(shard_metadata=shard1, node_id=NODE_A)], + NODE_B: [], # NODE_B has no downloads completed yet + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status=local_download_status, + global_download_status=global_download_status, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None diff --git a/src/exo/worker/tests/unittests/test_plan/test_runner_lifecycle.py b/src/exo/worker/tests/unittests/test_plan/test_runner_lifecycle.py new file mode 100644 index 00000000..944cb6db --- /dev/null +++ b/src/exo/worker/tests/unittests/test_plan/test_runner_lifecycle.py @@ -0,0 +1,199 @@ +from typing import Any + +import exo.worker.plan as plan_mod +from exo.shared.types.tasks import Shutdown +from exo.shared.types.worker.instances import BoundInstance, Instance, InstanceId +from exo.shared.types.worker.runners import ( + RunnerFailed, + RunnerId, + RunnerReady, + RunnerStatus, +) +from exo.worker.tests.constants import ( + INSTANCE_1_ID, + MODEL_A_ID, + NODE_A, + NODE_B, + RUNNER_1_ID, + RUNNER_2_ID, +) +from exo.worker.tests.unittests.conftest import ( + FakeRunnerSupervisor, + get_mlx_ring_instance, + get_pipeline_shard_metadata, +) + + +def test_plan_kills_runner_when_instance_missing(): + """ + If a local runner's instance is no longer present in state, + plan() should return a Shutdown for that runner. + """ + shard = get_pipeline_shard_metadata(model_id=MODEL_A_ID, device_rank=0) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID}, + runner_to_shard={RUNNER_1_ID: shard}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + runner = FakeRunnerSupervisor(bound_instance=bound_instance, status=RunnerReady()) + + runners = {RUNNER_1_ID: runner} + instances: dict[InstanceId, Instance] = {} + all_runners = {RUNNER_1_ID: RunnerReady()} + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert isinstance(result, Shutdown) + assert result.instance_id == INSTANCE_1_ID + assert result.runner_id == RUNNER_1_ID + + +def test_plan_kills_runner_when_sibling_failed(): + """ + If a sibling runner in the same instance has failed, the local runner + should be shut down. + """ + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard2 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard1, RUNNER_2_ID: shard2}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + runner = FakeRunnerSupervisor(bound_instance=bound_instance, status=RunnerReady()) + + runners = {RUNNER_1_ID: runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerReady(), + RUNNER_2_ID: RunnerFailed(error_message="boom"), + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert isinstance(result, Shutdown) + assert result.instance_id == INSTANCE_1_ID + assert result.runner_id == RUNNER_1_ID + + +def test_plan_creates_runner_when_missing_for_node(): + """ + If shard_assignments specify a runner for this node but we don't have + a local supervisor yet, plan() should emit a CreateRunner. + """ + shard = get_pipeline_shard_metadata(model_id=MODEL_A_ID, device_rank=0) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID}, + runner_to_shard={RUNNER_1_ID: shard}, + ) + + runners: dict[Any, Any] = {} # nothing local yet + instances = {INSTANCE_1_ID: instance} + all_runners: dict[Any, Any] = {} + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + # We patched plan_mod.CreateRunner → CreateRunner + assert isinstance(result, plan_mod.CreateRunner) + assert result.instance_id == INSTANCE_1_ID + assert isinstance(result.bound_instance, BoundInstance) + assert result.bound_instance.instance is instance + assert result.bound_instance.bound_runner_id == RUNNER_1_ID + + +def test_plan_does_not_create_runner_when_supervisor_already_present(): + """ + If we already have a local supervisor for the runner assigned to this node, + plan() should not emit a CreateRunner again. + """ + shard = get_pipeline_shard_metadata(model_id=MODEL_A_ID, device_rank=0) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID}, + runner_to_shard={RUNNER_1_ID: shard}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + runner = FakeRunnerSupervisor(bound_instance=bound_instance, status=RunnerReady()) + + runners = {RUNNER_1_ID: runner} + instances = {INSTANCE_1_ID: instance} + all_runners = {RUNNER_1_ID: RunnerReady()} + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None + + +def test_plan_does_not_create_runner_for_unassigned_node(): + """ + If this node does not appear in shard_assignments.node_to_runner, + plan() should not try to create a runner on this node. + """ + shard = get_pipeline_shard_metadata(model_id=MODEL_A_ID, device_rank=0) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_2_ID: shard}, + ) + + runners: dict[RunnerId, FakeRunnerSupervisor] = {} # no local runners + instances = {INSTANCE_1_ID: instance} + all_runners: dict[RunnerId, RunnerStatus] = {} + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None diff --git a/src/exo/worker/tests/unittests/test_plan/test_task_forwarding.py b/src/exo/worker/tests/unittests/test_plan/test_task_forwarding.py new file mode 100644 index 00000000..1bf985ac --- /dev/null +++ b/src/exo/worker/tests/unittests/test_plan/test_task_forwarding.py @@ -0,0 +1,271 @@ +from typing import cast + +import exo.worker.plan as plan_mod +from exo.shared.types.api import ChatCompletionTaskParams +from exo.shared.types.tasks import ChatCompletion, Task, TaskId, TaskStatus +from exo.shared.types.worker.instances import BoundInstance, InstanceId +from exo.shared.types.worker.runners import ( + RunnerReady, + RunnerRunning, + RunnerWaitingForModel, +) +from exo.worker.tests.constants import ( + COMMAND_1_ID, + INSTANCE_1_ID, + MODEL_A_ID, + NODE_A, + NODE_B, + RUNNER_1_ID, + RUNNER_2_ID, + TASK_1_ID, +) +from exo.worker.tests.unittests.conftest import ( + FakeRunnerSupervisor, + OtherTask, + get_mlx_ring_instance, + get_pipeline_shard_metadata, +) + + +def test_plan_forwards_pending_chat_completion_when_runner_ready(): + """ + When there is a pending ChatCompletion for the local instance and all + runners are Ready/Running, plan() should forward that task. + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerReady() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerReady(), + RUNNER_2_ID: RunnerReady(), + } + + task = ChatCompletion( + task_id=TASK_1_ID, + instance_id=INSTANCE_1_ID, + task_status=TaskStatus.Pending, + command_id=COMMAND_1_ID, + task_params=ChatCompletionTaskParams(model=MODEL_A_ID, messages=[]), + ) + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={TASK_1_ID: task}, + ) + + assert result is task + + +def test_plan_does_not_forward_chat_completion_if_any_runner_not_ready(): + """ + Even with a pending ChatCompletion, plan() should not forward it unless + all runners for the instance are Ready/Running. + """ + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard2 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard1, RUNNER_2_ID: shard2}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerReady() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerReady(), + RUNNER_2_ID: RunnerWaitingForModel(), + } + + task = ChatCompletion( + task_id=TASK_1_ID, + instance_id=INSTANCE_1_ID, + task_status=TaskStatus.Pending, + command_id=COMMAND_1_ID, + task_params=ChatCompletionTaskParams(model=MODEL_A_ID, messages=[]), + ) + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: [], NODE_B: []}, + instances=instances, + all_runners=all_runners, + tasks={TASK_1_ID: task}, + ) + + assert result is None + + +def test_plan_does_not_forward_tasks_for_other_instances(): + """ + plan() should ignore pending ChatCompletion tasks whose instance_id does + not match the local instance. + """ + shard = get_pipeline_shard_metadata(model_id=MODEL_A_ID, device_rank=0) + local_instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID}, + runner_to_shard={RUNNER_1_ID: shard}, + ) + bound_instance = BoundInstance( + instance=local_instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerReady() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: local_instance} + all_runners = {RUNNER_1_ID: RunnerReady()} + + other_instance_id = InstanceId("instance-2") + foreign_task = ChatCompletion( + task_id=TaskId("other-task"), + instance_id=other_instance_id, + task_status=TaskStatus.Pending, + command_id=COMMAND_1_ID, + task_params=ChatCompletionTaskParams(model=MODEL_A_ID, messages=[]), + ) + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={foreign_task.task_id: foreign_task}, + ) + + assert result is None + + +def test_plan_ignores_non_pending_or_non_chat_tasks(): + """ + _pending_tasks should not forward tasks that are either not ChatCompletion + or not in Pending/Running states. + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerReady() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerReady(), + RUNNER_2_ID: RunnerReady(), + } + + completed_task = ChatCompletion( + task_id=TASK_1_ID, + instance_id=INSTANCE_1_ID, + task_status=TaskStatus.Complete, + command_id=COMMAND_1_ID, + task_params=ChatCompletionTaskParams(model=MODEL_A_ID, messages=[]), + ) + + other_task_id = TaskId("other-task") + + other_task = cast( + Task, + cast( + object, + OtherTask( + task_id=other_task_id, + instance_id=INSTANCE_1_ID, + task_status=TaskStatus.Pending, + ), + ), + ) + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: [], NODE_B: []}, + instances=instances, + all_runners=all_runners, + tasks={TASK_1_ID: completed_task, other_task_id: other_task}, + ) + + assert result is None + + +def test_plan_returns_none_when_nothing_to_do(): + """ + If there are healthy runners, no downloads needed, and no pending tasks, + plan() should return None (steady state). + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerRunning() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerRunning(), + RUNNER_2_ID: RunnerRunning(), + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: [], NODE_B: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None diff --git a/src/exo/worker/tests/unittests/test_plan/test_warmup.py b/src/exo/worker/tests/unittests/test_plan/test_warmup.py new file mode 100644 index 00000000..f47d24c9 --- /dev/null +++ b/src/exo/worker/tests/unittests/test_plan/test_warmup.py @@ -0,0 +1,186 @@ +import exo.worker.plan as plan_mod +from exo.shared.types.tasks import StartWarmup +from exo.shared.types.worker.instances import BoundInstance +from exo.shared.types.worker.runners import ( + RunnerLoaded, + RunnerWaitingForModel, + RunnerWarmingUp, +) +from exo.worker.tests.constants import ( + INSTANCE_1_ID, + MODEL_A_ID, + NODE_A, + NODE_B, + RUNNER_1_ID, + RUNNER_2_ID, +) +from exo.worker.tests.unittests.conftest import ( + FakeRunnerSupervisor, + get_mlx_ring_instance, + get_pipeline_shard_metadata, +) + + +def test_plan_starts_warmup_for_non_zero_rank_when_all_loaded_or_warming(): + """ + For non-zero device_rank shards, StartWarmup should be emitted when all + shards in the instance are Loaded/WarmingUp. + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_2_ID, bound_node_id=NODE_B + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerLoaded() + ) + + runners = {RUNNER_2_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerLoaded(), + RUNNER_2_ID: RunnerLoaded(), + } + + result = plan_mod.plan( + node_id=NODE_B, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert isinstance(result, StartWarmup) + assert result.instance_id == INSTANCE_1_ID + + +def test_plan_starts_warmup_for_rank_zero_after_others_warming(): + """ + For device_rank == 0, StartWarmup should only be emitted once all the + other runners in the instance are already warming up. + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerLoaded() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerLoaded(), + RUNNER_2_ID: RunnerWarmingUp(), + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert isinstance(result, StartWarmup) + assert result.instance_id == INSTANCE_1_ID + + +def test_plan_does_not_start_warmup_for_non_zero_rank_until_all_loaded_or_warming(): + """ + Non-zero rank should not start warmup while any shard is not Loaded/WarmingUp. + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_2_ID, bound_node_id=NODE_B + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerLoaded() + ) + + runners = {RUNNER_2_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerWaitingForModel(), + RUNNER_2_ID: RunnerLoaded(), + } + + result = plan_mod.plan( + node_id=NODE_B, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: [], NODE_B: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None + + +def test_plan_does_not_start_warmup_for_rank_zero_until_others_warming(): + """ + Rank-zero shard should not start warmup until all non-zero ranks are + already WarmingUp. + """ + shard0 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=0, world_size=2) + shard1 = get_pipeline_shard_metadata(MODEL_A_ID, device_rank=1, world_size=2) + instance = get_mlx_ring_instance( + instance_id=INSTANCE_1_ID, + model_id=MODEL_A_ID, + node_to_runner={NODE_A: RUNNER_1_ID, NODE_B: RUNNER_2_ID}, + runner_to_shard={RUNNER_1_ID: shard0, RUNNER_2_ID: shard1}, + ) + + bound_instance = BoundInstance( + instance=instance, bound_runner_id=RUNNER_1_ID, bound_node_id=NODE_A + ) + local_runner = FakeRunnerSupervisor( + bound_instance=bound_instance, status=RunnerLoaded() + ) + + runners = {RUNNER_1_ID: local_runner} + instances = {INSTANCE_1_ID: instance} + all_runners = { + RUNNER_1_ID: RunnerLoaded(), + RUNNER_2_ID: RunnerLoaded(), + } + + result = plan_mod.plan( + node_id=NODE_A, + runners=runners, # type: ignore + download_status={}, + global_download_status={NODE_A: [], NODE_B: []}, + instances=instances, + all_runners=all_runners, + tasks={}, + ) + + assert result is None diff --git a/src/exo/worker/tests/unittests/test_runner/__init__.py b/src/exo/worker/tests/unittests/test_runner/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/src/exo/worker/utils/__init__.py b/src/exo/worker/utils/__init__.py new file mode 100644 index 00000000..9a94e028 --- /dev/null +++ b/src/exo/worker/utils/__init__.py @@ -0,0 +1,6 @@ +from .profile import start_polling_memory_metrics, start_polling_node_metrics + +__all__ = [ + "start_polling_node_metrics", + "start_polling_memory_metrics", +] diff --git a/src/exo/worker/utils/macmon.py b/src/exo/worker/utils/macmon.py new file mode 100644 index 00000000..3e4e29e1 --- /dev/null +++ b/src/exo/worker/utils/macmon.py @@ -0,0 +1,97 @@ +import platform +import shutil +from subprocess import CalledProcessError + +from anyio import run_process +from pydantic import BaseModel, ConfigDict, ValidationError + + +class MacMonError(Exception): + """Exception raised for errors in the MacMon functions.""" + + +def _get_binary_path() -> str: + """ + Get the path to the macmon binary. + + Raises: + MacMonError: If the binary doesn't exist or can't be made executable. + """ + # Check for macOS with ARM chip + system = platform.system().lower() + machine = platform.machine().lower() + + if system != "darwin" or not ( + "arm" in machine or "m1" in machine or "m2" in machine + ): + raise MacMonError("MacMon only supports macOS with Apple Silicon (ARM) chips") + + path = shutil.which("macmon") + + if path is None: + raise MacMonError("MacMon not found in PATH") + + return path + + +class TempMetrics(BaseModel): + """Temperature-related metrics returned by macmon.""" + + cpu_temp_avg: float + gpu_temp_avg: float + + model_config = ConfigDict(extra="ignore") + + +class Metrics(BaseModel): + """Complete set of metrics returned by macmon. + + Unknown fields are ignored for forward-compatibility. + """ + + all_power: float + ane_power: float + cpu_power: float + ecpu_usage: tuple[int, float] + gpu_power: float + gpu_ram_power: float + gpu_usage: tuple[int, float] + pcpu_usage: tuple[int, float] + ram_power: float + sys_power: float + temp: TempMetrics + timestamp: str + + model_config = ConfigDict(extra="ignore") + + +async def get_metrics_async() -> Metrics: + """ + Asynchronously run the binary and return the metrics as a Python dictionary. + + Args: + binary_path: Optional path to the binary. If not provided, will use the bundled binary. + + Returns: + A mapping containing system metrics. + + Raises: + MacMonError: If there's an error running the binary. + """ + path = _get_binary_path() + + result = None + try: + # TODO: Keep Macmon running in the background? + result = await run_process([path, "pipe", "-s", "1"]) + + return Metrics.model_validate_json(result.stdout.decode().strip()) + + except ValidationError as e: + raise MacMonError(f"Error parsing JSON output: {e}") from e + except CalledProcessError as e: + if result: + raise MacMonError( + f"MacMon failed with return code {result.returncode}" + ) from e + raise e diff --git a/src/exo/worker/utils/net_profile.py b/src/exo/worker/utils/net_profile.py new file mode 100644 index 00000000..1c8c5fe4 --- /dev/null +++ b/src/exo/worker/utils/net_profile.py @@ -0,0 +1,41 @@ +import socket + +from anyio import create_task_group, to_thread + +from exo.shared.topology import Topology +from exo.shared.types.common import NodeId + + +# TODO: ref. api port +async def check_reachability( + target_ip: str, target_node_id: NodeId, out: dict[NodeId, set[str]] +) -> None: + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.settimeout(1) # 1 second timeout + try: + result = await to_thread.run_sync(sock.connect_ex, (target_ip, 8000)) + except socket.gaierror: + # seems to throw on ipv6 loopback. oh well + # logger.warning(f"invalid {target_ip=}") + return + finally: + sock.close() + + if result == 0: + if target_node_id not in out: + out[target_node_id] = set() + out[target_node_id].add(target_ip) + + +async def check_reachable(topology: Topology) -> dict[NodeId, set[str]]: + reachable: dict[NodeId, set[str]] = {} + async with create_task_group() as tg: + for node in topology.list_nodes(): + if not node.node_profile: + continue + for iface in node.node_profile.network_interfaces: + tg.start_soon( + check_reachability, iface.ip_address, node.node_id, reachable + ) + + return reachable diff --git a/src/exo/worker/utils/profile.py b/src/exo/worker/utils/profile.py new file mode 100644 index 00000000..30aca08c --- /dev/null +++ b/src/exo/worker/utils/profile.py @@ -0,0 +1,113 @@ +import asyncio +import os +import platform +from typing import Any, Callable, Coroutine + +import anyio +from loguru import logger + +from exo.shared.types.memory import Memory +from exo.shared.types.profiling import ( + MemoryPerformanceProfile, + NodePerformanceProfile, + SystemPerformanceProfile, +) + +from .macmon import ( + MacMonError, + Metrics, +) +from .macmon import ( + get_metrics_async as macmon_get_metrics_async, +) +from .system_info import ( + get_friendly_name, + get_model_and_chip, + get_network_interfaces, +) + + +async def get_metrics_async() -> Metrics | None: + """Return detailed Metrics on macOS or a minimal fallback elsewhere.""" + + if platform.system().lower() == "darwin": + return await macmon_get_metrics_async() + + +def get_memory_profile() -> MemoryPerformanceProfile: + """Construct a MemoryPerformanceProfile using psutil""" + override_memory_env = os.getenv("OVERRIDE_MEMORY_MB") + override_memory: int | None = ( + Memory.from_mb(int(override_memory_env)).in_bytes + if override_memory_env + else None + ) + + return MemoryPerformanceProfile.from_psutil(override_memory=override_memory) + + +async def start_polling_memory_metrics( + callback: Callable[[MemoryPerformanceProfile], Coroutine[Any, Any, None]], + *, + poll_interval_s: float = 0.5, +) -> None: + """Continuously poll and emit memory-only metrics at a faster cadence. + + Parameters + - callback: coroutine called with a fresh MemoryPerformanceProfile each tick + - poll_interval_s: interval between polls + """ + while True: + try: + mem = get_memory_profile() + await callback(mem) + except MacMonError as e: + logger.opt(exception=e).error("Memory Monitor encountered error") + finally: + await anyio.sleep(poll_interval_s) + + +async def start_polling_node_metrics( + callback: Callable[[NodePerformanceProfile], Coroutine[Any, Any, None]], +): + poll_interval_s = 1.0 + while True: + try: + metrics = await get_metrics_async() + if metrics is None: + return + + network_interfaces = get_network_interfaces() + # these awaits could be joined but realistically they should be cached + model_id, chip_id = await get_model_and_chip() + friendly_name = await get_friendly_name() + + # do the memory profile last to get a fresh reading to not conflict with the other memory profiling loop + memory_profile = get_memory_profile() + + await callback( + NodePerformanceProfile( + model_id=model_id, + chip_id=chip_id, + friendly_name=friendly_name, + network_interfaces=network_interfaces, + memory=memory_profile, + system=SystemPerformanceProfile( + gpu_usage=metrics.gpu_usage[1], + temp=metrics.temp.gpu_temp_avg, + sys_power=metrics.sys_power, + pcpu_usage=metrics.pcpu_usage[1], + ecpu_usage=metrics.ecpu_usage[1], + ane_power=metrics.ane_power, + ), + ) + ) + + except asyncio.TimeoutError: + logger.warning( + "[resource_monitor] Operation timed out after 30s, skipping this cycle." + ) + except MacMonError as e: + logger.opt(exception=e).error("Resource Monitor encountered error") + finally: + await anyio.sleep(poll_interval_s) diff --git a/src/exo/worker/utils/system_info.py b/src/exo/worker/utils/system_info.py new file mode 100644 index 00000000..930d9428 --- /dev/null +++ b/src/exo/worker/utils/system_info.py @@ -0,0 +1,83 @@ +import socket +import sys +from subprocess import CalledProcessError + +import psutil +from anyio import run_process + +from exo.shared.types.profiling import NetworkInterfaceInfo + + +async def get_friendly_name() -> str: + """ + Asynchronously gets the 'Computer Name' (friendly name) of a Mac. + e.g., "John's MacBook Pro" + Returns the name as a string, or None if an error occurs or not on macOS. + """ + hostname = socket.gethostname() + + # TODO: better non mac support + if sys.platform != "darwin": # 'darwin' is the platform name for macOS + return hostname + + try: + process = await run_process(["scutil", "--get", "ComputerName"]) + except CalledProcessError: + return hostname + + return process.stdout.decode("utf-8", errors="replace").strip() or hostname + + +def get_network_interfaces() -> list[NetworkInterfaceInfo]: + """ + Retrieves detailed network interface information on macOS. + Parses output from 'networksetup -listallhardwareports' and 'ifconfig' + to determine interface names, IP addresses, and types (ethernet, wifi, vpn, other). + Returns a list of NetworkInterfaceInfo objects. + """ + interfaces_info: list[NetworkInterfaceInfo] = [] + + for iface, services in psutil.net_if_addrs().items(): + for service in services: + match service.family: + case socket.AF_INET | socket.AF_INET6: + interfaces_info.append( + NetworkInterfaceInfo(name=iface, ip_address=service.address) + ) + case _: + pass + + return interfaces_info + + +async def get_model_and_chip() -> tuple[str, str]: + """Get Mac system information using system_profiler.""" + model = "Unknown Model" + chip = "Unknown Chip" + + # TODO: better non mac support + if sys.platform != "darwin": + return (model, chip) + + try: + process = await run_process( + [ + "system_profiler", + "SPHardwareDataType", + ] + ) + except CalledProcessError: + return (model, chip) + + # less interested in errors here because this value should be hard coded + output = process.stdout.decode().strip() + + model_line = next( + (line for line in output.split("\n") if "Model Name" in line), None + ) + model = model_line.split(": ")[1] if model_line else "Unknown Model" + + chip_line = next((line for line in output.split("\n") if "Chip" in line), None) + chip = chip_line.split(": ")[1] if chip_line else "Unknown Chip" + + return (model, chip) diff --git a/tmp/disable_bridge_enable_dhcp.sh b/tmp/disable_bridge_enable_dhcp.sh new file mode 100755 index 00000000..8bce9333 --- /dev/null +++ b/tmp/disable_bridge_enable_dhcp.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +set -euo pipefail + +networksetup -listallnetworkservices | grep -q '^Thunderbolt Bridge$' \ + && echo "Disabling bridge in networksetup" \ + && networksetup -setnetworkserviceenabled "Thunderbolt Bridge" off + +networksetup -listallnetworkservices | grep -q '^\*Thunderbolt Bridge$' \ + && echo "Bridge disabled in networksetup" + +ifconfig bridge0 &>/dev/null && { + ifconfig bridge0 | grep -q 'member' && echo "Removing bridge members in ifconfig" && { + ifconfig bridge0 | \ + awk '/member/ {print $2}' | \ + xargs -n1 sudo ifconfig bridge0 deletem + } + ifconfig bridge0 | grep -q 'status: active' && sudo ifconfig bridge0 down + ifconfig bridge0 | grep -q 'status: inactive' && echo "Bridge disabled in ifconfig" +} + +for iface in $(seq 2 7); do + sudo ipconfig set "en$iface" dhcp && echo "enabled dhcp on en$iface" || echo "failed to enable dhcp on en$iface" +done + diff --git a/tmp/prompt.txt b/tmp/prompt.txt new file mode 100644 index 00000000..d566939c --- /dev/null +++ b/tmp/prompt.txt @@ -0,0 +1,47 @@ +Summarise this Wikipedia article for me: + +Transition from Republic to Empire + +Augustus of Prima Porta +Rome had begun expanding shortly after the founding of the Roman Republic in the 6th century BC, though not outside the Italian Peninsula until the 3rd century BC. The Republic was not a nation-state in the modern sense, but a network of self-ruled towns (with varying degrees of independence from the Senate) and provinces administered by military commanders. It was governed by annually elected magistrates (Roman consuls above all) in conjunction with the Senate.[22] The 1st century BC was a time of political and military upheaval, which ultimately led to rule by emperors.[23][24][25] The consuls' military power rested in the Roman legal concept of imperium, meaning "command" (typically in a military sense).[26] Occasionally, successful consuls or generals were given the honorary title imperator (commander); this is the origin of the word emperor, since this title was always bestowed to the early emperors.[27][g] + +Rome suffered a long series of internal conflicts, conspiracies, and civil wars from the late second century BC (see Crisis of the Roman Republic) while greatly extending its power beyond Italy. In 44 BC Julius Caesar was briefly perpetual dictator before being assassinated by a faction that opposed his concentration of power. This faction was driven from Rome and defeated at the Battle of Philippi in 42 BC by Mark Antony and Caesar's adopted son Octavian. Antony and Octavian divided the Roman world between them, but this did not last long. Octavian's forces defeated those of Mark Antony and Cleopatra at the Battle of Actium in 31 BC. In 27 BC the Senate gave him the title Augustus ("venerated") and made him princeps ("foremost") with proconsular imperium, thus beginning the Principate, the first epoch of Roman imperial history. Although the republic stood in name, Augustus had all meaningful authority.[29] During his 40-year rule, a new constitutional order emerged so that, upon his death, Tiberius would succeed him as the new de facto monarch.[30] + +Pax Romana +Main article: Pax Romana +The so-called "Five Good Emperors" of 96–180 AD + +Nerva (r. 96–98) + +Trajan (r. 98–117) + +Hadrian (r. 117–138) + +Antoninus Pius (r. 138–161) + +Marcus Aurelius (r. 161–180) +The 200 years that began with Augustus's rule are traditionally regarded as the Pax Romana ("Roman Peace"). The cohesion of the empire was furthered by a degree of social stability and economic prosperity that Rome had never before experienced. Uprisings in the provinces were infrequent and put down "mercilessly and swiftly".[31] The success of Augustus in establishing principles of dynastic succession was limited by his outliving a number of talented potential heirs. The Julio-Claudian dynasty lasted for four more emperors—Tiberius, Caligula, Claudius, and Nero—before it yielded in 69 AD to the strife-torn Year of the Four Emperors, from which Vespasian emerged as the victor. Vespasian became the founder of the brief Flavian dynasty, followed by the Nerva–Antonine dynasty which produced the "Five Good Emperors": Nerva, Trajan, Hadrian, Antoninus Pius, and Marcus Aurelius.[32] + +Among the so-called “Five Good Emperors,” Hadrian (r. 117–138) is particularly noted for consolidating the empire’s frontiers and embarking on ambitious building projects throughout the provinces.[33] In Judaea, which had long been the center of Jewish national and religious life, his reign marked a decisive turning point. After earlier Jewish resistance to Roman rule, Hadrian visited the region in 129/130 CE and refounded Jerusalem as the Roman colony Aelia Capitolina, naming it after his family (Aelius) and the Capitoline Triad.[34] The refoundation overlaid the destroyed Jewish city with a new Roman urban plan, and included the construction of a Temple to Jupiter on the site of the former Jewish Temple.[35] Later tradition and archaeological evidence also indicate a Temple of Venus near the site of the Holy Sepulchre.[36] + +Hadrian’s measures, combined with restrictions on Jewish practices, helped spark the Bar Kokhba Revolt (132–135 CE). After crushing the uprising, Roman forces expelled most Jews from Jerusalem, barring their entry except on certain days, and rebuilt the city as a statement of imperial power and domination.[33] Most scholars consider Hadrianic Aelia to have been unwalled, with free-standing gate complexes (such as the northern gate beneath today’s Damascus Gate) rather than a continuous defensive circuit.[37] + +Transition from classical to late antiquity +Main articles: Later Roman Empire and Fall of the Western Roman Empire +See also: Barbarian kingdoms and Byzantine Empire + +The Barbarian invasions consisted of the movement of (mainly) ancient Germanic peoples into Roman territory. Historically, this event marked the transition between classical antiquity and the Middle Ages. +In the view of contemporary Greek historian Cassius Dio, the accession of Commodus in 180 marked the descent "from a kingdom of gold to one of rust and iron",[38] a comment which has led some historians, notably Edward Gibbon, to take Commodus' reign as the beginning of the Empire's decline.[39][40] + +In 212, during the reign of Caracalla, Roman citizenship was granted to all freeborn inhabitants of the empire. The Severan dynasty was tumultuous; an emperor's reign was ended routinely by his murder or execution and, following its collapse, the Empire was engulfed by the Crisis of the Third Century, a period of invasions, civil strife, economic disorder, and plague.[41] In defining historical epochs, this crisis sometimes marks the transition from Classical to Late Antiquity. Aurelian (r. 270–275) stabilised the empire militarily and Diocletian reorganised and restored much of it in 285.[42] Diocletian's reign brought the empire's most concerted effort against the perceived threat of Christianity, the "Great Persecution".[43] + +Diocletian divided the empire into four regions, each ruled by a separate tetrarch.[44] Confident that he fixed the disorder plaguing Rome, he abdicated along with his co-emperor, but the Tetrarchy collapsed shortly after. Order was eventually restored by Constantine the Great, who became the first emperor to convert to Christianity, and who established Constantinople as the new capital of the Eastern Empire. During the decades of the Constantinian and Valentinian dynasties, the empire was divided along an east–west axis, with dual power centres in Constantinople and Rome. Julian, who under the influence of his adviser Mardonius attempted to restore Classical Roman and Hellenistic religion, only briefly interrupted the succession of Christian emperors. Theodosius I, the last emperor to rule over both East and West, died in 395 after making Christianity the state religion.[45] + + +The Roman Empire by 476, noting western and eastern divisions + +The administrative divisions of the Roman Empire in 395 AD +Fall in the West and survival in the East +The Western Roman Empire began to disintegrate in the early 5th century. The Romans fought off all invaders, most famously Attila,[46] but the empire had assimilated so many Germanic peoples of dubious loyalty to Rome that the empire started to dismember itself.[47] Most chronologies place the end of the Western Roman Empire in 476, when Romulus Augustulus was forced to abdicate to the Germanic warlord Odoacer.[48][49][50] + +Odoacer ended the Western Empire by declaring Zeno sole emperor and placing himself as Zeno's nominal subordinate. In reality, Italy was ruled by Odoacer alone.[48][49][51] The Eastern Roman Empire, called the Byzantine Empire by later historians, continued until the reign of Constantine XI Palaiologos, the last Roman emperor. He died in battle in 1453 against Mehmed II and his Ottoman forces during the siege of Constantinople. Mehmed II adopted the title of caesar in an attempt to claim a connection to the former Empire.[52][53] His claim was soon recognized by the Patriarchate of Constantinople, but not by European monarchs. \ No newline at end of file diff --git a/tmp/run_llm.py b/tmp/run_llm.py new file mode 100644 index 00000000..89a2e50b --- /dev/null +++ b/tmp/run_llm.py @@ -0,0 +1,86 @@ +#!/usr/bin/env python3 +import argparse +import json +import sys + +import requests + + +def stream_chat(host: str, query: str) -> None: + url = f"http://{host}:8000/v1/chat/completions" + headers = {"Content-Type": "application/json"} + payload = { + "model": "mlx-community/Llama-3.2-1B-Instruct-4bit", + # "model": "mlx-community/Llama-3_3-Nemotron-Super-49B-v1_5-mlx-4Bit", + "stream": True, + "messages": [{"role": "user", "content": query}], + } + + try: + with requests.post(url, headers=headers, json=payload, stream=True) as resp: + resp.raise_for_status() + for line in resp.iter_lines(decode_unicode=True): + if not line: + continue + + # SSE lines look like: "data: {...}" or "data: [DONE]" + if not line.startswith("data:"): + continue + + data = line[len("data:") :].strip() + if data == "[DONE]": + break + + try: + obj = json.loads(data) + except json.JSONDecodeError: + continue + + for choice in obj.get("choices", []): + delta = choice.get("delta") or {} + content = delta.get("content") + if content: + print(content, end="", flush=True) + + except requests.RequestException as e: + print(f"Request failed: {e}", file=sys.stderr) + sys.exit(1) + + print() + + +def main() -> None: + parser = argparse.ArgumentParser( + description="Stream chat completions from a local server." + ) + parser.add_argument("host", help="Hostname (without protocol), e.g. localhost") + parser.add_argument( + "-f", + "--file", + help="Path to a text file whose contents will be used as the query", + ) + parser.add_argument( + "query", + nargs="*", + help="Query text (if not using -f/--file). All remaining arguments are joined with spaces.", + ) + + args = parser.parse_args() + + if args.file: + try: + with open(args.file, "r", encoding="utf-8") as f: + query = f.read().strip() + except OSError as e: + print(f"Error reading file {args.file}: {e}", file=sys.stderr) + sys.exit(1) + elif args.query: + query = " ".join(args.query) + else: + parser.error("You must provide either a query or a file (-f/--file).") + + stream_chat(args.host, query) + + +if __name__ == "__main__": + main() diff --git a/tmp/run_llm.sh b/tmp/run_llm.sh new file mode 100755 index 00000000..b9dbb61b --- /dev/null +++ b/tmp/run_llm.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [ $# -lt 2 ]; then + echo "Usage: $0 " + exit 1 +fi + +HOST="$1" +shift +QUERY="$*" + +curl -sN -X POST "http://$HOST:8000/v1/chat/completions" \ + -H "Content-Type: application/json" \ + -d "{ + \"model\": \"mlx-community/Kimi-K2-Thinking\", + \"stream\": true, + \"messages\": [{ \"role\": \"user\", \"content\": \"$QUERY\"}] + }" | + grep --line-buffered '^data:' | + grep --line-buffered -v 'data: \[DONE\]' | + cut -d' ' -f2- | + jq -r --unbuffered '.choices[].delta.content // empty' | + awk '{ORS=""; print; fflush()} END {print "\n"}' diff --git a/uv.lock b/uv.lock new file mode 100644 index 00000000..d162d6b5 --- /dev/null +++ b/uv.lock @@ -0,0 +1,1640 @@ +version = 1 +revision = 3 +requires-python = ">=3.13" +resolution-markers = [ + "sys_platform == 'darwin'", + "sys_platform == 'linux'", +] +supported-markers = [ + "sys_platform == 'darwin'", + "sys_platform == 'linux'", +] + +[manifest] +members = [ + "exo", + "exo-pyo3-bindings", +] + +[[package]] +name = "aiofiles" +version = "25.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/41/c3/534eac40372d8ee36ef40df62ec129bee4fdb5ad9706e58a29be53b2c970/aiofiles-25.1.0.tar.gz", hash = "sha256:a8d728f0a29de45dc521f18f07297428d56992a742f0cd2701ba86e44d23d5b2", size = 46354, upload-time = "2025-10-09T20:51:04.358Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bc/8a/340a1555ae33d7354dbca4faa54948d76d89a27ceef032c8c3bc661d003e/aiofiles-25.1.0-py3-none-any.whl", hash = "sha256:abe311e527c862958650f9438e859c1fa7568a141b22abcd015e120e86a85695", size = 14668, upload-time = "2025-10-09T20:51:03.174Z" }, +] + +[[package]] +name = "aiohappyeyeballs" +version = "2.6.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/26/30/f84a107a9c4331c14b2b586036f40965c128aa4fee4dda5d3d51cb14ad54/aiohappyeyeballs-2.6.1.tar.gz", hash = "sha256:c3f9d0113123803ccadfdf3f0faa505bc78e6a72d1cc4806cbd719826e943558", size = 22760, upload-time = "2025-03-12T01:42:48.764Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0f/15/5bf3b99495fb160b63f95972b81750f18f7f4e02ad051373b669d17d44f2/aiohappyeyeballs-2.6.1-py3-none-any.whl", hash = "sha256:f349ba8f4b75cb25c99c5c2d84e997e485204d2902a9597802b0371f09331fb8", size = 15265, upload-time = "2025-03-12T01:42:47.083Z" }, +] + +[[package]] +name = "aiohttp" +version = "3.13.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "aiohappyeyeballs", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "aiosignal", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "attrs", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "frozenlist", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "multidict", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "propcache", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "yarl", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1c/ce/3b83ebba6b3207a7135e5fcaba49706f8a4b6008153b4e30540c982fae26/aiohttp-3.13.2.tar.gz", hash = "sha256:40176a52c186aefef6eb3cad2cdd30cd06e3afbe88fe8ab2af9c0b90f228daca", size = 7837994, upload-time = "2025-10-28T20:59:39.937Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bf/78/7e90ca79e5aa39f9694dcfd74f4720782d3c6828113bb1f3197f7e7c4a56/aiohttp-3.13.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:7519bdc7dfc1940d201651b52bf5e03f5503bda45ad6eacf64dda98be5b2b6be", size = 732139, upload-time = "2025-10-28T20:57:02.455Z" }, + { url = "https://files.pythonhosted.org/packages/db/ed/1f59215ab6853fbaa5c8495fa6cbc39edfc93553426152b75d82a5f32b76/aiohttp-3.13.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:088912a78b4d4f547a1f19c099d5a506df17eacec3c6f4375e2831ec1d995742", size = 490082, upload-time = "2025-10-28T20:57:04.784Z" }, + { url = "https://files.pythonhosted.org/packages/68/7b/fe0fe0f5e05e13629d893c760465173a15ad0039c0a5b0d0040995c8075e/aiohttp-3.13.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:5276807b9de9092af38ed23ce120539ab0ac955547b38563a9ba4f5b07b95293", size = 489035, upload-time = "2025-10-28T20:57:06.894Z" }, + { url = "https://files.pythonhosted.org/packages/d2/04/db5279e38471b7ac801d7d36a57d1230feeee130bbe2a74f72731b23c2b1/aiohttp-3.13.2-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1237c1375eaef0db4dcd7c2559f42e8af7b87ea7d295b118c60c36a6e61cb811", size = 1720387, upload-time = "2025-10-28T20:57:08.685Z" }, + { url = "https://files.pythonhosted.org/packages/31/07/8ea4326bd7dae2bd59828f69d7fdc6e04523caa55e4a70f4a8725a7e4ed2/aiohttp-3.13.2-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:96581619c57419c3d7d78703d5b78c1e5e5fc0172d60f555bdebaced82ded19a", size = 1688314, upload-time = "2025-10-28T20:57:10.693Z" }, + { url = "https://files.pythonhosted.org/packages/48/ab/3d98007b5b87ffd519d065225438cc3b668b2f245572a8cb53da5dd2b1bc/aiohttp-3.13.2-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a2713a95b47374169409d18103366de1050fe0ea73db358fc7a7acb2880422d4", size = 1756317, upload-time = "2025-10-28T20:57:12.563Z" }, + { url = "https://files.pythonhosted.org/packages/97/3d/801ca172b3d857fafb7b50c7c03f91b72b867a13abca982ed6b3081774ef/aiohttp-3.13.2-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:228a1cd556b3caca590e9511a89444925da87d35219a49ab5da0c36d2d943a6a", size = 1858539, upload-time = "2025-10-28T20:57:14.623Z" }, + { url = "https://files.pythonhosted.org/packages/f7/0d/4764669bdf47bd472899b3d3db91fffbe925c8e3038ec591a2fd2ad6a14d/aiohttp-3.13.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ac6cde5fba8d7d8c6ac963dbb0256a9854e9fafff52fbcc58fdf819357892c3e", size = 1739597, upload-time = "2025-10-28T20:57:16.399Z" }, + { url = "https://files.pythonhosted.org/packages/c4/52/7bd3c6693da58ba16e657eb904a5b6decfc48ecd06e9ac098591653b1566/aiohttp-3.13.2-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f2bef8237544f4e42878c61cef4e2839fee6346dc60f5739f876a9c50be7fcdb", size = 1555006, upload-time = "2025-10-28T20:57:18.288Z" }, + { url = "https://files.pythonhosted.org/packages/48/30/9586667acec5993b6f41d2ebcf96e97a1255a85f62f3c653110a5de4d346/aiohttp-3.13.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:16f15a4eac3bc2d76c45f7ebdd48a65d41b242eb6c31c2245463b40b34584ded", size = 1683220, upload-time = "2025-10-28T20:57:20.241Z" }, + { url = "https://files.pythonhosted.org/packages/71/01/3afe4c96854cfd7b30d78333852e8e851dceaec1c40fd00fec90c6402dd2/aiohttp-3.13.2-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:bb7fb776645af5cc58ab804c58d7eba545a97e047254a52ce89c157b5af6cd0b", size = 1712570, upload-time = "2025-10-28T20:57:22.253Z" }, + { url = "https://files.pythonhosted.org/packages/11/2c/22799d8e720f4697a9e66fd9c02479e40a49de3de2f0bbe7f9f78a987808/aiohttp-3.13.2-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:e1b4951125ec10c70802f2cb09736c895861cd39fd9dcb35107b4dc8ae6220b8", size = 1733407, upload-time = "2025-10-28T20:57:24.37Z" }, + { url = "https://files.pythonhosted.org/packages/34/cb/90f15dd029f07cebbd91f8238a8b363978b530cd128488085b5703683594/aiohttp-3.13.2-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:550bf765101ae721ee1d37d8095f47b1f220650f85fe1af37a90ce75bab89d04", size = 1550093, upload-time = "2025-10-28T20:57:26.257Z" }, + { url = "https://files.pythonhosted.org/packages/69/46/12dce9be9d3303ecbf4d30ad45a7683dc63d90733c2d9fe512be6716cd40/aiohttp-3.13.2-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:fe91b87fc295973096251e2d25a811388e7d8adf3bd2b97ef6ae78bc4ac6c476", size = 1758084, upload-time = "2025-10-28T20:57:28.349Z" }, + { url = "https://files.pythonhosted.org/packages/f9/c8/0932b558da0c302ffd639fc6362a313b98fdf235dc417bc2493da8394df7/aiohttp-3.13.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e0c8e31cfcc4592cb200160344b2fb6ae0f9e4effe06c644b5a125d4ae5ebe23", size = 1716987, upload-time = "2025-10-28T20:57:30.233Z" }, + { url = "https://files.pythonhosted.org/packages/9b/36/e2abae1bd815f01c957cbf7be817b3043304e1c87bad526292a0410fdcf9/aiohttp-3.13.2-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:2475391c29230e063ef53a66669b7b691c9bfc3f1426a0f7bcdf1216bdbac38b", size = 735234, upload-time = "2025-10-28T20:57:36.415Z" }, + { url = "https://files.pythonhosted.org/packages/ca/e3/1ee62dde9b335e4ed41db6bba02613295a0d5b41f74a783c142745a12763/aiohttp-3.13.2-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:f33c8748abef4d8717bb20e8fb1b3e07c6adacb7fd6beaae971a764cf5f30d61", size = 490733, upload-time = "2025-10-28T20:57:38.205Z" }, + { url = "https://files.pythonhosted.org/packages/1a/aa/7a451b1d6a04e8d15a362af3e9b897de71d86feac3babf8894545d08d537/aiohttp-3.13.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:ae32f24bbfb7dbb485a24b30b1149e2f200be94777232aeadba3eecece4d0aa4", size = 491303, upload-time = "2025-10-28T20:57:40.122Z" }, + { url = "https://files.pythonhosted.org/packages/57/1e/209958dbb9b01174870f6a7538cd1f3f28274fdbc88a750c238e2c456295/aiohttp-3.13.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5d7f02042c1f009ffb70067326ef183a047425bb2ff3bc434ead4dd4a4a66a2b", size = 1717965, upload-time = "2025-10-28T20:57:42.28Z" }, + { url = "https://files.pythonhosted.org/packages/08/aa/6a01848d6432f241416bc4866cae8dc03f05a5a884d2311280f6a09c73d6/aiohttp-3.13.2-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:93655083005d71cd6c072cdab54c886e6570ad2c4592139c3fb967bfc19e4694", size = 1667221, upload-time = "2025-10-28T20:57:44.869Z" }, + { url = "https://files.pythonhosted.org/packages/87/4f/36c1992432d31bbc789fa0b93c768d2e9047ec8c7177e5cd84ea85155f36/aiohttp-3.13.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0db1e24b852f5f664cd728db140cf11ea0e82450471232a394b3d1a540b0f906", size = 1757178, upload-time = "2025-10-28T20:57:47.216Z" }, + { url = "https://files.pythonhosted.org/packages/ac/b4/8e940dfb03b7e0f68a82b88fd182b9be0a65cb3f35612fe38c038c3112cf/aiohttp-3.13.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b009194665bcd128e23eaddef362e745601afa4641930848af4c8559e88f18f9", size = 1838001, upload-time = "2025-10-28T20:57:49.337Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ef/39f3448795499c440ab66084a9db7d20ca7662e94305f175a80f5b7e0072/aiohttp-3.13.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c038a8fdc8103cd51dbd986ecdce141473ffd9775a7a8057a6ed9c3653478011", size = 1716325, upload-time = "2025-10-28T20:57:51.327Z" }, + { url = "https://files.pythonhosted.org/packages/d7/51/b311500ffc860b181c05d91c59a1313bdd05c82960fdd4035a15740d431e/aiohttp-3.13.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:66bac29b95a00db411cd758fea0e4b9bdba6d549dfe333f9a945430f5f2cc5a6", size = 1547978, upload-time = "2025-10-28T20:57:53.554Z" }, + { url = "https://files.pythonhosted.org/packages/31/64/b9d733296ef79815226dab8c586ff9e3df41c6aff2e16c06697b2d2e6775/aiohttp-3.13.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:4ebf9cfc9ba24a74cf0718f04aac2a3bbe745902cc7c5ebc55c0f3b5777ef213", size = 1682042, upload-time = "2025-10-28T20:57:55.617Z" }, + { url = "https://files.pythonhosted.org/packages/3f/30/43d3e0f9d6473a6db7d472104c4eff4417b1e9df01774cb930338806d36b/aiohttp-3.13.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a4b88ebe35ce54205c7074f7302bd08a4cb83256a3e0870c72d6f68a3aaf8e49", size = 1680085, upload-time = "2025-10-28T20:57:57.59Z" }, + { url = "https://files.pythonhosted.org/packages/16/51/c709f352c911b1864cfd1087577760ced64b3e5bee2aa88b8c0c8e2e4972/aiohttp-3.13.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:98c4fb90bb82b70a4ed79ca35f656f4281885be076f3f970ce315402b53099ae", size = 1728238, upload-time = "2025-10-28T20:57:59.525Z" }, + { url = "https://files.pythonhosted.org/packages/19/e2/19bd4c547092b773caeb48ff5ae4b1ae86756a0ee76c16727fcfd281404b/aiohttp-3.13.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:ec7534e63ae0f3759df3a1ed4fa6bc8f75082a924b590619c0dd2f76d7043caa", size = 1544395, upload-time = "2025-10-28T20:58:01.914Z" }, + { url = "https://files.pythonhosted.org/packages/cf/87/860f2803b27dfc5ed7be532832a3498e4919da61299b4a1f8eb89b8ff44d/aiohttp-3.13.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:5b927cf9b935a13e33644cbed6c8c4b2d0f25b713d838743f8fe7191b33829c4", size = 1742965, upload-time = "2025-10-28T20:58:03.972Z" }, + { url = "https://files.pythonhosted.org/packages/67/7f/db2fc7618925e8c7a601094d5cbe539f732df4fb570740be88ed9e40e99a/aiohttp-3.13.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:88d6c017966a78c5265d996c19cdb79235be5e6412268d7e2ce7dee339471b7a", size = 1697585, upload-time = "2025-10-28T20:58:06.189Z" }, + { url = "https://files.pythonhosted.org/packages/c7/8e/3824ef98c039d3951cb65b9205a96dd2b20f22241ee17d89c5701557c826/aiohttp-3.13.2-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:f10d9c0b0188fe85398c61147bbd2a657d616c876863bfeff43376e0e3134673", size = 767360, upload-time = "2025-10-28T20:58:13.358Z" }, + { url = "https://files.pythonhosted.org/packages/a4/0f/6a03e3fc7595421274fa34122c973bde2d89344f8a881b728fa8c774e4f1/aiohttp-3.13.2-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:e7c952aefdf2460f4ae55c5e9c3e80aa72f706a6317e06020f80e96253b1accd", size = 504616, upload-time = "2025-10-28T20:58:15.339Z" }, + { url = "https://files.pythonhosted.org/packages/c6/aa/ed341b670f1bc8a6f2c6a718353d13b9546e2cef3544f573c6a1ff0da711/aiohttp-3.13.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c20423ce14771d98353d2e25e83591fa75dfa90a3c1848f3d7c68243b4fbded3", size = 509131, upload-time = "2025-10-28T20:58:17.693Z" }, + { url = "https://files.pythonhosted.org/packages/7f/f0/c68dac234189dae5c4bbccc0f96ce0cc16b76632cfc3a08fff180045cfa4/aiohttp-3.13.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e96eb1a34396e9430c19d8338d2ec33015e4a87ef2b4449db94c22412e25ccdf", size = 1864168, upload-time = "2025-10-28T20:58:20.113Z" }, + { url = "https://files.pythonhosted.org/packages/8f/65/75a9a76db8364b5d0e52a0c20eabc5d52297385d9af9c35335b924fafdee/aiohttp-3.13.2-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:23fb0783bc1a33640036465019d3bba069942616a6a2353c6907d7fe1ccdaf4e", size = 1719200, upload-time = "2025-10-28T20:58:22.583Z" }, + { url = "https://files.pythonhosted.org/packages/f5/55/8df2ed78d7f41d232f6bd3ff866b6f617026551aa1d07e2f03458f964575/aiohttp-3.13.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2e1a9bea6244a1d05a4e57c295d69e159a5c50d8ef16aa390948ee873478d9a5", size = 1843497, upload-time = "2025-10-28T20:58:24.672Z" }, + { url = "https://files.pythonhosted.org/packages/e9/e0/94d7215e405c5a02ccb6a35c7a3a6cfff242f457a00196496935f700cde5/aiohttp-3.13.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0a3d54e822688b56e9f6b5816fb3de3a3a64660efac64e4c2dc435230ad23bad", size = 1935703, upload-time = "2025-10-28T20:58:26.758Z" }, + { url = "https://files.pythonhosted.org/packages/0b/78/1eeb63c3f9b2d1015a4c02788fb543141aad0a03ae3f7a7b669b2483f8d4/aiohttp-3.13.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7a653d872afe9f33497215745da7a943d1dc15b728a9c8da1c3ac423af35178e", size = 1792738, upload-time = "2025-10-28T20:58:29.787Z" }, + { url = "https://files.pythonhosted.org/packages/41/75/aaf1eea4c188e51538c04cc568040e3082db263a57086ea74a7d38c39e42/aiohttp-3.13.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:56d36e80d2003fa3fc0207fac644216d8532e9504a785ef9a8fd013f84a42c61", size = 1624061, upload-time = "2025-10-28T20:58:32.529Z" }, + { url = "https://files.pythonhosted.org/packages/9b/c2/3b6034de81fbcc43de8aeb209073a2286dfb50b86e927b4efd81cf848197/aiohttp-3.13.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:78cd586d8331fb8e241c2dd6b2f4061778cc69e150514b39a9e28dd050475661", size = 1789201, upload-time = "2025-10-28T20:58:34.618Z" }, + { url = "https://files.pythonhosted.org/packages/c9/38/c15dcf6d4d890217dae79d7213988f4e5fe6183d43893a9cf2fe9e84ca8d/aiohttp-3.13.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:20b10bbfbff766294fe99987f7bb3b74fdd2f1a2905f2562132641ad434dcf98", size = 1776868, upload-time = "2025-10-28T20:58:38.835Z" }, + { url = "https://files.pythonhosted.org/packages/04/75/f74fd178ac81adf4f283a74847807ade5150e48feda6aef024403716c30c/aiohttp-3.13.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:9ec49dff7e2b3c85cdeaa412e9d438f0ecd71676fde61ec57027dd392f00c693", size = 1790660, upload-time = "2025-10-28T20:58:41.507Z" }, + { url = "https://files.pythonhosted.org/packages/e7/80/7368bd0d06b16b3aba358c16b919e9c46cf11587dc572091031b0e9e3ef0/aiohttp-3.13.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:94f05348c4406450f9d73d38efb41d669ad6cd90c7ee194810d0eefbfa875a7a", size = 1617548, upload-time = "2025-10-28T20:58:43.674Z" }, + { url = "https://files.pythonhosted.org/packages/7d/4b/a6212790c50483cb3212e507378fbe26b5086d73941e1ec4b56a30439688/aiohttp-3.13.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:fa4dcb605c6f82a80c7f95713c2b11c3b8e9893b3ebd2bc9bde93165ed6107be", size = 1817240, upload-time = "2025-10-28T20:58:45.787Z" }, + { url = "https://files.pythonhosted.org/packages/ff/f7/ba5f0ba4ea8d8f3c32850912944532b933acbf0f3a75546b89269b9b7dde/aiohttp-3.13.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cf00e5db968c3f67eccd2778574cf64d8b27d95b237770aa32400bd7a1ca4f6c", size = 1762334, upload-time = "2025-10-28T20:58:47.936Z" }, +] + +[[package]] +name = "aiosignal" +version = "1.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "frozenlist", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/61/62/06741b579156360248d1ec624842ad0edf697050bbaf7c3e46394e106ad1/aiosignal-1.4.0.tar.gz", hash = "sha256:f47eecd9468083c2029cc99945502cb7708b082c232f9aca65da147157b251c7", size = 25007, upload-time = "2025-07-03T22:54:43.528Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", hash = "sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", size = 7490, upload-time = "2025-07-03T22:54:42.156Z" }, +] + +[[package]] +name = "aiosqlite" +version = "0.21.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/13/7d/8bca2bf9a247c2c5dfeec1d7a5f40db6518f88d314b8bca9da29670d2671/aiosqlite-0.21.0.tar.gz", hash = "sha256:131bb8056daa3bc875608c631c678cda73922a2d4ba8aec373b19f18c17e7aa3", size = 13454, upload-time = "2025-02-03T07:30:16.235Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f5/10/6c25ed6de94c49f88a91fa5018cb4c0f3625f31d5be9f771ebe5cc7cd506/aiosqlite-0.21.0-py3-none-any.whl", hash = "sha256:2549cf4057f95f53dcba16f2b64e8e2791d7e1adedb13197dd8ed77bb226d7d0", size = 15792, upload-time = "2025-02-03T07:30:13.6Z" }, +] + +[[package]] +name = "annotated-doc" +version = "0.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/a6/dc46877b911e40c00d395771ea710d5e77b6de7bacd5fdcd78d70cc5a48f/annotated_doc-0.0.3.tar.gz", hash = "sha256:e18370014c70187422c33e945053ff4c286f453a984eba84d0dbfa0c935adeda", size = 5535, upload-time = "2025-10-24T14:57:10.718Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/02/b7/cf592cb5de5cb3bade3357f8d2cf42bf103bbe39f459824b4939fd212911/annotated_doc-0.0.3-py3-none-any.whl", hash = "sha256:348ec6664a76f1fd3be81f43dffbee4c7e8ce931ba71ec67cc7f4ade7fbbb580", size = 5488, upload-time = "2025-10-24T14:57:09.462Z" }, +] + +[[package]] +name = "annotated-types" +version = "0.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ee/67/531ea369ba64dcff5ec9c3402f9f51bf748cec26dde048a2f973a4eea7f5/annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89", size = 16081, upload-time = "2024-05-20T21:33:25.928Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" }, +] + +[[package]] +name = "anyio" +version = "4.11.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "idna", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "sniffio", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c6/78/7d432127c41b50bccba979505f272c16cbcadcc33645d5fa3a738110ae75/anyio-4.11.0.tar.gz", hash = "sha256:82a8d0b81e318cc5ce71a5f1f8b5c4e63619620b63141ef8c995fa0db95a57c4", size = 219094, upload-time = "2025-09-23T09:19:12.58Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/15/b3/9b1a8074496371342ec1e796a96f99c82c945a339cd81a8e73de28b4cf9e/anyio-4.11.0-py3-none-any.whl", hash = "sha256:0287e96f4d26d4149305414d4e3bc32f0dcd0862365a4bddea19d7a1ec38c4fc", size = 109097, upload-time = "2025-09-23T09:19:10.601Z" }, +] + +[[package]] +name = "attrs" +version = "25.4.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6b/5c/685e6633917e101e5dcb62b9dd76946cbb57c26e133bae9e0cd36033c0a9/attrs-25.4.0.tar.gz", hash = "sha256:16d5969b87f0859ef33a48b35d55ac1be6e42ae49d5e853b597db70c35c57e11", size = 934251, upload-time = "2025-10-06T13:54:44.725Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/3a/2a/7cc015f5b9f5db42b7d48157e23356022889fc354a2813c15934b7cb5c0e/attrs-25.4.0-py3-none-any.whl", hash = "sha256:adcf7e2a1fb3b36ac48d97835bb6d8ade15b8dcce26aba8bf1d14847b57a3373", size = 67615, upload-time = "2025-10-06T13:54:43.17Z" }, +] + +[[package]] +name = "base58" +version = "2.1.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7f/45/8ae61209bb9015f516102fa559a2914178da1d5868428bd86a1b4421141d/base58-2.1.1.tar.gz", hash = "sha256:c5d0cb3f5b6e81e8e35da5754388ddcc6d0d14b6c6a132cb93d69ed580a7278c", size = 6528, upload-time = "2021-10-30T22:12:17.858Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4a/45/ec96b29162a402fc4c1c5512d114d7b3787b9d1c2ec241d9568b4816ee23/base58-2.1.1-py3-none-any.whl", hash = "sha256:11a36f4d3ce51dfc1043f3218591ac4eb1ceb172919cebe05b52a5bcc8d245c2", size = 5621, upload-time = "2021-10-30T22:12:16.658Z" }, +] + +[[package]] +name = "bidict" +version = "0.23.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/9a/6e/026678aa5a830e07cd9498a05d3e7e650a4f56a42f267a53d22bcda1bdc9/bidict-0.23.1.tar.gz", hash = "sha256:03069d763bc387bbd20e7d49914e75fc4132a41937fa3405417e1a5a2d006d71", size = 29093, upload-time = "2024-02-18T19:09:05.748Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/99/37/e8730c3587a65eb5645d4aba2d27aae48e8003614d6aaf15dda67f702f1f/bidict-0.23.1-py3-none-any.whl", hash = "sha256:5dae8d4d79b552a71cbabc7deb25dfe8ce710b17ff41711e13010ead2abfc3e5", size = 32764, upload-time = "2024-02-18T19:09:04.156Z" }, +] + +[[package]] +name = "certifi" +version = "2025.10.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/4c/5b/b6ce21586237c77ce67d01dc5507039d444b630dd76611bbca2d8e5dcd91/certifi-2025.10.5.tar.gz", hash = "sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43", size = 164519, upload-time = "2025-10-05T04:12:15.808Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e4/37/af0d2ef3967ac0d6113837b44a4f0bfe1328c2b9763bd5b1744520e5cfed/certifi-2025.10.5-py3-none-any.whl", hash = "sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de", size = 163286, upload-time = "2025-10-05T04:12:14.03Z" }, +] + +[[package]] +name = "cffi" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pycparser", marker = "(implementation_name != 'PyPy' and sys_platform == 'darwin') or (implementation_name != 'PyPy' and sys_platform == 'linux')" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", size = 523588, upload-time = "2025-09-08T23:24:04.541Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", size = 185230, upload-time = "2025-09-08T23:23:00.879Z" }, + { url = "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", size = 181043, upload-time = "2025-09-08T23:23:02.231Z" }, + { url = "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", size = 212446, upload-time = "2025-09-08T23:23:03.472Z" }, + { url = "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", size = 220101, upload-time = "2025-09-08T23:23:04.792Z" }, + { url = "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", size = 207948, upload-time = "2025-09-08T23:23:06.127Z" }, + { url = "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", size = 206422, upload-time = "2025-09-08T23:23:07.753Z" }, + { url = "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", size = 219499, upload-time = "2025-09-08T23:23:09.648Z" }, + { url = "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", size = 222928, upload-time = "2025-09-08T23:23:10.928Z" }, + { url = "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", size = 221302, upload-time = "2025-09-08T23:23:12.42Z" }, + { url = "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", size = 185320, upload-time = "2025-09-08T23:23:18.087Z" }, + { url = "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", size = 181487, upload-time = "2025-09-08T23:23:19.622Z" }, + { url = "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", size = 220049, upload-time = "2025-09-08T23:23:20.853Z" }, + { url = "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", size = 207793, upload-time = "2025-09-08T23:23:22.08Z" }, + { url = "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", size = 206300, upload-time = "2025-09-08T23:23:23.314Z" }, + { url = "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", size = 219244, upload-time = "2025-09-08T23:23:24.541Z" }, + { url = "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", size = 222828, upload-time = "2025-09-08T23:23:26.143Z" }, + { url = "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", size = 220926, upload-time = "2025-09-08T23:23:27.873Z" }, + { url = "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", size = 188773, upload-time = "2025-09-08T23:23:29.347Z" }, + { url = "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", size = 185013, upload-time = "2025-09-08T23:23:30.63Z" }, + { url = "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", size = 221593, upload-time = "2025-09-08T23:23:31.91Z" }, + { url = "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", size = 209354, upload-time = "2025-09-08T23:23:33.214Z" }, + { url = "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", size = 208480, upload-time = "2025-09-08T23:23:34.495Z" }, + { url = "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", size = 221584, upload-time = "2025-09-08T23:23:36.096Z" }, + { url = "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", size = 224443, upload-time = "2025-09-08T23:23:37.328Z" }, + { url = "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", size = 223437, upload-time = "2025-09-08T23:23:38.945Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/13/69/33ddede1939fdd074bce5434295f38fae7136463422fe4fd3e0e89b98062/charset_normalizer-3.4.4.tar.gz", hash = "sha256:94537985111c35f28720e43603b8e7b43a6ecfb2ce1d3058bbe955b73404e21a", size = 129418, upload-time = "2025-10-14T04:42:32.879Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/97/45/4b3a1239bbacd321068ea6e7ac28875b03ab8bc0aa0966452db17cd36714/charset_normalizer-3.4.4-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:e1f185f86a6f3403aa2420e815904c67b2f9ebc443f045edd0de921108345794", size = 208091, upload-time = "2025-10-14T04:41:13.346Z" }, + { url = "https://files.pythonhosted.org/packages/7d/62/73a6d7450829655a35bb88a88fca7d736f9882a27eacdca2c6d505b57e2e/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b39f987ae8ccdf0d2642338faf2abb1862340facc796048b604ef14919e55ed", size = 147936, upload-time = "2025-10-14T04:41:14.461Z" }, + { url = "https://files.pythonhosted.org/packages/89/c5/adb8c8b3d6625bef6d88b251bbb0d95f8205831b987631ab0c8bb5d937c2/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3162d5d8ce1bb98dd51af660f2121c55d0fa541b46dff7bb9b9f86ea1d87de72", size = 144180, upload-time = "2025-10-14T04:41:15.588Z" }, + { url = "https://files.pythonhosted.org/packages/91/ed/9706e4070682d1cc219050b6048bfd293ccf67b3d4f5a4f39207453d4b99/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:81d5eb2a312700f4ecaa977a8235b634ce853200e828fbadf3a9c50bab278328", size = 161346, upload-time = "2025-10-14T04:41:16.738Z" }, + { url = "https://files.pythonhosted.org/packages/d5/0d/031f0d95e4972901a2f6f09ef055751805ff541511dc1252ba3ca1f80cf5/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5bd2293095d766545ec1a8f612559f6b40abc0eb18bb2f5d1171872d34036ede", size = 158874, upload-time = "2025-10-14T04:41:17.923Z" }, + { url = "https://files.pythonhosted.org/packages/f5/83/6ab5883f57c9c801ce5e5677242328aa45592be8a00644310a008d04f922/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8a8b89589086a25749f471e6a900d3f662d1d3b6e2e59dcecf787b1cc3a1894", size = 153076, upload-time = "2025-10-14T04:41:19.106Z" }, + { url = "https://files.pythonhosted.org/packages/75/1e/5ff781ddf5260e387d6419959ee89ef13878229732732ee73cdae01800f2/charset_normalizer-3.4.4-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc7637e2f80d8530ee4a78e878bce464f70087ce73cf7c1caf142416923b98f1", size = 150601, upload-time = "2025-10-14T04:41:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/d7/57/71be810965493d3510a6ca79b90c19e48696fb1ff964da319334b12677f0/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f8bf04158c6b607d747e93949aa60618b61312fe647a6369f88ce2ff16043490", size = 150376, upload-time = "2025-10-14T04:41:21.398Z" }, + { url = "https://files.pythonhosted.org/packages/e5/d5/c3d057a78c181d007014feb7e9f2e65905a6c4ef182c0ddf0de2924edd65/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:554af85e960429cf30784dd47447d5125aaa3b99a6f0683589dbd27e2f45da44", size = 144825, upload-time = "2025-10-14T04:41:22.583Z" }, + { url = "https://files.pythonhosted.org/packages/e6/8c/d0406294828d4976f275ffbe66f00266c4b3136b7506941d87c00cab5272/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:74018750915ee7ad843a774364e13a3db91682f26142baddf775342c3f5b1133", size = 162583, upload-time = "2025-10-14T04:41:23.754Z" }, + { url = "https://files.pythonhosted.org/packages/d7/24/e2aa1f18c8f15c4c0e932d9287b8609dd30ad56dbe41d926bd846e22fb8d/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:c0463276121fdee9c49b98908b3a89c39be45d86d1dbaa22957e38f6321d4ce3", size = 150366, upload-time = "2025-10-14T04:41:25.27Z" }, + { url = "https://files.pythonhosted.org/packages/e4/5b/1e6160c7739aad1e2df054300cc618b06bf784a7a164b0f238360721ab86/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:362d61fd13843997c1c446760ef36f240cf81d3ebf74ac62652aebaf7838561e", size = 160300, upload-time = "2025-10-14T04:41:26.725Z" }, + { url = "https://files.pythonhosted.org/packages/7a/10/f882167cd207fbdd743e55534d5d9620e095089d176d55cb22d5322f2afd/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9a26f18905b8dd5d685d6d07b0cdf98a79f3c7a918906af7cc143ea2e164c8bc", size = 154465, upload-time = "2025-10-14T04:41:28.322Z" }, + { url = "https://files.pythonhosted.org/packages/2a/35/7051599bd493e62411d6ede36fd5af83a38f37c4767b92884df7301db25d/charset_normalizer-3.4.4-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:da3326d9e65ef63a817ecbcc0df6e94463713b754fe293eaa03da99befb9a5bd", size = 207746, upload-time = "2025-10-14T04:41:33.773Z" }, + { url = "https://files.pythonhosted.org/packages/10/9a/97c8d48ef10d6cd4fcead2415523221624bf58bcf68a802721a6bc807c8f/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8af65f14dc14a79b924524b1e7fffe304517b2bff5a58bf64f30b98bbc5079eb", size = 147889, upload-time = "2025-10-14T04:41:34.897Z" }, + { url = "https://files.pythonhosted.org/packages/10/bf/979224a919a1b606c82bd2c5fa49b5c6d5727aa47b4312bb27b1734f53cd/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:74664978bb272435107de04e36db5a9735e78232b85b77d45cfb38f758efd33e", size = 143641, upload-time = "2025-10-14T04:41:36.116Z" }, + { url = "https://files.pythonhosted.org/packages/ba/33/0ad65587441fc730dc7bd90e9716b30b4702dc7b617e6ba4997dc8651495/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:752944c7ffbfdd10c074dc58ec2d5a8a4cd9493b314d367c14d24c17684ddd14", size = 160779, upload-time = "2025-10-14T04:41:37.229Z" }, + { url = "https://files.pythonhosted.org/packages/67/ed/331d6b249259ee71ddea93f6f2f0a56cfebd46938bde6fcc6f7b9a3d0e09/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d1f13550535ad8cff21b8d757a3257963e951d96e20ec82ab44bc64aeb62a191", size = 159035, upload-time = "2025-10-14T04:41:38.368Z" }, + { url = "https://files.pythonhosted.org/packages/67/ff/f6b948ca32e4f2a4576aa129d8bed61f2e0543bf9f5f2b7fc3758ed005c9/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ecaae4149d99b1c9e7b88bb03e3221956f68fd6d50be2ef061b2381b61d20838", size = 152542, upload-time = "2025-10-14T04:41:39.862Z" }, + { url = "https://files.pythonhosted.org/packages/16/85/276033dcbcc369eb176594de22728541a925b2632f9716428c851b149e83/charset_normalizer-3.4.4-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cb6254dc36b47a990e59e1068afacdcd02958bdcce30bb50cc1700a8b9d624a6", size = 149524, upload-time = "2025-10-14T04:41:41.319Z" }, + { url = "https://files.pythonhosted.org/packages/9e/f2/6a2a1f722b6aba37050e626530a46a68f74e63683947a8acff92569f979a/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c8ae8a0f02f57a6e61203a31428fa1d677cbe50c93622b4149d5c0f319c1d19e", size = 150395, upload-time = "2025-10-14T04:41:42.539Z" }, + { url = "https://files.pythonhosted.org/packages/60/bb/2186cb2f2bbaea6338cad15ce23a67f9b0672929744381e28b0592676824/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:47cc91b2f4dd2833fddaedd2893006b0106129d4b94fdb6af1f4ce5a9965577c", size = 143680, upload-time = "2025-10-14T04:41:43.661Z" }, + { url = "https://files.pythonhosted.org/packages/7d/a5/bf6f13b772fbb2a90360eb620d52ed8f796f3c5caee8398c3b2eb7b1c60d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:82004af6c302b5d3ab2cfc4cc5f29db16123b1a8417f2e25f9066f91d4411090", size = 162045, upload-time = "2025-10-14T04:41:44.821Z" }, + { url = "https://files.pythonhosted.org/packages/df/c5/d1be898bf0dc3ef9030c3825e5d3b83f2c528d207d246cbabe245966808d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7d8f6c26245217bd2ad053761201e9f9680f8ce52f0fcd8d0755aeae5b2152", size = 149687, upload-time = "2025-10-14T04:41:46.442Z" }, + { url = "https://files.pythonhosted.org/packages/a5/42/90c1f7b9341eef50c8a1cb3f098ac43b0508413f33affd762855f67a410e/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:799a7a5e4fb2d5898c60b640fd4981d6a25f1c11790935a44ce38c54e985f828", size = 160014, upload-time = "2025-10-14T04:41:47.631Z" }, + { url = "https://files.pythonhosted.org/packages/76/be/4d3ee471e8145d12795ab655ece37baed0929462a86e72372fd25859047c/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:99ae2cffebb06e6c22bdc25801d7b30f503cc87dbd283479e7b606f70aff57ec", size = 154044, upload-time = "2025-10-14T04:41:48.81Z" }, + { url = "https://files.pythonhosted.org/packages/0a/4c/925909008ed5a988ccbb72dcc897407e5d6d3bd72410d69e051fc0c14647/charset_normalizer-3.4.4-py3-none-any.whl", hash = "sha256:7a32c560861a02ff789ad905a2fe94e3f840803362c84fecf1851cb4cf3dc37f", size = 53402, upload-time = "2025-10-14T04:42:31.76Z" }, +] + +[[package]] +name = "cryptography" +version = "46.0.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cffi", marker = "(platform_python_implementation != 'PyPy' and sys_platform == 'darwin') or (platform_python_implementation != 'PyPy' and sys_platform == 'linux')" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9f/33/c00162f49c0e2fe8064a62cb92b93e50c74a72bc370ab92f86112b33ff62/cryptography-46.0.3.tar.gz", hash = "sha256:a8b17438104fed022ce745b362294d9ce35b4c2e45c1d958ad4a4b019285f4a1", size = 749258, upload-time = "2025-10-15T23:18:31.74Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1d/42/9c391dd801d6cf0d561b5890549d4b27bafcc53b39c31a817e69d87c625b/cryptography-46.0.3-cp311-abi3-macosx_10_9_universal2.whl", hash = "sha256:109d4ddfadf17e8e7779c39f9b18111a09efb969a301a31e987416a0191ed93a", size = 7225004, upload-time = "2025-10-15T23:16:52.239Z" }, + { url = "https://files.pythonhosted.org/packages/1c/67/38769ca6b65f07461eb200e85fc1639b438bdc667be02cf7f2cd6a64601c/cryptography-46.0.3-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:09859af8466b69bc3c27bdf4f5d84a665e0f7ab5088412e9e2ec49758eca5cbc", size = 4296667, upload-time = "2025-10-15T23:16:54.369Z" }, + { url = "https://files.pythonhosted.org/packages/5c/49/498c86566a1d80e978b42f0d702795f69887005548c041636df6ae1ca64c/cryptography-46.0.3-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:01ca9ff2885f3acc98c29f1860552e37f6d7c7d013d7334ff2a9de43a449315d", size = 4450807, upload-time = "2025-10-15T23:16:56.414Z" }, + { url = "https://files.pythonhosted.org/packages/4b/0a/863a3604112174c8624a2ac3c038662d9e59970c7f926acdcfaed8d61142/cryptography-46.0.3-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:6eae65d4c3d33da080cff9c4ab1f711b15c1d9760809dad6ea763f3812d254cb", size = 4299615, upload-time = "2025-10-15T23:16:58.442Z" }, + { url = "https://files.pythonhosted.org/packages/64/02/b73a533f6b64a69f3cd3872acb6ebc12aef924d8d103133bb3ea750dc703/cryptography-46.0.3-cp311-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:e5bf0ed4490068a2e72ac03d786693adeb909981cc596425d09032d372bcc849", size = 4016800, upload-time = "2025-10-15T23:17:00.378Z" }, + { url = "https://files.pythonhosted.org/packages/25/d5/16e41afbfa450cde85a3b7ec599bebefaef16b5c6ba4ec49a3532336ed72/cryptography-46.0.3-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:5ecfccd2329e37e9b7112a888e76d9feca2347f12f37918facbb893d7bb88ee8", size = 4984707, upload-time = "2025-10-15T23:17:01.98Z" }, + { url = "https://files.pythonhosted.org/packages/c9/56/e7e69b427c3878352c2fb9b450bd0e19ed552753491d39d7d0a2f5226d41/cryptography-46.0.3-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:a2c0cd47381a3229c403062f764160d57d4d175e022c1df84e168c6251a22eec", size = 4482541, upload-time = "2025-10-15T23:17:04.078Z" }, + { url = "https://files.pythonhosted.org/packages/78/f6/50736d40d97e8483172f1bb6e698895b92a223dba513b0ca6f06b2365339/cryptography-46.0.3-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:549e234ff32571b1f4076ac269fcce7a808d3bf98b76c8dd560e42dbc66d7d91", size = 4299464, upload-time = "2025-10-15T23:17:05.483Z" }, + { url = "https://files.pythonhosted.org/packages/00/de/d8e26b1a855f19d9994a19c702fa2e93b0456beccbcfe437eda00e0701f2/cryptography-46.0.3-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:c0a7bb1a68a5d3471880e264621346c48665b3bf1c3759d682fc0864c540bd9e", size = 4950838, upload-time = "2025-10-15T23:17:07.425Z" }, + { url = "https://files.pythonhosted.org/packages/8f/29/798fc4ec461a1c9e9f735f2fc58741b0daae30688f41b2497dcbc9ed1355/cryptography-46.0.3-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:10b01676fc208c3e6feeb25a8b83d81767e8059e1fe86e1dc62d10a3018fa926", size = 4481596, upload-time = "2025-10-15T23:17:09.343Z" }, + { url = "https://files.pythonhosted.org/packages/15/8d/03cd48b20a573adfff7652b76271078e3045b9f49387920e7f1f631d125e/cryptography-46.0.3-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0abf1ffd6e57c67e92af68330d05760b7b7efb243aab8377e583284dbab72c71", size = 4426782, upload-time = "2025-10-15T23:17:11.22Z" }, + { url = "https://files.pythonhosted.org/packages/fa/b1/ebacbfe53317d55cf33165bda24c86523497a6881f339f9aae5c2e13e57b/cryptography-46.0.3-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a04bee9ab6a4da801eb9b51f1b708a1b5b5c9eb48c03f74198464c66f0d344ac", size = 4698381, upload-time = "2025-10-15T23:17:12.829Z" }, + { url = "https://files.pythonhosted.org/packages/f5/e2/a510aa736755bffa9d2f75029c229111a1d02f8ecd5de03078f4c18d91a3/cryptography-46.0.3-cp314-cp314t-macosx_10_9_universal2.whl", hash = "sha256:00a5e7e87938e5ff9ff5447ab086a5706a957137e6e433841e9d24f38a065217", size = 7158012, upload-time = "2025-10-15T23:17:19.982Z" }, + { url = "https://files.pythonhosted.org/packages/73/dc/9aa866fbdbb95b02e7f9d086f1fccfeebf8953509b87e3f28fff927ff8a0/cryptography-46.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c8daeb2d2174beb4575b77482320303f3d39b8e81153da4f0fb08eb5fe86a6c5", size = 4288728, upload-time = "2025-10-15T23:17:21.527Z" }, + { url = "https://files.pythonhosted.org/packages/c5/fd/bc1daf8230eaa075184cbbf5f8cd00ba9db4fd32d63fb83da4671b72ed8a/cryptography-46.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:39b6755623145ad5eff1dab323f4eae2a32a77a7abef2c5089a04a3d04366715", size = 4435078, upload-time = "2025-10-15T23:17:23.042Z" }, + { url = "https://files.pythonhosted.org/packages/82/98/d3bd5407ce4c60017f8ff9e63ffee4200ab3e23fe05b765cab805a7db008/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:db391fa7c66df6762ee3f00c95a89e6d428f4d60e7abc8328f4fe155b5ac6e54", size = 4293460, upload-time = "2025-10-15T23:17:24.885Z" }, + { url = "https://files.pythonhosted.org/packages/26/e9/e23e7900983c2b8af7a08098db406cf989d7f09caea7897e347598d4cd5b/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:78a97cf6a8839a48c49271cdcbd5cf37ca2c1d6b7fdd86cc864f302b5e9bf459", size = 3995237, upload-time = "2025-10-15T23:17:26.449Z" }, + { url = "https://files.pythonhosted.org/packages/91/15/af68c509d4a138cfe299d0d7ddb14afba15233223ebd933b4bbdbc7155d3/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:dfb781ff7eaa91a6f7fd41776ec37c5853c795d3b358d4896fdbb5df168af422", size = 4967344, upload-time = "2025-10-15T23:17:28.06Z" }, + { url = "https://files.pythonhosted.org/packages/ca/e3/8643d077c53868b681af077edf6b3cb58288b5423610f21c62aadcbe99f4/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:6f61efb26e76c45c4a227835ddeae96d83624fb0d29eb5df5b96e14ed1a0afb7", size = 4466564, upload-time = "2025-10-15T23:17:29.665Z" }, + { url = "https://files.pythonhosted.org/packages/0e/43/c1e8726fa59c236ff477ff2b5dc071e54b21e5a1e51aa2cee1676f1c986f/cryptography-46.0.3-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:23b1a8f26e43f47ceb6d6a43115f33a5a37d57df4ea0ca295b780ae8546e8044", size = 4292415, upload-time = "2025-10-15T23:17:31.686Z" }, + { url = "https://files.pythonhosted.org/packages/42/f9/2f8fefdb1aee8a8e3256a0568cffc4e6d517b256a2fe97a029b3f1b9fe7e/cryptography-46.0.3-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:b419ae593c86b87014b9be7396b385491ad7f320bde96826d0dd174459e54665", size = 4931457, upload-time = "2025-10-15T23:17:33.478Z" }, + { url = "https://files.pythonhosted.org/packages/79/30/9b54127a9a778ccd6d27c3da7563e9f2d341826075ceab89ae3b41bf5be2/cryptography-46.0.3-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:50fc3343ac490c6b08c0cf0d704e881d0d660be923fd3076db3e932007e726e3", size = 4466074, upload-time = "2025-10-15T23:17:35.158Z" }, + { url = "https://files.pythonhosted.org/packages/ac/68/b4f4a10928e26c941b1b6a179143af9f4d27d88fe84a6a3c53592d2e76bf/cryptography-46.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:22d7e97932f511d6b0b04f2bfd818d73dcd5928db509460aaf48384778eb6d20", size = 4420569, upload-time = "2025-10-15T23:17:37.188Z" }, + { url = "https://files.pythonhosted.org/packages/a3/49/3746dab4c0d1979888f125226357d3262a6dd40e114ac29e3d2abdf1ec55/cryptography-46.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:d55f3dffadd674514ad19451161118fd010988540cee43d8bc20675e775925de", size = 4681941, upload-time = "2025-10-15T23:17:39.236Z" }, + { url = "https://files.pythonhosted.org/packages/fd/23/45fe7f376a7df8daf6da3556603b36f53475a99ce4faacb6ba2cf3d82021/cryptography-46.0.3-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:cb3d760a6117f621261d662bccc8ef5bc32ca673e037c83fbe565324f5c46936", size = 7218248, upload-time = "2025-10-15T23:17:46.294Z" }, + { url = "https://files.pythonhosted.org/packages/27/32/b68d27471372737054cbd34c84981f9edbc24fe67ca225d389799614e27f/cryptography-46.0.3-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:4b7387121ac7d15e550f5cb4a43aef2559ed759c35df7336c402bb8275ac9683", size = 4294089, upload-time = "2025-10-15T23:17:48.269Z" }, + { url = "https://files.pythonhosted.org/packages/26/42/fa8389d4478368743e24e61eea78846a0006caffaf72ea24a15159215a14/cryptography-46.0.3-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:15ab9b093e8f09daab0f2159bb7e47532596075139dd74365da52ecc9cb46c5d", size = 4440029, upload-time = "2025-10-15T23:17:49.837Z" }, + { url = "https://files.pythonhosted.org/packages/5f/eb/f483db0ec5ac040824f269e93dd2bd8a21ecd1027e77ad7bdf6914f2fd80/cryptography-46.0.3-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:46acf53b40ea38f9c6c229599a4a13f0d46a6c3fa9ef19fc1a124d62e338dfa0", size = 4297222, upload-time = "2025-10-15T23:17:51.357Z" }, + { url = "https://files.pythonhosted.org/packages/fd/cf/da9502c4e1912cb1da3807ea3618a6829bee8207456fbbeebc361ec38ba3/cryptography-46.0.3-cp38-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:10ca84c4668d066a9878890047f03546f3ae0a6b8b39b697457b7757aaf18dbc", size = 4012280, upload-time = "2025-10-15T23:17:52.964Z" }, + { url = "https://files.pythonhosted.org/packages/6b/8f/9adb86b93330e0df8b3dcf03eae67c33ba89958fc2e03862ef1ac2b42465/cryptography-46.0.3-cp38-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:36e627112085bb3b81b19fed209c05ce2a52ee8b15d161b7c643a7d5a88491f3", size = 4978958, upload-time = "2025-10-15T23:17:54.965Z" }, + { url = "https://files.pythonhosted.org/packages/d1/a0/5fa77988289c34bdb9f913f5606ecc9ada1adb5ae870bd0d1054a7021cc4/cryptography-46.0.3-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:1000713389b75c449a6e979ffc7dcc8ac90b437048766cef052d4d30b8220971", size = 4473714, upload-time = "2025-10-15T23:17:56.754Z" }, + { url = "https://files.pythonhosted.org/packages/14/e5/fc82d72a58d41c393697aa18c9abe5ae1214ff6f2a5c18ac470f92777895/cryptography-46.0.3-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:b02cf04496f6576afffef5ddd04a0cb7d49cf6be16a9059d793a30b035f6b6ac", size = 4296970, upload-time = "2025-10-15T23:17:58.588Z" }, + { url = "https://files.pythonhosted.org/packages/78/06/5663ed35438d0b09056973994f1aec467492b33bd31da36e468b01ec1097/cryptography-46.0.3-cp38-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:71e842ec9bc7abf543b47cf86b9a743baa95f4677d22baa4c7d5c69e49e9bc04", size = 4940236, upload-time = "2025-10-15T23:18:00.897Z" }, + { url = "https://files.pythonhosted.org/packages/fc/59/873633f3f2dcd8a053b8dd1d38f783043b5fce589c0f6988bf55ef57e43e/cryptography-46.0.3-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:402b58fc32614f00980b66d6e56a5b4118e6cb362ae8f3fda141ba4689bd4506", size = 4472642, upload-time = "2025-10-15T23:18:02.749Z" }, + { url = "https://files.pythonhosted.org/packages/3d/39/8e71f3930e40f6877737d6f69248cf74d4e34b886a3967d32f919cc50d3b/cryptography-46.0.3-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:ef639cb3372f69ec44915fafcd6698b6cc78fbe0c2ea41be867f6ed612811963", size = 4423126, upload-time = "2025-10-15T23:18:04.85Z" }, + { url = "https://files.pythonhosted.org/packages/cd/c7/f65027c2810e14c3e7268353b1681932b87e5a48e65505d8cc17c99e36ae/cryptography-46.0.3-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3b51b8ca4f1c6453d8829e1eb7299499ca7f313900dd4d89a24b8b87c0a780d4", size = 4686573, upload-time = "2025-10-15T23:18:06.908Z" }, +] + +[[package]] +name = "exo" +version = "0.3.0" +source = { editable = "." } +dependencies = [ + { name = "aiofiles", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "aiohttp", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "aiosqlite", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "anyio", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "base58", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "bidict", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "cryptography", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "exo-pyo3-bindings", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "fastapi", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "filelock", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "greenlet", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "huggingface-hub", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "hypercorn", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "loguru", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "mlx", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "mlx-lm", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "networkx", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pathlib", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "protobuf", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "psutil", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pydantic", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "rich", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "rustworkx", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "sqlalchemy", extra = ["asyncio"], marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "sqlmodel", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "textual", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "tiktoken", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "typeguard", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "types-aiofiles", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] + +[package.dev-dependencies] +dev = [ + { name = "pytest", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pytest-asyncio", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pytest-env", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "ruff", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] + +[package.metadata] +requires-dist = [ + { name = "aiofiles", specifier = ">=24.1.0" }, + { name = "aiohttp", specifier = ">=3.12.14" }, + { name = "aiosqlite", specifier = ">=0.21.0" }, + { name = "anyio", specifier = "==4.11.0" }, + { name = "base58", specifier = ">=2.1.1" }, + { name = "bidict", specifier = ">=0.23.1" }, + { name = "cryptography", specifier = ">=45.0.5" }, + { name = "exo-pyo3-bindings", editable = "rust/exo_pyo3_bindings" }, + { name = "fastapi", specifier = ">=0.116.1" }, + { name = "filelock", specifier = ">=3.18.0" }, + { name = "greenlet", specifier = ">=3.2.4" }, + { name = "huggingface-hub", specifier = ">=0.33.4" }, + { name = "hypercorn", specifier = ">=0.18.0" }, + { name = "loguru", specifier = ">=0.7.3" }, + { name = "mlx", specifier = ">=0.29.3" }, + { name = "mlx-lm", specifier = ">=0.28.3" }, + { name = "networkx", specifier = ">=3.5" }, + { name = "pathlib", specifier = ">=1.0.1" }, + { name = "protobuf", specifier = ">=6.32.0" }, + { name = "psutil", specifier = ">=7.0.0" }, + { name = "pydantic", specifier = ">=2.11.7" }, + { name = "rich", specifier = ">=14.1.0" }, + { name = "rustworkx", specifier = ">=0.17.1" }, + { name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0.43" }, + { name = "sqlmodel", specifier = ">=0.0.24" }, + { name = "textual", specifier = ">=5.3.0" }, + { name = "tiktoken", specifier = ">=0.12.0" }, + { name = "typeguard", specifier = ">=4.4.4" }, + { name = "types-aiofiles", specifier = ">=24.1.0.20250708" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "pytest", specifier = ">=8.4.0" }, + { name = "pytest-asyncio", specifier = ">=1.0.0" }, + { name = "pytest-env" }, + { name = "ruff", specifier = ">=0.11.13" }, +] + +[[package]] +name = "exo-pyo3-bindings" +version = "0.1.0" +source = { editable = "rust/exo_pyo3_bindings" } + +[package.dev-dependencies] +dev = [ + { name = "exo-pyo3-bindings", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pytest", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pytest-asyncio", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] + +[package.metadata] + +[package.metadata.requires-dev] +dev = [ + { name = "exo-pyo3-bindings", editable = "rust/exo_pyo3_bindings" }, + { name = "pytest", specifier = ">=8.4.0" }, + { name = "pytest-asyncio", specifier = ">=1.0.0" }, +] + +[[package]] +name = "fastapi" +version = "0.121.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "annotated-doc", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pydantic", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "starlette", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/8c/e3/77a2df0946703973b9905fd0cde6172c15e0781984320123b4f5079e7113/fastapi-0.121.0.tar.gz", hash = "sha256:06663356a0b1ee93e875bbf05a31fb22314f5bed455afaaad2b2dad7f26e98fa", size = 342412, upload-time = "2025-11-03T10:25:54.818Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/dd/2c/42277afc1ba1a18f8358561eee40785d27becab8f80a1f945c0a3051c6eb/fastapi-0.121.0-py3-none-any.whl", hash = "sha256:8bdf1b15a55f4e4b0d6201033da9109ea15632cb76cf156e7b8b4019f2172106", size = 109183, upload-time = "2025-11-03T10:25:53.27Z" }, +] + +[[package]] +name = "filelock" +version = "3.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/58/46/0028a82567109b5ef6e4d2a1f04a583fb513e6cf9527fcdd09afd817deeb/filelock-3.20.0.tar.gz", hash = "sha256:711e943b4ec6be42e1d4e6690b48dc175c822967466bb31c0c293f34334c13f4", size = 18922, upload-time = "2025-10-08T18:03:50.056Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/76/91/7216b27286936c16f5b4d0c530087e4a54eead683e6b0b73dd0c64844af6/filelock-3.20.0-py3-none-any.whl", hash = "sha256:339b4732ffda5cd79b13f4e2711a31b0365ce445d95d243bb996273d072546a2", size = 16054, upload-time = "2025-10-08T18:03:48.35Z" }, +] + +[[package]] +name = "frozenlist" +version = "1.8.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2d/f5/c831fac6cc817d26fd54c7eaccd04ef7e0288806943f7cc5bbf69f3ac1f0/frozenlist-1.8.0.tar.gz", hash = "sha256:3ede829ed8d842f6cd48fc7081d7a41001a56f1f38603f9d49bf3020d59a31ad", size = 45875, upload-time = "2025-10-06T05:38:17.865Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2d/40/0832c31a37d60f60ed79e9dfb5a92e1e2af4f40a16a29abcc7992af9edff/frozenlist-1.8.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8d92f1a84bb12d9e56f818b3a746f3efba93c1b63c8387a73dde655e1e42282a", size = 85717, upload-time = "2025-10-06T05:36:27.341Z" }, + { url = "https://files.pythonhosted.org/packages/30/ba/b0b3de23f40bc55a7057bd38434e25c34fa48e17f20ee273bbde5e0650f3/frozenlist-1.8.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:96153e77a591c8adc2ee805756c61f59fef4cf4073a9275ee86fe8cba41241f7", size = 49651, upload-time = "2025-10-06T05:36:28.855Z" }, + { url = "https://files.pythonhosted.org/packages/0c/ab/6e5080ee374f875296c4243c381bbdef97a9ac39c6e3ce1d5f7d42cb78d6/frozenlist-1.8.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:f21f00a91358803399890ab167098c131ec2ddd5f8f5fd5fe9c9f2c6fcd91e40", size = 49417, upload-time = "2025-10-06T05:36:29.877Z" }, + { url = "https://files.pythonhosted.org/packages/d5/4e/e4691508f9477ce67da2015d8c00acd751e6287739123113a9fca6f1604e/frozenlist-1.8.0-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:fb30f9626572a76dfe4293c7194a09fb1fe93ba94c7d4f720dfae3b646b45027", size = 234391, upload-time = "2025-10-06T05:36:31.301Z" }, + { url = "https://files.pythonhosted.org/packages/40/76/c202df58e3acdf12969a7895fd6f3bc016c642e6726aa63bd3025e0fc71c/frozenlist-1.8.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:eaa352d7047a31d87dafcacbabe89df0aa506abb5b1b85a2fb91bc3faa02d822", size = 233048, upload-time = "2025-10-06T05:36:32.531Z" }, + { url = "https://files.pythonhosted.org/packages/f9/c0/8746afb90f17b73ca5979c7a3958116e105ff796e718575175319b5bb4ce/frozenlist-1.8.0-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:03ae967b4e297f58f8c774c7eabcce57fe3c2434817d4385c50661845a058121", size = 226549, upload-time = "2025-10-06T05:36:33.706Z" }, + { url = "https://files.pythonhosted.org/packages/7e/eb/4c7eefc718ff72f9b6c4893291abaae5fbc0c82226a32dcd8ef4f7a5dbef/frozenlist-1.8.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f6292f1de555ffcc675941d65fffffb0a5bcd992905015f85d0592201793e0e5", size = 239833, upload-time = "2025-10-06T05:36:34.947Z" }, + { url = "https://files.pythonhosted.org/packages/c2/4e/e5c02187cf704224f8b21bee886f3d713ca379535f16893233b9d672ea71/frozenlist-1.8.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:29548f9b5b5e3460ce7378144c3010363d8035cea44bc0bf02d57f5a685e084e", size = 245363, upload-time = "2025-10-06T05:36:36.534Z" }, + { url = "https://files.pythonhosted.org/packages/1f/96/cb85ec608464472e82ad37a17f844889c36100eed57bea094518bf270692/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ec3cc8c5d4084591b4237c0a272cc4f50a5b03396a47d9caaf76f5d7b38a4f11", size = 229314, upload-time = "2025-10-06T05:36:38.582Z" }, + { url = "https://files.pythonhosted.org/packages/5d/6f/4ae69c550e4cee66b57887daeebe006fe985917c01d0fff9caab9883f6d0/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:517279f58009d0b1f2e7c1b130b377a349405da3f7621ed6bfae50b10adf20c1", size = 243365, upload-time = "2025-10-06T05:36:40.152Z" }, + { url = "https://files.pythonhosted.org/packages/7a/58/afd56de246cf11780a40a2c28dc7cbabbf06337cc8ddb1c780a2d97e88d8/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:db1e72ede2d0d7ccb213f218df6a078a9c09a7de257c2fe8fcef16d5925230b1", size = 237763, upload-time = "2025-10-06T05:36:41.355Z" }, + { url = "https://files.pythonhosted.org/packages/cb/36/cdfaf6ed42e2644740d4a10452d8e97fa1c062e2a8006e4b09f1b5fd7d63/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:b4dec9482a65c54a5044486847b8a66bf10c9cb4926d42927ec4e8fd5db7fed8", size = 240110, upload-time = "2025-10-06T05:36:42.716Z" }, + { url = "https://files.pythonhosted.org/packages/03/a8/9ea226fbefad669f11b52e864c55f0bd57d3c8d7eb07e9f2e9a0b39502e1/frozenlist-1.8.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:21900c48ae04d13d416f0e1e0c4d81f7931f73a9dfa0b7a8746fb2fe7dd970ed", size = 233717, upload-time = "2025-10-06T05:36:44.251Z" }, + { url = "https://files.pythonhosted.org/packages/d2/5c/3bbfaa920dfab09e76946a5d2833a7cbdf7b9b4a91c714666ac4855b88b4/frozenlist-1.8.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:e25ac20a2ef37e91c1b39938b591457666a0fa835c7783c3a8f33ea42870db94", size = 89235, upload-time = "2025-10-06T05:36:48.78Z" }, + { url = "https://files.pythonhosted.org/packages/d2/d6/f03961ef72166cec1687e84e8925838442b615bd0b8854b54923ce5b7b8a/frozenlist-1.8.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:07cdca25a91a4386d2e76ad992916a85038a9b97561bf7a3fd12d5d9ce31870c", size = 50742, upload-time = "2025-10-06T05:36:49.837Z" }, + { url = "https://files.pythonhosted.org/packages/1e/bb/a6d12b7ba4c3337667d0e421f7181c82dda448ce4e7ad7ecd249a16fa806/frozenlist-1.8.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:4e0c11f2cc6717e0a741f84a527c52616140741cd812a50422f83dc31749fb52", size = 51725, upload-time = "2025-10-06T05:36:50.851Z" }, + { url = "https://files.pythonhosted.org/packages/bc/71/d1fed0ffe2c2ccd70b43714c6cab0f4188f09f8a67a7914a6b46ee30f274/frozenlist-1.8.0-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b3210649ee28062ea6099cfda39e147fa1bc039583c8ee4481cb7811e2448c51", size = 284533, upload-time = "2025-10-06T05:36:51.898Z" }, + { url = "https://files.pythonhosted.org/packages/c9/1f/fb1685a7b009d89f9bf78a42d94461bc06581f6e718c39344754a5d9bada/frozenlist-1.8.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:581ef5194c48035a7de2aefc72ac6539823bb71508189e5de01d60c9dcd5fa65", size = 292506, upload-time = "2025-10-06T05:36:53.101Z" }, + { url = "https://files.pythonhosted.org/packages/e6/3b/b991fe1612703f7e0d05c0cf734c1b77aaf7c7d321df4572e8d36e7048c8/frozenlist-1.8.0-cp313-cp313t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3ef2d026f16a2b1866e1d86fc4e1291e1ed8a387b2c333809419a2f8b3a77b82", size = 274161, upload-time = "2025-10-06T05:36:54.309Z" }, + { url = "https://files.pythonhosted.org/packages/ca/ec/c5c618767bcdf66e88945ec0157d7f6c4a1322f1473392319b7a2501ded7/frozenlist-1.8.0-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5500ef82073f599ac84d888e3a8c1f77ac831183244bfd7f11eaa0289fb30714", size = 294676, upload-time = "2025-10-06T05:36:55.566Z" }, + { url = "https://files.pythonhosted.org/packages/7c/ce/3934758637d8f8a88d11f0585d6495ef54b2044ed6ec84492a91fa3b27aa/frozenlist-1.8.0-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:50066c3997d0091c411a66e710f4e11752251e6d2d73d70d8d5d4c76442a199d", size = 300638, upload-time = "2025-10-06T05:36:56.758Z" }, + { url = "https://files.pythonhosted.org/packages/fc/4f/a7e4d0d467298f42de4b41cbc7ddaf19d3cfeabaf9ff97c20c6c7ee409f9/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:5c1c8e78426e59b3f8005e9b19f6ff46e5845895adbde20ece9218319eca6506", size = 283067, upload-time = "2025-10-06T05:36:57.965Z" }, + { url = "https://files.pythonhosted.org/packages/dc/48/c7b163063d55a83772b268e6d1affb960771b0e203b632cfe09522d67ea5/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:eefdba20de0d938cec6a89bd4d70f346a03108a19b9df4248d3cf0d88f1b0f51", size = 292101, upload-time = "2025-10-06T05:36:59.237Z" }, + { url = "https://files.pythonhosted.org/packages/9f/d0/2366d3c4ecdc2fd391e0afa6e11500bfba0ea772764d631bbf82f0136c9d/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:cf253e0e1c3ceb4aaff6df637ce033ff6535fb8c70a764a8f46aafd3d6ab798e", size = 289901, upload-time = "2025-10-06T05:37:00.811Z" }, + { url = "https://files.pythonhosted.org/packages/b8/94/daff920e82c1b70e3618a2ac39fbc01ae3e2ff6124e80739ce5d71c9b920/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:032efa2674356903cd0261c4317a561a6850f3ac864a63fc1583147fb05a79b0", size = 289395, upload-time = "2025-10-06T05:37:02.115Z" }, + { url = "https://files.pythonhosted.org/packages/e3/20/bba307ab4235a09fdcd3cc5508dbabd17c4634a1af4b96e0f69bfe551ebd/frozenlist-1.8.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:6da155091429aeba16851ecb10a9104a108bcd32f6c1642867eadaee401c1c41", size = 283659, upload-time = "2025-10-06T05:37:03.711Z" }, + { url = "https://files.pythonhosted.org/packages/f1/c8/85da824b7e7b9b6e7f7705b2ecaf9591ba6f79c1177f324c2735e41d36a2/frozenlist-1.8.0-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:cee686f1f4cadeb2136007ddedd0aaf928ab95216e7691c63e50a8ec066336d0", size = 86127, upload-time = "2025-10-06T05:37:08.438Z" }, + { url = "https://files.pythonhosted.org/packages/8e/e8/a1185e236ec66c20afd72399522f142c3724c785789255202d27ae992818/frozenlist-1.8.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:119fb2a1bd47307e899c2fac7f28e85b9a543864df47aa7ec9d3c1b4545f096f", size = 49698, upload-time = "2025-10-06T05:37:09.48Z" }, + { url = "https://files.pythonhosted.org/packages/a1/93/72b1736d68f03fda5fdf0f2180fb6caaae3894f1b854d006ac61ecc727ee/frozenlist-1.8.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4970ece02dbc8c3a92fcc5228e36a3e933a01a999f7094ff7c23fbd2beeaa67c", size = 49749, upload-time = "2025-10-06T05:37:10.569Z" }, + { url = "https://files.pythonhosted.org/packages/a7/b2/fabede9fafd976b991e9f1b9c8c873ed86f202889b864756f240ce6dd855/frozenlist-1.8.0-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:cba69cb73723c3f329622e34bdbf5ce1f80c21c290ff04256cff1cd3c2036ed2", size = 231298, upload-time = "2025-10-06T05:37:11.993Z" }, + { url = "https://files.pythonhosted.org/packages/3a/3b/d9b1e0b0eed36e70477ffb8360c49c85c8ca8ef9700a4e6711f39a6e8b45/frozenlist-1.8.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:778a11b15673f6f1df23d9586f83c4846c471a8af693a22e066508b77d201ec8", size = 232015, upload-time = "2025-10-06T05:37:13.194Z" }, + { url = "https://files.pythonhosted.org/packages/dc/94/be719d2766c1138148564a3960fc2c06eb688da592bdc25adcf856101be7/frozenlist-1.8.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:0325024fe97f94c41c08872db482cf8ac4800d80e79222c6b0b7b162d5b13686", size = 225038, upload-time = "2025-10-06T05:37:14.577Z" }, + { url = "https://files.pythonhosted.org/packages/e4/09/6712b6c5465f083f52f50cf74167b92d4ea2f50e46a9eea0523d658454ae/frozenlist-1.8.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:97260ff46b207a82a7567b581ab4190bd4dfa09f4db8a8b49d1a958f6aa4940e", size = 240130, upload-time = "2025-10-06T05:37:15.781Z" }, + { url = "https://files.pythonhosted.org/packages/f8/d4/cd065cdcf21550b54f3ce6a22e143ac9e4836ca42a0de1022da8498eac89/frozenlist-1.8.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:54b2077180eb7f83dd52c40b2750d0a9f175e06a42e3213ce047219de902717a", size = 242845, upload-time = "2025-10-06T05:37:17.037Z" }, + { url = "https://files.pythonhosted.org/packages/62/c3/f57a5c8c70cd1ead3d5d5f776f89d33110b1addae0ab010ad774d9a44fb9/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:2f05983daecab868a31e1da44462873306d3cbfd76d1f0b5b69c473d21dbb128", size = 229131, upload-time = "2025-10-06T05:37:18.221Z" }, + { url = "https://files.pythonhosted.org/packages/6c/52/232476fe9cb64f0742f3fde2b7d26c1dac18b6d62071c74d4ded55e0ef94/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:33f48f51a446114bc5d251fb2954ab0164d5be02ad3382abcbfe07e2531d650f", size = 240542, upload-time = "2025-10-06T05:37:19.771Z" }, + { url = "https://files.pythonhosted.org/packages/5f/85/07bf3f5d0fb5414aee5f47d33c6f5c77bfe49aac680bfece33d4fdf6a246/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:154e55ec0655291b5dd1b8731c637ecdb50975a2ae70c606d100750a540082f7", size = 237308, upload-time = "2025-10-06T05:37:20.969Z" }, + { url = "https://files.pythonhosted.org/packages/11/99/ae3a33d5befd41ac0ca2cc7fd3aa707c9c324de2e89db0e0f45db9a64c26/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:4314debad13beb564b708b4a496020e5306c7333fa9a3ab90374169a20ffab30", size = 238210, upload-time = "2025-10-06T05:37:22.252Z" }, + { url = "https://files.pythonhosted.org/packages/b2/60/b1d2da22f4970e7a155f0adde9b1435712ece01b3cd45ba63702aea33938/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:073f8bf8becba60aa931eb3bc420b217bb7d5b8f4750e6f8b3be7f3da85d38b7", size = 231972, upload-time = "2025-10-06T05:37:23.5Z" }, + { url = "https://files.pythonhosted.org/packages/c0/c7/43200656ecc4e02d3f8bc248df68256cd9572b3f0017f0a0c4e93440ae23/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d3bb933317c52d7ea5004a1c442eef86f426886fba134ef8cf4226ea6ee1821d", size = 89238, upload-time = "2025-10-06T05:37:29.373Z" }, + { url = "https://files.pythonhosted.org/packages/d1/29/55c5f0689b9c0fb765055629f472c0de484dcaf0acee2f7707266ae3583c/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:8009897cdef112072f93a0efdce29cd819e717fd2f649ee3016efd3cd885a7ed", size = 50738, upload-time = "2025-10-06T05:37:30.792Z" }, + { url = "https://files.pythonhosted.org/packages/ba/7d/b7282a445956506fa11da8c2db7d276adcbf2b17d8bb8407a47685263f90/frozenlist-1.8.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2c5dcbbc55383e5883246d11fd179782a9d07a986c40f49abe89ddf865913930", size = 51739, upload-time = "2025-10-06T05:37:32.127Z" }, + { url = "https://files.pythonhosted.org/packages/62/1c/3d8622e60d0b767a5510d1d3cf21065b9db874696a51ea6d7a43180a259c/frozenlist-1.8.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:39ecbc32f1390387d2aa4f5a995e465e9e2f79ba3adcac92d68e3e0afae6657c", size = 284186, upload-time = "2025-10-06T05:37:33.21Z" }, + { url = "https://files.pythonhosted.org/packages/2d/14/aa36d5f85a89679a85a1d44cd7a6657e0b1c75f61e7cad987b203d2daca8/frozenlist-1.8.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92db2bf818d5cc8d9c1f1fc56b897662e24ea5adb36ad1f1d82875bd64e03c24", size = 292196, upload-time = "2025-10-06T05:37:36.107Z" }, + { url = "https://files.pythonhosted.org/packages/05/23/6bde59eb55abd407d34f77d39a5126fb7b4f109a3f611d3929f14b700c66/frozenlist-1.8.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2dc43a022e555de94c3b68a4ef0b11c4f747d12c024a520c7101709a2144fb37", size = 273830, upload-time = "2025-10-06T05:37:37.663Z" }, + { url = "https://files.pythonhosted.org/packages/d2/3f/22cff331bfad7a8afa616289000ba793347fcd7bc275f3b28ecea2a27909/frozenlist-1.8.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb89a7f2de3602cfed448095bab3f178399646ab7c61454315089787df07733a", size = 294289, upload-time = "2025-10-06T05:37:39.261Z" }, + { url = "https://files.pythonhosted.org/packages/a4/89/5b057c799de4838b6c69aa82b79705f2027615e01be996d2486a69ca99c4/frozenlist-1.8.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:33139dc858c580ea50e7e60a1b0ea003efa1fd42e6ec7fdbad78fff65fad2fd2", size = 300318, upload-time = "2025-10-06T05:37:43.213Z" }, + { url = "https://files.pythonhosted.org/packages/30/de/2c22ab3eb2a8af6d69dc799e48455813bab3690c760de58e1bf43b36da3e/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:168c0969a329b416119507ba30b9ea13688fafffac1b7822802537569a1cb0ef", size = 282814, upload-time = "2025-10-06T05:37:45.337Z" }, + { url = "https://files.pythonhosted.org/packages/59/f7/970141a6a8dbd7f556d94977858cfb36fa9b66e0892c6dd780d2219d8cd8/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:28bd570e8e189d7f7b001966435f9dac6718324b5be2990ac496cf1ea9ddb7fe", size = 291762, upload-time = "2025-10-06T05:37:46.657Z" }, + { url = "https://files.pythonhosted.org/packages/c1/15/ca1adae83a719f82df9116d66f5bb28bb95557b3951903d39135620ef157/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b2a095d45c5d46e5e79ba1e5b9cb787f541a8dee0433836cea4b96a2c439dcd8", size = 289470, upload-time = "2025-10-06T05:37:47.946Z" }, + { url = "https://files.pythonhosted.org/packages/ac/83/dca6dc53bf657d371fbc88ddeb21b79891e747189c5de990b9dfff2ccba1/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:eab8145831a0d56ec9c4139b6c3e594c7a83c2c8be25d5bcf2d86136a532287a", size = 289042, upload-time = "2025-10-06T05:37:49.499Z" }, + { url = "https://files.pythonhosted.org/packages/96/52/abddd34ca99be142f354398700536c5bd315880ed0a213812bc491cff5e4/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:974b28cf63cc99dfb2188d8d222bc6843656188164848c4f679e63dae4b0708e", size = 283148, upload-time = "2025-10-06T05:37:50.745Z" }, + { url = "https://files.pythonhosted.org/packages/9a/9a/e35b4a917281c0b8419d4207f4334c8e8c5dbf4f3f5f9ada73958d937dcc/frozenlist-1.8.0-py3-none-any.whl", hash = "sha256:0c18a16eab41e82c295618a77502e17b195883241c563b00f0aa5106fc4eaa0d", size = 13409, upload-time = "2025-10-06T05:38:16.721Z" }, +] + +[[package]] +name = "fsspec" +version = "2025.10.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/24/7f/2747c0d332b9acfa75dc84447a066fdf812b5a6b8d30472b74d309bfe8cb/fsspec-2025.10.0.tar.gz", hash = "sha256:b6789427626f068f9a83ca4e8a3cc050850b6c0f71f99ddb4f542b8266a26a59", size = 309285, upload-time = "2025-10-30T14:58:44.036Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/eb/02/a6b21098b1d5d6249b7c5ab69dde30108a71e4e819d4a9778f1de1d5b70d/fsspec-2025.10.0-py3-none-any.whl", hash = "sha256:7c7712353ae7d875407f97715f0e1ffcc21e33d5b24556cb1e090ae9409ec61d", size = 200966, upload-time = "2025-10-30T14:58:42.53Z" }, +] + +[[package]] +name = "greenlet" +version = "3.2.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/03/b8/704d753a5a45507a7aab61f18db9509302ed3d0a27ac7e0359ec2905b1a6/greenlet-3.2.4.tar.gz", hash = "sha256:0dca0d95ff849f9a364385f36ab49f50065d76964944638be9691e1832e9f86d", size = 188260, upload-time = "2025-08-07T13:24:33.51Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/e8/58c7f85958bda41dafea50497cbd59738c5c43dbbea5ee83d651234398f4/greenlet-3.2.4-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:1a921e542453fe531144e91e1feedf12e07351b1cf6c9e8a3325ea600a715a31", size = 272814, upload-time = "2025-08-07T13:15:50.011Z" }, + { url = "https://files.pythonhosted.org/packages/62/dd/b9f59862e9e257a16e4e610480cfffd29e3fae018a68c2332090b53aac3d/greenlet-3.2.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cd3c8e693bff0fff6ba55f140bf390fa92c994083f838fece0f63be121334945", size = 641073, upload-time = "2025-08-07T13:42:57.23Z" }, + { url = "https://files.pythonhosted.org/packages/f7/0b/bc13f787394920b23073ca3b6c4a7a21396301ed75a655bcb47196b50e6e/greenlet-3.2.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:710638eb93b1fa52823aa91bf75326f9ecdfd5e0466f00789246a5280f4ba0fc", size = 655191, upload-time = "2025-08-07T13:45:29.752Z" }, + { url = "https://files.pythonhosted.org/packages/f2/d6/6adde57d1345a8d0f14d31e4ab9c23cfe8e2cd39c3baf7674b4b0338d266/greenlet-3.2.4-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:c5111ccdc9c88f423426df3fd1811bfc40ed66264d35aa373420a34377efc98a", size = 649516, upload-time = "2025-08-07T13:53:16.314Z" }, + { url = "https://files.pythonhosted.org/packages/7f/3b/3a3328a788d4a473889a2d403199932be55b1b0060f4ddd96ee7cdfcad10/greenlet-3.2.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d76383238584e9711e20ebe14db6c88ddcedc1829a9ad31a584389463b5aa504", size = 652169, upload-time = "2025-08-07T13:18:32.861Z" }, + { url = "https://files.pythonhosted.org/packages/ee/43/3cecdc0349359e1a527cbf2e3e28e5f8f06d3343aaf82ca13437a9aa290f/greenlet-3.2.4-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:23768528f2911bcd7e475210822ffb5254ed10d71f4028387e5a99b4c6699671", size = 610497, upload-time = "2025-08-07T13:18:31.636Z" }, + { url = "https://files.pythonhosted.org/packages/b8/19/06b6cf5d604e2c382a6f31cafafd6f33d5dea706f4db7bdab184bad2b21d/greenlet-3.2.4-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:00fadb3fedccc447f517ee0d3fd8fe49eae949e1cd0f6a611818f4f6fb7dc83b", size = 1121662, upload-time = "2025-08-07T13:42:41.117Z" }, + { url = "https://files.pythonhosted.org/packages/a2/15/0d5e4e1a66fab130d98168fe984c509249c833c1a3c16806b90f253ce7b9/greenlet-3.2.4-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:d25c5091190f2dc0eaa3f950252122edbbadbb682aa7b1ef2f8af0f8c0afefae", size = 1149210, upload-time = "2025-08-07T13:18:24.072Z" }, + { url = "https://files.pythonhosted.org/packages/1c/53/f9c440463b3057485b8594d7a638bed53ba531165ef0ca0e6c364b5cc807/greenlet-3.2.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6e343822feb58ac4d0a1211bd9399de2b3a04963ddeec21530fc426cc121f19b", size = 1564759, upload-time = "2025-11-04T12:42:19.395Z" }, + { url = "https://files.pythonhosted.org/packages/47/e4/3bb4240abdd0a8d23f4f88adec746a3099f0d86bfedb623f063b2e3b4df0/greenlet-3.2.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ca7f6f1f2649b89ce02f6f229d7c19f680a6238af656f61e0115b24857917929", size = 1634288, upload-time = "2025-11-04T12:42:21.174Z" }, + { url = "https://files.pythonhosted.org/packages/22/5c/85273fd7cc388285632b0498dbbab97596e04b154933dfe0f3e68156c68c/greenlet-3.2.4-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:49a30d5fda2507ae77be16479bdb62a660fa51b1eb4928b524975b3bde77b3c0", size = 273586, upload-time = "2025-08-07T13:16:08.004Z" }, + { url = "https://files.pythonhosted.org/packages/d1/75/10aeeaa3da9332c2e761e4c50d4c3556c21113ee3f0afa2cf5769946f7a3/greenlet-3.2.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:299fd615cd8fc86267b47597123e3f43ad79c9d8a22bebdce535e53550763e2f", size = 686346, upload-time = "2025-08-07T13:42:59.944Z" }, + { url = "https://files.pythonhosted.org/packages/c0/aa/687d6b12ffb505a4447567d1f3abea23bd20e73a5bed63871178e0831b7a/greenlet-3.2.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:c17b6b34111ea72fc5a4e4beec9711d2226285f0386ea83477cbb97c30a3f3a5", size = 699218, upload-time = "2025-08-07T13:45:30.969Z" }, + { url = "https://files.pythonhosted.org/packages/dc/8b/29aae55436521f1d6f8ff4e12fb676f3400de7fcf27fccd1d4d17fd8fecd/greenlet-3.2.4-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:b4a1870c51720687af7fa3e7cda6d08d801dae660f75a76f3845b642b4da6ee1", size = 694659, upload-time = "2025-08-07T13:53:17.759Z" }, + { url = "https://files.pythonhosted.org/packages/92/2e/ea25914b1ebfde93b6fc4ff46d6864564fba59024e928bdc7de475affc25/greenlet-3.2.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:061dc4cf2c34852b052a8620d40f36324554bc192be474b9e9770e8c042fd735", size = 695355, upload-time = "2025-08-07T13:18:34.517Z" }, + { url = "https://files.pythonhosted.org/packages/72/60/fc56c62046ec17f6b0d3060564562c64c862948c9d4bc8aa807cf5bd74f4/greenlet-3.2.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:44358b9bf66c8576a9f57a590d5f5d6e72fa4228b763d0e43fee6d3b06d3a337", size = 657512, upload-time = "2025-08-07T13:18:33.969Z" }, + { url = "https://files.pythonhosted.org/packages/23/6e/74407aed965a4ab6ddd93a7ded3180b730d281c77b765788419484cdfeef/greenlet-3.2.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:2917bdf657f5859fbf3386b12d68ede4cf1f04c90c3a6bc1f013dd68a22e2269", size = 1612508, upload-time = "2025-11-04T12:42:23.427Z" }, + { url = "https://files.pythonhosted.org/packages/0d/da/343cd760ab2f92bac1845ca07ee3faea9fe52bee65f7bcb19f16ad7de08b/greenlet-3.2.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:015d48959d4add5d6c9f6c5210ee3803a830dce46356e3bc326d6776bde54681", size = 1680760, upload-time = "2025-11-04T12:42:25.341Z" }, +] + +[[package]] +name = "h11" +version = "0.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" }, +] + +[[package]] +name = "h2" +version = "4.3.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "hpack", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "hyperframe", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1d/17/afa56379f94ad0fe8defd37d6eb3f89a25404ffc71d4d848893d270325fc/h2-4.3.0.tar.gz", hash = "sha256:6c59efe4323fa18b47a632221a1888bd7fde6249819beda254aeca909f221bf1", size = 2152026, upload-time = "2025-08-23T18:12:19.778Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/69/b2/119f6e6dcbd96f9069ce9a2665e0146588dc9f88f29549711853645e736a/h2-4.3.0-py3-none-any.whl", hash = "sha256:c438f029a25f7945c69e0ccf0fb951dc3f73a5f6412981daee861431b70e2bdd", size = 61779, upload-time = "2025-08-23T18:12:17.779Z" }, +] + +[[package]] +name = "hf-xet" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5e/6e/0f11bacf08a67f7fb5ee09740f2ca54163863b07b70d579356e9222ce5d8/hf_xet-1.2.0.tar.gz", hash = "sha256:a8c27070ca547293b6890c4bf389f713f80e8c478631432962bb7f4bc0bd7d7f", size = 506020, upload-time = "2025-10-24T19:04:32.129Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9e/a5/85ef910a0aa034a2abcfadc360ab5ac6f6bc4e9112349bd40ca97551cff0/hf_xet-1.2.0-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:ceeefcd1b7aed4956ae8499e2199607765fbd1c60510752003b6cc0b8413b649", size = 2861870, upload-time = "2025-10-24T19:04:11.422Z" }, + { url = "https://files.pythonhosted.org/packages/ea/40/e2e0a7eb9a51fe8828ba2d47fe22a7e74914ea8a0db68a18c3aa7449c767/hf_xet-1.2.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:b70218dd548e9840224df5638fdc94bd033552963cfa97f9170829381179c813", size = 2717584, upload-time = "2025-10-24T19:04:09.586Z" }, + { url = "https://files.pythonhosted.org/packages/a5/7d/daf7f8bc4594fdd59a8a596f9e3886133fdc68e675292218a5e4c1b7e834/hf_xet-1.2.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7d40b18769bb9a8bc82a9ede575ce1a44c75eb80e7375a01d76259089529b5dc", size = 3315004, upload-time = "2025-10-24T19:04:00.314Z" }, + { url = "https://files.pythonhosted.org/packages/b1/ba/45ea2f605fbf6d81c8b21e4d970b168b18a53515923010c312c06cd83164/hf_xet-1.2.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:cd3a6027d59cfb60177c12d6424e31f4b5ff13d8e3a1247b3a584bf8977e6df5", size = 3222636, upload-time = "2025-10-24T19:03:58.111Z" }, + { url = "https://files.pythonhosted.org/packages/4a/1d/04513e3cab8f29ab8c109d309ddd21a2705afab9d52f2ba1151e0c14f086/hf_xet-1.2.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:6de1fc44f58f6dd937956c8d304d8c2dea264c80680bcfa61ca4a15e7b76780f", size = 3408448, upload-time = "2025-10-24T19:04:20.951Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7c/60a2756d7feec7387db3a1176c632357632fbe7849fce576c5559d4520c7/hf_xet-1.2.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:f182f264ed2acd566c514e45da9f2119110e48a87a327ca271027904c70c5832", size = 3503401, upload-time = "2025-10-24T19:04:22.549Z" }, + { url = "https://files.pythonhosted.org/packages/e2/51/f7e2caae42f80af886db414d4e9885fac959330509089f97cccb339c6b87/hf_xet-1.2.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:10bfab528b968c70e062607f663e21e34e2bba349e8038db546646875495179e", size = 2861861, upload-time = "2025-10-24T19:04:19.01Z" }, + { url = "https://files.pythonhosted.org/packages/6e/1d/a641a88b69994f9371bd347f1dd35e5d1e2e2460a2e350c8d5165fc62005/hf_xet-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2a212e842647b02eb6a911187dc878e79c4aa0aa397e88dd3b26761676e8c1f8", size = 2717699, upload-time = "2025-10-24T19:04:17.306Z" }, + { url = "https://files.pythonhosted.org/packages/df/e0/e5e9bba7d15f0318955f7ec3f4af13f92e773fbb368c0b8008a5acbcb12f/hf_xet-1.2.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:30e06daccb3a7d4c065f34fc26c14c74f4653069bb2b194e7f18f17cbe9939c0", size = 3314885, upload-time = "2025-10-24T19:04:07.642Z" }, + { url = "https://files.pythonhosted.org/packages/21/90/b7fe5ff6f2b7b8cbdf1bd56145f863c90a5807d9758a549bf3d916aa4dec/hf_xet-1.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:29c8fc913a529ec0a91867ce3d119ac1aac966e098cf49501800c870328cc090", size = 3221550, upload-time = "2025-10-24T19:04:05.55Z" }, + { url = "https://files.pythonhosted.org/packages/6f/cb/73f276f0a7ce46cc6a6ec7d6c7d61cbfe5f2e107123d9bbd0193c355f106/hf_xet-1.2.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e159cbfcfbb29f920db2c09ed8b660eb894640d284f102ada929b6e3dc410a", size = 3408010, upload-time = "2025-10-24T19:04:28.598Z" }, + { url = "https://files.pythonhosted.org/packages/b8/1e/d642a12caa78171f4be64f7cd9c40e3ca5279d055d0873188a58c0f5fbb9/hf_xet-1.2.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9c91d5ae931510107f148874e9e2de8a16052b6f1b3ca3c1b12f15ccb491390f", size = 3503264, upload-time = "2025-10-24T19:04:30.397Z" }, + { url = "https://files.pythonhosted.org/packages/96/2d/22338486473df5923a9ab7107d375dbef9173c338ebef5098ef593d2b560/hf_xet-1.2.0-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:46740d4ac024a7ca9b22bebf77460ff43332868b661186a8e46c227fdae01848", size = 2866099, upload-time = "2025-10-24T19:04:15.366Z" }, + { url = "https://files.pythonhosted.org/packages/7f/8c/c5becfa53234299bc2210ba314eaaae36c2875e0045809b82e40a9544f0c/hf_xet-1.2.0-cp37-abi3-macosx_11_0_arm64.whl", hash = "sha256:27df617a076420d8845bea087f59303da8be17ed7ec0cd7ee3b9b9f579dff0e4", size = 2722178, upload-time = "2025-10-24T19:04:13.695Z" }, + { url = "https://files.pythonhosted.org/packages/9a/92/cf3ab0b652b082e66876d08da57fcc6fa2f0e6c70dfbbafbd470bb73eb47/hf_xet-1.2.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3651fd5bfe0281951b988c0facbe726aa5e347b103a675f49a3fa8144c7968fd", size = 3320214, upload-time = "2025-10-24T19:04:03.596Z" }, + { url = "https://files.pythonhosted.org/packages/46/92/3f7ec4a1b6a65bf45b059b6d4a5d38988f63e193056de2f420137e3c3244/hf_xet-1.2.0-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:d06fa97c8562fb3ee7a378dd9b51e343bc5bc8190254202c9771029152f5e08c", size = 3229054, upload-time = "2025-10-24T19:04:01.949Z" }, + { url = "https://files.pythonhosted.org/packages/0b/dd/7ac658d54b9fb7999a0ccb07ad863b413cbaf5cf172f48ebcd9497ec7263/hf_xet-1.2.0-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:4c1428c9ae73ec0939410ec73023c4f842927f39db09b063b9482dac5a3bb737", size = 3413812, upload-time = "2025-10-24T19:04:24.585Z" }, + { url = "https://files.pythonhosted.org/packages/92/68/89ac4e5b12a9ff6286a12174c8538a5930e2ed662091dd2572bbe0a18c8a/hf_xet-1.2.0-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a55558084c16b09b5ed32ab9ed38421e2d87cf3f1f89815764d1177081b99865", size = 3508920, upload-time = "2025-10-24T19:04:26.927Z" }, +] + +[[package]] +name = "hpack" +version = "4.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2c/48/71de9ed269fdae9c8057e5a4c0aa7402e8bb16f2c6e90b3aa53327b113f8/hpack-4.1.0.tar.gz", hash = "sha256:ec5eca154f7056aa06f196a557655c5b009b382873ac8d1e66e79e87535f1dca", size = 51276, upload-time = "2025-01-22T21:44:58.347Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/07/c6/80c95b1b2b94682a72cbdbfb85b81ae2daffa4291fbfa1b1464502ede10d/hpack-4.1.0-py3-none-any.whl", hash = "sha256:157ac792668d995c657d93111f46b4535ed114f0c9c8d672271bbec7eae1b496", size = 34357, upload-time = "2025-01-22T21:44:56.92Z" }, +] + +[[package]] +name = "huggingface-hub" +version = "0.36.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "filelock", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "fsspec", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "hf-xet", marker = "(platform_machine == 'aarch64' and sys_platform == 'darwin') or (platform_machine == 'amd64' and sys_platform == 'darwin') or (platform_machine == 'arm64' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'amd64' and sys_platform == 'linux') or (platform_machine == 'arm64' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux')" }, + { name = "packaging", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pyyaml", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "requests", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "tqdm", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/98/63/4910c5fa9128fdadf6a9c5ac138e8b1b6cee4ca44bf7915bbfbce4e355ee/huggingface_hub-0.36.0.tar.gz", hash = "sha256:47b3f0e2539c39bf5cde015d63b72ec49baff67b6931c3d97f3f84532e2b8d25", size = 463358, upload-time = "2025-10-23T12:12:01.413Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/bd/1a875e0d592d447cbc02805fd3fe0f497714d6a2583f59d14fa9ebad96eb/huggingface_hub-0.36.0-py3-none-any.whl", hash = "sha256:7bcc9ad17d5b3f07b57c78e79d527102d08313caa278a641993acddcb894548d", size = 566094, upload-time = "2025-10-23T12:11:59.557Z" }, +] + +[[package]] +name = "hypercorn" +version = "0.18.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "h11", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "h2", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "priority", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "wsproto", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/44/01/39f41a014b83dd5c795217362f2ca9071cf243e6a75bdcd6cd5b944658cc/hypercorn-0.18.0.tar.gz", hash = "sha256:d63267548939c46b0247dc8e5b45a9947590e35e64ee73a23c074aa3cf88e9da", size = 68420, upload-time = "2025-11-08T13:54:04.78Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/93/35/850277d1b17b206bd10874c8a9a3f52e059452fb49bb0d22cbb908f6038b/hypercorn-0.18.0-py3-none-any.whl", hash = "sha256:225e268f2c1c2f28f6d8f6db8f40cb8c992963610c5725e13ccfcddccb24b1cd", size = 61640, upload-time = "2025-11-08T13:54:03.202Z" }, +] + +[[package]] +name = "hyperframe" +version = "6.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/02/e7/94f8232d4a74cc99514c13a9f995811485a6903d48e5d952771ef6322e30/hyperframe-6.1.0.tar.gz", hash = "sha256:f630908a00854a7adeabd6382b43923a4c4cd4b821fcb527e6ab9e15382a3b08", size = 26566, upload-time = "2025-01-22T21:41:49.302Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/48/30/47d0bf6072f7252e6521f3447ccfa40b421b6824517f82854703d0f5a98b/hyperframe-6.1.0-py3-none-any.whl", hash = "sha256:b03380493a519fce58ea5af42e4a42317bf9bd425596f7a0835ffce80f1a42e5", size = 13007, upload-time = "2025-01-22T21:41:47.295Z" }, +] + +[[package]] +name = "idna" +version = "3.11" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582, upload-time = "2025-10-12T14:55:20.501Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008, upload-time = "2025-10-12T14:55:18.883Z" }, +] + +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, +] + +[[package]] +name = "linkify-it-py" +version = "2.0.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "uc-micro-py", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/2a/ae/bb56c6828e4797ba5a4821eec7c43b8bf40f69cda4d4f5f8c8a2810ec96a/linkify-it-py-2.0.3.tar.gz", hash = "sha256:68cda27e162e9215c17d786649d1da0021a451bdc436ef9e0fa0ba5234b9b048", size = 27946, upload-time = "2024-02-04T14:48:04.179Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/1e/b832de447dee8b582cac175871d2f6c3d5077cc56d5575cadba1fd1cccfa/linkify_it_py-2.0.3-py3-none-any.whl", hash = "sha256:6bcbc417b0ac14323382aef5c5192c0075bf8a9d6b41820a2b66371eac6b6d79", size = 19820, upload-time = "2024-02-04T14:48:02.496Z" }, +] + +[[package]] +name = "loguru" +version = "0.7.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559, upload-time = "2024-12-06T11:20:56.608Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595, upload-time = "2024-12-06T11:20:54.538Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "4.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mdurl", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/5b/f5/4ec618ed16cc4f8fb3b701563655a69816155e79e24a17b651541804721d/markdown_it_py-4.0.0.tar.gz", hash = "sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3", size = 73070, upload-time = "2025-08-11T12:57:52.854Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/94/54/e7d793b573f298e1c9013b8c4dade17d481164aa517d1d7148619c2cedbf/markdown_it_py-4.0.0-py3-none-any.whl", hash = "sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147", size = 87321, upload-time = "2025-08-11T12:57:51.923Z" }, +] + +[package.optional-dependencies] +linkify = [ + { name = "linkify-it-py", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b2/fd/a756d36c0bfba5f6e39a1cdbdbfdd448dc02692467d83816dff4592a1ebc/mdit_py_plugins-0.5.0.tar.gz", hash = "sha256:f4918cb50119f50446560513a8e311d574ff6aaed72606ddae6d35716fe809c6", size = 44655, upload-time = "2025-08-11T07:25:49.083Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fb/86/dd6e5db36df29e76c7a7699123569a4a18c1623ce68d826ed96c62643cae/mdit_py_plugins-0.5.0-py3-none-any.whl", hash = "sha256:07a08422fc1936a5d26d146759e9155ea466e842f5ab2f7d2266dd084c8dab1f", size = 57205, upload-time = "2025-08-11T07:25:47.597Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + +[[package]] +name = "mlx" +version = "0.29.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mlx-metal", marker = "sys_platform == 'darwin'" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/fe/a2/078152b45aa8a23949a1b09601d0044f8bb4ab85e909e4475a440c21aaea/mlx-0.29.3-cp313-cp313-macosx_13_0_arm64.whl", hash = "sha256:d59eccf6a1e1e131becc5a3910504507862da3a4e9b7bd9e73a625515d767844", size = 549585, upload-time = "2025-10-17T19:17:01.872Z" }, + { url = "https://files.pythonhosted.org/packages/ae/bb/869eaac4efaae033c13db5fddd6a8907b5d667d135a35a2e482b1af402ee/mlx-0.29.3-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:6642aa0a6dc2242c024fb8274d00631a7e7ffbdcef26148afd299b877c1e6a4a", size = 549586, upload-time = "2025-10-17T19:16:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/ad/76/196c248c2b2a471f795356564ad1d7dc40284160c8b66370ffadfd991fa1/mlx-0.29.3-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:ec0aef311fab10cb5f2c274afa6edf6c482636096a5f7886aba43676454aa462", size = 549586, upload-time = "2025-10-17T19:16:39.912Z" }, + { url = "https://files.pythonhosted.org/packages/f2/90/d481dd70b351e28718cfc9a0deb229a75e140abda3ed59284cf635f93f12/mlx-0.29.3-cp313-cp313-manylinux_2_35_x86_64.whl", hash = "sha256:e217a99ece66832a2e631131df32e9feb047276b68ac59ca0ad63735842f6dd0", size = 649781, upload-time = "2025-10-17T19:21:26.075Z" }, +] + +[[package]] +name = "mlx-lm" +version = "0.28.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "mlx", marker = "sys_platform == 'darwin'" }, + { name = "numpy", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "protobuf", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pyyaml", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "transformers", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/51/f6/15e002d52c28d8c544ec3aaf9053677468333e6ef0e76ea68579fd77b76d/mlx_lm-0.28.3.tar.gz", hash = "sha256:75df2b925d343ebaf50b63008dede4fe98cd3b02b1b24b7da71ebeb198d674f0", size = 214455, upload-time = "2025-10-17T21:44:33.921Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/a6/db3b44a5ac1a1174605628b0a477fbe4632d4fad1f94cf08647e27cc79ad/mlx_lm-0.28.3-py3-none-any.whl", hash = "sha256:ec103e2c9a06bd2cbafd41aafc975e40262176f7360d4f53ec342cebb9e0e6ea", size = 294506, upload-time = "2025-10-17T21:44:32.447Z" }, +] + +[[package]] +name = "mlx-metal" +version = "0.29.3" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/41/95/a00054a006df82bb1b5b8f666ae44a676b259146fadbff90fe654309fefc/mlx_metal-0.29.3-py3-none-macosx_13_0_arm64.whl", hash = "sha256:27b5a4d905202a71e84d9fd559ea0236813f6f960ef494e5cafe9c45df4c9d7c", size = 36817352, upload-time = "2025-10-17T19:19:25.801Z" }, + { url = "https://files.pythonhosted.org/packages/c0/d8/5ee91eac16dfcf0334103120b47d4abd8c890ccc0d73d3eee4770ce8810f/mlx_metal-0.29.3-py3-none-macosx_14_0_arm64.whl", hash = "sha256:f426d4b67f96b4d6f0ed50d5992933595aadb370dc3e9ed2410bafbc16229882", size = 36555573, upload-time = "2025-10-17T19:18:42.098Z" }, + { url = "https://files.pythonhosted.org/packages/cd/9a/39b7ecdf21cf2a39ced8d7933eed65c6cb38295cadfd0907dd1abd4d1ded/mlx_metal-0.29.3-py3-none-macosx_15_0_arm64.whl", hash = "sha256:106616f7f825851043c53d3dc186965c003985da9cbb6e5c034f35108fc1fc27", size = 36549163, upload-time = "2025-10-17T19:18:37.701Z" }, +] + +[[package]] +name = "multidict" +version = "6.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/80/1e/5492c365f222f907de1039b91f922b93fa4f764c713ee858d235495d8f50/multidict-6.7.0.tar.gz", hash = "sha256:c6e99d9a65ca282e578dfea819cfa9c0a62b2499d8677392e09feaf305e9e6f5", size = 101834, upload-time = "2025-10-06T14:52:30.657Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/86/33272a544eeb36d66e4d9a920602d1a2f57d4ebea4ef3cdfe5a912574c95/multidict-6.7.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:bee7c0588aa0076ce77c0ea5d19a68d76ad81fcd9fe8501003b9a24f9d4000f6", size = 76135, upload-time = "2025-10-06T14:49:54.26Z" }, + { url = "https://files.pythonhosted.org/packages/91/1c/eb97db117a1ebe46d457a3d235a7b9d2e6dcab174f42d1b67663dd9e5371/multidict-6.7.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7ef6b61cad77091056ce0e7ce69814ef72afacb150b7ac6a3e9470def2198159", size = 45117, upload-time = "2025-10-06T14:49:55.82Z" }, + { url = "https://files.pythonhosted.org/packages/f1/d8/6c3442322e41fb1dd4de8bd67bfd11cd72352ac131f6368315617de752f1/multidict-6.7.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:9c0359b1ec12b1d6849c59f9d319610b7f20ef990a6d454ab151aa0e3b9f78ca", size = 43472, upload-time = "2025-10-06T14:49:57.048Z" }, + { url = "https://files.pythonhosted.org/packages/75/3f/e2639e80325af0b6c6febdf8e57cc07043ff15f57fa1ef808f4ccb5ac4cd/multidict-6.7.0-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:cd240939f71c64bd658f186330603aac1a9a81bf6273f523fca63673cb7378a8", size = 249342, upload-time = "2025-10-06T14:49:58.368Z" }, + { url = "https://files.pythonhosted.org/packages/5d/cc/84e0585f805cbeaa9cbdaa95f9a3d6aed745b9d25700623ac89a6ecff400/multidict-6.7.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a60a4d75718a5efa473ebd5ab685786ba0c67b8381f781d1be14da49f1a2dc60", size = 257082, upload-time = "2025-10-06T14:49:59.89Z" }, + { url = "https://files.pythonhosted.org/packages/b0/9c/ac851c107c92289acbbf5cfb485694084690c1b17e555f44952c26ddc5bd/multidict-6.7.0-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:53a42d364f323275126aff81fb67c5ca1b7a04fda0546245730a55c8c5f24bc4", size = 240704, upload-time = "2025-10-06T14:50:01.485Z" }, + { url = "https://files.pythonhosted.org/packages/50/cc/5f93e99427248c09da95b62d64b25748a5f5c98c7c2ab09825a1d6af0e15/multidict-6.7.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3b29b980d0ddbecb736735ee5bef69bb2ddca56eff603c86f3f29a1128299b4f", size = 266355, upload-time = "2025-10-06T14:50:02.955Z" }, + { url = "https://files.pythonhosted.org/packages/ec/0c/2ec1d883ceb79c6f7f6d7ad90c919c898f5d1c6ea96d322751420211e072/multidict-6.7.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f8a93b1c0ed2d04b97a5e9336fd2d33371b9a6e29ab7dd6503d63407c20ffbaf", size = 267259, upload-time = "2025-10-06T14:50:04.446Z" }, + { url = "https://files.pythonhosted.org/packages/c6/2d/f0b184fa88d6630aa267680bdb8623fb69cb0d024b8c6f0d23f9a0f406d3/multidict-6.7.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9ff96e8815eecacc6645da76c413eb3b3d34cfca256c70b16b286a687d013c32", size = 254903, upload-time = "2025-10-06T14:50:05.98Z" }, + { url = "https://files.pythonhosted.org/packages/06/c9/11ea263ad0df7dfabcad404feb3c0dd40b131bc7f232d5537f2fb1356951/multidict-6.7.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7516c579652f6a6be0e266aec0acd0db80829ca305c3d771ed898538804c2036", size = 252365, upload-time = "2025-10-06T14:50:07.511Z" }, + { url = "https://files.pythonhosted.org/packages/41/88/d714b86ee2c17d6e09850c70c9d310abac3d808ab49dfa16b43aba9d53fd/multidict-6.7.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:040f393368e63fb0f3330e70c26bfd336656bed925e5cbe17c9da839a6ab13ec", size = 250062, upload-time = "2025-10-06T14:50:09.074Z" }, + { url = "https://files.pythonhosted.org/packages/15/fe/ad407bb9e818c2b31383f6131ca19ea7e35ce93cf1310fce69f12e89de75/multidict-6.7.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:b3bc26a951007b1057a1c543af845f1c7e3e71cc240ed1ace7bf4484aa99196e", size = 249683, upload-time = "2025-10-06T14:50:10.714Z" }, + { url = "https://files.pythonhosted.org/packages/8c/a4/a89abdb0229e533fb925e7c6e5c40201c2873efebc9abaf14046a4536ee6/multidict-6.7.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:7b022717c748dd1992a83e219587aabe45980d88969f01b316e78683e6285f64", size = 261254, upload-time = "2025-10-06T14:50:12.28Z" }, + { url = "https://files.pythonhosted.org/packages/8d/aa/0e2b27bd88b40a4fb8dc53dd74eecac70edaa4c1dd0707eb2164da3675b3/multidict-6.7.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:9600082733859f00d79dee64effc7aef1beb26adb297416a4ad2116fd61374bd", size = 257967, upload-time = "2025-10-06T14:50:14.16Z" }, + { url = "https://files.pythonhosted.org/packages/d0/8e/0c67b7120d5d5f6d874ed85a085f9dc770a7f9d8813e80f44a9fec820bb7/multidict-6.7.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:94218fcec4d72bc61df51c198d098ce2b378e0ccbac41ddbed5ef44092913288", size = 250085, upload-time = "2025-10-06T14:50:15.639Z" }, + { url = "https://files.pythonhosted.org/packages/e8/68/7b3a5170a382a340147337b300b9eb25a9ddb573bcdfff19c0fa3f31ffba/multidict-6.7.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:ad9ce259f50abd98a1ca0aa6e490b58c316a0fce0617f609723e40804add2c00", size = 83114, upload-time = "2025-10-06T14:50:21.223Z" }, + { url = "https://files.pythonhosted.org/packages/55/5c/3fa2d07c84df4e302060f555bbf539310980362236ad49f50eeb0a1c1eb9/multidict-6.7.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:07f5594ac6d084cbb5de2df218d78baf55ef150b91f0ff8a21cc7a2e3a5a58eb", size = 48442, upload-time = "2025-10-06T14:50:22.871Z" }, + { url = "https://files.pythonhosted.org/packages/fc/56/67212d33239797f9bd91962bb899d72bb0f4c35a8652dcdb8ed049bef878/multidict-6.7.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:0591b48acf279821a579282444814a2d8d0af624ae0bc600aa4d1b920b6e924b", size = 46885, upload-time = "2025-10-06T14:50:24.258Z" }, + { url = "https://files.pythonhosted.org/packages/46/d1/908f896224290350721597a61a69cd19b89ad8ee0ae1f38b3f5cd12ea2ac/multidict-6.7.0-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:749a72584761531d2b9467cfbdfd29487ee21124c304c4b6cb760d8777b27f9c", size = 242588, upload-time = "2025-10-06T14:50:25.716Z" }, + { url = "https://files.pythonhosted.org/packages/ab/67/8604288bbd68680eee0ab568fdcb56171d8b23a01bcd5cb0c8fedf6e5d99/multidict-6.7.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b4c3d199f953acd5b446bf7c0de1fe25d94e09e79086f8dc2f48a11a129cdf1", size = 249966, upload-time = "2025-10-06T14:50:28.192Z" }, + { url = "https://files.pythonhosted.org/packages/20/33/9228d76339f1ba51e3efef7da3ebd91964d3006217aae13211653193c3ff/multidict-6.7.0-cp313-cp313t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:9fb0211dfc3b51efea2f349ec92c114d7754dd62c01f81c3e32b765b70c45c9b", size = 228618, upload-time = "2025-10-06T14:50:29.82Z" }, + { url = "https://files.pythonhosted.org/packages/f8/2d/25d9b566d10cab1c42b3b9e5b11ef79c9111eaf4463b8c257a3bd89e0ead/multidict-6.7.0-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a027ec240fe73a8d6281872690b988eed307cd7d91b23998ff35ff577ca688b5", size = 257539, upload-time = "2025-10-06T14:50:31.731Z" }, + { url = "https://files.pythonhosted.org/packages/b6/b1/8d1a965e6637fc33de3c0d8f414485c2b7e4af00f42cab3d84e7b955c222/multidict-6.7.0-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d1d964afecdf3a8288789df2f5751dc0a8261138c3768d9af117ed384e538fad", size = 256345, upload-time = "2025-10-06T14:50:33.26Z" }, + { url = "https://files.pythonhosted.org/packages/ba/0c/06b5a8adbdeedada6f4fb8d8f193d44a347223b11939b42953eeb6530b6b/multidict-6.7.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:caf53b15b1b7df9fbd0709aa01409000a2b4dd03a5f6f5cc548183c7c8f8b63c", size = 247934, upload-time = "2025-10-06T14:50:34.808Z" }, + { url = "https://files.pythonhosted.org/packages/8f/31/b2491b5fe167ca044c6eb4b8f2c9f3b8a00b24c432c365358eadac5d7625/multidict-6.7.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:654030da3197d927f05a536a66186070e98765aa5142794c9904555d3a9d8fb5", size = 245243, upload-time = "2025-10-06T14:50:36.436Z" }, + { url = "https://files.pythonhosted.org/packages/61/1a/982913957cb90406c8c94f53001abd9eafc271cb3e70ff6371590bec478e/multidict-6.7.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:2090d3718829d1e484706a2f525e50c892237b2bf9b17a79b059cb98cddc2f10", size = 235878, upload-time = "2025-10-06T14:50:37.953Z" }, + { url = "https://files.pythonhosted.org/packages/be/c0/21435d804c1a1cf7a2608593f4d19bca5bcbd7a81a70b253fdd1c12af9c0/multidict-6.7.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:2d2cfeec3f6f45651b3d408c4acec0ebf3daa9bc8a112a084206f5db5d05b754", size = 243452, upload-time = "2025-10-06T14:50:39.574Z" }, + { url = "https://files.pythonhosted.org/packages/54/0a/4349d540d4a883863191be6eb9a928846d4ec0ea007d3dcd36323bb058ac/multidict-6.7.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:4ef089f985b8c194d341eb2c24ae6e7408c9a0e2e5658699c92f497437d88c3c", size = 252312, upload-time = "2025-10-06T14:50:41.612Z" }, + { url = "https://files.pythonhosted.org/packages/26/64/d5416038dbda1488daf16b676e4dbfd9674dde10a0cc8f4fc2b502d8125d/multidict-6.7.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:e93a0617cd16998784bf4414c7e40f17a35d2350e5c6f0bd900d3a8e02bd3762", size = 246935, upload-time = "2025-10-06T14:50:43.972Z" }, + { url = "https://files.pythonhosted.org/packages/9f/8c/8290c50d14e49f35e0bd4abc25e1bc7711149ca9588ab7d04f886cdf03d9/multidict-6.7.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:f0feece2ef8ebc42ed9e2e8c78fc4aa3cf455733b507c09ef7406364c94376c6", size = 243385, upload-time = "2025-10-06T14:50:45.648Z" }, + { url = "https://files.pythonhosted.org/packages/e2/b1/3da6934455dd4b261d4c72f897e3a5728eba81db59959f3a639245891baa/multidict-6.7.0-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:3bab1e4aff7adaa34410f93b1f8e57c4b36b9af0426a76003f441ee1d3c7e842", size = 75128, upload-time = "2025-10-06T14:50:51.92Z" }, + { url = "https://files.pythonhosted.org/packages/14/2c/f069cab5b51d175a1a2cb4ccdf7a2c2dabd58aa5bd933fa036a8d15e2404/multidict-6.7.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:b8512bac933afc3e45fb2b18da8e59b78d4f408399a960339598374d4ae3b56b", size = 44410, upload-time = "2025-10-06T14:50:53.275Z" }, + { url = "https://files.pythonhosted.org/packages/42/e2/64bb41266427af6642b6b128e8774ed84c11b80a90702c13ac0a86bb10cc/multidict-6.7.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:79dcf9e477bc65414ebfea98ffd013cb39552b5ecd62908752e0e413d6d06e38", size = 43205, upload-time = "2025-10-06T14:50:54.911Z" }, + { url = "https://files.pythonhosted.org/packages/02/68/6b086fef8a3f1a8541b9236c594f0c9245617c29841f2e0395d979485cde/multidict-6.7.0-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:31bae522710064b5cbeddaf2e9f32b1abab70ac6ac91d42572502299e9953128", size = 245084, upload-time = "2025-10-06T14:50:56.369Z" }, + { url = "https://files.pythonhosted.org/packages/15/ee/f524093232007cd7a75c1d132df70f235cfd590a7c9eaccd7ff422ef4ae8/multidict-6.7.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4a0df7ff02397bb63e2fd22af2c87dfa39e8c7f12947bc524dbdc528282c7e34", size = 252667, upload-time = "2025-10-06T14:50:57.991Z" }, + { url = "https://files.pythonhosted.org/packages/02/a5/eeb3f43ab45878f1895118c3ef157a480db58ede3f248e29b5354139c2c9/multidict-6.7.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:7a0222514e8e4c514660e182d5156a415c13ef0aabbd71682fc714e327b95e99", size = 233590, upload-time = "2025-10-06T14:50:59.589Z" }, + { url = "https://files.pythonhosted.org/packages/6a/1e/76d02f8270b97269d7e3dbd45644b1785bda457b474315f8cf999525a193/multidict-6.7.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2397ab4daaf2698eb51a76721e98db21ce4f52339e535725de03ea962b5a3202", size = 264112, upload-time = "2025-10-06T14:51:01.183Z" }, + { url = "https://files.pythonhosted.org/packages/76/0b/c28a70ecb58963847c2a8efe334904cd254812b10e535aefb3bcce513918/multidict-6.7.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:8891681594162635948a636c9fe0ff21746aeb3dd5463f6e25d9bea3a8a39ca1", size = 261194, upload-time = "2025-10-06T14:51:02.794Z" }, + { url = "https://files.pythonhosted.org/packages/b4/63/2ab26e4209773223159b83aa32721b4021ffb08102f8ac7d689c943fded1/multidict-6.7.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:18706cc31dbf402a7945916dd5cddf160251b6dab8a2c5f3d6d5a55949f676b3", size = 248510, upload-time = "2025-10-06T14:51:04.724Z" }, + { url = "https://files.pythonhosted.org/packages/93/cd/06c1fa8282af1d1c46fd55c10a7930af652afdce43999501d4d68664170c/multidict-6.7.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:f844a1bbf1d207dd311a56f383f7eda2d0e134921d45751842d8235e7778965d", size = 248395, upload-time = "2025-10-06T14:51:06.306Z" }, + { url = "https://files.pythonhosted.org/packages/99/ac/82cb419dd6b04ccf9e7e61befc00c77614fc8134362488b553402ecd55ce/multidict-6.7.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:d4393e3581e84e5645506923816b9cc81f5609a778c7e7534054091acc64d1c6", size = 239520, upload-time = "2025-10-06T14:51:08.091Z" }, + { url = "https://files.pythonhosted.org/packages/fa/f3/a0f9bf09493421bd8716a362e0cd1d244f5a6550f5beffdd6b47e885b331/multidict-6.7.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:fbd18dc82d7bf274b37aa48d664534330af744e03bccf696d6f4c6042e7d19e7", size = 245479, upload-time = "2025-10-06T14:51:10.365Z" }, + { url = "https://files.pythonhosted.org/packages/8d/01/476d38fc73a212843f43c852b0eee266b6971f0e28329c2184a8df90c376/multidict-6.7.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:b6234e14f9314731ec45c42fc4554b88133ad53a09092cc48a88e771c125dadb", size = 258903, upload-time = "2025-10-06T14:51:12.466Z" }, + { url = "https://files.pythonhosted.org/packages/49/6d/23faeb0868adba613b817d0e69c5f15531b24d462af8012c4f6de4fa8dc3/multidict-6.7.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:08d4379f9744d8f78d98c8673c06e202ffa88296f009c71bbafe8a6bf847d01f", size = 252333, upload-time = "2025-10-06T14:51:14.48Z" }, + { url = "https://files.pythonhosted.org/packages/1e/cc/48d02ac22b30fa247f7dad82866e4b1015431092f4ba6ebc7e77596e0b18/multidict-6.7.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:9fe04da3f79387f450fd0061d4dd2e45a72749d31bf634aecc9e27f24fdc4b3f", size = 243411, upload-time = "2025-10-06T14:51:16.072Z" }, + { url = "https://files.pythonhosted.org/packages/8b/40/cd499bd0dbc5f1136726db3153042a735fffd0d77268e2ee20d5f33c010f/multidict-6.7.0-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:c1dcc7524066fa918c6a27d61444d4ee7900ec635779058571f70d042d86ed63", size = 82326, upload-time = "2025-10-06T14:51:21.588Z" }, + { url = "https://files.pythonhosted.org/packages/13/8a/18e031eca251c8df76daf0288e6790561806e439f5ce99a170b4af30676b/multidict-6.7.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:27e0b36c2d388dc7b6ced3406671b401e84ad7eb0656b8f3a2f46ed0ce483718", size = 48065, upload-time = "2025-10-06T14:51:22.93Z" }, + { url = "https://files.pythonhosted.org/packages/40/71/5e6701277470a87d234e433fb0a3a7deaf3bcd92566e421e7ae9776319de/multidict-6.7.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2a7baa46a22e77f0988e3b23d4ede5513ebec1929e34ee9495be535662c0dfe2", size = 46475, upload-time = "2025-10-06T14:51:24.352Z" }, + { url = "https://files.pythonhosted.org/packages/fe/6a/bab00cbab6d9cfb57afe1663318f72ec28289ea03fd4e8236bb78429893a/multidict-6.7.0-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:7bf77f54997a9166a2f5675d1201520586439424c2511723a7312bdb4bcc034e", size = 239324, upload-time = "2025-10-06T14:51:25.822Z" }, + { url = "https://files.pythonhosted.org/packages/2a/5f/8de95f629fc22a7769ade8b41028e3e5a822c1f8904f618d175945a81ad3/multidict-6.7.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e011555abada53f1578d63389610ac8a5400fc70ce71156b0aa30d326f1a5064", size = 246877, upload-time = "2025-10-06T14:51:27.604Z" }, + { url = "https://files.pythonhosted.org/packages/23/b4/38881a960458f25b89e9f4a4fdcb02ac101cfa710190db6e5528841e67de/multidict-6.7.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:28b37063541b897fd6a318007373930a75ca6d6ac7c940dbe14731ffdd8d498e", size = 225824, upload-time = "2025-10-06T14:51:29.664Z" }, + { url = "https://files.pythonhosted.org/packages/1e/39/6566210c83f8a261575f18e7144736059f0c460b362e96e9cf797a24b8e7/multidict-6.7.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:05047ada7a2fde2631a0ed706f1fd68b169a681dfe5e4cf0f8e4cb6618bbc2cd", size = 253558, upload-time = "2025-10-06T14:51:31.684Z" }, + { url = "https://files.pythonhosted.org/packages/00/a3/67f18315100f64c269f46e6c0319fa87ba68f0f64f2b8e7fd7c72b913a0b/multidict-6.7.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:716133f7d1d946a4e1b91b1756b23c088881e70ff180c24e864c26192ad7534a", size = 252339, upload-time = "2025-10-06T14:51:33.699Z" }, + { url = "https://files.pythonhosted.org/packages/c8/2a/1cb77266afee2458d82f50da41beba02159b1d6b1f7973afc9a1cad1499b/multidict-6.7.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d1bed1b467ef657f2a0ae62844a607909ef1c6889562de5e1d505f74457d0b96", size = 244895, upload-time = "2025-10-06T14:51:36.189Z" }, + { url = "https://files.pythonhosted.org/packages/dd/72/09fa7dd487f119b2eb9524946ddd36e2067c08510576d43ff68469563b3b/multidict-6.7.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:ca43bdfa5d37bd6aee89d85e1d0831fb86e25541be7e9d376ead1b28974f8e5e", size = 241862, upload-time = "2025-10-06T14:51:41.291Z" }, + { url = "https://files.pythonhosted.org/packages/65/92/bc1f8bd0853d8669300f732c801974dfc3702c3eeadae2f60cef54dc69d7/multidict-6.7.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:44b546bd3eb645fd26fb949e43c02a25a2e632e2ca21a35e2e132c8105dc8599", size = 232376, upload-time = "2025-10-06T14:51:43.55Z" }, + { url = "https://files.pythonhosted.org/packages/09/86/ac39399e5cb9d0c2ac8ef6e10a768e4d3bc933ac808d49c41f9dc23337eb/multidict-6.7.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:a6ef16328011d3f468e7ebc326f24c1445f001ca1dec335b2f8e66bed3006394", size = 240272, upload-time = "2025-10-06T14:51:45.265Z" }, + { url = "https://files.pythonhosted.org/packages/3d/b6/fed5ac6b8563ec72df6cb1ea8dac6d17f0a4a1f65045f66b6d3bf1497c02/multidict-6.7.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:5aa873cbc8e593d361ae65c68f85faadd755c3295ea2c12040ee146802f23b38", size = 248774, upload-time = "2025-10-06T14:51:46.836Z" }, + { url = "https://files.pythonhosted.org/packages/6b/8d/b954d8c0dc132b68f760aefd45870978deec6818897389dace00fcde32ff/multidict-6.7.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:3d7b6ccce016e29df4b7ca819659f516f0bc7a4b3efa3bb2012ba06431b044f9", size = 242731, upload-time = "2025-10-06T14:51:48.541Z" }, + { url = "https://files.pythonhosted.org/packages/16/9d/a2dac7009125d3540c2f54e194829ea18ac53716c61b655d8ed300120b0f/multidict-6.7.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:171b73bd4ee683d307599b66793ac80981b06f069b62eea1c9e29c9241aa66b0", size = 240193, upload-time = "2025-10-06T14:51:50.355Z" }, + { url = "https://files.pythonhosted.org/packages/b7/da/7d22601b625e241d4f23ef1ebff8acfc60da633c9e7e7922e24d10f592b3/multidict-6.7.0-py3-none-any.whl", hash = "sha256:394fc5c42a333c9ffc3e421a4c85e08580d990e08b99f6bf35b4132114c5dcb3", size = 12317, upload-time = "2025-10-06T14:52:29.272Z" }, +] + +[[package]] +name = "networkx" +version = "3.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6c/4f/ccdb8ad3a38e583f214547fd2f7ff1fc160c43a75af88e6aec213404b96a/networkx-3.5.tar.gz", hash = "sha256:d4c6f9cf81f52d69230866796b82afbccdec3db7ae4fbd1b65ea750feed50037", size = 2471065, upload-time = "2025-05-29T11:35:07.804Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/eb/8d/776adee7bbf76365fdd7f2552710282c79a4ead5d2a46408c9043a2b70ba/networkx-3.5-py3-none-any.whl", hash = "sha256:0030d386a9a06dee3565298b4a734b68589749a544acbb6c412dc9e2489ec6ec", size = 2034406, upload-time = "2025-05-29T11:35:04.961Z" }, +] + +[[package]] +name = "numpy" +version = "2.3.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b5/f4/098d2270d52b41f1bd7db9fc288aaa0400cb48c2a3e2af6fa365d9720947/numpy-2.3.4.tar.gz", hash = "sha256:a7d018bfedb375a8d979ac758b120ba846a7fe764911a64465fd87b8729f4a6a", size = 20582187, upload-time = "2025-10-15T16:18:11.77Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/57/7e/b72610cc91edf138bc588df5150957a4937221ca6058b825b4725c27be62/numpy-2.3.4-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:c090d4860032b857d94144d1a9976b8e36709e40386db289aaf6672de2a81966", size = 20950335, upload-time = "2025-10-15T16:16:10.304Z" }, + { url = "https://files.pythonhosted.org/packages/3e/46/bdd3370dcea2f95ef14af79dbf81e6927102ddf1cc54adc0024d61252fd9/numpy-2.3.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a13fc473b6db0be619e45f11f9e81260f7302f8d180c49a22b6e6120022596b3", size = 14179878, upload-time = "2025-10-15T16:16:12.595Z" }, + { url = "https://files.pythonhosted.org/packages/ac/01/5a67cb785bda60f45415d09c2bc245433f1c68dd82eef9c9002c508b5a65/numpy-2.3.4-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:3634093d0b428e6c32c3a69b78e554f0cd20ee420dcad5a9f3b2a63762ce4197", size = 5108673, upload-time = "2025-10-15T16:16:14.877Z" }, + { url = "https://files.pythonhosted.org/packages/c2/cd/8428e23a9fcebd33988f4cb61208fda832800ca03781f471f3727a820704/numpy-2.3.4-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:043885b4f7e6e232d7df4f51ffdef8c36320ee9d5f227b380ea636722c7ed12e", size = 6641438, upload-time = "2025-10-15T16:16:16.805Z" }, + { url = "https://files.pythonhosted.org/packages/3e/d1/913fe563820f3c6b079f992458f7331278dcd7ba8427e8e745af37ddb44f/numpy-2.3.4-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4ee6a571d1e4f0ea6d5f22d6e5fbd6ed1dc2b18542848e1e7301bd190500c9d7", size = 14281290, upload-time = "2025-10-15T16:16:18.764Z" }, + { url = "https://files.pythonhosted.org/packages/9e/7e/7d306ff7cb143e6d975cfa7eb98a93e73495c4deabb7d1b5ecf09ea0fd69/numpy-2.3.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc8a63918b04b8571789688b2780ab2b4a33ab44bfe8ccea36d3eba51228c953", size = 16636543, upload-time = "2025-10-15T16:16:21.072Z" }, + { url = "https://files.pythonhosted.org/packages/47/6a/8cfc486237e56ccfb0db234945552a557ca266f022d281a2f577b98e955c/numpy-2.3.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:40cc556d5abbc54aabe2b1ae287042d7bdb80c08edede19f0c0afb36ae586f37", size = 16056117, upload-time = "2025-10-15T16:16:23.369Z" }, + { url = "https://files.pythonhosted.org/packages/b1/0e/42cb5e69ea901e06ce24bfcc4b5664a56f950a70efdcf221f30d9615f3f3/numpy-2.3.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ecb63014bb7f4ce653f8be7f1df8cbc6093a5a2811211770f6606cc92b5a78fd", size = 18577788, upload-time = "2025-10-15T16:16:27.496Z" }, + { url = "https://files.pythonhosted.org/packages/11/83/66ac031464ec1767ea3ed48ce40f615eb441072945e98693bec0bcd056cc/numpy-2.3.4-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:86966db35c4040fdca64f0816a1c1dd8dbd027d90fca5a57e00e1ca4cd41b879", size = 21049003, upload-time = "2025-10-15T16:16:36.101Z" }, + { url = "https://files.pythonhosted.org/packages/5f/99/5b14e0e686e61371659a1d5bebd04596b1d72227ce36eed121bb0aeab798/numpy-2.3.4-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:838f045478638b26c375ee96ea89464d38428c69170360b23a1a50fa4baa3562", size = 14302980, upload-time = "2025-10-15T16:16:39.124Z" }, + { url = "https://files.pythonhosted.org/packages/2c/44/e9486649cd087d9fc6920e3fc3ac2aba10838d10804b1e179fb7cbc4e634/numpy-2.3.4-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:d7315ed1dab0286adca467377c8381cd748f3dc92235f22a7dfc42745644a96a", size = 5231472, upload-time = "2025-10-15T16:16:41.168Z" }, + { url = "https://files.pythonhosted.org/packages/3e/51/902b24fa8887e5fe2063fd61b1895a476d0bbf46811ab0c7fdf4bd127345/numpy-2.3.4-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:84f01a4d18b2cc4ade1814a08e5f3c907b079c847051d720fad15ce37aa930b6", size = 6739342, upload-time = "2025-10-15T16:16:43.777Z" }, + { url = "https://files.pythonhosted.org/packages/34/f1/4de9586d05b1962acdcdb1dc4af6646361a643f8c864cef7c852bf509740/numpy-2.3.4-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:817e719a868f0dacde4abdfc5c1910b301877970195db9ab6a5e2c4bd5b121f7", size = 14354338, upload-time = "2025-10-15T16:16:46.081Z" }, + { url = "https://files.pythonhosted.org/packages/1f/06/1c16103b425de7969d5a76bdf5ada0804b476fed05d5f9e17b777f1cbefd/numpy-2.3.4-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:85e071da78d92a214212cacea81c6da557cab307f2c34b5f85b628e94803f9c0", size = 16702392, upload-time = "2025-10-15T16:16:48.455Z" }, + { url = "https://files.pythonhosted.org/packages/34/b2/65f4dc1b89b5322093572b6e55161bb42e3e0487067af73627f795cc9d47/numpy-2.3.4-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:2ec646892819370cf3558f518797f16597b4e4669894a2ba712caccc9da53f1f", size = 16134998, upload-time = "2025-10-15T16:16:51.114Z" }, + { url = "https://files.pythonhosted.org/packages/d4/11/94ec578896cdb973aaf56425d6c7f2aff4186a5c00fac15ff2ec46998b46/numpy-2.3.4-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:035796aaaddfe2f9664b9a9372f089cfc88bd795a67bd1bfe15e6e770934cf64", size = 18651574, upload-time = "2025-10-15T16:16:53.429Z" }, + { url = "https://files.pythonhosted.org/packages/72/71/ae6170143c115732470ae3a2d01512870dd16e0953f8a6dc89525696069b/numpy-2.3.4-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:81c3e6d8c97295a7360d367f9f8553973651b76907988bb6066376bc2252f24e", size = 20955580, upload-time = "2025-10-15T16:17:02.509Z" }, + { url = "https://files.pythonhosted.org/packages/af/39/4be9222ffd6ca8a30eda033d5f753276a9c3426c397bb137d8e19dedd200/numpy-2.3.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:7c26b0b2bf58009ed1f38a641f3db4be8d960a417ca96d14e5b06df1506d41ff", size = 14188056, upload-time = "2025-10-15T16:17:04.873Z" }, + { url = "https://files.pythonhosted.org/packages/6c/3d/d85f6700d0a4aa4f9491030e1021c2b2b7421b2b38d01acd16734a2bfdc7/numpy-2.3.4-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:62b2198c438058a20b6704351b35a1d7db881812d8512d67a69c9de1f18ca05f", size = 5116555, upload-time = "2025-10-15T16:17:07.499Z" }, + { url = "https://files.pythonhosted.org/packages/bf/04/82c1467d86f47eee8a19a464c92f90a9bb68ccf14a54c5224d7031241ffb/numpy-2.3.4-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:9d729d60f8d53a7361707f4b68a9663c968882dd4f09e0d58c044c8bf5faee7b", size = 6643581, upload-time = "2025-10-15T16:17:09.774Z" }, + { url = "https://files.pythonhosted.org/packages/0c/d3/c79841741b837e293f48bd7db89d0ac7a4f2503b382b78a790ef1dc778a5/numpy-2.3.4-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bd0c630cf256b0a7fd9d0a11c9413b42fef5101219ce6ed5a09624f5a65392c7", size = 14299186, upload-time = "2025-10-15T16:17:11.937Z" }, + { url = "https://files.pythonhosted.org/packages/e8/7e/4a14a769741fbf237eec5a12a2cbc7a4c4e061852b6533bcb9e9a796c908/numpy-2.3.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d5e081bc082825f8b139f9e9fe42942cb4054524598aaeb177ff476cc76d09d2", size = 16638601, upload-time = "2025-10-15T16:17:14.391Z" }, + { url = "https://files.pythonhosted.org/packages/93/87/1c1de269f002ff0a41173fe01dcc925f4ecff59264cd8f96cf3b60d12c9b/numpy-2.3.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:15fb27364ed84114438fff8aaf998c9e19adbeba08c0b75409f8c452a8692c52", size = 16074219, upload-time = "2025-10-15T16:17:17.058Z" }, + { url = "https://files.pythonhosted.org/packages/cd/28/18f72ee77408e40a76d691001ae599e712ca2a47ddd2c4f695b16c65f077/numpy-2.3.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:85d9fb2d8cd998c84d13a79a09cc0c1091648e848e4e6249b0ccd7f6b487fa26", size = 18576702, upload-time = "2025-10-15T16:17:19.379Z" }, + { url = "https://files.pythonhosted.org/packages/83/4b/c4a5f0841f92536f6b9592694a5b5f68c9ab37b775ff342649eadf9055d3/numpy-2.3.4-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:22758999b256b595cf0b1d102b133bb61866ba5ceecf15f759623b64c020c9ec", size = 21052280, upload-time = "2025-10-15T16:17:29.638Z" }, + { url = "https://files.pythonhosted.org/packages/3e/80/90308845fc93b984d2cc96d83e2324ce8ad1fd6efea81b324cba4b673854/numpy-2.3.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:9cb177bc55b010b19798dc5497d540dea67fd13a8d9e882b2dae71de0cf09eb3", size = 14302930, upload-time = "2025-10-15T16:17:32.384Z" }, + { url = "https://files.pythonhosted.org/packages/3d/4e/07439f22f2a3b247cec4d63a713faae55e1141a36e77fb212881f7cda3fb/numpy-2.3.4-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:0f2bcc76f1e05e5ab58893407c63d90b2029908fa41f9f1cc51eecce936c3365", size = 5231504, upload-time = "2025-10-15T16:17:34.515Z" }, + { url = "https://files.pythonhosted.org/packages/ab/de/1e11f2547e2fe3d00482b19721855348b94ada8359aef5d40dd57bfae9df/numpy-2.3.4-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:8dc20bde86802df2ed8397a08d793da0ad7a5fd4ea3ac85d757bf5dd4ad7c252", size = 6739405, upload-time = "2025-10-15T16:17:36.128Z" }, + { url = "https://files.pythonhosted.org/packages/3b/40/8cd57393a26cebe2e923005db5134a946c62fa56a1087dc7c478f3e30837/numpy-2.3.4-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e199c087e2aa71c8f9ce1cb7a8e10677dc12457e7cc1be4798632da37c3e86e", size = 14354866, upload-time = "2025-10-15T16:17:38.884Z" }, + { url = "https://files.pythonhosted.org/packages/93/39/5b3510f023f96874ee6fea2e40dfa99313a00bf3ab779f3c92978f34aace/numpy-2.3.4-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:85597b2d25ddf655495e2363fe044b0ae999b75bc4d630dc0d886484b03a5eb0", size = 16703296, upload-time = "2025-10-15T16:17:41.564Z" }, + { url = "https://files.pythonhosted.org/packages/41/0d/19bb163617c8045209c1996c4e427bccbc4bbff1e2c711f39203c8ddbb4a/numpy-2.3.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:04a69abe45b49c5955923cf2c407843d1c85013b424ae8a560bba16c92fe44a0", size = 16136046, upload-time = "2025-10-15T16:17:43.901Z" }, + { url = "https://files.pythonhosted.org/packages/e2/c1/6dba12fdf68b02a21ac411c9df19afa66bed2540f467150ca64d246b463d/numpy-2.3.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:e1708fac43ef8b419c975926ce1eaf793b0c13b7356cfab6ab0dc34c0a02ac0f", size = 18652691, upload-time = "2025-10-15T16:17:46.247Z" }, +] + +[[package]] +name = "packaging" +version = "25.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a1/d4/1fc4078c65507b51b96ca8f8c3ba19e6a61c8253c72794544580a7b6c24d/packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f", size = 165727, upload-time = "2025-04-19T11:48:59.673Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/20/12/38679034af332785aac8774540895e234f4d07f7545804097de4b666afd8/packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484", size = 66469, upload-time = "2025-04-19T11:48:57.875Z" }, +] + +[[package]] +name = "pathlib" +version = "1.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ac/aa/9b065a76b9af472437a0059f77e8f962fe350438b927cb80184c32f075eb/pathlib-1.0.1.tar.gz", hash = "sha256:6940718dfc3eff4258203ad5021090933e5c04707d5ca8cc9e73c94a7894ea9f", size = 49298, upload-time = "2014-09-03T15:41:57.18Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/78/f9/690a8600b93c332de3ab4a344a4ac34f00c8f104917061f779db6a918ed6/pathlib-1.0.1-py3-none-any.whl", hash = "sha256:f35f95ab8b0f59e6d354090350b44a80a80635d22efdedfa84c7ad1cf0a74147", size = 14363, upload-time = "2022-05-04T13:37:20.585Z" }, +] + +[[package]] +name = "platformdirs" +version = "4.5.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/61/33/9611380c2bdb1225fdef633e2a9610622310fed35ab11dac9620972ee088/platformdirs-4.5.0.tar.gz", hash = "sha256:70ddccdd7c99fc5942e9fc25636a8b34d04c24b335100223152c2803e4063312", size = 21632, upload-time = "2025-10-08T17:44:48.791Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/73/cb/ac7874b3e5d58441674fb70742e6c374b28b0c7cb988d37d991cde47166c/platformdirs-4.5.0-py3-none-any.whl", hash = "sha256:e578a81bb873cbb89a41fcc904c7ef523cc18284b7e3b3ccf06aca1403b7ebd3", size = 18651, upload-time = "2025-10-08T17:44:47.223Z" }, +] + +[[package]] +name = "pluggy" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, +] + +[[package]] +name = "priority" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f5/3c/eb7c35f4dcede96fca1842dac5f4f5d15511aa4b52f3a961219e68ae9204/priority-2.0.0.tar.gz", hash = "sha256:c965d54f1b8d0d0b19479db3924c7c36cf672dbf2aec92d43fbdaf4492ba18c0", size = 24792, upload-time = "2021-06-27T10:15:05.487Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5e/5f/82c8074f7e84978129347c2c6ec8b6c59f3584ff1a20bc3c940a3e061790/priority-2.0.0-py3-none-any.whl", hash = "sha256:6f8eefce5f3ad59baf2c080a664037bb4725cd0a790d53d59ab4059288faf6aa", size = 8946, upload-time = "2021-06-27T10:15:03.856Z" }, +] + +[[package]] +name = "propcache" +version = "0.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/9e/da/e9fc233cf63743258bff22b3dfa7ea5baef7b5bc324af47a0ad89b8ffc6f/propcache-0.4.1.tar.gz", hash = "sha256:f48107a8c637e80362555f37ecf49abe20370e557cc4ab374f04ec4423c97c3d", size = 46442, upload-time = "2025-10-08T19:49:02.291Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bf/df/6d9c1b6ac12b003837dde8a10231a7344512186e87b36e855bef32241942/propcache-0.4.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:43eedf29202c08550aac1d14e0ee619b0430aaef78f85864c1a892294fbc28cf", size = 77750, upload-time = "2025-10-08T19:47:07.648Z" }, + { url = "https://files.pythonhosted.org/packages/8b/e8/677a0025e8a2acf07d3418a2e7ba529c9c33caf09d3c1f25513023c1db56/propcache-0.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:d62cdfcfd89ccb8de04e0eda998535c406bf5e060ffd56be6c586cbcc05b3311", size = 44780, upload-time = "2025-10-08T19:47:08.851Z" }, + { url = "https://files.pythonhosted.org/packages/89/a4/92380f7ca60f99ebae761936bc48a72a639e8a47b29050615eef757cb2a7/propcache-0.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:cae65ad55793da34db5f54e4029b89d3b9b9490d8abe1b4c7ab5d4b8ec7ebf74", size = 46308, upload-time = "2025-10-08T19:47:09.982Z" }, + { url = "https://files.pythonhosted.org/packages/2d/48/c5ac64dee5262044348d1d78a5f85dd1a57464a60d30daee946699963eb3/propcache-0.4.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:333ddb9031d2704a301ee3e506dc46b1fe5f294ec198ed6435ad5b6a085facfe", size = 208182, upload-time = "2025-10-08T19:47:11.319Z" }, + { url = "https://files.pythonhosted.org/packages/c6/0c/cd762dd011a9287389a6a3eb43aa30207bde253610cca06824aeabfe9653/propcache-0.4.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:fd0858c20f078a32cf55f7e81473d96dcf3b93fd2ccdb3d40fdf54b8573df3af", size = 211215, upload-time = "2025-10-08T19:47:13.146Z" }, + { url = "https://files.pythonhosted.org/packages/30/3e/49861e90233ba36890ae0ca4c660e95df565b2cd15d4a68556ab5865974e/propcache-0.4.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:678ae89ebc632c5c204c794f8dab2837c5f159aeb59e6ed0539500400577298c", size = 218112, upload-time = "2025-10-08T19:47:14.913Z" }, + { url = "https://files.pythonhosted.org/packages/f1/8b/544bc867e24e1bd48f3118cecd3b05c694e160a168478fa28770f22fd094/propcache-0.4.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d472aeb4fbf9865e0c6d622d7f4d54a4e101a89715d8904282bb5f9a2f476c3f", size = 204442, upload-time = "2025-10-08T19:47:16.277Z" }, + { url = "https://files.pythonhosted.org/packages/50/a6/4282772fd016a76d3e5c0df58380a5ea64900afd836cec2c2f662d1b9bb3/propcache-0.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4d3df5fa7e36b3225954fba85589da77a0fe6a53e3976de39caf04a0db4c36f1", size = 199398, upload-time = "2025-10-08T19:47:17.962Z" }, + { url = "https://files.pythonhosted.org/packages/3e/ec/d8a7cd406ee1ddb705db2139f8a10a8a427100347bd698e7014351c7af09/propcache-0.4.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:ee17f18d2498f2673e432faaa71698032b0127ebf23ae5974eeaf806c279df24", size = 196920, upload-time = "2025-10-08T19:47:19.355Z" }, + { url = "https://files.pythonhosted.org/packages/f6/6c/f38ab64af3764f431e359f8baf9e0a21013e24329e8b85d2da32e8ed07ca/propcache-0.4.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:580e97762b950f993ae618e167e7be9256b8353c2dcd8b99ec100eb50f5286aa", size = 203748, upload-time = "2025-10-08T19:47:21.338Z" }, + { url = "https://files.pythonhosted.org/packages/d6/e3/fa846bd70f6534d647886621388f0a265254d30e3ce47e5c8e6e27dbf153/propcache-0.4.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:501d20b891688eb8e7aa903021f0b72d5a55db40ffaab27edefd1027caaafa61", size = 205877, upload-time = "2025-10-08T19:47:23.059Z" }, + { url = "https://files.pythonhosted.org/packages/e2/39/8163fc6f3133fea7b5f2827e8eba2029a0277ab2c5beee6c1db7b10fc23d/propcache-0.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9a0bd56e5b100aef69bd8562b74b46254e7c8812918d3baa700c8a8009b0af66", size = 199437, upload-time = "2025-10-08T19:47:24.445Z" }, + { url = "https://files.pythonhosted.org/packages/83/ce/a31bbdfc24ee0dcbba458c8175ed26089cf109a55bbe7b7640ed2470cfe9/propcache-0.4.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:92d1935ee1f8d7442da9c0c4fa7ac20d07e94064184811b685f5c4fada64553b", size = 81451, upload-time = "2025-10-08T19:47:29.445Z" }, + { url = "https://files.pythonhosted.org/packages/25/9c/442a45a470a68456e710d96cacd3573ef26a1d0a60067e6a7d5e655621ed/propcache-0.4.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:473c61b39e1460d386479b9b2f337da492042447c9b685f28be4f74d3529e566", size = 46374, upload-time = "2025-10-08T19:47:30.579Z" }, + { url = "https://files.pythonhosted.org/packages/f4/bf/b1d5e21dbc3b2e889ea4327044fb16312a736d97640fb8b6aa3f9c7b3b65/propcache-0.4.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:c0ef0aaafc66fbd87842a3fe3902fd889825646bc21149eafe47be6072725835", size = 48396, upload-time = "2025-10-08T19:47:31.79Z" }, + { url = "https://files.pythonhosted.org/packages/f4/04/5b4c54a103d480e978d3c8a76073502b18db0c4bc17ab91b3cb5092ad949/propcache-0.4.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f95393b4d66bfae908c3ca8d169d5f79cd65636ae15b5e7a4f6e67af675adb0e", size = 275950, upload-time = "2025-10-08T19:47:33.481Z" }, + { url = "https://files.pythonhosted.org/packages/b4/c1/86f846827fb969c4b78b0af79bba1d1ea2156492e1b83dea8b8a6ae27395/propcache-0.4.1-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c07fda85708bc48578467e85099645167a955ba093be0a2dcba962195676e859", size = 273856, upload-time = "2025-10-08T19:47:34.906Z" }, + { url = "https://files.pythonhosted.org/packages/36/1d/fc272a63c8d3bbad6878c336c7a7dea15e8f2d23a544bda43205dfa83ada/propcache-0.4.1-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:af223b406d6d000830c6f65f1e6431783fc3f713ba3e6cc8c024d5ee96170a4b", size = 280420, upload-time = "2025-10-08T19:47:36.338Z" }, + { url = "https://files.pythonhosted.org/packages/07/0c/01f2219d39f7e53d52e5173bcb09c976609ba30209912a0680adfb8c593a/propcache-0.4.1-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a78372c932c90ee474559c5ddfffd718238e8673c340dc21fe45c5b8b54559a0", size = 263254, upload-time = "2025-10-08T19:47:37.692Z" }, + { url = "https://files.pythonhosted.org/packages/2d/18/cd28081658ce597898f0c4d174d4d0f3c5b6d4dc27ffafeef835c95eb359/propcache-0.4.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:564d9f0d4d9509e1a870c920a89b2fec951b44bf5ba7d537a9e7c1ccec2c18af", size = 261205, upload-time = "2025-10-08T19:47:39.659Z" }, + { url = "https://files.pythonhosted.org/packages/7a/71/1f9e22eb8b8316701c2a19fa1f388c8a3185082607da8e406a803c9b954e/propcache-0.4.1-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:17612831fda0138059cc5546f4d12a2aacfb9e47068c06af35c400ba58ba7393", size = 247873, upload-time = "2025-10-08T19:47:41.084Z" }, + { url = "https://files.pythonhosted.org/packages/4a/65/3d4b61f36af2b4eddba9def857959f1016a51066b4f1ce348e0cf7881f58/propcache-0.4.1-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:41a89040cb10bd345b3c1a873b2bf36413d48da1def52f268a055f7398514874", size = 262739, upload-time = "2025-10-08T19:47:42.51Z" }, + { url = "https://files.pythonhosted.org/packages/2a/42/26746ab087faa77c1c68079b228810436ccd9a5ce9ac85e2b7307195fd06/propcache-0.4.1-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:e35b88984e7fa64aacecea39236cee32dd9bd8c55f57ba8a75cf2399553f9bd7", size = 263514, upload-time = "2025-10-08T19:47:43.927Z" }, + { url = "https://files.pythonhosted.org/packages/94/13/630690fe201f5502d2403dd3cfd451ed8858fe3c738ee88d095ad2ff407b/propcache-0.4.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:6f8b465489f927b0df505cbe26ffbeed4d6d8a2bbc61ce90eb074ff129ef0ab1", size = 257781, upload-time = "2025-10-08T19:47:45.448Z" }, + { url = "https://files.pythonhosted.org/packages/8e/5c/bca52d654a896f831b8256683457ceddd490ec18d9ec50e97dfd8fc726a8/propcache-0.4.1-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:3f7124c9d820ba5548d431afb4632301acf965db49e666aa21c305cbe8c6de12", size = 78152, upload-time = "2025-10-08T19:47:51.051Z" }, + { url = "https://files.pythonhosted.org/packages/65/9b/03b04e7d82a5f54fb16113d839f5ea1ede58a61e90edf515f6577c66fa8f/propcache-0.4.1-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:c0d4b719b7da33599dfe3b22d3db1ef789210a0597bc650b7cee9c77c2be8c5c", size = 44869, upload-time = "2025-10-08T19:47:52.594Z" }, + { url = "https://files.pythonhosted.org/packages/b2/fa/89a8ef0468d5833a23fff277b143d0573897cf75bd56670a6d28126c7d68/propcache-0.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:9f302f4783709a78240ebc311b793f123328716a60911d667e0c036bc5dcbded", size = 46596, upload-time = "2025-10-08T19:47:54.073Z" }, + { url = "https://files.pythonhosted.org/packages/86/bd/47816020d337f4a746edc42fe8d53669965138f39ee117414c7d7a340cfe/propcache-0.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c80ee5802e3fb9ea37938e7eecc307fb984837091d5fd262bb37238b1ae97641", size = 206981, upload-time = "2025-10-08T19:47:55.715Z" }, + { url = "https://files.pythonhosted.org/packages/df/f6/c5fa1357cc9748510ee55f37173eb31bfde6d94e98ccd9e6f033f2fc06e1/propcache-0.4.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ed5a841e8bb29a55fb8159ed526b26adc5bdd7e8bd7bf793ce647cb08656cdf4", size = 211490, upload-time = "2025-10-08T19:47:57.499Z" }, + { url = "https://files.pythonhosted.org/packages/80/1e/e5889652a7c4a3846683401a48f0f2e5083ce0ec1a8a5221d8058fbd1adf/propcache-0.4.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:55c72fd6ea2da4c318e74ffdf93c4fe4e926051133657459131a95c846d16d44", size = 215371, upload-time = "2025-10-08T19:47:59.317Z" }, + { url = "https://files.pythonhosted.org/packages/b2/f2/889ad4b2408f72fe1a4f6a19491177b30ea7bf1a0fd5f17050ca08cfc882/propcache-0.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8326e144341460402713f91df60ade3c999d601e7eb5ff8f6f7862d54de0610d", size = 201424, upload-time = "2025-10-08T19:48:00.67Z" }, + { url = "https://files.pythonhosted.org/packages/27/73/033d63069b57b0812c8bd19f311faebeceb6ba31b8f32b73432d12a0b826/propcache-0.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:060b16ae65bc098da7f6d25bf359f1f31f688384858204fe5d652979e0015e5b", size = 197566, upload-time = "2025-10-08T19:48:02.604Z" }, + { url = "https://files.pythonhosted.org/packages/dc/89/ce24f3dc182630b4e07aa6d15f0ff4b14ed4b9955fae95a0b54c58d66c05/propcache-0.4.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:89eb3fa9524f7bec9de6e83cf3faed9d79bffa560672c118a96a171a6f55831e", size = 193130, upload-time = "2025-10-08T19:48:04.499Z" }, + { url = "https://files.pythonhosted.org/packages/a9/24/ef0d5fd1a811fb5c609278d0209c9f10c35f20581fcc16f818da959fc5b4/propcache-0.4.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:dee69d7015dc235f526fe80a9c90d65eb0039103fe565776250881731f06349f", size = 202625, upload-time = "2025-10-08T19:48:06.213Z" }, + { url = "https://files.pythonhosted.org/packages/f5/02/98ec20ff5546f68d673df2f7a69e8c0d076b5abd05ca882dc7ee3a83653d/propcache-0.4.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:5558992a00dfd54ccbc64a32726a3357ec93825a418a401f5cc67df0ac5d9e49", size = 204209, upload-time = "2025-10-08T19:48:08.432Z" }, + { url = "https://files.pythonhosted.org/packages/a0/87/492694f76759b15f0467a2a93ab68d32859672b646aa8a04ce4864e7932d/propcache-0.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:c9b822a577f560fbd9554812526831712c1436d2c046cedee4c3796d3543b144", size = 197797, upload-time = "2025-10-08T19:48:09.968Z" }, + { url = "https://files.pythonhosted.org/packages/99/85/9ff785d787ccf9bbb3f3106f79884a130951436f58392000231b4c737c80/propcache-0.4.1-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:824e908bce90fb2743bd6b59db36eb4f45cd350a39637c9f73b1c1ea66f5b75f", size = 81455, upload-time = "2025-10-08T19:48:15.16Z" }, + { url = "https://files.pythonhosted.org/packages/90/85/2431c10c8e7ddb1445c1f7c4b54d886e8ad20e3c6307e7218f05922cad67/propcache-0.4.1-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:c2b5e7db5328427c57c8e8831abda175421b709672f6cfc3d630c3b7e2146393", size = 46372, upload-time = "2025-10-08T19:48:16.424Z" }, + { url = "https://files.pythonhosted.org/packages/01/20/b0972d902472da9bcb683fa595099911f4d2e86e5683bcc45de60dd05dc3/propcache-0.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:6f6ff873ed40292cd4969ef5310179afd5db59fdf055897e282485043fc80ad0", size = 48411, upload-time = "2025-10-08T19:48:17.577Z" }, + { url = "https://files.pythonhosted.org/packages/e2/e3/7dc89f4f21e8f99bad3d5ddb3a3389afcf9da4ac69e3deb2dcdc96e74169/propcache-0.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:49a2dc67c154db2c1463013594c458881a069fcf98940e61a0569016a583020a", size = 275712, upload-time = "2025-10-08T19:48:18.901Z" }, + { url = "https://files.pythonhosted.org/packages/20/67/89800c8352489b21a8047c773067644e3897f02ecbbd610f4d46b7f08612/propcache-0.4.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:005f08e6a0529984491e37d8dbc3dd86f84bd78a8ceb5fa9a021f4c48d4984be", size = 273557, upload-time = "2025-10-08T19:48:20.762Z" }, + { url = "https://files.pythonhosted.org/packages/e2/a1/b52b055c766a54ce6d9c16d9aca0cad8059acd9637cdf8aa0222f4a026ef/propcache-0.4.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5c3310452e0d31390da9035c348633b43d7e7feb2e37be252be6da45abd1abcc", size = 280015, upload-time = "2025-10-08T19:48:22.592Z" }, + { url = "https://files.pythonhosted.org/packages/48/c8/33cee30bd890672c63743049f3c9e4be087e6780906bfc3ec58528be59c1/propcache-0.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4c3c70630930447f9ef1caac7728c8ad1c56bc5015338b20fed0d08ea2480b3a", size = 262880, upload-time = "2025-10-08T19:48:23.947Z" }, + { url = "https://files.pythonhosted.org/packages/0c/b1/8f08a143b204b418285c88b83d00edbd61afbc2c6415ffafc8905da7038b/propcache-0.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8e57061305815dfc910a3634dcf584f08168a8836e6999983569f51a8544cd89", size = 260938, upload-time = "2025-10-08T19:48:25.656Z" }, + { url = "https://files.pythonhosted.org/packages/cf/12/96e4664c82ca2f31e1c8dff86afb867348979eb78d3cb8546a680287a1e9/propcache-0.4.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:521a463429ef54143092c11a77e04056dd00636f72e8c45b70aaa3140d639726", size = 247641, upload-time = "2025-10-08T19:48:27.207Z" }, + { url = "https://files.pythonhosted.org/packages/18/ed/e7a9cfca28133386ba52278136d42209d3125db08d0a6395f0cba0c0285c/propcache-0.4.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:120c964da3fdc75e3731aa392527136d4ad35868cc556fd09bb6d09172d9a367", size = 262510, upload-time = "2025-10-08T19:48:28.65Z" }, + { url = "https://files.pythonhosted.org/packages/f5/76/16d8bf65e8845dd62b4e2b57444ab81f07f40caa5652b8969b87ddcf2ef6/propcache-0.4.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:d8f353eb14ee3441ee844ade4277d560cdd68288838673273b978e3d6d2c8f36", size = 263161, upload-time = "2025-10-08T19:48:30.133Z" }, + { url = "https://files.pythonhosted.org/packages/e7/70/c99e9edb5d91d5ad8a49fa3c1e8285ba64f1476782fed10ab251ff413ba1/propcache-0.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ab2943be7c652f09638800905ee1bab2c544e537edb57d527997a24c13dc1455", size = 257393, upload-time = "2025-10-08T19:48:31.567Z" }, + { url = "https://files.pythonhosted.org/packages/5b/5a/bc7b4a4ef808fa59a816c17b20c4bef6884daebbdf627ff2a161da67da19/propcache-0.4.1-py3-none-any.whl", hash = "sha256:af2a6052aeb6cf17d3e46ee169099044fd8224cbaf75c76a2ef596e8163e2237", size = 13305, upload-time = "2025-10-08T19:49:00.792Z" }, +] + +[[package]] +name = "protobuf" +version = "6.33.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/19/ff/64a6c8f420818bb873713988ca5492cba3a7946be57e027ac63495157d97/protobuf-6.33.0.tar.gz", hash = "sha256:140303d5c8d2037730c548f8c7b93b20bb1dc301be280c378b82b8894589c954", size = 443463, upload-time = "2025-10-15T20:39:52.159Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e1/a9/b6eee662a6951b9c3640e8e452ab3e09f117d99fc10baa32d1581a0d4099/protobuf-6.33.0-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:905b07a65f1a4b72412314082c7dbfae91a9e8b68a0cc1577515f8df58ecf455", size = 427521, upload-time = "2025-10-15T20:39:43.803Z" }, + { url = "https://files.pythonhosted.org/packages/10/35/16d31e0f92c6d2f0e77c2a3ba93185130ea13053dd16200a57434c882f2b/protobuf-6.33.0-cp39-abi3-manylinux2014_aarch64.whl", hash = "sha256:e0697ece353e6239b90ee43a9231318302ad8353c70e6e45499fa52396debf90", size = 324445, upload-time = "2025-10-15T20:39:44.932Z" }, + { url = "https://files.pythonhosted.org/packages/e6/eb/2a981a13e35cda8b75b5585aaffae2eb904f8f351bdd3870769692acbd8a/protobuf-6.33.0-cp39-abi3-manylinux2014_s390x.whl", hash = "sha256:e0a1715e4f27355afd9570f3ea369735afc853a6c3951a6afe1f80d8569ad298", size = 339159, upload-time = "2025-10-15T20:39:46.186Z" }, + { url = "https://files.pythonhosted.org/packages/21/51/0b1cbad62074439b867b4e04cc09b93f6699d78fd191bed2bbb44562e077/protobuf-6.33.0-cp39-abi3-manylinux2014_x86_64.whl", hash = "sha256:35be49fd3f4fefa4e6e2aacc35e8b837d6703c37a2168a55ac21e9b1bc7559ef", size = 323172, upload-time = "2025-10-15T20:39:47.465Z" }, + { url = "https://files.pythonhosted.org/packages/07/d1/0a28c21707807c6aacd5dc9c3704b2aa1effbf37adebd8caeaf68b17a636/protobuf-6.33.0-py3-none-any.whl", hash = "sha256:25c9e1963c6734448ea2d308cfa610e692b801304ba0908d7bfa564ac5132995", size = 170477, upload-time = "2025-10-15T20:39:51.311Z" }, +] + +[[package]] +name = "psutil" +version = "7.1.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e1/88/bdd0a41e5857d5d703287598cbf08dad90aed56774ea52ae071bae9071b6/psutil-7.1.3.tar.gz", hash = "sha256:6c86281738d77335af7aec228328e944b30930899ea760ecf33a4dba66be5e74", size = 489059, upload-time = "2025-11-02T12:25:54.619Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bd/93/0c49e776b8734fef56ec9c5c57f923922f2cf0497d62e0f419465f28f3d0/psutil-7.1.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0005da714eee687b4b8decd3d6cc7c6db36215c9e74e5ad2264b90c3df7d92dc", size = 239751, upload-time = "2025-11-02T12:25:58.161Z" }, + { url = "https://files.pythonhosted.org/packages/6f/8d/b31e39c769e70780f007969815195a55c81a63efebdd4dbe9e7a113adb2f/psutil-7.1.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:19644c85dcb987e35eeeaefdc3915d059dac7bd1167cdcdbf27e0ce2df0c08c0", size = 240368, upload-time = "2025-11-02T12:26:00.491Z" }, + { url = "https://files.pythonhosted.org/packages/62/61/23fd4acc3c9eebbf6b6c78bcd89e5d020cfde4acf0a9233e9d4e3fa698b4/psutil-7.1.3-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:95ef04cf2e5ba0ab9eaafc4a11eaae91b44f4ef5541acd2ee91d9108d00d59a7", size = 287134, upload-time = "2025-11-02T12:26:02.613Z" }, + { url = "https://files.pythonhosted.org/packages/30/1c/f921a009ea9ceb51aa355cb0cc118f68d354db36eae18174bab63affb3e6/psutil-7.1.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1068c303be3a72f8e18e412c5b2a8f6d31750fb152f9cb106b54090296c9d251", size = 289904, upload-time = "2025-11-02T12:26:05.207Z" }, + { url = "https://files.pythonhosted.org/packages/2e/bb/6670bded3e3236eb4287c7bcdc167e9fae6e1e9286e437f7111caed2f909/psutil-7.1.3-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:b403da1df4d6d43973dc004d19cee3b848e998ae3154cc8097d139b77156c353", size = 239843, upload-time = "2025-11-02T12:26:11.968Z" }, + { url = "https://files.pythonhosted.org/packages/b8/66/853d50e75a38c9a7370ddbeefabdd3d3116b9c31ef94dc92c6729bc36bec/psutil-7.1.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ad81425efc5e75da3f39b3e636293360ad8d0b49bed7df824c79764fb4ba9b8b", size = 240369, upload-time = "2025-11-02T12:26:14.358Z" }, + { url = "https://files.pythonhosted.org/packages/41/bd/313aba97cb5bfb26916dc29cf0646cbe4dd6a89ca69e8c6edce654876d39/psutil-7.1.3-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8f33a3702e167783a9213db10ad29650ebf383946e91bc77f28a5eb083496bc9", size = 288210, upload-time = "2025-11-02T12:26:16.699Z" }, + { url = "https://files.pythonhosted.org/packages/c2/fa/76e3c06e760927a0cfb5705eb38164254de34e9bd86db656d4dbaa228b04/psutil-7.1.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fac9cd332c67f4422504297889da5ab7e05fd11e3c4392140f7370f4208ded1f", size = 291182, upload-time = "2025-11-02T12:26:18.848Z" }, + { url = "https://files.pythonhosted.org/packages/ef/94/46b9154a800253e7ecff5aaacdf8ebf43db99de4a2dfa18575b02548654e/psutil-7.1.3-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:2bdbcd0e58ca14996a42adf3621a6244f1bb2e2e528886959c72cf1e326677ab", size = 238359, upload-time = "2025-11-02T12:26:25.284Z" }, + { url = "https://files.pythonhosted.org/packages/68/3a/9f93cff5c025029a36d9a92fef47220ab4692ee7f2be0fba9f92813d0cb8/psutil-7.1.3-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:bc31fa00f1fbc3c3802141eede66f3a2d51d89716a194bf2cd6fc68310a19880", size = 239171, upload-time = "2025-11-02T12:26:27.23Z" }, + { url = "https://files.pythonhosted.org/packages/ce/b1/5f49af514f76431ba4eea935b8ad3725cdeb397e9245ab919dbc1d1dc20f/psutil-7.1.3-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3bb428f9f05c1225a558f53e30ccbad9930b11c3fc206836242de1091d3e7dd3", size = 263261, upload-time = "2025-11-02T12:26:29.48Z" }, + { url = "https://files.pythonhosted.org/packages/e0/95/992c8816a74016eb095e73585d747e0a8ea21a061ed3689474fabb29a395/psutil-7.1.3-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:56d974e02ca2c8eb4812c3f76c30e28836fffc311d55d979f1465c1feeb2b68b", size = 264635, upload-time = "2025-11-02T12:26:31.74Z" }, +] + +[[package]] +name = "pycparser" +version = "2.23" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/fe/cf/d2d3b9f5699fb1e4615c8e32ff220203e43b248e1dfcc6736ad9057731ca/pycparser-2.23.tar.gz", hash = "sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2", size = 173734, upload-time = "2025-09-09T13:23:47.91Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/e3/59cd50310fc9b59512193629e1984c1f95e5c8ae6e5d8c69532ccc65a7fe/pycparser-2.23-py3-none-any.whl", hash = "sha256:e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934", size = 118140, upload-time = "2025-09-09T13:23:46.651Z" }, +] + +[[package]] +name = "pydantic" +version = "2.12.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "annotated-types", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pydantic-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "typing-inspection", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f3/1e/4f0a3233767010308f2fd6bd0814597e3f63f1dc98304a9112b8759df4ff/pydantic-2.12.3.tar.gz", hash = "sha256:1da1c82b0fc140bb0103bc1441ffe062154c8d38491189751ee00fd8ca65ce74", size = 819383, upload-time = "2025-10-17T15:04:21.222Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a1/6b/83661fa77dcefa195ad5f8cd9af3d1a7450fd57cc883ad04d65446ac2029/pydantic-2.12.3-py3-none-any.whl", hash = "sha256:6986454a854bc3bc6e5443e1369e06a3a456af9d339eda45510f517d9ea5c6bf", size = 462431, upload-time = "2025-10-17T15:04:19.346Z" }, +] + +[[package]] +name = "pydantic-core" +version = "2.41.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/18/d0944e8eaaa3efd0a91b0f1fc537d3be55ad35091b6a87638211ba691964/pydantic_core-2.41.4.tar.gz", hash = "sha256:70e47929a9d4a1905a67e4b687d5946026390568a8e952b92824118063cee4d5", size = 457557, upload-time = "2025-10-14T10:23:47.909Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/13/d0/c20adabd181a029a970738dfe23710b52a31f1258f591874fcdec7359845/pydantic_core-2.41.4-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:85e050ad9e5f6fe1004eec65c914332e52f429bc0ae12d6fa2092407a462c746", size = 2105688, upload-time = "2025-10-14T10:20:54.448Z" }, + { url = "https://files.pythonhosted.org/packages/00/b6/0ce5c03cec5ae94cca220dfecddc453c077d71363b98a4bbdb3c0b22c783/pydantic_core-2.41.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:e7393f1d64792763a48924ba31d1e44c2cfbc05e3b1c2c9abb4ceeadd912cced", size = 1910807, upload-time = "2025-10-14T10:20:56.115Z" }, + { url = "https://files.pythonhosted.org/packages/68/3e/800d3d02c8beb0b5c069c870cbb83799d085debf43499c897bb4b4aaff0d/pydantic_core-2.41.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:94dab0940b0d1fb28bcab847adf887c66a27a40291eedf0b473be58761c9799a", size = 1956669, upload-time = "2025-10-14T10:20:57.874Z" }, + { url = "https://files.pythonhosted.org/packages/60/a4/24271cc71a17f64589be49ab8bd0751f6a0a03046c690df60989f2f95c2c/pydantic_core-2.41.4-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:de7c42f897e689ee6f9e93c4bec72b99ae3b32a2ade1c7e4798e690ff5246e02", size = 2051629, upload-time = "2025-10-14T10:21:00.006Z" }, + { url = "https://files.pythonhosted.org/packages/68/de/45af3ca2f175d91b96bfb62e1f2d2f1f9f3b14a734afe0bfeff079f78181/pydantic_core-2.41.4-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:664b3199193262277b8b3cd1e754fb07f2c6023289c815a1e1e8fb415cb247b1", size = 2224049, upload-time = "2025-10-14T10:21:01.801Z" }, + { url = "https://files.pythonhosted.org/packages/af/8f/ae4e1ff84672bf869d0a77af24fd78387850e9497753c432875066b5d622/pydantic_core-2.41.4-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d95b253b88f7d308b1c0b417c4624f44553ba4762816f94e6986819b9c273fb2", size = 2342409, upload-time = "2025-10-14T10:21:03.556Z" }, + { url = "https://files.pythonhosted.org/packages/18/62/273dd70b0026a085c7b74b000394e1ef95719ea579c76ea2f0cc8893736d/pydantic_core-2.41.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a1351f5bbdbbabc689727cb91649a00cb9ee7203e0a6e54e9f5ba9e22e384b84", size = 2069635, upload-time = "2025-10-14T10:21:05.385Z" }, + { url = "https://files.pythonhosted.org/packages/30/03/cf485fff699b4cdaea469bc481719d3e49f023241b4abb656f8d422189fc/pydantic_core-2.41.4-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1affa4798520b148d7182da0615d648e752de4ab1a9566b7471bc803d88a062d", size = 2194284, upload-time = "2025-10-14T10:21:07.122Z" }, + { url = "https://files.pythonhosted.org/packages/f9/7e/c8e713db32405dfd97211f2fc0a15d6bf8adb7640f3d18544c1f39526619/pydantic_core-2.41.4-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:7b74e18052fea4aa8dea2fb7dbc23d15439695da6cbe6cfc1b694af1115df09d", size = 2137566, upload-time = "2025-10-14T10:21:08.981Z" }, + { url = "https://files.pythonhosted.org/packages/04/f7/db71fd4cdccc8b75990f79ccafbbd66757e19f6d5ee724a6252414483fb4/pydantic_core-2.41.4-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:285b643d75c0e30abda9dc1077395624f314a37e3c09ca402d4015ef5979f1a2", size = 2316809, upload-time = "2025-10-14T10:21:10.805Z" }, + { url = "https://files.pythonhosted.org/packages/76/63/a54973ddb945f1bca56742b48b144d85c9fc22f819ddeb9f861c249d5464/pydantic_core-2.41.4-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:f52679ff4218d713b3b33f88c89ccbf3a5c2c12ba665fb80ccc4192b4608dbab", size = 2311119, upload-time = "2025-10-14T10:21:12.583Z" }, + { url = "https://files.pythonhosted.org/packages/36/0d/b5706cacb70a8414396efdda3d72ae0542e050b591119e458e2490baf035/pydantic_core-2.41.4-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:ed97fd56a561f5eb5706cebe94f1ad7c13b84d98312a05546f2ad036bafe87f4", size = 1877324, upload-time = "2025-10-14T10:21:20.363Z" }, + { url = "https://files.pythonhosted.org/packages/de/2d/cba1fa02cfdea72dfb3a9babb067c83b9dff0bbcb198368e000a6b756ea7/pydantic_core-2.41.4-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a870c307bf1ee91fc58a9a61338ff780d01bfae45922624816878dce784095d2", size = 1884515, upload-time = "2025-10-14T10:21:22.339Z" }, + { url = "https://files.pythonhosted.org/packages/07/ea/3df927c4384ed9b503c9cc2d076cf983b4f2adb0c754578dfb1245c51e46/pydantic_core-2.41.4-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d25e97bc1f5f8f7985bdc2335ef9e73843bb561eb1fa6831fdfc295c1c2061cf", size = 2042819, upload-time = "2025-10-14T10:21:26.683Z" }, + { url = "https://files.pythonhosted.org/packages/54/28/d3325da57d413b9819365546eb9a6e8b7cbd9373d9380efd5f74326143e6/pydantic_core-2.41.4-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:e9205d97ed08a82ebb9a307e92914bb30e18cdf6f6b12ca4bedadb1588a0bfe1", size = 2102022, upload-time = "2025-10-14T10:21:32.809Z" }, + { url = "https://files.pythonhosted.org/packages/9e/24/b58a1bc0d834bf1acc4361e61233ee217169a42efbdc15a60296e13ce438/pydantic_core-2.41.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:82df1f432b37d832709fbcc0e24394bba04a01b6ecf1ee87578145c19cde12ac", size = 1905495, upload-time = "2025-10-14T10:21:34.812Z" }, + { url = "https://files.pythonhosted.org/packages/fb/a4/71f759cc41b7043e8ecdaab81b985a9b6cad7cec077e0b92cff8b71ecf6b/pydantic_core-2.41.4-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fc3b4cc4539e055cfa39a3763c939f9d409eb40e85813257dcd761985a108554", size = 1956131, upload-time = "2025-10-14T10:21:36.924Z" }, + { url = "https://files.pythonhosted.org/packages/b0/64/1e79ac7aa51f1eec7c4cda8cbe456d5d09f05fdd68b32776d72168d54275/pydantic_core-2.41.4-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b1eb1754fce47c63d2ff57fdb88c351a6c0150995890088b33767a10218eaa4e", size = 2052236, upload-time = "2025-10-14T10:21:38.927Z" }, + { url = "https://files.pythonhosted.org/packages/e9/e3/a3ffc363bd4287b80f1d43dc1c28ba64831f8dfc237d6fec8f2661138d48/pydantic_core-2.41.4-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e6ab5ab30ef325b443f379ddb575a34969c333004fca5a1daa0133a6ffaad616", size = 2223573, upload-time = "2025-10-14T10:21:41.574Z" }, + { url = "https://files.pythonhosted.org/packages/28/27/78814089b4d2e684a9088ede3790763c64693c3d1408ddc0a248bc789126/pydantic_core-2.41.4-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:31a41030b1d9ca497634092b46481b937ff9397a86f9f51bd41c4767b6fc04af", size = 2342467, upload-time = "2025-10-14T10:21:44.018Z" }, + { url = "https://files.pythonhosted.org/packages/92/97/4de0e2a1159cb85ad737e03306717637842c88c7fd6d97973172fb183149/pydantic_core-2.41.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a44ac1738591472c3d020f61c6df1e4015180d6262ebd39bf2aeb52571b60f12", size = 2063754, upload-time = "2025-10-14T10:21:46.466Z" }, + { url = "https://files.pythonhosted.org/packages/0f/50/8cb90ce4b9efcf7ae78130afeb99fd1c86125ccdf9906ef64b9d42f37c25/pydantic_core-2.41.4-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d72f2b5e6e82ab8f94ea7d0d42f83c487dc159c5240d8f83beae684472864e2d", size = 2196754, upload-time = "2025-10-14T10:21:48.486Z" }, + { url = "https://files.pythonhosted.org/packages/34/3b/ccdc77af9cd5082723574a1cc1bcae7a6acacc829d7c0a06201f7886a109/pydantic_core-2.41.4-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:c4d1e854aaf044487d31143f541f7aafe7b482ae72a022c664b2de2e466ed0ad", size = 2137115, upload-time = "2025-10-14T10:21:50.63Z" }, + { url = "https://files.pythonhosted.org/packages/ca/ba/e7c7a02651a8f7c52dc2cff2b64a30c313e3b57c7d93703cecea76c09b71/pydantic_core-2.41.4-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:b568af94267729d76e6ee5ececda4e283d07bbb28e8148bb17adad93d025d25a", size = 2317400, upload-time = "2025-10-14T10:21:52.959Z" }, + { url = "https://files.pythonhosted.org/packages/2c/ba/6c533a4ee8aec6b812c643c49bb3bd88d3f01e3cebe451bb85512d37f00f/pydantic_core-2.41.4-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:6d55fb8b1e8929b341cc313a81a26e0d48aa3b519c1dbaadec3a6a2b4fcad025", size = 2312070, upload-time = "2025-10-14T10:21:55.419Z" }, + { url = "https://files.pythonhosted.org/packages/0d/c2/472f2e31b95eff099961fa050c376ab7156a81da194f9edb9f710f68787b/pydantic_core-2.41.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:6c1fe4c5404c448b13188dd8bd2ebc2bdd7e6727fa61ff481bcc2cca894018da", size = 1876904, upload-time = "2025-10-14T10:22:04.062Z" }, + { url = "https://files.pythonhosted.org/packages/4a/07/ea8eeb91173807ecdae4f4a5f4b150a520085b35454350fc219ba79e66a3/pydantic_core-2.41.4-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:523e7da4d43b113bf8e7b49fa4ec0c35bf4fe66b2230bfc5c13cc498f12c6c3e", size = 1882538, upload-time = "2025-10-14T10:22:06.39Z" }, + { url = "https://files.pythonhosted.org/packages/1e/29/b53a9ca6cd366bfc928823679c6a76c7a4c69f8201c0ba7903ad18ebae2f/pydantic_core-2.41.4-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5729225de81fb65b70fdb1907fcf08c75d498f4a6f15af005aabb1fdadc19dfa", size = 2041183, upload-time = "2025-10-14T10:22:08.812Z" }, +] + +[[package]] +name = "pygments" +version = "2.19.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" }, +] + +[[package]] +name = "pytest" +version = "8.4.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "iniconfig", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "packaging", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pluggy", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pygments", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a3/5c/00a0e072241553e1a7496d638deababa67c5058571567b92a7eaa258397c/pytest-8.4.2.tar.gz", hash = "sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01", size = 1519618, upload-time = "2025-09-04T14:34:22.711Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a8/a4/20da314d277121d6534b3a980b29035dcd51e6744bd79075a6ce8fa4eb8d/pytest-8.4.2-py3-none-any.whl", hash = "sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79", size = 365750, upload-time = "2025-09-04T14:34:20.226Z" }, +] + +[[package]] +name = "pytest-asyncio" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/86/9e3c5f48f7b7b638b216e4b9e645f54d199d7abbbab7a64a13b4e12ba10f/pytest_asyncio-1.2.0.tar.gz", hash = "sha256:c609a64a2a8768462d0c99811ddb8bd2583c33fd33cf7f21af1c142e824ffb57", size = 50119, upload-time = "2025-09-12T07:33:53.816Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/93/2fa34714b7a4ae72f2f8dad66ba17dd9a2c793220719e736dda28b7aec27/pytest_asyncio-1.2.0-py3-none-any.whl", hash = "sha256:8e17ae5e46d8e7efe51ab6494dd2010f4ca8dae51652aa3c8d55acf50bfb2e99", size = 15095, upload-time = "2025-09-12T07:33:52.639Z" }, +] + +[[package]] +name = "pytest-env" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/13/12/9c87d0ca45d5992473208bcef2828169fa7d39b8d7fc6e3401f5c08b8bf7/pytest_env-1.2.0.tar.gz", hash = "sha256:475e2ebe8626cee01f491f304a74b12137742397d6c784ea4bc258f069232b80", size = 8973, upload-time = "2025-10-09T19:15:47.42Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/98/822b924a4a3eb58aacba84444c7439fce32680592f394de26af9c76e2569/pytest_env-1.2.0-py3-none-any.whl", hash = "sha256:d7e5b7198f9b83c795377c09feefa45d56083834e60d04767efd64819fc9da00", size = 6251, upload-time = "2025-10-09T19:15:46.077Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, +] + +[[package]] +name = "regex" +version = "2025.11.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cc/a9/546676f25e573a4cf00fe8e119b78a37b6a8fe2dc95cda877b30889c9c45/regex-2025.11.3.tar.gz", hash = "sha256:1fedc720f9bb2494ce31a58a1631f9c82df6a09b49c19517ea5cc280b4541e01", size = 414669, upload-time = "2025-11-03T21:34:22.089Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e1/a7/dda24ebd49da46a197436ad96378f17df30ceb40e52e859fc42cac45b850/regex-2025.11.3-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:c1e448051717a334891f2b9a620fe36776ebf3dd8ec46a0b877c8ae69575feb4", size = 489081, upload-time = "2025-11-03T21:31:55.9Z" }, + { url = "https://files.pythonhosted.org/packages/19/22/af2dc751aacf88089836aa088a1a11c4f21a04707eb1b0478e8e8fb32847/regex-2025.11.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:9b5aca4d5dfd7fbfbfbdaf44850fcc7709a01146a797536a8f84952e940cca76", size = 291123, upload-time = "2025-11-03T21:31:57.758Z" }, + { url = "https://files.pythonhosted.org/packages/a3/88/1a3ea5672f4b0a84802ee9891b86743438e7c04eb0b8f8c4e16a42375327/regex-2025.11.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:04d2765516395cf7dda331a244a3282c0f5ae96075f728629287dfa6f76ba70a", size = 288814, upload-time = "2025-11-03T21:32:01.12Z" }, + { url = "https://files.pythonhosted.org/packages/fb/8c/f5987895bf42b8ddeea1b315c9fedcfe07cadee28b9c98cf50d00adcb14d/regex-2025.11.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5d9903ca42bfeec4cebedba8022a7c97ad2aab22e09573ce9976ba01b65e4361", size = 798592, upload-time = "2025-11-03T21:32:03.006Z" }, + { url = "https://files.pythonhosted.org/packages/99/2a/6591ebeede78203fa77ee46a1c36649e02df9eaa77a033d1ccdf2fcd5d4e/regex-2025.11.3-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:639431bdc89d6429f6721625e8129413980ccd62e9d3f496be618a41d205f160", size = 864122, upload-time = "2025-11-03T21:32:04.553Z" }, + { url = "https://files.pythonhosted.org/packages/94/d6/be32a87cf28cf8ed064ff281cfbd49aefd90242a83e4b08b5a86b38e8eb4/regex-2025.11.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f117efad42068f9715677c8523ed2be1518116d1c49b1dd17987716695181efe", size = 912272, upload-time = "2025-11-03T21:32:06.148Z" }, + { url = "https://files.pythonhosted.org/packages/62/11/9bcef2d1445665b180ac7f230406ad80671f0fc2a6ffb93493b5dd8cd64c/regex-2025.11.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4aecb6f461316adf9f1f0f6a4a1a3d79e045f9b71ec76055a791affa3b285850", size = 803497, upload-time = "2025-11-03T21:32:08.162Z" }, + { url = "https://files.pythonhosted.org/packages/e5/a7/da0dc273d57f560399aa16d8a68ae7f9b57679476fc7ace46501d455fe84/regex-2025.11.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:3b3a5f320136873cc5561098dfab677eea139521cb9a9e8db98b7e64aef44cbc", size = 787892, upload-time = "2025-11-03T21:32:09.769Z" }, + { url = "https://files.pythonhosted.org/packages/da/4b/732a0c5a9736a0b8d6d720d4945a2f1e6f38f87f48f3173559f53e8d5d82/regex-2025.11.3-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:75fa6f0056e7efb1f42a1c34e58be24072cb9e61a601340cc1196ae92326a4f9", size = 858462, upload-time = "2025-11-03T21:32:11.769Z" }, + { url = "https://files.pythonhosted.org/packages/0c/f5/a2a03df27dc4c2d0c769220f5110ba8c4084b0bfa9ab0f9b4fcfa3d2b0fc/regex-2025.11.3-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:dbe6095001465294f13f1adcd3311e50dd84e5a71525f20a10bd16689c61ce0b", size = 850528, upload-time = "2025-11-03T21:32:13.906Z" }, + { url = "https://files.pythonhosted.org/packages/d6/09/e1cd5bee3841c7f6eb37d95ca91cdee7100b8f88b81e41c2ef426910891a/regex-2025.11.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:454d9b4ae7881afbc25015b8627c16d88a597479b9dea82b8c6e7e2e07240dc7", size = 789866, upload-time = "2025-11-03T21:32:15.748Z" }, + { url = "https://files.pythonhosted.org/packages/20/28/fd0c63357caefe5680b8ea052131acbd7f456893b69cc2a90cc3e0dc90d4/regex-2025.11.3-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:1eb1ebf6822b756c723e09f5186473d93236c06c579d2cc0671a722d2ab14281", size = 491984, upload-time = "2025-11-03T21:32:23.466Z" }, + { url = "https://files.pythonhosted.org/packages/df/ec/7014c15626ab46b902b3bcc4b28a7bae46d8f281fc7ea9c95e22fcaaa917/regex-2025.11.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:1e00ec2970aab10dc5db34af535f21fcf32b4a31d99e34963419636e2f85ae39", size = 292673, upload-time = "2025-11-03T21:32:25.034Z" }, + { url = "https://files.pythonhosted.org/packages/23/ab/3b952ff7239f20d05f1f99e9e20188513905f218c81d52fb5e78d2bf7634/regex-2025.11.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:a4cb042b615245d5ff9b3794f56be4138b5adc35a4166014d31d1814744148c7", size = 291029, upload-time = "2025-11-03T21:32:26.528Z" }, + { url = "https://files.pythonhosted.org/packages/21/7e/3dc2749fc684f455f162dcafb8a187b559e2614f3826877d3844a131f37b/regex-2025.11.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:44f264d4bf02f3176467d90b294d59bf1db9fe53c141ff772f27a8b456b2a9ed", size = 807437, upload-time = "2025-11-03T21:32:28.363Z" }, + { url = "https://files.pythonhosted.org/packages/1b/0b/d529a85ab349c6a25d1ca783235b6e3eedf187247eab536797021f7126c6/regex-2025.11.3-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7be0277469bf3bd7a34a9c57c1b6a724532a0d235cd0dc4e7f4316f982c28b19", size = 873368, upload-time = "2025-11-03T21:32:30.4Z" }, + { url = "https://files.pythonhosted.org/packages/7d/18/2d868155f8c9e3e9d8f9e10c64e9a9f496bb8f7e037a88a8bed26b435af6/regex-2025.11.3-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0d31e08426ff4b5b650f68839f5af51a92a5b51abd8554a60c2fbc7c71f25d0b", size = 914921, upload-time = "2025-11-03T21:32:32.123Z" }, + { url = "https://files.pythonhosted.org/packages/2d/71/9d72ff0f354fa783fe2ba913c8734c3b433b86406117a8db4ea2bf1c7a2f/regex-2025.11.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e43586ce5bd28f9f285a6e729466841368c4a0353f6fd08d4ce4630843d3648a", size = 812708, upload-time = "2025-11-03T21:32:34.305Z" }, + { url = "https://files.pythonhosted.org/packages/e7/19/ce4bf7f5575c97f82b6e804ffb5c4e940c62609ab2a0d9538d47a7fdf7d4/regex-2025.11.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:0f9397d561a4c16829d4e6ff75202c1c08b68a3bdbfe29dbfcdb31c9830907c6", size = 795472, upload-time = "2025-11-03T21:32:36.364Z" }, + { url = "https://files.pythonhosted.org/packages/03/86/fd1063a176ffb7b2315f9a1b08d17b18118b28d9df163132615b835a26ee/regex-2025.11.3-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:dd16e78eb18ffdb25ee33a0682d17912e8cc8a770e885aeee95020046128f1ce", size = 868341, upload-time = "2025-11-03T21:32:38.042Z" }, + { url = "https://files.pythonhosted.org/packages/12/43/103fb2e9811205e7386366501bc866a164a0430c79dd59eac886a2822950/regex-2025.11.3-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:ffcca5b9efe948ba0661e9df0fa50d2bc4b097c70b9810212d6b62f05d83b2dd", size = 854666, upload-time = "2025-11-03T21:32:40.079Z" }, + { url = "https://files.pythonhosted.org/packages/7d/22/e392e53f3869b75804762c7c848bd2dd2abf2b70fb0e526f58724638bd35/regex-2025.11.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c56b4d162ca2b43318ac671c65bd4d563e841a694ac70e1a976ac38fcf4ca1d2", size = 799473, upload-time = "2025-11-03T21:32:42.148Z" }, + { url = "https://files.pythonhosted.org/packages/31/e9/f6e13de7e0983837f7b6d238ad9458800a874bf37c264f7923e63409944c/regex-2025.11.3-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:9697a52e57576c83139d7c6f213d64485d3df5bf84807c35fa409e6c970801c6", size = 489089, upload-time = "2025-11-03T21:32:50.027Z" }, + { url = "https://files.pythonhosted.org/packages/a3/5c/261f4a262f1fa65141c1b74b255988bd2fa020cc599e53b080667d591cfc/regex-2025.11.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:e18bc3f73bd41243c9b38a6d9f2366cd0e0137a9aebe2d8ff76c5b67d4c0a3f4", size = 291059, upload-time = "2025-11-03T21:32:51.682Z" }, + { url = "https://files.pythonhosted.org/packages/8e/57/f14eeb7f072b0e9a5a090d1712741fd8f214ec193dba773cf5410108bb7d/regex-2025.11.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:61a08bcb0ec14ff4e0ed2044aad948d0659604f824cbd50b55e30b0ec6f09c73", size = 288900, upload-time = "2025-11-03T21:32:53.569Z" }, + { url = "https://files.pythonhosted.org/packages/3c/6b/1d650c45e99a9b327586739d926a1cd4e94666b1bd4af90428b36af66dc7/regex-2025.11.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c9c30003b9347c24bcc210958c5d167b9e4f9be786cb380a7d32f14f9b84674f", size = 799010, upload-time = "2025-11-03T21:32:55.222Z" }, + { url = "https://files.pythonhosted.org/packages/99/ee/d66dcbc6b628ce4e3f7f0cbbb84603aa2fc0ffc878babc857726b8aab2e9/regex-2025.11.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4e1e592789704459900728d88d41a46fe3969b82ab62945560a31732ffc19a6d", size = 864893, upload-time = "2025-11-03T21:32:57.239Z" }, + { url = "https://files.pythonhosted.org/packages/bf/2d/f238229f1caba7ac87a6c4153d79947fb0261415827ae0f77c304260c7d3/regex-2025.11.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6538241f45eb5a25aa575dbba1069ad786f68a4f2773a29a2bd3dd1f9de787be", size = 911522, upload-time = "2025-11-03T21:32:59.274Z" }, + { url = "https://files.pythonhosted.org/packages/bd/3d/22a4eaba214a917c80e04f6025d26143690f0419511e0116508e24b11c9b/regex-2025.11.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bce22519c989bb72a7e6b36a199384c53db7722fe669ba891da75907fe3587db", size = 803272, upload-time = "2025-11-03T21:33:01.393Z" }, + { url = "https://files.pythonhosted.org/packages/84/b1/03188f634a409353a84b5ef49754b97dbcc0c0f6fd6c8ede505a8960a0a4/regex-2025.11.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:66d559b21d3640203ab9075797a55165d79017520685fb407b9234d72ab63c62", size = 787958, upload-time = "2025-11-03T21:33:03.379Z" }, + { url = "https://files.pythonhosted.org/packages/99/6a/27d072f7fbf6fadd59c64d210305e1ff865cc3b78b526fd147db768c553b/regex-2025.11.3-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:669dcfb2e38f9e8c69507bace46f4889e3abbfd9b0c29719202883c0a603598f", size = 859289, upload-time = "2025-11-03T21:33:05.374Z" }, + { url = "https://files.pythonhosted.org/packages/9a/70/1b3878f648e0b6abe023172dacb02157e685564853cc363d9961bcccde4e/regex-2025.11.3-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:32f74f35ff0f25a5021373ac61442edcb150731fbaa28286bbc8bb1582c89d02", size = 850026, upload-time = "2025-11-03T21:33:07.131Z" }, + { url = "https://files.pythonhosted.org/packages/dd/d5/68e25559b526b8baab8e66839304ede68ff6727237a47727d240006bd0ff/regex-2025.11.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e6c7a21dffba883234baefe91bc3388e629779582038f75d2a5be918e250f0ed", size = 789499, upload-time = "2025-11-03T21:33:09.141Z" }, + { url = "https://files.pythonhosted.org/packages/c3/06/49b198550ee0f5e4184271cee87ba4dfd9692c91ec55289e6282f0f86ccf/regex-2025.11.3-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:ba0d8a5d7f04f73ee7d01d974d47c5834f8a1b0224390e4fe7c12a3a92a78ecc", size = 491985, upload-time = "2025-11-03T21:33:16.555Z" }, + { url = "https://files.pythonhosted.org/packages/ce/bf/abdafade008f0b1c9da10d934034cb670432d6cf6cbe38bbb53a1cfd6cf8/regex-2025.11.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:442d86cf1cfe4faabf97db7d901ef58347efd004934da045c745e7b5bd57ac49", size = 292669, upload-time = "2025-11-03T21:33:18.32Z" }, + { url = "https://files.pythonhosted.org/packages/f9/ef/0c357bb8edbd2ad8e273fcb9e1761bc37b8acbc6e1be050bebd6475f19c1/regex-2025.11.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:fd0a5e563c756de210bb964789b5abe4f114dacae9104a47e1a649b910361536", size = 291030, upload-time = "2025-11-03T21:33:20.048Z" }, + { url = "https://files.pythonhosted.org/packages/79/06/edbb67257596649b8fb088d6aeacbcb248ac195714b18a65e018bf4c0b50/regex-2025.11.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bf3490bcbb985a1ae97b2ce9ad1c0f06a852d5b19dde9b07bdf25bf224248c95", size = 807674, upload-time = "2025-11-03T21:33:21.797Z" }, + { url = "https://files.pythonhosted.org/packages/f4/d9/ad4deccfce0ea336296bd087f1a191543bb99ee1c53093dcd4c64d951d00/regex-2025.11.3-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3809988f0a8b8c9dcc0f92478d6501fac7200b9ec56aecf0ec21f4a2ec4b6009", size = 873451, upload-time = "2025-11-03T21:33:23.741Z" }, + { url = "https://files.pythonhosted.org/packages/13/75/a55a4724c56ef13e3e04acaab29df26582f6978c000ac9cd6810ad1f341f/regex-2025.11.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f4ff94e58e84aedb9c9fce66d4ef9f27a190285b451420f297c9a09f2b9abee9", size = 914980, upload-time = "2025-11-03T21:33:25.999Z" }, + { url = "https://files.pythonhosted.org/packages/67/1e/a1657ee15bd9116f70d4a530c736983eed997b361e20ecd8f5ca3759d5c5/regex-2025.11.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7eb542fd347ce61e1321b0a6b945d5701528dca0cd9759c2e3bb8bd57e47964d", size = 812852, upload-time = "2025-11-03T21:33:27.852Z" }, + { url = "https://files.pythonhosted.org/packages/b8/6f/f7516dde5506a588a561d296b2d0044839de06035bb486b326065b4c101e/regex-2025.11.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d6c2d5919075a1f2e413c00b056ea0c2f065b3f5fe83c3d07d325ab92dce51d6", size = 795566, upload-time = "2025-11-03T21:33:32.364Z" }, + { url = "https://files.pythonhosted.org/packages/d9/dd/3d10b9e170cc16fb34cb2cef91513cf3df65f440b3366030631b2984a264/regex-2025.11.3-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:3f8bf11a4827cc7ce5a53d4ef6cddd5ad25595d3c1435ef08f76825851343154", size = 868463, upload-time = "2025-11-03T21:33:34.459Z" }, + { url = "https://files.pythonhosted.org/packages/f5/8e/935e6beff1695aa9085ff83195daccd72acc82c81793df480f34569330de/regex-2025.11.3-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:22c12d837298651e5550ac1d964e4ff57c3f56965fc1812c90c9fb2028eaf267", size = 854694, upload-time = "2025-11-03T21:33:36.793Z" }, + { url = "https://files.pythonhosted.org/packages/92/12/10650181a040978b2f5720a6a74d44f841371a3d984c2083fc1752e4acf6/regex-2025.11.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:62ba394a3dda9ad41c7c780f60f6e4a70988741415ae96f6d1bf6c239cf01379", size = 799691, upload-time = "2025-11-03T21:33:39.079Z" }, +] + +[[package]] +name = "requests" +version = "2.32.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "charset-normalizer", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "idna", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "urllib3", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, +] + +[[package]] +name = "rich" +version = "14.2.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pygments", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/fb/d2/8920e102050a0de7bfabeb4c4614a49248cf8d5d7a8d01885fbb24dc767a/rich-14.2.0.tar.gz", hash = "sha256:73ff50c7c0c1c77c8243079283f4edb376f0f6442433aecb8ce7e6d0b92d1fe4", size = 219990, upload-time = "2025-10-09T14:16:53.064Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/25/7a/b0178788f8dc6cafce37a212c99565fa1fe7872c70c6c9c1e1a372d9d88f/rich-14.2.0-py3-none-any.whl", hash = "sha256:76bc51fe2e57d2b1be1f96c524b890b816e334ab4c1e45888799bfaab0021edd", size = 243393, upload-time = "2025-10-09T14:16:51.245Z" }, +] + +[[package]] +name = "ruff" +version = "0.14.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/75/62/50b7727004dfe361104dfbf898c45a9a2fdfad8c72c04ae62900224d6ecf/ruff-0.14.3.tar.gz", hash = "sha256:4ff876d2ab2b161b6de0aa1f5bd714e8e9b4033dc122ee006925fbacc4f62153", size = 5558687, upload-time = "2025-10-31T00:26:26.878Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ce/8e/0c10ff1ea5d4360ab8bfca4cb2c9d979101a391f3e79d2616c9bf348cd26/ruff-0.14.3-py3-none-linux_armv6l.whl", hash = "sha256:876b21e6c824f519446715c1342b8e60f97f93264012de9d8d10314f8a79c371", size = 12535613, upload-time = "2025-10-31T00:25:44.302Z" }, + { url = "https://files.pythonhosted.org/packages/d3/c8/6724f4634c1daf52409fbf13fefda64aa9c8f81e44727a378b7b73dc590b/ruff-0.14.3-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:b6fd8c79b457bedd2abf2702b9b472147cd860ed7855c73a5247fa55c9117654", size = 12855812, upload-time = "2025-10-31T00:25:47.793Z" }, + { url = "https://files.pythonhosted.org/packages/de/03/db1bce591d55fd5f8a08bb02517fa0b5097b2ccabd4ea1ee29aa72b67d96/ruff-0.14.3-py3-none-macosx_11_0_arm64.whl", hash = "sha256:71ff6edca490c308f083156938c0c1a66907151263c4abdcb588602c6e696a14", size = 11944026, upload-time = "2025-10-31T00:25:49.657Z" }, + { url = "https://files.pythonhosted.org/packages/0b/75/4f8dbd48e03272715d12c87dc4fcaaf21b913f0affa5f12a4e9c6f8a0582/ruff-0.14.3-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:786ee3ce6139772ff9272aaf43296d975c0217ee1b97538a98171bf0d21f87ed", size = 12356818, upload-time = "2025-10-31T00:25:51.949Z" }, + { url = "https://files.pythonhosted.org/packages/ec/9b/506ec5b140c11d44a9a4f284ea7c14ebf6f8b01e6e8917734a3325bff787/ruff-0.14.3-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:cd6291d0061811c52b8e392f946889916757610d45d004e41140d81fb6cd5ddc", size = 12336745, upload-time = "2025-10-31T00:25:54.248Z" }, + { url = "https://files.pythonhosted.org/packages/c7/e1/c560d254048c147f35e7f8131d30bc1f63a008ac61595cf3078a3e93533d/ruff-0.14.3-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a497ec0c3d2c88561b6d90f9c29f5ae68221ac00d471f306fa21fa4264ce5fcd", size = 13101684, upload-time = "2025-10-31T00:25:56.253Z" }, + { url = "https://files.pythonhosted.org/packages/a5/32/e310133f8af5cd11f8cc30f52522a3ebccc5ea5bff4b492f94faceaca7a8/ruff-0.14.3-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:e231e1be58fc568950a04fbe6887c8e4b85310e7889727e2b81db205c45059eb", size = 14535000, upload-time = "2025-10-31T00:25:58.397Z" }, + { url = "https://files.pythonhosted.org/packages/a2/a1/7b0470a22158c6d8501eabc5e9b6043c99bede40fa1994cadf6b5c2a61c7/ruff-0.14.3-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:469e35872a09c0e45fecf48dd960bfbce056b5db2d5e6b50eca329b4f853ae20", size = 14156450, upload-time = "2025-10-31T00:26:00.889Z" }, + { url = "https://files.pythonhosted.org/packages/0a/96/24bfd9d1a7f532b560dcee1a87096332e461354d3882124219bcaff65c09/ruff-0.14.3-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3d6bc90307c469cb9d28b7cfad90aaa600b10d67c6e22026869f585e1e8a2db0", size = 13568414, upload-time = "2025-10-31T00:26:03.291Z" }, + { url = "https://files.pythonhosted.org/packages/a7/e7/138b883f0dfe4ad5b76b58bf4ae675f4d2176ac2b24bdd81b4d966b28c61/ruff-0.14.3-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0e2f8a0bbcffcfd895df39c9a4ecd59bb80dca03dc43f7fb63e647ed176b741e", size = 13315293, upload-time = "2025-10-31T00:26:05.708Z" }, + { url = "https://files.pythonhosted.org/packages/33/f4/c09bb898be97b2eb18476b7c950df8815ef14cf956074177e9fbd40b7719/ruff-0.14.3-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:678fdd7c7d2d94851597c23ee6336d25f9930b460b55f8598e011b57c74fd8c5", size = 13539444, upload-time = "2025-10-31T00:26:08.09Z" }, + { url = "https://files.pythonhosted.org/packages/9c/aa/b30a1db25fc6128b1dd6ff0741fa4abf969ded161599d07ca7edd0739cc0/ruff-0.14.3-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:1ec1ac071e7e37e0221d2f2dbaf90897a988c531a8592a6a5959f0603a1ecf5e", size = 12252581, upload-time = "2025-10-31T00:26:10.297Z" }, + { url = "https://files.pythonhosted.org/packages/da/13/21096308f384d796ffe3f2960b17054110a9c3828d223ca540c2b7cc670b/ruff-0.14.3-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:afcdc4b5335ef440d19e7df9e8ae2ad9f749352190e96d481dc501b753f0733e", size = 12307503, upload-time = "2025-10-31T00:26:12.646Z" }, + { url = "https://files.pythonhosted.org/packages/cb/cc/a350bac23f03b7dbcde3c81b154706e80c6f16b06ff1ce28ed07dc7b07b0/ruff-0.14.3-py3-none-musllinux_1_2_i686.whl", hash = "sha256:7bfc42f81862749a7136267a343990f865e71fe2f99cf8d2958f684d23ce3dfa", size = 12675457, upload-time = "2025-10-31T00:26:15.044Z" }, + { url = "https://files.pythonhosted.org/packages/cb/76/46346029fa2f2078826bc88ef7167e8c198e58fe3126636e52f77488cbba/ruff-0.14.3-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:a65e448cfd7e9c59fae8cf37f9221585d3354febaad9a07f29158af1528e165f", size = 13403980, upload-time = "2025-10-31T00:26:17.81Z" }, +] + +[[package]] +name = "rustworkx" +version = "0.17.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e7/b0/66d96f02120f79eeed86b5c5be04029b6821155f31ed4907a4e9f1460671/rustworkx-0.17.1.tar.gz", hash = "sha256:59ea01b4e603daffa4e8827316c1641eef18ae9032f0b1b14aa0181687e3108e", size = 399407, upload-time = "2025-09-15T16:29:46.429Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/20/24/8972ed631fa05fdec05a7bb7f1fc0f8e78ee761ab37e8a93d1ed396ba060/rustworkx-0.17.1-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:c08fb8db041db052da404839b064ebfb47dcce04ba9a3e2eb79d0c65ab011da4", size = 2257491, upload-time = "2025-08-13T01:43:31.466Z" }, + { url = "https://files.pythonhosted.org/packages/23/ae/7b6bbae5e0487ee42072dc6a46edf5db9731a0701ed648db22121fb7490c/rustworkx-0.17.1-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:4ef8e327dadf6500edd76fedb83f6d888b9266c58bcdbffd5a40c33835c9dd26", size = 2040175, upload-time = "2025-08-13T01:43:33.762Z" }, + { url = "https://files.pythonhosted.org/packages/cd/ea/c17fb9428c8f0dcc605596f9561627a5b9ef629d356204ee5088cfcf52c6/rustworkx-0.17.1-cp39-abi3-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5b809e0aa2927c68574b196f993233e269980918101b0dd235289c4f3ddb2115", size = 2324771, upload-time = "2025-08-13T01:43:35.553Z" }, + { url = "https://files.pythonhosted.org/packages/d7/40/ec8b3b8b0f8c0b768690c454b8dcc2781b4f2c767f9f1215539c7909e35b/rustworkx-0.17.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c7e82c46a92fb0fd478b7372e15ca524c287485fdecaed37b8bb68f4df2720f2", size = 2068584, upload-time = "2025-08-13T01:43:37.261Z" }, + { url = "https://files.pythonhosted.org/packages/d9/22/713b900d320d06ce8677e71bba0ec5df0037f1d83270bff5db3b271c10d7/rustworkx-0.17.1-cp39-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:42170075d8a7319e89ff63062c2f1d1116ced37b6f044f3bf36d10b60a107aa4", size = 2380949, upload-time = "2025-08-13T01:52:17.435Z" }, + { url = "https://files.pythonhosted.org/packages/20/4b/54be84b3b41a19caf0718a2b6bb280dde98c8626c809c969f16aad17458f/rustworkx-0.17.1-cp39-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:65cba97fa95470239e2d65eb4db1613f78e4396af9f790ff771b0e5476bfd887", size = 2562069, upload-time = "2025-08-13T02:09:27.222Z" }, + { url = "https://files.pythonhosted.org/packages/39/5b/281bb21d091ab4e36cf377088366d55d0875fa2347b3189c580ec62b44c7/rustworkx-0.17.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:246cc252053f89e36209535b9c58755960197e6ae08d48d3973760141c62ac95", size = 2221186, upload-time = "2025-08-13T01:43:38.598Z" }, + { url = "https://files.pythonhosted.org/packages/cc/2d/30a941a21b81e9db50c4c3ef8a64c5ee1c8eea3a90506ca0326ce39d021f/rustworkx-0.17.1-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:c10d25e9f0e87d6a273d1ea390b636b4fb3fede2094bf0cb3fe565d696a91b48", size = 2123510, upload-time = "2025-08-13T01:43:40.288Z" }, + { url = "https://files.pythonhosted.org/packages/4f/ef/c9199e4b6336ee5a9f1979c11b5779c5cf9ab6f8386e0b9a96c8ffba7009/rustworkx-0.17.1-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:48784a673cf8d04f3cd246fa6b53fd1ccc4d83304503463bd561c153517bccc1", size = 2302783, upload-time = "2025-08-13T01:43:42.073Z" }, +] + +[[package]] +name = "safetensors" +version = "0.6.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ac/cc/738f3011628920e027a11754d9cae9abec1aed00f7ae860abbf843755233/safetensors-0.6.2.tar.gz", hash = "sha256:43ff2aa0e6fa2dc3ea5524ac7ad93a9839256b8703761e76e2d0b2a3fa4f15d9", size = 197968, upload-time = "2025-08-08T13:13:58.654Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4d/b1/3f5fd73c039fc87dba3ff8b5d528bfc5a32b597fea8e7a6a4800343a17c7/safetensors-0.6.2-cp38-abi3-macosx_10_12_x86_64.whl", hash = "sha256:9c85ede8ec58f120bad982ec47746981e210492a6db876882aa021446af8ffba", size = 454797, upload-time = "2025-08-08T13:13:52.066Z" }, + { url = "https://files.pythonhosted.org/packages/8c/c9/bb114c158540ee17907ec470d01980957fdaf87b4aa07914c24eba87b9c6/safetensors-0.6.2-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:d6675cf4b39c98dbd7d940598028f3742e0375a6b4d4277e76beb0c35f4b843b", size = 432206, upload-time = "2025-08-08T13:13:50.931Z" }, + { url = "https://files.pythonhosted.org/packages/d3/8e/f70c34e47df3110e8e0bb268d90db8d4be8958a54ab0336c9be4fe86dac8/safetensors-0.6.2-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1d2d2b3ce1e2509c68932ca03ab8f20570920cd9754b05063d4368ee52833ecd", size = 473261, upload-time = "2025-08-08T13:13:41.259Z" }, + { url = "https://files.pythonhosted.org/packages/2a/f5/be9c6a7c7ef773e1996dc214e73485286df1836dbd063e8085ee1976f9cb/safetensors-0.6.2-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:93de35a18f46b0f5a6a1f9e26d91b442094f2df02e9fd7acf224cfec4238821a", size = 485117, upload-time = "2025-08-08T13:13:43.506Z" }, + { url = "https://files.pythonhosted.org/packages/c9/55/23f2d0a2c96ed8665bf17a30ab4ce5270413f4d74b6d87dd663258b9af31/safetensors-0.6.2-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:89a89b505f335640f9120fac65ddeb83e40f1fd081cb8ed88b505bdccec8d0a1", size = 616154, upload-time = "2025-08-08T13:13:45.096Z" }, + { url = "https://files.pythonhosted.org/packages/98/c6/affb0bd9ce02aa46e7acddbe087912a04d953d7a4d74b708c91b5806ef3f/safetensors-0.6.2-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:fc4d0d0b937e04bdf2ae6f70cd3ad51328635fe0e6214aa1fc811f3b576b3bda", size = 520713, upload-time = "2025-08-08T13:13:46.25Z" }, + { url = "https://files.pythonhosted.org/packages/fe/5d/5a514d7b88e310c8b146e2404e0dc161282e78634d9358975fd56dfd14be/safetensors-0.6.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8045db2c872db8f4cbe3faa0495932d89c38c899c603f21e9b6486951a5ecb8f", size = 485835, upload-time = "2025-08-08T13:13:49.373Z" }, + { url = "https://files.pythonhosted.org/packages/7a/7b/4fc3b2ba62c352b2071bea9cfbad330fadda70579f617506ae1a2f129cab/safetensors-0.6.2-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:81e67e8bab9878bb568cffbc5f5e655adb38d2418351dc0859ccac158f753e19", size = 521503, upload-time = "2025-08-08T13:13:47.651Z" }, + { url = "https://files.pythonhosted.org/packages/5a/50/0057e11fe1f3cead9254315a6c106a16dd4b1a19cd247f7cc6414f6b7866/safetensors-0.6.2-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:b0e4d029ab0a0e0e4fdf142b194514695b1d7d3735503ba700cf36d0fc7136ce", size = 652256, upload-time = "2025-08-08T13:13:53.167Z" }, + { url = "https://files.pythonhosted.org/packages/e9/29/473f789e4ac242593ac1656fbece6e1ecd860bb289e635e963667807afe3/safetensors-0.6.2-cp38-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:fa48268185c52bfe8771e46325a1e21d317207bcabcb72e65c6e28e9ffeb29c7", size = 747281, upload-time = "2025-08-08T13:13:54.656Z" }, + { url = "https://files.pythonhosted.org/packages/68/52/f7324aad7f2df99e05525c84d352dc217e0fa637a4f603e9f2eedfbe2c67/safetensors-0.6.2-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:d83c20c12c2d2f465997c51b7ecb00e407e5f94d7dec3ea0cc11d86f60d3fde5", size = 692286, upload-time = "2025-08-08T13:13:55.884Z" }, + { url = "https://files.pythonhosted.org/packages/ad/fe/cad1d9762868c7c5dc70c8620074df28ebb1a8e4c17d4c0cb031889c457e/safetensors-0.6.2-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:d944cea65fad0ead848b6ec2c37cc0b197194bec228f8020054742190e9312ac", size = 655957, upload-time = "2025-08-08T13:13:57.029Z" }, +] + +[[package]] +name = "sniffio" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a2/87/a6771e1546d97e7e041b6ae58d80074f81b7d5121207425c964ddf5cfdbd/sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc", size = 20372, upload-time = "2024-02-25T23:20:04.057Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" }, +] + +[[package]] +name = "sqlalchemy" +version = "2.0.44" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "greenlet", marker = "(platform_machine == 'AMD64' and sys_platform == 'darwin') or (platform_machine == 'WIN32' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'darwin') or (platform_machine == 'amd64' and sys_platform == 'darwin') or (platform_machine == 'ppc64le' and sys_platform == 'darwin') or (platform_machine == 'win32' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and sys_platform == 'darwin') or (platform_machine == 'AMD64' and sys_platform == 'linux') or (platform_machine == 'WIN32' and sys_platform == 'linux') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'amd64' and sys_platform == 'linux') or (platform_machine == 'ppc64le' and sys_platform == 'linux') or (platform_machine == 'win32' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux')" }, + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f0/f2/840d7b9496825333f532d2e3976b8eadbf52034178aac53630d09fe6e1ef/sqlalchemy-2.0.44.tar.gz", hash = "sha256:0ae7454e1ab1d780aee69fd2aae7d6b8670a581d8847f2d1e0f7ddfbf47e5a22", size = 9819830, upload-time = "2025-10-10T14:39:12.935Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/45/d3/c67077a2249fdb455246e6853166360054c331db4613cda3e31ab1cadbef/sqlalchemy-2.0.44-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ff486e183d151e51b1d694c7aa1695747599bb00b9f5f604092b54b74c64a8e1", size = 2135479, upload-time = "2025-10-10T16:03:37.671Z" }, + { url = "https://files.pythonhosted.org/packages/2b/91/eabd0688330d6fd114f5f12c4f89b0d02929f525e6bf7ff80aa17ca802af/sqlalchemy-2.0.44-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0b1af8392eb27b372ddb783b317dea0f650241cea5bd29199b22235299ca2e45", size = 2123212, upload-time = "2025-10-10T16:03:41.755Z" }, + { url = "https://files.pythonhosted.org/packages/b0/bb/43e246cfe0e81c018076a16036d9b548c4cc649de241fa27d8d9ca6f85ab/sqlalchemy-2.0.44-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2b61188657e3a2b9ac4e8f04d6cf8e51046e28175f79464c67f2fd35bceb0976", size = 3255353, upload-time = "2025-10-10T15:35:31.221Z" }, + { url = "https://files.pythonhosted.org/packages/b9/96/c6105ed9a880abe346b64d3b6ddef269ddfcab04f7f3d90a0bf3c5a88e82/sqlalchemy-2.0.44-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b87e7b91a5d5973dda5f00cd61ef72ad75a1db73a386b62877d4875a8840959c", size = 3260222, upload-time = "2025-10-10T15:43:50.124Z" }, + { url = "https://files.pythonhosted.org/packages/44/16/1857e35a47155b5ad927272fee81ae49d398959cb749edca6eaa399b582f/sqlalchemy-2.0.44-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:15f3326f7f0b2bfe406ee562e17f43f36e16167af99c4c0df61db668de20002d", size = 3189614, upload-time = "2025-10-10T15:35:32.578Z" }, + { url = "https://files.pythonhosted.org/packages/88/ee/4afb39a8ee4fc786e2d716c20ab87b5b1fb33d4ac4129a1aaa574ae8a585/sqlalchemy-2.0.44-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:1e77faf6ff919aa8cd63f1c4e561cac1d9a454a191bb864d5dd5e545935e5a40", size = 3226248, upload-time = "2025-10-10T15:43:51.862Z" }, + { url = "https://files.pythonhosted.org/packages/9c/5e/6a29fa884d9fb7ddadf6b69490a9d45fded3b38541713010dad16b77d015/sqlalchemy-2.0.44-py3-none-any.whl", hash = "sha256:19de7ca1246fbef9f9d1bff8f1ab25641569df226364a0e40457dc5457c54b05", size = 1928718, upload-time = "2025-10-10T15:29:45.32Z" }, +] + +[package.optional-dependencies] +asyncio = [ + { name = "greenlet", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] + +[[package]] +name = "sqlmodel" +version = "0.0.27" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pydantic", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "sqlalchemy", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/90/5a/693d90866233e837d182da76082a6d4c2303f54d3aaaa5c78e1238c5d863/sqlmodel-0.0.27.tar.gz", hash = "sha256:ad1227f2014a03905aef32e21428640848ac09ff793047744a73dfdd077ff620", size = 118053, upload-time = "2025-10-08T16:39:11.938Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8c/92/c35e036151fe53822893979f8a13e6f235ae8191f4164a79ae60a95d66aa/sqlmodel-0.0.27-py3-none-any.whl", hash = "sha256:667fe10aa8ff5438134668228dc7d7a08306f4c5c4c7e6ad3ad68defa0e7aa49", size = 29131, upload-time = "2025-10-08T16:39:10.917Z" }, +] + +[[package]] +name = "starlette" +version = "0.49.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/de/1a/608df0b10b53b0beb96a37854ee05864d182ddd4b1156a22f1ad3860425a/starlette-0.49.3.tar.gz", hash = "sha256:1c14546f299b5901a1ea0e34410575bc33bbd741377a10484a54445588d00284", size = 2655031, upload-time = "2025-11-01T15:12:26.13Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a3/e0/021c772d6a662f43b63044ab481dc6ac7592447605b5b35a957785363122/starlette-0.49.3-py3-none-any.whl", hash = "sha256:b579b99715fdc2980cf88c8ec96d3bf1ce16f5a8051a7c2b84ef9b1cdecaea2f", size = 74340, upload-time = "2025-11-01T15:12:24.387Z" }, +] + +[[package]] +name = "textual" +version = "6.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py", extra = ["linkify"], marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "mdit-py-plugins", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "platformdirs", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pygments", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "rich", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/af/90/59757aa887ddcea61428820274f1a2d1f986feb7880374a5420ab5d37132/textual-6.5.0.tar.gz", hash = "sha256:e5f152cdd47db48a635d23b839721bae4d0e8b6d855e3fede7285218289294e3", size = 1574116, upload-time = "2025-10-31T17:21:53.4Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/42/37/1deba011782a49ea249c73adcf703a39b0249ac9b0e17d1a2e4074df8d57/textual-6.5.0-py3-none-any.whl", hash = "sha256:c5505be7fe606b8054fb88431279885f88352bddca64832f6acd293ef7d9b54f", size = 711848, upload-time = "2025-10-31T17:21:51.134Z" }, +] + +[[package]] +name = "tiktoken" +version = "0.12.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "regex", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "requests", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/7d/ab/4d017d0f76ec3171d469d80fc03dfbb4e48a4bcaddaa831b31d526f05edc/tiktoken-0.12.0.tar.gz", hash = "sha256:b18ba7ee2b093863978fcb14f74b3707cdc8d4d4d3836853ce7ec60772139931", size = 37806, upload-time = "2025-10-06T20:22:45.419Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/00/61/441588ee21e6b5cdf59d6870f86beb9789e532ee9718c251b391b70c68d6/tiktoken-0.12.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:775c2c55de2310cc1bc9a3ad8826761cbdc87770e586fd7b6da7d4589e13dab3", size = 1050802, upload-time = "2025-10-06T20:22:00.96Z" }, + { url = "https://files.pythonhosted.org/packages/1f/05/dcf94486d5c5c8d34496abe271ac76c5b785507c8eae71b3708f1ad9b45a/tiktoken-0.12.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a01b12f69052fbe4b080a2cfb867c4de12c704b56178edf1d1d7b273561db160", size = 993995, upload-time = "2025-10-06T20:22:02.788Z" }, + { url = "https://files.pythonhosted.org/packages/a0/70/5163fe5359b943f8db9946b62f19be2305de8c3d78a16f629d4165e2f40e/tiktoken-0.12.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:01d99484dc93b129cd0964f9d34eee953f2737301f18b3c7257bf368d7615baa", size = 1128948, upload-time = "2025-10-06T20:22:03.814Z" }, + { url = "https://files.pythonhosted.org/packages/0c/da/c028aa0babf77315e1cef357d4d768800c5f8a6de04d0eac0f377cb619fa/tiktoken-0.12.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:4a1a4fcd021f022bfc81904a911d3df0f6543b9e7627b51411da75ff2fe7a1be", size = 1151986, upload-time = "2025-10-06T20:22:05.173Z" }, + { url = "https://files.pythonhosted.org/packages/a0/5a/886b108b766aa53e295f7216b509be95eb7d60b166049ce2c58416b25f2a/tiktoken-0.12.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:981a81e39812d57031efdc9ec59fa32b2a5a5524d20d4776574c4b4bd2e9014a", size = 1194222, upload-time = "2025-10-06T20:22:06.265Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f8/4db272048397636ac7a078d22773dd2795b1becee7bc4922fe6207288d57/tiktoken-0.12.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9baf52f84a3f42eef3ff4e754a0db79a13a27921b457ca9832cf944c6be4f8f3", size = 1255097, upload-time = "2025-10-06T20:22:07.403Z" }, + { url = "https://files.pythonhosted.org/packages/ce/76/994fc868f88e016e6d05b0da5ac24582a14c47893f4474c3e9744283f1d5/tiktoken-0.12.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:d5f89ea5680066b68bcb797ae85219c72916c922ef0fcdd3480c7d2315ffff16", size = 1050309, upload-time = "2025-10-06T20:22:10.939Z" }, + { url = "https://files.pythonhosted.org/packages/f6/b8/57ef1456504c43a849821920d582a738a461b76a047f352f18c0b26c6516/tiktoken-0.12.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:b4e7ed1c6a7a8a60a3230965bdedba8cc58f68926b835e519341413370e0399a", size = 993712, upload-time = "2025-10-06T20:22:12.115Z" }, + { url = "https://files.pythonhosted.org/packages/72/90/13da56f664286ffbae9dbcfadcc625439142675845baa62715e49b87b68b/tiktoken-0.12.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:fc530a28591a2d74bce821d10b418b26a094bf33839e69042a6e86ddb7a7fb27", size = 1128725, upload-time = "2025-10-06T20:22:13.541Z" }, + { url = "https://files.pythonhosted.org/packages/05/df/4f80030d44682235bdaecd7346c90f67ae87ec8f3df4a3442cb53834f7e4/tiktoken-0.12.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:06a9f4f49884139013b138920a4c393aa6556b2f8f536345f11819389c703ebb", size = 1151875, upload-time = "2025-10-06T20:22:14.559Z" }, + { url = "https://files.pythonhosted.org/packages/22/1f/ae535223a8c4ef4c0c1192e3f9b82da660be9eb66b9279e95c99288e9dab/tiktoken-0.12.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:04f0e6a985d95913cabc96a741c5ffec525a2c72e9df086ff17ebe35985c800e", size = 1194451, upload-time = "2025-10-06T20:22:15.545Z" }, + { url = "https://files.pythonhosted.org/packages/78/a7/f8ead382fce0243cb625c4f266e66c27f65ae65ee9e77f59ea1653b6d730/tiktoken-0.12.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:0ee8f9ae00c41770b5f9b0bb1235474768884ae157de3beb5439ca0fd70f3e25", size = 1253794, upload-time = "2025-10-06T20:22:16.624Z" }, + { url = "https://files.pythonhosted.org/packages/72/05/3abc1db5d2c9aadc4d2c76fa5640134e475e58d9fbb82b5c535dc0de9b01/tiktoken-0.12.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:a90388128df3b3abeb2bfd1895b0681412a8d7dc644142519e6f0a97c2111646", size = 1050188, upload-time = "2025-10-06T20:22:19.563Z" }, + { url = "https://files.pythonhosted.org/packages/e3/7b/50c2f060412202d6c95f32b20755c7a6273543b125c0985d6fa9465105af/tiktoken-0.12.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:da900aa0ad52247d8794e307d6446bd3cdea8e192769b56276695d34d2c9aa88", size = 993978, upload-time = "2025-10-06T20:22:20.702Z" }, + { url = "https://files.pythonhosted.org/packages/14/27/bf795595a2b897e271771cd31cb847d479073497344c637966bdf2853da1/tiktoken-0.12.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:285ba9d73ea0d6171e7f9407039a290ca77efcdb026be7769dccc01d2c8d7fff", size = 1129271, upload-time = "2025-10-06T20:22:22.06Z" }, + { url = "https://files.pythonhosted.org/packages/f5/de/9341a6d7a8f1b448573bbf3425fa57669ac58258a667eb48a25dfe916d70/tiktoken-0.12.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:d186a5c60c6a0213f04a7a802264083dea1bbde92a2d4c7069e1a56630aef830", size = 1151216, upload-time = "2025-10-06T20:22:23.085Z" }, + { url = "https://files.pythonhosted.org/packages/75/0d/881866647b8d1be4d67cb24e50d0c26f9f807f994aa1510cb9ba2fe5f612/tiktoken-0.12.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:604831189bd05480f2b885ecd2d1986dc7686f609de48208ebbbddeea071fc0b", size = 1194860, upload-time = "2025-10-06T20:22:24.602Z" }, + { url = "https://files.pythonhosted.org/packages/b3/1e/b651ec3059474dab649b8d5b69f5c65cd8fcd8918568c1935bd4136c9392/tiktoken-0.12.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:8f317e8530bb3a222547b85a58583238c8f74fd7a7408305f9f63246d1a0958b", size = 1254567, upload-time = "2025-10-06T20:22:25.671Z" }, + { url = "https://files.pythonhosted.org/packages/ac/a4/72eed53e8976a099539cdd5eb36f241987212c29629d0a52c305173e0a68/tiktoken-0.12.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:c2c714c72bc00a38ca969dae79e8266ddec999c7ceccd603cc4f0d04ccd76365", size = 1050473, upload-time = "2025-10-06T20:22:27.775Z" }, + { url = "https://files.pythonhosted.org/packages/e6/d7/0110b8f54c008466b19672c615f2168896b83706a6611ba6e47313dbc6e9/tiktoken-0.12.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:cbb9a3ba275165a2cb0f9a83f5d7025afe6b9d0ab01a22b50f0e74fee2ad253e", size = 993855, upload-time = "2025-10-06T20:22:28.799Z" }, + { url = "https://files.pythonhosted.org/packages/5f/77/4f268c41a3957c418b084dd576ea2fad2e95da0d8e1ab705372892c2ca22/tiktoken-0.12.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:dfdfaa5ffff8993a3af94d1125870b1d27aed7cb97aa7eb8c1cefdbc87dbee63", size = 1129022, upload-time = "2025-10-06T20:22:29.981Z" }, + { url = "https://files.pythonhosted.org/packages/4e/2b/fc46c90fe5028bd094cd6ee25a7db321cb91d45dc87531e2bdbb26b4867a/tiktoken-0.12.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:584c3ad3d0c74f5269906eb8a659c8bfc6144a52895d9261cdaf90a0ae5f4de0", size = 1150736, upload-time = "2025-10-06T20:22:30.996Z" }, + { url = "https://files.pythonhosted.org/packages/28/c0/3c7a39ff68022ddfd7d93f3337ad90389a342f761c4d71de99a3ccc57857/tiktoken-0.12.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:54c891b416a0e36b8e2045b12b33dd66fb34a4fe7965565f1b482da50da3e86a", size = 1194908, upload-time = "2025-10-06T20:22:32.073Z" }, + { url = "https://files.pythonhosted.org/packages/ab/0d/c1ad6f4016a3968c048545f5d9b8ffebf577774b2ede3e2e352553b685fe/tiktoken-0.12.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5edb8743b88d5be814b1a8a8854494719080c28faaa1ccbef02e87354fe71ef0", size = 1253706, upload-time = "2025-10-06T20:22:33.385Z" }, +] + +[[package]] +name = "tokenizers" +version = "0.22.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "huggingface-hub", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1c/46/fb6854cec3278fbfa4a75b50232c77622bc517ac886156e6afbfa4d8fc6e/tokenizers-0.22.1.tar.gz", hash = "sha256:61de6522785310a309b3407bac22d99c4db5dba349935e99e4d15ea2226af2d9", size = 363123, upload-time = "2025-09-19T09:49:23.424Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bf/33/f4b2d94ada7ab297328fc671fed209368ddb82f965ec2224eb1892674c3a/tokenizers-0.22.1-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:59fdb013df17455e5f950b4b834a7b3ee2e0271e6378ccb33aa74d178b513c73", size = 3069318, upload-time = "2025-09-19T09:49:11.848Z" }, + { url = "https://files.pythonhosted.org/packages/1c/58/2aa8c874d02b974990e89ff95826a4852a8b2a273c7d1b4411cdd45a4565/tokenizers-0.22.1-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:8d4e484f7b0827021ac5f9f71d4794aaef62b979ab7608593da22b1d2e3c4edc", size = 2926478, upload-time = "2025-09-19T09:49:09.759Z" }, + { url = "https://files.pythonhosted.org/packages/1e/3b/55e64befa1e7bfea963cf4b787b2cea1011362c4193f5477047532ce127e/tokenizers-0.22.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:19d2962dd28bc67c1f205ab180578a78eef89ac60ca7ef7cbe9635a46a56422a", size = 3256994, upload-time = "2025-09-19T09:48:56.701Z" }, + { url = "https://files.pythonhosted.org/packages/71/0b/fbfecf42f67d9b7b80fde4aabb2b3110a97fac6585c9470b5bff103a80cb/tokenizers-0.22.1-cp39-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:38201f15cdb1f8a6843e6563e6e79f4abd053394992b9bbdf5213ea3469b4ae7", size = 3153141, upload-time = "2025-09-19T09:48:59.749Z" }, + { url = "https://files.pythonhosted.org/packages/17/a9/b38f4e74e0817af8f8ef925507c63c6ae8171e3c4cb2d5d4624bf58fca69/tokenizers-0.22.1-cp39-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d1cbe5454c9a15df1b3443c726063d930c16f047a3cc724b9e6e1a91140e5a21", size = 3508049, upload-time = "2025-09-19T09:49:05.868Z" }, + { url = "https://files.pythonhosted.org/packages/d2/48/dd2b3dac46bb9134a88e35d72e1aa4869579eacc1a27238f1577270773ff/tokenizers-0.22.1-cp39-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e7d094ae6312d69cc2a872b54b91b309f4f6fbce871ef28eb27b52a98e4d0214", size = 3710730, upload-time = "2025-09-19T09:49:01.832Z" }, + { url = "https://files.pythonhosted.org/packages/93/0e/ccabc8d16ae4ba84a55d41345207c1e2ea88784651a5a487547d80851398/tokenizers-0.22.1-cp39-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:afd7594a56656ace95cdd6df4cca2e4059d294c5cfb1679c57824b605556cb2f", size = 3412560, upload-time = "2025-09-19T09:49:03.867Z" }, + { url = "https://files.pythonhosted.org/packages/d0/c6/dc3a0db5a6766416c32c034286d7c2d406da1f498e4de04ab1b8959edd00/tokenizers-0.22.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e2ef6063d7a84994129732b47e7915e8710f27f99f3a3260b8a38fc7ccd083f4", size = 3250221, upload-time = "2025-09-19T09:49:07.664Z" }, + { url = "https://files.pythonhosted.org/packages/d7/a6/2c8486eef79671601ff57b093889a345dd3d576713ef047776015dc66de7/tokenizers-0.22.1-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:ba0a64f450b9ef412c98f6bcd2a50c6df6e2443b560024a09fa6a03189726879", size = 9345569, upload-time = "2025-09-19T09:49:14.214Z" }, + { url = "https://files.pythonhosted.org/packages/6b/16/32ce667f14c35537f5f605fe9bea3e415ea1b0a646389d2295ec348d5657/tokenizers-0.22.1-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:331d6d149fa9c7d632cde4490fb8bbb12337fa3a0232e77892be656464f4b446", size = 9271599, upload-time = "2025-09-19T09:49:16.639Z" }, + { url = "https://files.pythonhosted.org/packages/51/7c/a5f7898a3f6baa3fc2685c705e04c98c1094c523051c805cdd9306b8f87e/tokenizers-0.22.1-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:607989f2ea68a46cb1dfbaf3e3aabdf3f21d8748312dbeb6263d1b3b66c5010a", size = 9533862, upload-time = "2025-09-19T09:49:19.146Z" }, + { url = "https://files.pythonhosted.org/packages/36/65/7e75caea90bc73c1dd8d40438adf1a7bc26af3b8d0a6705ea190462506e1/tokenizers-0.22.1-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a0f307d490295717726598ef6fa4f24af9d484809223bbc253b201c740a06390", size = 9681250, upload-time = "2025-09-19T09:49:21.501Z" }, +] + +[[package]] +name = "tqdm" +version = "4.67.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a8/4b/29b4ef32e036bb34e4ab51796dd745cdba7ed47ad142a9f4a1eb8e0c744d/tqdm-4.67.1.tar.gz", hash = "sha256:f8aef9c52c08c13a65f30ea34f4e5aac3fd1a34959879d7e59e63027286627f2", size = 169737, upload-time = "2024-11-24T20:12:22.481Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl", hash = "sha256:26445eca388f82e72884e0d580d5464cd801a3ea01e63e5601bdff9ba6a48de2", size = 78540, upload-time = "2024-11-24T20:12:19.698Z" }, +] + +[[package]] +name = "transformers" +version = "4.57.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "filelock", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "huggingface-hub", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "numpy", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "packaging", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "pyyaml", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "regex", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "requests", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "safetensors", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "tokenizers", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "tqdm", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/d6/68/a39307bcc4116a30b2106f2e689130a48de8bd8a1e635b5e1030e46fcd9e/transformers-4.57.1.tar.gz", hash = "sha256:f06c837959196c75039809636cd964b959f6604b75b8eeec6fdfc0440b89cc55", size = 10142511, upload-time = "2025-10-14T15:39:26.18Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/71/d3/c16c3b3cf7655a67db1144da94b021c200ac1303f82428f2beef6c2e72bb/transformers-4.57.1-py3-none-any.whl", hash = "sha256:b10d05da8fa67dc41644dbbf9bc45a44cb86ae33da6f9295f5fbf5b7890bd267", size = 11990925, upload-time = "2025-10-14T15:39:23.085Z" }, +] + +[[package]] +name = "typeguard" +version = "4.4.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/68/71c1a15b5f65f40e91b65da23b8224dad41349894535a97f63a52e462196/typeguard-4.4.4.tar.gz", hash = "sha256:3a7fd2dffb705d4d0efaed4306a704c89b9dee850b688f060a8b1615a79e5f74", size = 75203, upload-time = "2025-06-18T09:56:07.624Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1b/a9/e3aee762739c1d7528da1c3e06d518503f8b6c439c35549b53735ba52ead/typeguard-4.4.4-py3-none-any.whl", hash = "sha256:b5f562281b6bfa1f5492470464730ef001646128b180769880468bd84b68b09e", size = 34874, upload-time = "2025-06-18T09:56:05.999Z" }, +] + +[[package]] +name = "types-aiofiles" +version = "25.1.0.20251011" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/84/6c/6d23908a8217e36704aa9c79d99a620f2fdd388b66a4b7f72fbc6b6ff6c6/types_aiofiles-25.1.0.20251011.tar.gz", hash = "sha256:1c2b8ab260cb3cd40c15f9d10efdc05a6e1e6b02899304d80dfa0410e028d3ff", size = 14535, upload-time = "2025-10-11T02:44:51.237Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/71/0f/76917bab27e270bb6c32addd5968d69e558e5b6f7fb4ac4cbfa282996a96/types_aiofiles-25.1.0.20251011-py3-none-any.whl", hash = "sha256:8ff8de7f9d42739d8f0dadcceeb781ce27cd8d8c4152d4a7c52f6b20edb8149c", size = 14338, upload-time = "2025-10-11T02:44:50.054Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.15.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" }, +] + +[[package]] +name = "typing-inspection" +version = "0.4.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac53367ae42139cf4b1ca5f36bb3dc6c9d33acdb43655/typing_inspection-0.4.2.tar.gz", hash = "sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464", size = 75949, upload-time = "2025-10-01T02:14:41.687Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" }, +] + +[[package]] +name = "uc-micro-py" +version = "1.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/91/7a/146a99696aee0609e3712f2b44c6274566bc368dfe8375191278045186b8/uc-micro-py-1.0.3.tar.gz", hash = "sha256:d321b92cff673ec58027c04015fcaa8bb1e005478643ff4a500882eaab88c48a", size = 6043, upload-time = "2024-02-09T16:52:01.654Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/37/87/1f677586e8ac487e29672e4b17455758fce261de06a0d086167bb760361a/uc_micro_py-1.0.3-py3-none-any.whl", hash = "sha256:db1dffff340817673d7b466ec86114a9dc0e9d4d9b5ba229d9d60e5c12600cd5", size = 6229, upload-time = "2024-02-09T16:52:00.371Z" }, +] + +[[package]] +name = "urllib3" +version = "2.5.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/15/22/9ee70a2574a4f4599c47dd506532914ce044817c7752a79b6a51286319bc/urllib3-2.5.0.tar.gz", hash = "sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760", size = 393185, upload-time = "2025-06-18T14:07:41.644Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a7/c2/fe1e52489ae3122415c51f387e221dd0773709bad6c6cdaa599e8a2c5185/urllib3-2.5.0-py3-none-any.whl", hash = "sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc", size = 129795, upload-time = "2025-06-18T14:07:40.39Z" }, +] + +[[package]] +name = "wsproto" +version = "1.3.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "h11", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/79/12135bdf8b9c9367b8701c2c19a14c913c120b882d50b014ca0d38083c2c/wsproto-1.3.2.tar.gz", hash = "sha256:b86885dcf294e15204919950f666e06ffc6c7c114ca900b060d6e16293528294", size = 50116, upload-time = "2025-11-20T18:18:01.871Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a4/f5/10b68b7b1544245097b2a1b8238f66f2fc6dcaeb24ba5d917f52bd2eed4f/wsproto-1.3.2-py3-none-any.whl", hash = "sha256:61eea322cdf56e8cc904bd3ad7573359a242ba65688716b0710a5eb12beab584", size = 24405, upload-time = "2025-11-20T18:18:00.454Z" }, +] + +[[package]] +name = "yarl" +version = "1.22.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "idna", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "multidict", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, + { name = "propcache", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/57/63/0c6ebca57330cd313f6102b16dd57ffaf3ec4c83403dcb45dbd15c6f3ea1/yarl-1.22.0.tar.gz", hash = "sha256:bebf8557577d4401ba8bd9ff33906f1376c877aa78d1fe216ad01b4d6745af71", size = 187169, upload-time = "2025-10-06T14:12:55.963Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ea/f3/d67de7260456ee105dc1d162d43a019ecad6b91e2f51809d6cddaa56690e/yarl-1.22.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8dee9c25c74997f6a750cd317b8ca63545169c098faee42c84aa5e506c819b53", size = 139980, upload-time = "2025-10-06T14:10:14.601Z" }, + { url = "https://files.pythonhosted.org/packages/01/88/04d98af0b47e0ef42597b9b28863b9060bb515524da0a65d5f4db160b2d5/yarl-1.22.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:01e73b85a5434f89fc4fe27dcda2aff08ddf35e4d47bbbea3bdcd25321af538a", size = 93424, upload-time = "2025-10-06T14:10:16.115Z" }, + { url = "https://files.pythonhosted.org/packages/18/91/3274b215fd8442a03975ce6bee5fe6aa57a8326b29b9d3d56234a1dca244/yarl-1.22.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:22965c2af250d20c873cdbee8ff958fb809940aeb2e74ba5f20aaf6b7ac8c70c", size = 93821, upload-time = "2025-10-06T14:10:17.993Z" }, + { url = "https://files.pythonhosted.org/packages/61/3a/caf4e25036db0f2da4ca22a353dfeb3c9d3c95d2761ebe9b14df8fc16eb0/yarl-1.22.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b4f15793aa49793ec8d1c708ab7f9eded1aa72edc5174cae703651555ed1b601", size = 373243, upload-time = "2025-10-06T14:10:19.44Z" }, + { url = "https://files.pythonhosted.org/packages/6e/9e/51a77ac7516e8e7803b06e01f74e78649c24ee1021eca3d6a739cb6ea49c/yarl-1.22.0-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:e5542339dcf2747135c5c85f68680353d5cb9ffd741c0f2e8d832d054d41f35a", size = 342361, upload-time = "2025-10-06T14:10:21.124Z" }, + { url = "https://files.pythonhosted.org/packages/d4/f8/33b92454789dde8407f156c00303e9a891f1f51a0330b0fad7c909f87692/yarl-1.22.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5c401e05ad47a75869c3ab3e35137f8468b846770587e70d71e11de797d113df", size = 387036, upload-time = "2025-10-06T14:10:22.902Z" }, + { url = "https://files.pythonhosted.org/packages/d9/9a/c5db84ea024f76838220280f732970aa4ee154015d7f5c1bfb60a267af6f/yarl-1.22.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:243dda95d901c733f5b59214d28b0120893d91777cb8aa043e6ef059d3cddfe2", size = 397671, upload-time = "2025-10-06T14:10:24.523Z" }, + { url = "https://files.pythonhosted.org/packages/11/c9/cd8538dc2e7727095e0c1d867bad1e40c98f37763e6d995c1939f5fdc7b1/yarl-1.22.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bec03d0d388060058f5d291a813f21c011041938a441c593374da6077fe21b1b", size = 377059, upload-time = "2025-10-06T14:10:26.406Z" }, + { url = "https://files.pythonhosted.org/packages/a1/b9/ab437b261702ced75122ed78a876a6dec0a1b0f5e17a4ac7a9a2482d8abe/yarl-1.22.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:b0748275abb8c1e1e09301ee3cf90c8a99678a4e92e4373705f2a2570d581273", size = 365356, upload-time = "2025-10-06T14:10:28.461Z" }, + { url = "https://files.pythonhosted.org/packages/b2/9d/8e1ae6d1d008a9567877b08f0ce4077a29974c04c062dabdb923ed98e6fe/yarl-1.22.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:47fdb18187e2a4e18fda2c25c05d8251a9e4a521edaed757fef033e7d8498d9a", size = 361331, upload-time = "2025-10-06T14:10:30.541Z" }, + { url = "https://files.pythonhosted.org/packages/ca/5a/09b7be3905962f145b73beb468cdd53db8aa171cf18c80400a54c5b82846/yarl-1.22.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:c7044802eec4524fde550afc28edda0dd5784c4c45f0be151a2d3ba017daca7d", size = 382590, upload-time = "2025-10-06T14:10:33.352Z" }, + { url = "https://files.pythonhosted.org/packages/aa/7f/59ec509abf90eda5048b0bc3e2d7b5099dffdb3e6b127019895ab9d5ef44/yarl-1.22.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:139718f35149ff544caba20fce6e8a2f71f1e39b92c700d8438a0b1d2a631a02", size = 385316, upload-time = "2025-10-06T14:10:35.034Z" }, + { url = "https://files.pythonhosted.org/packages/e5/84/891158426bc8036bfdfd862fabd0e0fa25df4176ec793e447f4b85cf1be4/yarl-1.22.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e1b51bebd221006d3d2f95fbe124b22b247136647ae5dcc8c7acafba66e5ee67", size = 374431, upload-time = "2025-10-06T14:10:37.76Z" }, + { url = "https://files.pythonhosted.org/packages/88/fc/6908f062a2f77b5f9f6d69cecb1747260831ff206adcbc5b510aff88df91/yarl-1.22.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:719ae08b6972befcba4310e49edb1161a88cdd331e3a694b84466bd938a6ab10", size = 146209, upload-time = "2025-10-06T14:10:44.643Z" }, + { url = "https://files.pythonhosted.org/packages/65/47/76594ae8eab26210b4867be6f49129861ad33da1f1ebdf7051e98492bf62/yarl-1.22.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:47d8a5c446df1c4db9d21b49619ffdba90e77c89ec6e283f453856c74b50b9e3", size = 95966, upload-time = "2025-10-06T14:10:46.554Z" }, + { url = "https://files.pythonhosted.org/packages/ab/ce/05e9828a49271ba6b5b038b15b3934e996980dd78abdfeb52a04cfb9467e/yarl-1.22.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:cfebc0ac8333520d2d0423cbbe43ae43c8838862ddb898f5ca68565e395516e9", size = 97312, upload-time = "2025-10-06T14:10:48.007Z" }, + { url = "https://files.pythonhosted.org/packages/d1/c5/7dffad5e4f2265b29c9d7ec869c369e4223166e4f9206fc2243ee9eea727/yarl-1.22.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4398557cbf484207df000309235979c79c4356518fd5c99158c7d38203c4da4f", size = 361967, upload-time = "2025-10-06T14:10:49.997Z" }, + { url = "https://files.pythonhosted.org/packages/50/b2/375b933c93a54bff7fc041e1a6ad2c0f6f733ffb0c6e642ce56ee3b39970/yarl-1.22.0-cp313-cp313t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2ca6fd72a8cd803be290d42f2dec5cdcd5299eeb93c2d929bf060ad9efaf5de0", size = 323949, upload-time = "2025-10-06T14:10:52.004Z" }, + { url = "https://files.pythonhosted.org/packages/66/50/bfc2a29a1d78644c5a7220ce2f304f38248dc94124a326794e677634b6cf/yarl-1.22.0-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ca1f59c4e1ab6e72f0a23c13fca5430f889634166be85dbf1013683e49e3278e", size = 361818, upload-time = "2025-10-06T14:10:54.078Z" }, + { url = "https://files.pythonhosted.org/packages/46/96/f3941a46af7d5d0f0498f86d71275696800ddcdd20426298e572b19b91ff/yarl-1.22.0-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c5010a52015e7c70f86eb967db0f37f3c8bd503a695a49f8d45700144667708", size = 372626, upload-time = "2025-10-06T14:10:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/c1/42/8b27c83bb875cd89448e42cd627e0fb971fa1675c9ec546393d18826cb50/yarl-1.22.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9d7672ecf7557476642c88497c2f8d8542f8e36596e928e9bcba0e42e1e7d71f", size = 341129, upload-time = "2025-10-06T14:10:57.985Z" }, + { url = "https://files.pythonhosted.org/packages/49/36/99ca3122201b382a3cf7cc937b95235b0ac944f7e9f2d5331d50821ed352/yarl-1.22.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:3b7c88eeef021579d600e50363e0b6ee4f7f6f728cd3486b9d0f3ee7b946398d", size = 346776, upload-time = "2025-10-06T14:10:59.633Z" }, + { url = "https://files.pythonhosted.org/packages/85/b4/47328bf996acd01a4c16ef9dcd2f59c969f495073616586f78cd5f2efb99/yarl-1.22.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:f4afb5c34f2c6fecdcc182dfcfc6af6cccf1aa923eed4d6a12e9d96904e1a0d8", size = 334879, upload-time = "2025-10-06T14:11:01.454Z" }, + { url = "https://files.pythonhosted.org/packages/c2/ad/b77d7b3f14a4283bffb8e92c6026496f6de49751c2f97d4352242bba3990/yarl-1.22.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:59c189e3e99a59cf8d83cbb31d4db02d66cda5a1a4374e8a012b51255341abf5", size = 350996, upload-time = "2025-10-06T14:11:03.452Z" }, + { url = "https://files.pythonhosted.org/packages/81/c8/06e1d69295792ba54d556f06686cbd6a7ce39c22307100e3fb4a2c0b0a1d/yarl-1.22.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:5a3bf7f62a289fa90f1990422dc8dff5a458469ea71d1624585ec3a4c8d6960f", size = 356047, upload-time = "2025-10-06T14:11:05.115Z" }, + { url = "https://files.pythonhosted.org/packages/4b/b8/4c0e9e9f597074b208d18cef227d83aac36184bfbc6eab204ea55783dbc5/yarl-1.22.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:de6b9a04c606978fdfe72666fa216ffcf2d1a9f6a381058d4378f8d7b1e5de62", size = 342947, upload-time = "2025-10-06T14:11:08.137Z" }, + { url = "https://files.pythonhosted.org/packages/46/b3/e20ef504049f1a1c54a814b4b9bed96d1ac0e0610c3b4da178f87209db05/yarl-1.22.0-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:34b36c2c57124530884d89d50ed2c1478697ad7473efd59cfd479945c95650e4", size = 140520, upload-time = "2025-10-06T14:11:15.465Z" }, + { url = "https://files.pythonhosted.org/packages/e4/04/3532d990fdbab02e5ede063676b5c4260e7f3abea2151099c2aa745acc4c/yarl-1.22.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:0dd9a702591ca2e543631c2a017e4a547e38a5c0f29eece37d9097e04a7ac683", size = 93504, upload-time = "2025-10-06T14:11:17.106Z" }, + { url = "https://files.pythonhosted.org/packages/11/63/ff458113c5c2dac9a9719ac68ee7c947cb621432bcf28c9972b1c0e83938/yarl-1.22.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:594fcab1032e2d2cc3321bb2e51271e7cd2b516c7d9aee780ece81b07ff8244b", size = 94282, upload-time = "2025-10-06T14:11:19.064Z" }, + { url = "https://files.pythonhosted.org/packages/a7/bc/315a56aca762d44a6aaaf7ad253f04d996cb6b27bad34410f82d76ea8038/yarl-1.22.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f3d7a87a78d46a2e3d5b72587ac14b4c16952dd0887dbb051451eceac774411e", size = 372080, upload-time = "2025-10-06T14:11:20.996Z" }, + { url = "https://files.pythonhosted.org/packages/3f/3f/08e9b826ec2e099ea6e7c69a61272f4f6da62cb5b1b63590bb80ca2e4a40/yarl-1.22.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:852863707010316c973162e703bddabec35e8757e67fcb8ad58829de1ebc8590", size = 338696, upload-time = "2025-10-06T14:11:22.847Z" }, + { url = "https://files.pythonhosted.org/packages/e3/9f/90360108e3b32bd76789088e99538febfea24a102380ae73827f62073543/yarl-1.22.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:131a085a53bfe839a477c0845acf21efc77457ba2bcf5899618136d64f3303a2", size = 387121, upload-time = "2025-10-06T14:11:24.889Z" }, + { url = "https://files.pythonhosted.org/packages/98/92/ab8d4657bd5b46a38094cfaea498f18bb70ce6b63508fd7e909bd1f93066/yarl-1.22.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:078a8aefd263f4d4f923a9677b942b445a2be970ca24548a8102689a3a8ab8da", size = 394080, upload-time = "2025-10-06T14:11:27.307Z" }, + { url = "https://files.pythonhosted.org/packages/f5/e7/d8c5a7752fef68205296201f8ec2bf718f5c805a7a7e9880576c67600658/yarl-1.22.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bca03b91c323036913993ff5c738d0842fc9c60c4648e5c8d98331526df89784", size = 372661, upload-time = "2025-10-06T14:11:29.387Z" }, + { url = "https://files.pythonhosted.org/packages/b6/2e/f4d26183c8db0bb82d491b072f3127fb8c381a6206a3a56332714b79b751/yarl-1.22.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:68986a61557d37bb90d3051a45b91fa3d5c516d177dfc6dd6f2f436a07ff2b6b", size = 364645, upload-time = "2025-10-06T14:11:31.423Z" }, + { url = "https://files.pythonhosted.org/packages/80/7c/428e5812e6b87cd00ee8e898328a62c95825bf37c7fa87f0b6bb2ad31304/yarl-1.22.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:4792b262d585ff0dff6bcb787f8492e40698443ec982a3568c2096433660c694", size = 355361, upload-time = "2025-10-06T14:11:33.055Z" }, + { url = "https://files.pythonhosted.org/packages/ec/2a/249405fd26776f8b13c067378ef4d7dd49c9098d1b6457cdd152a99e96a9/yarl-1.22.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:ebd4549b108d732dba1d4ace67614b9545b21ece30937a63a65dd34efa19732d", size = 381451, upload-time = "2025-10-06T14:11:35.136Z" }, + { url = "https://files.pythonhosted.org/packages/67/a8/fb6b1adbe98cf1e2dd9fad71003d3a63a1bc22459c6e15f5714eb9323b93/yarl-1.22.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:f87ac53513d22240c7d59203f25cc3beac1e574c6cd681bbfd321987b69f95fd", size = 383814, upload-time = "2025-10-06T14:11:37.094Z" }, + { url = "https://files.pythonhosted.org/packages/d9/f9/3aa2c0e480fb73e872ae2814c43bc1e734740bb0d54e8cb2a95925f98131/yarl-1.22.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:22b029f2881599e2f1b06f8f1db2ee63bd309e2293ba2d566e008ba12778b8da", size = 370799, upload-time = "2025-10-06T14:11:38.83Z" }, + { url = "https://files.pythonhosted.org/packages/06/5e/a15eb13db90abd87dfbefb9760c0f3f257ac42a5cac7e75dbc23bed97a9f/yarl-1.22.0-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:45c2842ff0e0d1b35a6bf1cd6c690939dacb617a70827f715232b2e0494d55d1", size = 146223, upload-time = "2025-10-06T14:11:46.796Z" }, + { url = "https://files.pythonhosted.org/packages/18/82/9665c61910d4d84f41a5bf6837597c89e665fa88aa4941080704645932a9/yarl-1.22.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:d947071e6ebcf2e2bee8fce76e10faca8f7a14808ca36a910263acaacef08eca", size = 95981, upload-time = "2025-10-06T14:11:48.845Z" }, + { url = "https://files.pythonhosted.org/packages/5d/9a/2f65743589809af4d0a6d3aa749343c4b5f4c380cc24a8e94a3c6625a808/yarl-1.22.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:334b8721303e61b00019474cc103bdac3d7b1f65e91f0bfedeec2d56dfe74b53", size = 97303, upload-time = "2025-10-06T14:11:50.897Z" }, + { url = "https://files.pythonhosted.org/packages/b0/ab/5b13d3e157505c43c3b43b5a776cbf7b24a02bc4cccc40314771197e3508/yarl-1.22.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1e7ce67c34138a058fd092f67d07a72b8e31ff0c9236e751957465a24b28910c", size = 361820, upload-time = "2025-10-06T14:11:52.549Z" }, + { url = "https://files.pythonhosted.org/packages/fb/76/242a5ef4677615cf95330cfc1b4610e78184400699bdda0acb897ef5e49a/yarl-1.22.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:d77e1b2c6d04711478cb1c4ab90db07f1609ccf06a287d5607fcd90dc9863acf", size = 323203, upload-time = "2025-10-06T14:11:54.225Z" }, + { url = "https://files.pythonhosted.org/packages/8c/96/475509110d3f0153b43d06164cf4195c64d16999e0c7e2d8a099adcd6907/yarl-1.22.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c4647674b6150d2cae088fc07de2738a84b8bcedebef29802cf0b0a82ab6face", size = 363173, upload-time = "2025-10-06T14:11:56.069Z" }, + { url = "https://files.pythonhosted.org/packages/c9/66/59db471aecfbd559a1fd48aedd954435558cd98c7d0da8b03cc6c140a32c/yarl-1.22.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:efb07073be061c8f79d03d04139a80ba33cbd390ca8f0297aae9cce6411e4c6b", size = 373562, upload-time = "2025-10-06T14:11:58.783Z" }, + { url = "https://files.pythonhosted.org/packages/03/1f/c5d94abc91557384719da10ff166b916107c1b45e4d0423a88457071dd88/yarl-1.22.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e51ac5435758ba97ad69617e13233da53908beccc6cfcd6c34bbed8dcbede486", size = 339828, upload-time = "2025-10-06T14:12:00.686Z" }, + { url = "https://files.pythonhosted.org/packages/5f/97/aa6a143d3afba17b6465733681c70cf175af89f76ec8d9286e08437a7454/yarl-1.22.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:33e32a0dd0c8205efa8e83d04fc9f19313772b78522d1bdc7d9aed706bfd6138", size = 347551, upload-time = "2025-10-06T14:12:02.628Z" }, + { url = "https://files.pythonhosted.org/packages/43/3c/45a2b6d80195959239a7b2a8810506d4eea5487dce61c2a3393e7fc3c52e/yarl-1.22.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:bf4a21e58b9cde0e401e683ebd00f6ed30a06d14e93f7c8fd059f8b6e8f87b6a", size = 334512, upload-time = "2025-10-06T14:12:04.871Z" }, + { url = "https://files.pythonhosted.org/packages/86/a0/c2ab48d74599c7c84cb104ebd799c5813de252bea0f360ffc29d270c2caa/yarl-1.22.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:e4b582bab49ac33c8deb97e058cd67c2c50dac0dd134874106d9c774fd272529", size = 352400, upload-time = "2025-10-06T14:12:06.624Z" }, + { url = "https://files.pythonhosted.org/packages/32/75/f8919b2eafc929567d3d8411f72bdb1a2109c01caaab4ebfa5f8ffadc15b/yarl-1.22.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:0b5bcc1a9c4839e7e30b7b30dd47fe5e7e44fb7054ec29b5bb8d526aa1041093", size = 357140, upload-time = "2025-10-06T14:12:08.362Z" }, + { url = "https://files.pythonhosted.org/packages/cf/72/6a85bba382f22cf78add705d8c3731748397d986e197e53ecc7835e76de7/yarl-1.22.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:c0232bce2170103ec23c454e54a57008a9a72b5d1c3105dc2496750da8cfa47c", size = 341473, upload-time = "2025-10-06T14:12:10.994Z" }, + { url = "https://files.pythonhosted.org/packages/73/ae/b48f95715333080afb75a4504487cbe142cae1268afc482d06692d605ae6/yarl-1.22.0-py3-none-any.whl", hash = "sha256:1380560bdba02b6b6c90de54133c81c9f2a453dee9912fe58c1dcced1edb7cff", size = 46814, upload-time = "2025-10-06T14:12:53.872Z" }, +]