Compare commits

...

70 Commits

Author SHA1 Message Date
Alex Cheema 6a1f6d734a feat: auto-discover models from HuggingFace Hub cache and custom paths
Searches ~/.cache/huggingface/hub by default so models downloaded by
other tools (HF CLI, transformers, LM Studio, etc.) are used without
re-downloading. Adds EXO_MODELS_PATH env var support and a Model Search
Paths setting in the macOS app.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 09:56:02 -08:00
Alex Cheema 2cf31a195b fix: remove default parameters and log ValueError per PR review
- Remove `= None` defaults from `on_layer_loaded`, `on_timeout`, and
  `timeout_seconds` parameters to be intentional about caller intent
- Replace silent `except ValueError: pass` with logger.debug so we
  know when a model architecture doesn't support layer-by-layer
  progress tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 07:20:04 -08:00
Alex Cheema 296ec9eeab fix: address PR review comments — onboarding file, layer callback indexing, RDMA warning
- Move onboarding_complete file to EXO_CACHE_HOME and make it a module-level constant
- Fix layer loaded callback to use 0-indexed layer count
- Update runner's current_status in on_layer_loaded callback via nonlocal
- Apply RDMA "NOT ENABLED" warning styling from main (yellow + tooltip)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:10:57 -08:00
Alex Cheema d6af5126bc fix: toast system reflects real state transitions, remove model picker auto-launch
- Remove auto-launch from handleModelPickerSelect() — selecting a model
  no longer immediately launches it, user must click LAUNCH
- Change launch toast from misleading "success" to "info" acknowledgment
- Add $effect tracking instance status transitions to fire toasts on real
  state changes: download complete, model ready, model failed, shutdown
- Add connection lost/recovered toast notifications
- Update download status message copy

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:08:15 -08:00
Alex Cheema c3da520a84 style: apply nix fmt formatting fixes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:08:02 -08:00
Alex Cheema ee7db70d6c feat: show real topology on onboarding step 1, simulated device indicator
- Step 1 now shows the actual TopologyGraph with all connected devices
- Cross-fade transition from real topology to device pair animation on step 2
- Simulated Mac Studio has dashed outline border and "(example)" label

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema bcf491b166 fix: rename interconnect label from "Standard" to "TCP/IP"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 441b12e3ee fix: tensor parallel loading progress and remove _eval_layers_with_progress
- Add on_layer_loaded callback to tensor_auto_parallel and all 8 sharding
  strategies so per-layer progress fires for tensor parallel models
- Remove _eval_layers_with_progress helper, inline layer eval in single-device path
- Fix dashboard aggregation: use max/min for tensor parallel (all runners load
  all layers), sum for pipeline parallel (disjoint slices)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 86e42b8926 fix: single-device loading progress, warmup status text, yellow unlock animation
- Load model lazily on single device and eval layers one-by-one with progress callback
- Separate LOADING from WARMING UP status so warmup shows "Warming up..." not "Loading model into memory..."
- Add warm yellow glow effect to unlock animation chips with peak-and-settle highlight

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema ebf3cdfd72 feat: model loading progress bars with layer-by-layer tracking
Emit RunnerLoading progress (layers_loaded/total_layers) during pipeline
parallel model loading and display progress bars in both onboarding step 8
and the main instances view.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 885a84c4d1 fix: make models unlocked animation elegant instead of flashy
Remove glow filters, burst rings, scale pops, gold flashing, and dramatic
header scale-in. Replace with clean staggered fade-in of subtle chips with
white/transparent styling that fits the overall aesthetic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 95efd291d4 fix: onboarding step advancement only tracks the launched model's instance
Was checking ALL instances, so an already-running Llama instance would
make the step logic think the newly-launched model was ready, skipping
download and loading steps. Now filters by onboardingModelId.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema f628110a98 fix: onboarding no longer skips download/loading steps for uncached models
The step 7→8→9 advancement was using download data availability as a proxy
for readiness. When download data hadn't arrived yet, it defaulted to
"RUNNING" status, causing immediate skip to the chat input.

Now uses actual runner status (deriveInstanceStatus) as the source of truth:
- Only advances to step 9 when runners report READY/LOADED/RUNNING
- Only advances to step 8 when runners report LOADING/WARMING UP
- Stays on step 7 while downloading or in early states (PREPARING, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema f5c424c162 fix: bump launch defaults localStorage key to v2
Avoids stale MlxIbv values from old cache.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 20bd82cdd4 fix: sanitize MlxIbv from cached launch defaults in localStorage
Old localStorage values may contain MlxIbv — normalize to MlxJaccl on load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 6019174175 fix: remove all remaining MlxIbv references across dashboard
Missed ModelCard, ChatSidebar, and app store — now fully MlxJaccl everywhere.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema ab9d7ccf1f fix: use /node_id endpoint to identify own device in onboarding
Fetch our node_id on mount and use it to look up the correct node
in the topology, so the device name/icon matches the actual machine.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema e0cfa1574e fix: replace all MlxIbv references with MlxJaccl
MlxIbv is no longer a valid instance type. Use MlxJaccl everywhere.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 50f10f43c8 fix: combined GB shows current device + simulated 256GB Mac Studio only
Instead of summing all cluster nodes, use just the current device's memory
plus the simulated 256GB Mac Studio for the "XGB combined" display.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 5aceedbbe8 fix: revert topology view on step 1, show single device with correct icon
- Remove TopologyGraph from step 1, back to single device icon (with correct
  device type e.g. Mac Studio)
- Change label from "N Devices in your EXO network" to "Your EXO Network"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema eb761c154a fix: onboarding polish — topology on step 1, 2L/2M/2S model selection, device type fix
- Show actual TopologyGraph on step 1 instead of single device icon
- Select 6 models across size tiers (2 large, 2 medium, 2 small) with Llama 3.2 3B pinned
- Fix device type detection to use system_info.model_id (Mac Studio shows correctly)
- Fix launchInstance to use /instance or /place_instance instead of nonexistent /meta_instance
- Slim LLM/shard rectangles, adjust shard spacing below device names

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema f1bbe74bc6 fix: shard label spacing + show real models in unlock animation
- Reduced shard split distance so Shard 1/2 and Shard 2/2 labels don't
  overlap with device names
- Unlocked models now filter to models with a known family, deduplicate
  by base_model, and pick smallest quant per base — shows recognizable
  models like Llama-3.3-70B, Qwen3-Next-80B instead of obscure quants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema ce638fffb5 feat: persist onboarding completion to ~/.exo so it survives browser clears
Added GET/POST /onboarding API endpoints that read/write
~/.exo/onboarding_complete. Dashboard checks server first, falls back
to localStorage. Onboarding only shows once from terminal.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 7d6d2626f6 feat: show dashboard popover on every launch with 10s countdown
- Popover now appears on every launch, not just first time
- Countdown reduced from 30s to 10s
- Text updated: "EXO is running" + "Launching in N secs..."

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema f303b396f3 fix: model dropdown positioning on onboarding chat screen + sort models biggest-first
The model selector dropdown on step 9 (chat input) was appearing in the
top-right corner because CSS transform in the onb-fade-in animation
created a containing block for fixed-position elements. Added a new
onb-fade-opacity keyframe (opacity only, no transform) for step 9.

Also simplified onboarding model list to show biggest-to-smallest models
that fit in available memory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema c72a543088 feat: onboarding polish — smooth transitions, game-style model unlock, replay button
- Smooth step transitions: title/subtitle crossfade with tweened opacity instead of
  display:none or CSS animation remounts (no layout jumps between steps)
- Game-style "MODELS UNLOCKED" animation on step 2: chips appear locked (grey),
  then sequentially light up gold with glow filter and scale pulse
- Settled chip state uses white text on dark bg for readability (no yellow-on-yellow)
- Chips fade out gently (600ms) before step 3 model block appears (no overlap)
- Added Replay button (restart icon) next to Skip button (skip-forward icon)
- Fixed onb-fade-in CSS: transform:none instead of translateY(0) to avoid
  containing block issues with fixed-positioned dropdowns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 76c058cf97 Revert "fix: eliminate brownish yellow tints and enforce dark color scheme"
This reverts commit 0d9c48ce603f9947f8c49cd3678a44671e1df8f5.
2026-02-21 11:07:46 -08:00
Alex Cheema e5fb35d08e fix: eliminate brownish yellow tints and enforce dark color scheme
Replace all low-opacity yellow backgrounds (bg-yellow-500/10,
bg-amber-500/15, bg-exo-yellow/10-20) that created muddy brown tints
with clean neutral alternatives (bg-exo-medium-gray/50, bg-white/[0.06]).

- Add color-scheme: dark to HTML meta tag and CSS to force dark browser
  chrome on macOS light mode
- Replace text-yellow-200/300/400 and text-amber-400 with text-exo-yellow
  for consistent brand color usage
- Fix TopologyGraph node fills from transparent yellow to neutral white
  tints to eliminate brownish hover/selection states
- Clean up border colors from border-yellow-500/50 to border-exo-yellow/20
- Affects 15 files across dashboard components

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema f30849ea7a fix: onboarding UX overhaul — smooth transitions, bigger graphics, correct API
- Add smooth fade transitions between steps (advanceStep with 350ms cross-fade)
- Increase device graphics from size 75 to 110 in SVG canvas (420px tall)
- Use exo LOGO (w-20 h-20) prominently on step 1, not text
- Rewrite step 1 as Headspace-style: "Your home AI cluster" with clear explanation
- Change step 2 title to "More devices. Larger models." with auto-advance
- Steps 2-4 all auto-animate (removed unnecessary continue buttons)
- Step 5 title: "exo self-heals when devices fail" with detailed subtitle
- Redesign step 9: clean "You're all set" with Start Chatting button
- Fix dashboard layout: move welcome overlay to bottom of topology (no more overlap)
- Replace "exo" text overlay with just the Choose a Model button
- Add fade-out transition when completing onboarding
- Use cubicInOut easing for smoother device/model animations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 4d5b976812 fix: onboarding UX overhaul — smooth transitions, bigger graphics, correct API
1. Smooth transitions: CSS keyframe animations instead of janky fade/appear
2. Device graphics bigger: size 55→75, viewBox 300→360px height
3. Exo logo: replaced text "exo" with actual logo image
4. Device context: Headspace-style intro text explaining distributed inference
5. Better messaging: "More devices = Larger models", memory combining explained
6. Auto-advance: steps 3 (model split) and 4 (disconnect) auto-advance, no click
7. Self-healing detail: explicit text about automatic redistribution
8. LLM not "LLM Model": fixed redundancy, cleaner final chat step
9. Dashboard layout: fixed overflow-visible causing post-skip layout jumbling
10. CRITICAL: Fixed API call using standard placement API, not meta-instance

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 7aaf041823 feat: onboarding animation redesign - 5-step device join/leave/heal sequence
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 46f4065a52 fix: onboarding UX improvements - DMG layout, NSPopover, sidebar toggle, icon border
- Restore simple DMG creation in CI (AppleScript doesn't work headless)
- Improve NSPopover reliability: more retries, broader window detection, fallback
- Fix sidebar toggle visibility with brighter border
- Reduce exo logo drop-shadow from 20px to 4px

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema d039c4c304 feat: add Skip button to all onboarding steps
Subtle text link at the bottom of every onboarding step that skips
directly to the dashboard. Positioned unobtrusively so it doesn't
compete with primary actions but is available for advanced users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:46 -08:00
Alex Cheema 912841bf0c fix: upgrade onboarding step 3 devices to match TopologyGraph quality
Replace flat SVG boxes with high-fidelity device renderings matching the
TopologyGraph component: MacBook Pro with screen, Apple logo, keyboard,
trackpad, and memory fill; Mac Studio with cube body, front panel slots,
and memory fill. Adjust layout spacing so "228 GB combined" label sits
cleanly between devices.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema ff05afc5b6 fix: merge main and update onboarding to use /instance API
Main reverted MetaInstance layer; update onboarding model launch to use
the /instance endpoint instead of the removed /meta_instance. Apply
nix fmt formatting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 0ef31a9a54 fix: convert onboarding wizard from light mode to dark theme
The onboarding wizard was using bg-white, text-gray-*, bg-gray-*, and
rgba(0,0,0,...) SVG fills designed for light backgrounds, inconsistent
with the dark-themed dashboard. Converted all 33+ instances to dark
equivalents (bg-exo-black, text-white/*, bg-white/*, rgba(255,255,255,...)).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 0d372741ec fix: onboarding flow order, auto-download, sidebar visibility
- Reorder onboarding: Welcome → Your Devices (topology) → Add More
  Devices (animation) → Choose Model, so users see their own cluster
  before the "add more" pitch
- Auto-launch model on select in main dashboard model picker so
  downloads begin immediately
- Hide chat sidebar and its toggle on the home/welcome view for a
  cleaner first impression (only show when a chat is active)
- Fix step 7 branding text contrast on white background

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 85fcbfa55f chore: retrigger CI 2026-02-21 11:07:45 -08:00
Alex Cheema 67f78e6145 chore: trigger CI 2026-02-21 11:07:45 -08:00
Alex Cheema 1436af1d55 fix: use meta_instance API and remove invalid derived assignment
The merge resolution kept pre-#1447 code that assigned to `instanceData`
(a Svelte 5 $derived constant) and used the old /instance endpoint.
Switch both launchInstance and onboardingLaunchModel to POST /meta_instance.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 10ac96be0b style: fix Svelte formatting for CI treefmt check
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 349a88aa3b chore: remove temporary screenshot files 2026-02-21 11:07:45 -08:00
Alex Cheema fc1122051f temp: add onboarding screenshots for PR comment 2026-02-21 11:07:45 -08:00
Alex Cheema 312b7a7580 feat: revert to light background, fix DMG layout arrow
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 47e3714e31 fix: revert DMG background, icon border, popover positioning, onboarding topology
Address Gary's Feb 16 feedback on the better-onboarding PR:

- Revert DMG background to white Ollama-style (was changed to dark)
- Revert app icon to original (remove added white border)
- Fix DMG text size (10 → 12) for readability on white background
- Fix first-launch popover: add retry mechanism for status bar button
  detection, search view hierarchy for NSStatusBarButton, auto-dismiss
  after 30s countdown
- Scale up onboarding step 2 SVG animation: devices ~1.5x larger,
  model cards ~1.35x larger, increased font sizes, taller container

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 7a91a51336 fix: hide main model picker during onboarding so Browse All Models triggers download
During onboarding, two ModelPickerModal instances were rendered — the
onboarding-specific one (which calls onboardingLaunchModel to start
downloading) and the always-rendered main one (which only selects a
preview). The main modal intercepted the selection, so Browse All Models
never triggered a download. Wrapping the main modal with {#if !showOnboarding}
ensures only the onboarding modal is active during the onboarding flow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 8d3a06b53b fix: anchor welcome popover to menu bar icon as NSPopover
Replaces standalone NSPanel with NSPopover anchored to menu bar status item.
Adds findStatusItemButton() helper, UserDefaults persistence in dismiss().
2026-02-21 11:07:45 -08:00
Alex Cheema d847fa9b68 fix: DMG design — white text labels and thinner icon border
Bake white "EXO.app" and "Applications" labels into the DMG background
image so they are legible on the dark background (Finder renders its own
labels in black which is invisible on dark). Reduce Finder text size to
10 (minimum) to minimise the native black labels.

Reduce the app icon white border from ~11px to ~2px at 1024 scale and
regenerate all icon sizes from the updated master.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 1403d7dfc8 feat: add onboarding step 2 with device cluster animation
New animated step shows "Add more devices, run bigger models" with a
sequenced SVG animation: MacBook appears, Mac Studio flies in, a
connection line forms, and locked model cards (30B, 72B, 405B) unlock
with golden glow as combined compute increases. Existing steps 2-6
renumbered to 3-7.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 857cda595b fix: make sidebar toggle chevron visible on dark background
The collapsed-state chevron used text-exo-medium-gray (oklch 0.22)
on bg-exo-dark-gray (oklch 0.16), making it nearly invisible.
Changed to text-exo-light-gray (oklch 0.6) to match other inactive
header elements like Home and Downloads.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 18da2a66b1 fix: first-launch popover not appearing on fresh install
The .onReceive(controller.$isFirstLaunchReady) was inside the
MenuBarExtra content closure, which is lazily rendered only when
the user clicks the menu bar icon. Moved it to the label view
(always rendered), added 1s delay for status bar setup, and
switched to EXOOnboardingCompleted flag that is only set on
dismiss (so a failed show retries next launch).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 5e4531bc8b fix: track onboarding completion separately, extend popover to 30s
Replace EXOHasLaunchedBefore (set immediately on launch) with
EXOOnboardingCompleted (set only when user interacts with the popout).
This ensures onboarding re-triggers on reinstall if never completed.

Extend auto-open timer from 5s to 30s and remove auto-dismiss so the
popover persists until the user clicks Open Dashboard or X.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 5db9030535 style: dark DMG background and white icon border
Dark (#1a1a1a) DMG installer background with inverted white arrow,
matching exo's dark theme. Added light border outline around EXO.app
icon so it stands out against the dark background.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema 7933d4cd77 feat: add skeleton loading, download progress in header, instance status badges
- Add "Connecting to cluster..." spinner overlay before first data fetch arrives
- Show welcome overlay only after initial connection (prevents flash of empty state)
- Add compact circular download progress indicator in header nav (replaces
  static download icon when downloads are active, shows count + ring progress)
- Add colored status badge (DOWNLOADING/LOADING/READY/RUNNING/FAILED) on
  instance cards in both welcome and chat views for clearer state visibility
- Improve chat empty state with keyboard shortcut hint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema bf17e2a4f5 feat: add sharding/runtime tooltips, ARIA landmarks, download status, recommended models
- Add explanatory tooltips for Pipeline/Tensor sharding and Ring/RDMA runtime options in ModelCard
- Render download progress bar (percentage, speed, ETA) when model is downloading in ModelCard
- Add ARIA landmarks: complementary role on sidebars, aria-labels on icon buttons,
  aria-live region on chat messages, aria-expanded on toggle buttons, nav element in header
- Add "Recommended for your cluster" section in model picker Available tab that visually
  separates models that fit in available memory from those that don't

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:45 -08:00
Alex Cheema b8e792be96 style: apply nix fmt formatting to dashboard files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 928cdd87fa feat: dashboard polish — toast notifications, connection banner, visual improvements
- Add global toast notification system (success/error/warning/info) with
  auto-dismiss progress bar and stacked bottom-right layout
- Add connection loss detection (3 consecutive failures) with persistent
  red banner: "Connection lost — Reconnecting to backend..."
- Wire toasts to model launch (success + error), placement errors, and
  instance deletion failures
- Enhance welcome overlay: dynamic device count message, fade-in
  animation, quick-hint links
- Add slide transitions on instance cards (appear/disappear)
- Add hover glow effect on instance cards (border brightens on hover)
- Add prefers-reduced-motion support: disables shooting stars, graph
  flow animation, pulse effects, and forces minimal transition durations
- Add ARIA landmarks to toast container (role="log", aria-live="polite")
  and connection banner (role="alert", aria-live="assertive")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 74451ea362 style: apply nix fmt to FirstLaunchPopout.swift
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 72487f7a73 style: polish onboarding popout and dashboard UI
FirstLaunchPopout: rename to WelcomeCalloutView, use transparent panel
with ultraThinMaterial instead of HUD style, add fade in/out animations,
friendlier copy ("Welcome to EXO!", "Run your first model here:"),
smaller 280×100 window, dismiss on Open Dashboard click.

Dashboard onboarding: switch body text from font-mono to font-sans,
rounded-full buttons with softer hover states, subtler secondary text
opacity, cleaner copy ("Your devices" vs "Here are your devices"),
rounded-xl model cards, refined spacing throughout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema cde03fbcfd fix: use Ollama-style DMG background (white, hand-drawn arrow, retina)
Replace procedural dark background with Ollama's MIT-licensed static
background asset: white canvas, hand-drawn curved arrow, yellow bookmark
accents. Window resized to 800×400 with app left / Applications right
matching standard macOS DMG conventions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 35c8021bc8 fix: move DMG icons and labels up 30px to reduce top padding
Shift icon center from y=160 to y=130, labels from y=232 to y=202,
and "Drag to install" from y=340 to y=310 for better vertical centering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 046e530d2f fix: DMG AppleScript text size 1 → 10 (Finder minimum is 10)
Finder rejects `set text size of opts to 1` with AppleEvent handler
failed (-10000). The minimum valid text size is 10.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 239315586f style: format generate-background.py for ruff line length
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 84f7e89b22 fix: move chat input to onboarding step 6, fix DMG white text and layout
- Chat input with suggestion chips now appears as final onboarding step
  (step 6) instead of in the main dashboard — first message seamlessly
  transitions to dashboard
- Remove chat home screen from dashboard, revert HeaderNav gating
- Add onSend callback to ChatForm for onboarding integration
- DMG: increase window to 660×440, move icons up (y=160) to reduce
  top whitespace
- DMG: bake white "EXO" and "Applications" labels into background PNG,
  hide Finder's black labels via text size 1
- DMG: increase "Drag to install" to scale=3 for readability

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 4da92e5778 feat: Perplexity chat home, menu bar overhaul, Settings Advanced tab
- Show centered chat input with suggestion chips when model is loaded
  (replaces topology dashboard as default post-onboarding view)
- Redesign menu bar dropdown: remove yellow background, use SF Symbols,
  clean dividers, move debug/uninstall to Settings
- Add Advanced tab in Settings with Reset Onboarding, Debug Info,
  and Uninstall (moved from menu bar)
- Add multi-device message in onboarding step 2
- Fix timer task leak in FirstLaunchPopout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 34c181035e fix: DMG background — solid arrow + white text replacing broken Y-shape
Replace the chevron-style arrow (rendered as Y-shape) with a bold filled
triangle arrowhead + thick shaft. Use fully opaque white for text instead
of semi-transparent. Simplify background to solid dark color.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 51854d9063 fix: 5 CTO review issues — double open, derived bug, cached race, error UX, sharding visibility
1. Suppress Python webbrowser.open() when running inside native macOS app
   (EXO_RUNTIME_DIR set) to prevent double browser open with FirstLaunchPopout
2. Fix $derived(() => ...) → $derived.by(() => ...) for onboardingModels and
   onboardingDownloadProgress so they cache properly instead of returning thunks
3. Handle cached/already-downloaded models: step 4 effect checks for READY
   status and skips directly to step 6 instead of flashing download UI
4. Show error banner on step 3 when placement or launch fails instead of
   silently reverting
5. Remove debugEnabled guard from sharding/instanceType info — always visible
   for power users in both welcome and chat sidebars

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema a67de3b078 feat: complete onboarding rewrite, native settings, DMG polish, menu bar UX
- Full-screen Apple-style onboarding wizard (Welcome → Devices → Pick Model
  → Downloading → Loading → Ready) replacing the broken layered approach
- Native macOS Settings window (General/Model/About tabs) replacing the
  cramped Advanced dropdown section
- First-launch floating popout with 5s countdown that auto-opens dashboard
- Clean DMG installer: dark gradient, white anti-aliased arrow, no grid
- Menu bar: "Web Dashboard" with link icon, Base URL copy (localhost:52415/v1)
- UI renames: Sharding Strategy, Interconnect, Load Model
- Helpful "no model loaded" message instead of raw error on chat submit
- Reduce launch delay from 15s to 5s

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:07:16 -08:00
Alex Cheema 73f23f07e3 fix: parse hdiutil mount point correctly in create-dmg.sh
The awk command was matching the first '/' in the line (from /dev/diskXsY)
instead of the mount path (/Volumes/...). Use tab field separator and
extract the last field to get the correct mount directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:04:25 -08:00
Alex Cheema ca97d287c9 feat: better onboarding UX for new users
- Auto-open dashboard in browser on first launch
- Welcome overlay with "Choose a Model" button when no model is running
- Tutorial progress messages during download/loading/ready stages
- Fix conversation sidebar text contrast (white text on dark background)
- Hide sharding/instance type/min nodes behind "Advanced Options" toggle
- Polished DMG installer with drag-to-Applications layout and custom background
- Simplify technical jargon: rename labels, hide Strategy behind debug mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 11:04:25 -08:00
33 changed files with 4109 additions and 822 deletions
+59 -421
View File
@@ -15,18 +15,12 @@ struct ContentView: View {
@EnvironmentObject private var localNetworkChecker: LocalNetworkChecker
@EnvironmentObject private var updater: SparkleUpdater
@EnvironmentObject private var thunderboltBridgeService: ThunderboltBridgeService
@EnvironmentObject private var settingsWindowController: SettingsWindowController
@State private var focusedNode: NodeViewModel?
@State private var deletingInstanceIDs: Set<String> = []
@State private var showAllNodes = false
@State private var showAllInstances = false
@State private var showAdvanced = false
@State private var showDebugInfo = false
@State private var bugReportInFlight = false
@State private var bugReportMessage: String?
@State private var uninstallInProgress = false
@State private var pendingNamespace: String = ""
@State private var pendingHFToken: String = ""
@State private var pendingEnableImageModels = false
@State private var baseURLCopied = false
var body: some View {
VStack(alignment: .leading, spacing: 12) {
@@ -258,139 +252,79 @@ struct ContentView: View {
VStack(alignment: .leading, spacing: 0) {
if controller.status != .stopped {
dashboardButton
baseURLRow
Divider()
.padding(.vertical, 8)
} else {
Divider()
.padding(.vertical, 4)
}
advancedSection
.padding(.bottom, 8)
controlButton(title: "Quit", tint: .secondary) {
HoverButton(
title: "Settings",
tint: .primary,
trailingSystemImage: "gear"
) {
settingsWindowController.open(
controller: controller,
updater: updater,
networkStatusService: networkStatusService,
thunderboltBridgeService: thunderboltBridgeService,
stateService: stateService
)
}
HoverButton(
title: "Check for Updates",
tint: .primary,
trailingSystemImage: "arrow.triangle.2.circlepath"
) {
updater.checkForUpdates()
}
.padding(.bottom, 8)
HoverButton(title: "Quit", tint: .secondary) {
controller.stop()
NSApplication.shared.terminate(nil)
}
}
}
private var advancedSection: some View {
VStack(alignment: .leading, spacing: 6) {
HStack {
Text("Advanced")
.font(.caption)
.foregroundColor(.secondary)
Spacer()
collapseButton(isExpanded: $showAdvanced)
}
.animation(nil, value: showAdvanced)
if showAdvanced {
VStack(alignment: .leading, spacing: 8) {
VStack(alignment: .leading, spacing: 4) {
Text("Cluster Namespace")
.font(.caption2)
.foregroundColor(.secondary)
HStack {
TextField("optional", text: $pendingNamespace)
.textFieldStyle(.roundedBorder)
.font(.caption2)
.onAppear {
pendingNamespace = controller.customNamespace
}
Button("Save & Restart") {
controller.customNamespace = pendingNamespace
if controller.status == .running || controller.status == .starting {
controller.restart()
}
}
.font(.caption2)
.disabled(pendingNamespace == controller.customNamespace)
}
}
VStack(alignment: .leading, spacing: 4) {
Text("HuggingFace Token")
.font(.caption2)
.foregroundColor(.secondary)
HStack {
SecureField("optional", text: $pendingHFToken)
.textFieldStyle(.roundedBorder)
.font(.caption2)
.onAppear {
pendingHFToken = controller.hfToken
}
Button("Save & Restart") {
controller.hfToken = pendingHFToken
if controller.status == .running || controller.status == .starting {
controller.restart()
}
}
.font(.caption2)
.disabled(pendingHFToken == controller.hfToken)
}
}
Divider()
HStack {
Toggle(
"Enable Image Models (experimental)", isOn: $pendingEnableImageModels
)
.toggleStyle(.switch)
.font(.caption2)
.onAppear {
pendingEnableImageModels = controller.enableImageModels
}
Spacer()
Button("Save & Restart") {
controller.enableImageModels = pendingEnableImageModels
if controller.status == .running || controller.status == .starting {
controller.restart()
}
}
.font(.caption2)
.disabled(pendingEnableImageModels == controller.enableImageModels)
}
HoverButton(title: "Check for Updates", small: true) {
updater.checkForUpdates()
}
debugSection
HoverButton(title: "Uninstall", tint: .red, small: true) {
showUninstallConfirmationAlert()
}
.disabled(uninstallInProgress)
}
.transition(.opacity)
}
}
.animation(.easeInOut(duration: 0.25), value: showAdvanced)
}
private func controlButton(title: String, tint: Color = .primary, action: @escaping () -> Void)
-> some View
{
HoverButton(title: title, tint: tint, trailingSystemImage: nil, action: action)
}
private var dashboardButton: some View {
Button {
HoverButton(
title: "Web Dashboard",
tint: .primary,
trailingSystemImage: "arrow.up.right"
) {
guard let url = URL(string: "http://localhost:52415/") else { return }
NSWorkspace.shared.open(url)
} label: {
HStack {
Image(systemName: "arrow.up.right.square")
.imageScale(.small)
Text("Dashboard")
.fontWeight(.medium)
Spacer()
}
.padding(.vertical, 8)
.padding(.horizontal, 10)
.background(
RoundedRectangle(cornerRadius: 8, style: .continuous)
.fill(Color(red: 1.0, green: 0.87, blue: 0.0).opacity(0.2))
)
}
.buttonStyle(.plain)
.padding(.bottom, 4)
}
private var baseURLRow: some View {
HStack(spacing: 6) {
Image(systemName: "link")
.imageScale(.small)
.foregroundColor(.secondary)
Text("localhost:52415/v1")
.font(.system(.caption, design: .monospaced))
.foregroundColor(.primary)
Spacer()
Button {
NSPasteboard.general.clearContents()
NSPasteboard.general.setString("http://localhost:52415/v1", forType: .string)
baseURLCopied = true
DispatchQueue.main.asyncAfter(deadline: .now() + 2) {
baseURLCopied = false
}
} label: {
Image(systemName: baseURLCopied ? "checkmark" : "doc.on.doc")
.imageScale(.small)
.foregroundColor(baseURLCopied ? .green : .secondary)
.contentTransition(.symbolEffect(.replace))
}
.buttonStyle(.plain)
.help("Copy API base URL")
}
.padding(.vertical, 4)
.padding(.horizontal, 8)
}
private func collapseButton(isExpanded: Binding<Bool>) -> some View {
@@ -445,207 +379,6 @@ struct ContentView: View {
}
}
private var thunderboltStatusText: String {
switch networkStatusService.status.thunderboltBridgeState {
case .some(.disabled):
return "Thunderbolt Bridge: Disabled"
case .some(.deleted):
return "Thunderbolt Bridge: Deleted"
case .some(.enabled):
return "Thunderbolt Bridge: Enabled"
case nil:
return "Thunderbolt Bridge: Unknown"
}
}
private var thunderboltStatusColor: Color {
switch networkStatusService.status.thunderboltBridgeState {
case .some(.disabled), .some(.deleted):
return .green
case .some(.enabled):
return .red
case nil:
return .secondary
}
}
/// Shows TB bridge status for all nodes from exo cluster state
private var clusterThunderboltBridgeView: some View {
let bridgeStatuses = stateService.latestSnapshot?.nodeThunderboltBridge ?? [:]
let localNodeId = stateService.localNodeId
let nodeProfiles = stateService.latestSnapshot?.nodeProfiles ?? [:]
return VStack(alignment: .leading, spacing: 1) {
if bridgeStatuses.isEmpty {
Text("Cluster TB Bridge: No data")
.font(.caption2)
.foregroundColor(.secondary)
} else {
Text("Cluster TB Bridge Status:")
.font(.caption2)
.foregroundColor(.secondary)
ForEach(Array(bridgeStatuses.keys.sorted()), id: \.self) { nodeId in
if let status = bridgeStatuses[nodeId] {
let nodeName =
nodeProfiles[nodeId]?.friendlyName ?? String(nodeId.prefix(8))
let isLocal = nodeId == localNodeId
let prefix = isLocal ? " \(nodeName) (local):" : " \(nodeName):"
let statusText =
!status.exists
? "N/A"
: (status.enabled ? "Enabled" : "Disabled")
let color: Color =
!status.exists
? .secondary
: (status.enabled ? .red : .green)
Text("\(prefix) \(statusText)")
.font(.caption2)
.foregroundColor(color)
}
}
}
}
}
private var interfaceIpList: some View {
let statuses = networkStatusService.status.interfaceStatuses
return VStack(alignment: .leading, spacing: 1) {
Text("Interfaces (en0en7):")
.font(.caption2)
.foregroundColor(.secondary)
if statuses.isEmpty {
Text(" Unknown")
.font(.caption2)
.foregroundColor(.secondary)
} else {
ForEach(statuses, id: \.interfaceName) { status in
let ipText = status.ipAddress ?? "No IP"
Text(" \(status.interfaceName): \(ipText)")
.font(.caption2)
.foregroundColor(status.ipAddress == nil ? .red : .green)
}
}
}
}
private var debugSection: some View {
VStack(alignment: .leading, spacing: 4) {
HoverButton(
title: "Debug Info",
tint: .primary,
trailingSystemImage: showDebugInfo ? "chevron.up" : "chevron.down",
small: true
) {
showDebugInfo.toggle()
}
if showDebugInfo {
VStack(alignment: .leading, spacing: 4) {
Text("Version: \(buildTag)")
.font(.caption2)
.foregroundColor(.secondary)
Text("Commit: \(buildCommit)")
.font(.caption2)
.foregroundColor(.secondary)
Text(thunderboltStatusText)
.font(.caption2)
.foregroundColor(thunderboltStatusColor)
clusterThunderboltBridgeView
interfaceIpList
rdmaStatusView
sendBugReportButton
.padding(.top, 6)
}
.padding(.leading, 8)
.transition(.opacity)
}
}
.animation(.easeInOut(duration: 0.25), value: showDebugInfo)
}
private var rdmaStatusView: some View {
let rdmaStatuses = stateService.latestSnapshot?.nodeRdmaCtl ?? [:]
let localNodeId = stateService.localNodeId
let nodeProfiles = stateService.latestSnapshot?.nodeProfiles ?? [:]
let localDevices = networkStatusService.status.localRdmaDevices
let localPorts = networkStatusService.status.localRdmaActivePorts
return VStack(alignment: .leading, spacing: 1) {
if rdmaStatuses.isEmpty {
Text("Cluster RDMA: No data")
.font(.caption2)
.foregroundColor(.secondary)
} else {
Text("Cluster RDMA Status:")
.font(.caption2)
.foregroundColor(.secondary)
ForEach(Array(rdmaStatuses.keys.sorted()), id: \.self) { nodeId in
if let status = rdmaStatuses[nodeId] {
let nodeName =
nodeProfiles[nodeId]?.friendlyName ?? String(nodeId.prefix(8))
let isLocal = nodeId == localNodeId
let prefix = isLocal ? " \(nodeName) (local):" : " \(nodeName):"
let statusText = status.enabled ? "Enabled" : "Disabled"
let color: Color = status.enabled ? .green : .orange
Text("\(prefix) \(statusText)")
.font(.caption2)
.foregroundColor(color)
}
}
}
if !localDevices.isEmpty {
Text(" Local Devices: \(localDevices.joined(separator: ", "))")
.font(.caption2)
.foregroundColor(.secondary)
}
if !localPorts.isEmpty {
Text(" Local Active Ports:")
.font(.caption2)
.foregroundColor(.secondary)
ForEach(localPorts, id: \.device) { port in
Text(" \(port.device) port \(port.port): \(port.state)")
.font(.caption2)
.foregroundColor(.green)
}
}
}
}
private var sendBugReportButton: some View {
VStack(alignment: .leading, spacing: 4) {
Button {
Task {
await sendBugReport()
}
} label: {
HStack {
if bugReportInFlight {
ProgressView()
.scaleEffect(0.6)
}
Text("Send Bug Report")
.font(.caption)
.fontWeight(.semibold)
Spacer()
}
.padding(.vertical, 6)
.padding(.horizontal, 8)
.background(
RoundedRectangle(cornerRadius: 6)
.fill(Color.accentColor.opacity(0.12))
)
}
.buttonStyle(.plain)
.disabled(bugReportInFlight)
if let message = bugReportMessage {
Text(message)
.font(.caption2)
.foregroundColor(.secondary)
.fixedSize(horizontal: false, vertical: true)
}
}
}
private var processToggleBinding: Binding<Bool> {
Binding(
get: {
@@ -686,101 +419,6 @@ struct ContentView: View {
)
}
private func sendBugReport() async {
bugReportInFlight = true
bugReportMessage = "Collecting logs..."
let service = BugReportService()
do {
let outcome = try await service.sendReport(isManual: true)
bugReportMessage = outcome.message
} catch {
bugReportMessage = error.localizedDescription
}
bugReportInFlight = false
}
private func showUninstallConfirmationAlert() {
let alert = NSAlert()
alert.messageText = "Uninstall EXO"
alert.informativeText = """
This will remove EXO and all its system components:
• Network configuration daemon
• Launch at login registration
• EXO network location
The app will be moved to Trash.
"""
alert.alertStyle = .warning
alert.addButton(withTitle: "Uninstall")
alert.addButton(withTitle: "Cancel")
// Style the Uninstall button as destructive
if let uninstallButton = alert.buttons.first {
uninstallButton.hasDestructiveAction = true
}
let response = alert.runModal()
if response == .alertFirstButtonReturn {
performUninstall()
}
}
private func performUninstall() {
uninstallInProgress = true
// Stop EXO process first
controller.cancelPendingLaunch()
controller.stop()
stateService.stopPolling()
// Run the privileged uninstall on a background thread
// Using .utility QoS to avoid priority inversion with NSAppleScript's subprocess
DispatchQueue.global(qos: .utility).async {
do {
// Remove network setup daemon and components (requires admin privileges)
try NetworkSetupHelper.uninstall()
DispatchQueue.main.async {
// Unregister from launch at login
LaunchAtLoginHelper.disable()
// Move app to trash
self.moveAppToTrash()
// Quit the app
DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) {
NSApplication.shared.terminate(nil)
}
}
} catch {
DispatchQueue.main.async {
self.showErrorAlert(message: error.localizedDescription)
self.uninstallInProgress = false
}
}
}
}
private func showErrorAlert(message: String) {
let alert = NSAlert()
alert.messageText = "Uninstall Failed"
alert.informativeText = message
alert.alertStyle = .critical
alert.addButton(withTitle: "OK")
alert.runModal()
}
private func moveAppToTrash() {
guard let appURL = Bundle.main.bundleURL as URL? else { return }
do {
try FileManager.default.trashItem(at: appURL, resultingItemURL: nil)
} catch {
// If we can't trash the app, that's OK - user can do it manually
// The important system components have already been cleaned up
}
}
private var buildTag: String {
Bundle.main.infoDictionary?["EXOBuildTag"] as? String ?? "unknown"
}
+15 -1
View File
@@ -21,7 +21,9 @@ struct EXOApp: App {
@StateObject private var localNetworkChecker: LocalNetworkChecker
@StateObject private var updater: SparkleUpdater
@StateObject private var thunderboltBridgeService: ThunderboltBridgeService
@StateObject private var settingsWindowController: SettingsWindowController
private let terminationObserver: TerminationObserver
private let firstLaunchPopout = FirstLaunchPopout()
private let ciContext = CIContext(options: nil)
init() {
@@ -43,12 +45,13 @@ struct EXOApp: App {
_updater = StateObject(wrappedValue: updater)
let thunderboltBridge = ThunderboltBridgeService(clusterStateService: service)
_thunderboltBridgeService = StateObject(wrappedValue: thunderboltBridge)
_settingsWindowController = StateObject(wrappedValue: SettingsWindowController())
enableLaunchAtLoginIfNeeded()
// Install LaunchDaemon to disable Thunderbolt Bridge on startup (prevents network loops)
NetworkSetupHelper.promptAndInstallIfNeeded()
// Check local network access periodically (warning disappears when user grants permission)
localNetwork.startPeriodicChecking(interval: 10)
controller.scheduleLaunch(after: 15)
controller.scheduleLaunch(after: 5)
service.startPolling()
networkStatus.startPolling()
}
@@ -62,8 +65,19 @@ struct EXOApp: App {
.environmentObject(localNetworkChecker)
.environmentObject(updater)
.environmentObject(thunderboltBridgeService)
.environmentObject(settingsWindowController)
} label: {
menuBarIcon
.onReceive(controller.$isFirstLaunchReady) { ready in
if ready {
DispatchQueue.main.asyncAfter(deadline: .now() + 3.0) {
self.firstLaunchPopout.onComplete = { [weak controller] in
controller?.markOnboardingCompleted()
}
self.firstLaunchPopout.show()
}
}
}
}
.menuBarExtraStyle(.window)
}
+30
View File
@@ -5,6 +5,8 @@ import Foundation
private let customNamespaceKey = "EXOCustomNamespace"
private let hfTokenKey = "EXOHFToken"
private let enableImageModelsKey = "EXOEnableImageModels"
private let onboardingCompletedKey = "EXOOnboardingCompleted"
private let modelSearchPathsKey = "EXOModelSearchPaths"
@MainActor
final class ExoProcessController: ObservableObject {
@@ -59,6 +61,17 @@ final class ExoProcessController: ObservableObject {
UserDefaults.standard.set(enableImageModels, forKey: enableImageModelsKey)
}
}
@Published var modelSearchPaths: String = {
return UserDefaults.standard.string(forKey: modelSearchPathsKey) ?? ""
}()
{
didSet {
UserDefaults.standard.set(modelSearchPaths, forKey: modelSearchPathsKey)
}
}
/// Fires once when EXO transitions to `.running` for the very first time (fresh install).
@Published private(set) var isFirstLaunchReady = false
private var process: Process?
private var runtimeDirectoryURL: URL?
@@ -113,6 +126,9 @@ final class ExoProcessController: ObservableObject {
try child.run()
process = child
status = .running
// Show welcome popout on every launch
isFirstLaunchReady = true
} catch {
process = nil
status = .failed(message: "Launch error")
@@ -164,6 +180,17 @@ final class ExoProcessController: ObservableObject {
launch()
}
/// Mark onboarding as completed (user interacted with the welcome popout).
func markOnboardingCompleted() {
UserDefaults.standard.set(true, forKey: onboardingCompletedKey)
}
/// Reset onboarding so the welcome popout appears on next launch.
func resetOnboarding() {
UserDefaults.standard.removeObject(forKey: onboardingCompletedKey)
isFirstLaunchReady = false
}
func scheduleLaunch(after seconds: TimeInterval) {
cancelPendingLaunch()
let start = max(1, Int(ceil(seconds)))
@@ -249,6 +276,9 @@ final class ExoProcessController: ObservableObject {
if enableImageModels {
environment["EXO_ENABLE_IMAGE_MODELS"] = "true"
}
if !modelSearchPaths.isEmpty {
environment["EXO_MODELS_PATH"] = modelSearchPaths
}
var paths: [String] = []
if let existing = environment["PATH"], !existing.isEmpty {
+200
View File
@@ -0,0 +1,200 @@
import AppKit
import SwiftUI
/// A popover callout anchored to the menu bar icon on every launch,
/// pointing the user to the web dashboard with an arrow connecting to the icon.
@MainActor
final class FirstLaunchPopout {
private var popover: NSPopover?
private var countdownTask: Task<Void, Never>?
private static let dashboardURL = "http://localhost:52415/"
/// Called when the user completes onboarding (clicks Open Dashboard or dismisses).
var onComplete: (() -> Void)?
func show() {
guard popover == nil else { return }
// The status bar button may not exist yet on first launch; retry generously.
showWithRetry(attemptsRemaining: 15)
}
private func showWithRetry(attemptsRemaining: Int) {
guard attemptsRemaining > 0 else {
// Exhausted retries fall back to just opening the dashboard directly.
openDashboard()
onComplete?()
return
}
guard let button = Self.findStatusItemButton() else {
DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) { [weak self] in
self?.showWithRetry(attemptsRemaining: attemptsRemaining - 1)
}
return
}
let pop = NSPopover()
pop.behavior = .applicationDefined
pop.animates = true
pop.contentSize = NSSize(width: 280, height: 120)
pop.contentViewController = NSHostingController(
rootView: WelcomeCalloutView(
countdownDuration: 10,
onDismiss: { [weak self] in
self?.onComplete?()
self?.dismiss()
},
onOpen: { [weak self] in
self?.openDashboard()
self?.onComplete?()
self?.dismiss()
}
)
)
self.popover = pop
pop.show(relativeTo: button.bounds, of: button, preferredEdge: .minY)
// Auto-open dashboard after 10s then dismiss
countdownTask = Task {
try? await Task.sleep(nanoseconds: 10_000_000_000)
if !Task.isCancelled {
openDashboard()
onComplete?()
dismiss()
}
}
}
func dismiss() {
countdownTask?.cancel()
countdownTask = nil
guard let pop = popover else { return }
popover = nil
pop.performClose(nil)
}
private func openDashboard() {
guard let url = URL(string: Self.dashboardURL) else { return }
NSWorkspace.shared.open(url)
}
/// Finds the NSStatusBarButton created by SwiftUI's MenuBarExtra.
/// Walks the view hierarchy to find the actual button rather than the content view.
private static func findStatusItemButton() -> NSView? {
for window in NSApp.windows {
let className = NSStringFromClass(type(of: window))
// Match NSStatusBarWindow or any internal SwiftUI status bar window
guard className.contains("StatusBar") || className.contains("MenuBarExtra") else {
continue
}
if let content = window.contentView {
if let button = findButton(in: content) {
return button
}
// Fall back to the content view itself if it has a non-zero frame
if content.frame.width > 0 {
return content
}
}
}
return nil
}
/// Recursively searches the view hierarchy for an NSStatusBarButton.
private static func findButton(in view: NSView) -> NSView? {
let className = NSStringFromClass(type(of: view))
if className.contains("StatusBarButton") || className.contains("StatusItem") {
return view
}
for subview in view.subviews {
if let found = findButton(in: subview) {
return found
}
}
return nil
}
}
/// Minimal welcome callout friendly pointer, not a wall of text.
/// Rendered inside the NSPopover which provides its own chrome and arrow.
private struct WelcomeCalloutView: View {
let countdownDuration: Int
let onDismiss: () -> Void
let onOpen: () -> Void
@State private var countdown: Int
@State private var timerTask: Task<Void, Never>?
init(countdownDuration: Int, onDismiss: @escaping () -> Void, onOpen: @escaping () -> Void) {
self.countdownDuration = countdownDuration
self.onDismiss = onDismiss
self.onOpen = onOpen
self._countdown = State(initialValue: countdownDuration)
}
var body: some View {
VStack(alignment: .leading, spacing: 10) {
HStack(alignment: .top) {
Text("EXO is running")
.font(.system(.headline, design: .rounded))
.fontWeight(.semibold)
.foregroundColor(.primary)
Spacer()
Button {
onDismiss()
} label: {
Image(systemName: "xmark.circle.fill")
.font(.system(size: 14))
.foregroundStyle(.tertiary)
}
.buttonStyle(.plain)
}
Text("Run your first model here:")
.font(.system(.subheadline, design: .default))
.foregroundColor(.secondary)
HStack {
Button {
onOpen()
} label: {
Label("Open Dashboard", systemImage: "arrow.up.right.square")
.font(.system(.caption, design: .default))
.fontWeight(.medium)
}
.buttonStyle(.borderedProminent)
.tint(.accentColor)
.controlSize(.small)
Spacer()
if countdown > 0 {
Text("Launching in \(countdown) secs...")
.font(.system(.caption2, design: .default))
.foregroundColor(.secondary.opacity(0.6))
.monospacedDigit()
}
}
}
.padding(14)
.onAppear {
startCountdown()
}
.onDisappear {
timerTask?.cancel()
timerTask = nil
}
}
private func startCountdown() {
timerTask = Task {
while countdown > 0 {
try? await Task.sleep(nanoseconds: 1_000_000_000)
if !Task.isCancelled {
countdown -= 1
}
}
}
}
}
+495
View File
@@ -0,0 +1,495 @@
import AppKit
import SwiftUI
/// Native macOS Settings window following Apple HIG.
/// Organized into General, Model, Advanced, and About sections.
struct SettingsView: View {
@EnvironmentObject private var controller: ExoProcessController
@EnvironmentObject private var updater: SparkleUpdater
@EnvironmentObject private var networkStatusService: NetworkStatusService
@EnvironmentObject private var thunderboltBridgeService: ThunderboltBridgeService
@EnvironmentObject private var stateService: ClusterStateService
@State private var pendingNamespace: String = ""
@State private var pendingHFToken: String = ""
@State private var pendingEnableImageModels = false
@State private var pendingModelSearchPaths: String = ""
@State private var needsRestart = false
@State private var bugReportInFlight = false
@State private var bugReportMessage: String?
@State private var uninstallInProgress = false
var body: some View {
TabView {
generalTab
.tabItem {
Label("General", systemImage: "gear")
}
modelTab
.tabItem {
Label("Model", systemImage: "cube")
}
advancedTab
.tabItem {
Label("Advanced", systemImage: "wrench.and.screwdriver")
}
aboutTab
.tabItem {
Label("About", systemImage: "info.circle")
}
}
.frame(width: 450, height: 400)
.onAppear {
pendingNamespace = controller.customNamespace
pendingHFToken = controller.hfToken
pendingEnableImageModels = controller.enableImageModels
pendingModelSearchPaths = controller.modelSearchPaths
needsRestart = false
}
}
// MARK: - General Tab
private var generalTab: some View {
Form {
Section {
LabeledContent("Cluster Namespace") {
TextField("default", text: $pendingNamespace)
.textFieldStyle(.roundedBorder)
.frame(width: 200)
}
Text("Nodes with the same namespace form a cluster. Leave empty for default.")
.font(.caption)
.foregroundColor(.secondary)
}
Section {
LabeledContent("HuggingFace Token") {
SecureField("optional", text: $pendingHFToken)
.textFieldStyle(.roundedBorder)
.frame(width: 200)
}
Text("Required for gated models. Get yours at huggingface.co/settings/tokens")
.font(.caption)
.foregroundColor(.secondary)
}
Section {
HStack {
Spacer()
Button("Save & Restart") {
applyGeneralSettings()
}
.disabled(!hasGeneralChanges)
}
}
}
.formStyle(.grouped)
.padding()
}
// MARK: - Model Tab
private var modelTab: some View {
Form {
Section {
Toggle("Enable Image Models (experimental)", isOn: $pendingEnableImageModels)
Text("Allow text-to-image and image-to-image models in the model picker.")
.font(.caption)
.foregroundColor(.secondary)
}
Section {
LabeledContent("Model Search Paths") {
TextField("/path/one:/path/two", text: $pendingModelSearchPaths)
.textFieldStyle(.roundedBorder)
.frame(width: 200)
}
Text(
"Extra directories to search for pre-downloaded models (colon-separated). HuggingFace cache (~/.cache/huggingface/hub) is checked automatically."
)
.font(.caption)
.foregroundColor(.secondary)
}
Section {
HStack {
Spacer()
Button("Save & Restart") {
applyModelSettings()
}
.disabled(!hasModelChanges)
}
}
}
.formStyle(.grouped)
.padding()
}
// MARK: - Advanced Tab
private var advancedTab: some View {
Form {
Section("Onboarding") {
HStack {
VStack(alignment: .leading) {
Text("Reset Onboarding")
Text("Opens the dashboard and resets the onboarding wizard.")
.font(.caption)
.foregroundColor(.secondary)
}
Spacer()
Button("Reset") {
guard let url = URL(string: "http://localhost:52415/?reset-onboarding")
else { return }
NSWorkspace.shared.open(url)
}
}
}
Section("Debug Info") {
LabeledContent("Thunderbolt Bridge") {
Text(thunderboltStatusText)
.foregroundColor(thunderboltStatusColor)
}
VStack(alignment: .leading, spacing: 2) {
clusterThunderboltBridgeView
}
VStack(alignment: .leading, spacing: 2) {
interfaceIpList
}
VStack(alignment: .leading, spacing: 2) {
rdmaStatusView
}
sendBugReportButton
}
Section("Danger Zone") {
Button(role: .destructive) {
showUninstallConfirmationAlert()
} label: {
HStack {
Text("Uninstall EXO")
Spacer()
Image(systemName: "trash")
.imageScale(.small)
}
}
.disabled(uninstallInProgress)
}
}
.formStyle(.grouped)
.padding()
}
// MARK: - About Tab
private var aboutTab: some View {
Form {
Section {
LabeledContent("Version") {
Text(buildTag)
.textSelection(.enabled)
}
LabeledContent("Commit") {
Text(buildCommit)
.font(.system(.body, design: .monospaced))
.textSelection(.enabled)
}
}
Section {
Button("Check for Updates") {
updater.checkForUpdates()
}
}
}
.formStyle(.grouped)
.padding()
}
// MARK: - Debug Info Views (moved from ContentView)
private var thunderboltStatusText: String {
switch networkStatusService.status.thunderboltBridgeState {
case .some(.disabled):
return "Disabled"
case .some(.deleted):
return "Deleted"
case .some(.enabled):
return "Enabled"
case nil:
return "Unknown"
}
}
private var thunderboltStatusColor: Color {
switch networkStatusService.status.thunderboltBridgeState {
case .some(.disabled), .some(.deleted):
return .green
case .some(.enabled):
return .red
case nil:
return .secondary
}
}
private var clusterThunderboltBridgeView: some View {
let bridgeStatuses = stateService.latestSnapshot?.nodeThunderboltBridge ?? [:]
let localNodeId = stateService.localNodeId
let nodeProfiles = stateService.latestSnapshot?.nodeProfiles ?? [:]
return VStack(alignment: .leading, spacing: 1) {
if bridgeStatuses.isEmpty {
Text("Cluster TB Bridge: No data")
.font(.caption2)
.foregroundColor(.secondary)
} else {
Text("Cluster TB Bridge Status:")
.font(.caption2)
.foregroundColor(.secondary)
ForEach(Array(bridgeStatuses.keys.sorted()), id: \.self) { nodeId in
if let status = bridgeStatuses[nodeId] {
let nodeName =
nodeProfiles[nodeId]?.friendlyName ?? String(nodeId.prefix(8))
let isLocal = nodeId == localNodeId
let prefix = isLocal ? " \(nodeName) (local):" : " \(nodeName):"
let statusText =
!status.exists
? "N/A"
: (status.enabled ? "Enabled" : "Disabled")
let color: Color =
!status.exists
? .secondary
: (status.enabled ? .red : .green)
Text("\(prefix) \(statusText)")
.font(.caption2)
.foregroundColor(color)
}
}
}
}
}
private var interfaceIpList: some View {
let statuses = networkStatusService.status.interfaceStatuses
return VStack(alignment: .leading, spacing: 1) {
Text("Interfaces (en0en7):")
.font(.caption2)
.foregroundColor(.secondary)
if statuses.isEmpty {
Text(" Unknown")
.font(.caption2)
.foregroundColor(.secondary)
} else {
ForEach(statuses, id: \.interfaceName) { status in
let ipText = status.ipAddress ?? "No IP"
Text(" \(status.interfaceName): \(ipText)")
.font(.caption2)
.foregroundColor(status.ipAddress == nil ? .red : .green)
}
}
}
}
private var rdmaStatusView: some View {
let rdmaStatuses = stateService.latestSnapshot?.nodeRdmaCtl ?? [:]
let localNodeId = stateService.localNodeId
let nodeProfiles = stateService.latestSnapshot?.nodeProfiles ?? [:]
let localDevices = networkStatusService.status.localRdmaDevices
let localPorts = networkStatusService.status.localRdmaActivePorts
return VStack(alignment: .leading, spacing: 1) {
if rdmaStatuses.isEmpty {
Text("Cluster RDMA: No data")
.font(.caption2)
.foregroundColor(.secondary)
} else {
Text("Cluster RDMA Status:")
.font(.caption2)
.foregroundColor(.secondary)
ForEach(Array(rdmaStatuses.keys.sorted()), id: \.self) { nodeId in
if let status = rdmaStatuses[nodeId] {
let nodeName =
nodeProfiles[nodeId]?.friendlyName ?? String(nodeId.prefix(8))
let isLocal = nodeId == localNodeId
let prefix = isLocal ? " \(nodeName) (local):" : " \(nodeName):"
let statusText = status.enabled ? "Enabled" : "Disabled"
let color: Color = status.enabled ? .green : .orange
Text("\(prefix) \(statusText)")
.font(.caption2)
.foregroundColor(color)
}
}
}
if !localDevices.isEmpty {
Text(" Local Devices: \(localDevices.joined(separator: ", "))")
.font(.caption2)
.foregroundColor(.secondary)
}
if !localPorts.isEmpty {
Text(" Local Active Ports:")
.font(.caption2)
.foregroundColor(.secondary)
ForEach(localPorts, id: \.device) { port in
Text(" \(port.device) port \(port.port): \(port.state)")
.font(.caption2)
.foregroundColor(.green)
}
}
}
}
private var sendBugReportButton: some View {
VStack(alignment: .leading, spacing: 4) {
Button {
Task {
await sendBugReport()
}
} label: {
HStack {
if bugReportInFlight {
ProgressView()
.scaleEffect(0.6)
}
Text("Send Bug Report")
.font(.caption)
.fontWeight(.semibold)
Spacer()
}
}
.disabled(bugReportInFlight)
if let message = bugReportMessage {
Text(message)
.font(.caption2)
.foregroundColor(.secondary)
.fixedSize(horizontal: false, vertical: true)
}
}
}
// MARK: - Actions
private func sendBugReport() async {
bugReportInFlight = true
bugReportMessage = "Collecting logs..."
let service = BugReportService()
do {
let outcome = try await service.sendReport(isManual: true)
bugReportMessage = outcome.message
} catch {
bugReportMessage = error.localizedDescription
}
bugReportInFlight = false
}
private func showUninstallConfirmationAlert() {
let alert = NSAlert()
alert.messageText = "Uninstall EXO"
alert.informativeText = """
This will remove EXO and all its system components:
• Network configuration daemon
• Launch at login registration
• EXO network location
The app will be moved to Trash.
"""
alert.alertStyle = .warning
alert.addButton(withTitle: "Uninstall")
alert.addButton(withTitle: "Cancel")
if let uninstallButton = alert.buttons.first {
uninstallButton.hasDestructiveAction = true
}
let response = alert.runModal()
if response == .alertFirstButtonReturn {
performUninstall()
}
}
private func performUninstall() {
uninstallInProgress = true
controller.cancelPendingLaunch()
controller.stop()
stateService.stopPolling()
DispatchQueue.global(qos: .utility).async {
do {
try NetworkSetupHelper.uninstall()
DispatchQueue.main.async {
LaunchAtLoginHelper.disable()
self.moveAppToTrash()
DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) {
NSApplication.shared.terminate(nil)
}
}
} catch {
DispatchQueue.main.async {
let errorAlert = NSAlert()
errorAlert.messageText = "Uninstall Failed"
errorAlert.informativeText = error.localizedDescription
errorAlert.alertStyle = .critical
errorAlert.addButton(withTitle: "OK")
errorAlert.runModal()
self.uninstallInProgress = false
}
}
}
}
private func moveAppToTrash() {
guard let appURL = Bundle.main.bundleURL as URL? else { return }
do {
try FileManager.default.trashItem(at: appURL, resultingItemURL: nil)
} catch {
// If we can't trash the app, that's OK - user can do it manually
}
}
// MARK: - Helpers
private var hasGeneralChanges: Bool {
pendingNamespace != controller.customNamespace || pendingHFToken != controller.hfToken
}
private var hasModelChanges: Bool {
pendingEnableImageModels != controller.enableImageModels
|| pendingModelSearchPaths != controller.modelSearchPaths
}
private func applyGeneralSettings() {
controller.customNamespace = pendingNamespace
controller.hfToken = pendingHFToken
restartIfRunning()
}
private func applyModelSettings() {
controller.enableImageModels = pendingEnableImageModels
controller.modelSearchPaths = pendingModelSearchPaths
restartIfRunning()
}
private func restartIfRunning() {
if controller.status == .running || controller.status == .starting {
controller.restart()
}
}
private var buildTag: String {
Bundle.main.infoDictionary?["EXOBuildTag"] as? String ?? "unknown"
}
private var buildCommit: String {
Bundle.main.infoDictionary?["EXOBuildCommit"] as? String ?? "unknown"
}
}
@@ -0,0 +1,47 @@
import AppKit
import SwiftUI
/// Manages a standalone native macOS Settings window.
/// Ensures only one instance exists and brings it to front on repeated opens.
@MainActor
final class SettingsWindowController: ObservableObject {
private var window: NSWindow?
func open(
controller: ExoProcessController,
updater: SparkleUpdater,
networkStatusService: NetworkStatusService,
thunderboltBridgeService: ThunderboltBridgeService,
stateService: ClusterStateService
) {
if let existing = window, existing.isVisible {
existing.makeKeyAndOrderFront(nil)
NSApp.activate(ignoringOtherApps: true)
return
}
let settingsView = SettingsView()
.environmentObject(controller)
.environmentObject(updater)
.environmentObject(networkStatusService)
.environmentObject(thunderboltBridgeService)
.environmentObject(stateService)
let hostingView = NSHostingView(rootView: settingsView)
let newWindow = NSWindow(
contentRect: NSRect(x: 0, y: 0, width: 450, height: 400),
styleMask: [.titled, .closable],
backing: .buffered,
defer: false
)
newWindow.title = "EXO Settings"
newWindow.contentView = hostingView
newWindow.center()
newWindow.isReleasedWhenClosed = false
newWindow.makeKeyAndOrderFront(nil)
NSApp.activate(ignoringOtherApps: true)
window = newWindow
}
}
+65
View File
@@ -202,6 +202,23 @@
filter: drop-shadow(0 0 3px oklch(0.85 0.18 85 / 0.5));
}
/* Onboarding step 2: connection line between devices */
.onboarding-connection-line {
stroke: oklch(0.85 0.18 85 / 0.5);
stroke-width: 1.5px;
stroke-dasharray: 6, 6;
animation: flowAnimation 1s linear infinite;
filter: drop-shadow(0 0 4px oklch(0.85 0.18 85 / 0.4));
}
/* Onboarding step 4: red connection line for disconnect */
.onboarding-connection-line-red {
stroke: rgba(220, 38, 38, 0.7);
stroke-width: 1.5px;
stroke-dasharray: 6, 6;
filter: drop-shadow(0 0 2px rgba(220, 38, 38, 0.3));
}
.graph-link-active {
stroke: oklch(0.85 0.18 85 / 0.8);
stroke-width: 2px;
@@ -320,3 +337,51 @@ input:focus, textarea:focus {
transform: translate(400px, 400px);
}
}
/* Onboarding smooth fade-in */
@keyframes onb-fade-in {
from { opacity: 0; transform: translateY(8px); }
to { opacity: 1; transform: none; }
}
/* Opacity-only fade-in — no transform, safe for containers with fixed-position children */
@keyframes onb-fade-opacity {
from { opacity: 0; }
to { opacity: 1; }
}
/* Respect reduced motion preference */
@media (prefers-reduced-motion: reduce) {
.shooting-star,
.shooting-star::before {
animation: none !important;
opacity: 0 !important;
}
.graph-link {
animation: none;
}
.status-pulse {
animation: none;
}
.cursor-blink {
animation: none;
}
.onboarding-connection-line {
animation: none;
}
.onboarding-connection-line-red {
animation: none;
}
[style*="onb-fade-in"],
[style*="onb-fade-opacity"] {
animation: none !important;
opacity: 1 !important;
}
*,
*::before,
*::after {
transition-duration: 0.01ms !important;
animation-duration: 0.01ms !important;
animation-iteration-count: 1 !important;
}
}
@@ -29,6 +29,7 @@
showModelSelector?: boolean;
modelTasks?: Record<string, string[]>;
modelCapabilities?: Record<string, string[]>;
onSend?: () => void;
}
let {
@@ -39,6 +40,7 @@
showModelSelector = false,
modelTasks = {},
modelCapabilities = {},
onSend,
}: Props = $props();
let message = $state("");
@@ -306,6 +308,8 @@
);
}
onSend?.();
// Refocus the textarea after sending
setTimeout(() => textareaRef?.focus(), 10);
}
@@ -807,8 +807,8 @@
>
AWAITING INPUT
</p>
<p class="text-sm sm:text-xs text-exo-light-gray tracking-wider mt-1">
ENTER A QUERY TO BEGIN
<p class="text-xs text-white/30 tracking-wider mt-1.5 font-mono">
Type a message below &middot; Shift+Enter for newline
</p>
</div>
{/if}
@@ -823,6 +823,7 @@
onclick={scrollToBottom}
class="sticky bottom-4 left-1/2 -translate-x-1/2 w-10 h-10 rounded-full bg-exo-dark-gray/90 border border-exo-medium-gray/50 flex items-center justify-center text-exo-light-gray hover:text-exo-yellow hover:border-exo-yellow/50 transition-all shadow-lg cursor-pointer z-10"
title="Scroll to bottom"
aria-label="Scroll to bottom of messages"
>
<svg
class="w-5 h-5"
+20 -26
View File
@@ -185,11 +185,7 @@
let instanceType: string | null = null;
if (instanceTag === "MlxRingInstance") instanceType = "MLX Ring";
else if (
instanceTag === "MlxIbvInstance" ||
instanceTag === "MlxJacclInstance"
)
instanceType = "MLX RDMA";
else if (instanceTag === "MlxJacclInstance") instanceType = "MLX RDMA";
let sharding: string | null = null;
const inst = instance as {
@@ -307,7 +303,7 @@
<div class="py-2">
<div class="px-4 py-2">
<span
class="text-sm text-white/70 font-mono tracking-wider uppercase"
class="text-xs text-exo-light-gray font-mono tracking-wider uppercase"
>
{searchQuery ? "SEARCH RESULTS" : "CONVERSATIONS"}
</span>
@@ -376,39 +372,37 @@
onkeydown={(e) =>
e.key === "Enter" &&
handleSelectConversation(conversation.id)}
class="group w-full flex items-center justify-between p-2 rounded mb-1 transition-all text-left cursor-pointer
class="group w-full flex items-center justify-between p-2.5 rounded-lg mb-1 transition-all text-left cursor-pointer
{activeId === conversation.id
? 'bg-transparent border border-exo-yellow/30'
: 'hover:border-exo-yellow/20 border border-transparent'}"
? 'bg-exo-yellow/5 border border-exo-yellow/30'
: 'hover:bg-white/[0.03] hover:border-white/10 border border-transparent'}"
>
<div class="flex-1 min-w-0 pr-2">
<div
class="text-sm truncate {activeId === conversation.id
class="text-sm font-medium truncate {activeId ===
conversation.id
? 'text-exo-yellow'
: 'text-white/90'}"
: 'text-white'}"
>
{conversation.name}
</div>
<div class="text-sm text-white/50 mt-0.5">
<div class="text-xs text-white/60 mt-0.5">
{formatDate(conversation.updatedAt)}
</div>
<div class="text-sm text-white/70 truncate">
<div class="text-xs text-exo-light-gray truncate">
{info.modelLabel}
</div>
<div class="text-xs text-white/60 font-mono">
Strategy: <span class="text-white/80"
>{info.strategyLabel}</span
>
</div>
{#if stats}
<div class="text-xs text-white/60 font-mono mt-1">
{#if stats.ttftMs}<span class="text-white/40">TTFT</span>
{stats.ttftMs.toFixed(
0,
)}ms{/if}{#if stats.ttftMs && stats.tps}<span
class="text-white/30 mx-1.5"></span
>{/if}{#if stats.tps}{stats.tps.toFixed(1)}
<span class="text-white/40">tok/s</span>{/if}
<div class="text-xs text-white/70 font-mono mt-1">
{#if stats.ttftMs}<span class="text-white/50">TTFT</span>
<span class="text-exo-yellow/80"
>{stats.ttftMs.toFixed(0)}ms</span
>{/if}{#if stats.ttftMs && stats.tps}<span
class="text-white/30 mx-1.5">·</span
>{/if}{#if stats.tps}<span class="text-exo-yellow/80"
>{stats.tps.toFixed(1)}</span
>
<span class="text-white/50">tok/s</span>{/if}
</div>
{/if}
</div>
@@ -0,0 +1,20 @@
<script lang="ts">
import { isConnected } from "$lib/stores/app.svelte";
import { slide } from "svelte/transition";
const connected = $derived(isConnected());
</script>
{#if !connected}
<div
transition:slide={{ duration: 200 }}
class="relative z-50 flex items-center justify-center gap-2 px-4 py-2 bg-red-950/80 border-b border-red-500/30"
role="alert"
aria-live="assertive"
>
<div class="w-2 h-2 bg-red-500 rounded-full animate-pulse"></div>
<span class="text-xs font-mono text-red-300 tracking-wider uppercase">
Connection lost &mdash; Reconnecting to backend&hellip;
</span>
</div>
{/if}
@@ -0,0 +1,277 @@
<script lang="ts">
/**
* DeviceIcon — renders a device icon as an SVG <g> element.
* Uses the exact same proportional math as TopologyGraph.svelte
* so that devices look identical in both the topology view and
* the onboarding animation.
*
* Must be placed inside an <svg> element.
*/
interface Props {
/** "macbook pro" | "mac studio" | "mac mini" etc. */
deviceType: string;
/** Center X coordinate in SVG space */
cx: number;
/** Center Y coordinate in SVG space */
cy: number;
/** Base sizing factor (equivalent to TopologyGraph's nodeRadius) */
size?: number;
/** RAM usage 0100 */
ramPercent?: number;
/** Unique id suffix for clip-path ids */
uid?: string;
}
let {
deviceType,
cx,
cy,
size = 60,
ramPercent = 60,
uid = "dev",
}: Props = $props();
// Apple logo path — same constant used by TopologyGraph
const APPLE_LOGO_PATH =
"M788.1 340.9c-5.8 4.5-108.2 62.2-108.2 190.5 0 148.4 130.3 200.9 134.2 202.2-.6 3.2-20.7 71.9-68.7 141.9-42.8 61.6-87.5 123.1-155.5 123.1s-85.5-39.5-164-39.5c-76.5 0-103.7 40.8-165.9 40.8s-105.6-57-155.5-127C46.7 790.7 0 663 0 541.8c0-194.4 126.4-297.5 250.8-297.5 66.1 0 121.2 43.4 162.7 43.4 39.5 0 101.1-46 176.3-46 28.5 0 130.9 2.6 198.3 99.2zm-234-181.5c31.1-36.9 53.1-88.1 53.1-139.3 0-7.1-.6-14.3-1.9-20.1-50.6 1.9-110.8 33.7-147.1 75.8-28.5 32.4-55.1 83.6-55.1 135.5 0 7.8 1.3 15.6 1.9 18.1 3.2.6 8.4 1.3 13.6 1.3 45.4 0 102.5-30.4 135.5-71.3z";
const LOGO_NATIVE_WIDTH = 814;
const LOGO_NATIVE_HEIGHT = 1000;
const wireColor = "rgba(179,179,179,0.8)";
const strokeWidth = 1.5;
const modelLower = $derived(deviceType.toLowerCase());
// ── Mac Studio dimensions (same ratios as TopologyGraph) ──
const studioW = $derived(size * 1.25);
const studioH = $derived(size * 0.85);
const studioX = $derived(cx - studioW / 2);
const studioY = $derived(cy - studioH / 2);
const studioCorner = 4;
const studioTopH = $derived(studioH * 0.15);
// Studio front panel details
const studioSlotH = $derived(studioH * 0.14);
const studioVSlotW = $derived(studioW * 0.05);
const studioVSlotY = $derived(
studioY + studioTopH + (studioH - studioTopH) * 0.6,
);
const studioVSlot1X = $derived(studioX + studioW * 0.18);
const studioVSlot2X = $derived(studioX + studioW * 0.28);
const studioHSlotW = $derived(studioW * 0.2);
const studioHSlotX = $derived(studioX + studioW * 0.5 - studioHSlotW / 2);
// Studio memory fill
const studioMemTotalH = $derived(studioH - studioTopH);
const studioMemH = $derived((ramPercent / 100) * studioMemTotalH);
// ── MacBook dimensions (same ratios as TopologyGraph) ──
const mbW = $derived(size * 1.6);
const mbH = $derived(size * 1.15);
const mbX = $derived(cx - mbW / 2);
const mbY = $derived(cy - mbH / 2);
const mbScreenH = $derived(mbH * 0.7);
const mbBaseH = $derived(mbH * 0.3);
const mbScreenW = $derived(mbW * 0.85);
const mbScreenX = $derived(cx - mbScreenW / 2);
const mbBezel = 3;
// MacBook memory fill
const mbMemTotalH = $derived(mbScreenH - mbBezel * 2);
const mbMemH = $derived((ramPercent / 100) * mbMemTotalH);
// Apple logo sizing
const mbLogoTargetH = $derived(mbScreenH * 0.22);
const mbLogoScale = $derived(mbLogoTargetH / LOGO_NATIVE_HEIGHT);
const mbLogoX = $derived(cx - (LOGO_NATIVE_WIDTH * mbLogoScale) / 2);
const mbLogoY = $derived(
mbY + mbScreenH / 2 - (LOGO_NATIVE_HEIGHT * mbLogoScale) / 2,
);
// MacBook base (trapezoidal)
const mbBaseY = $derived(mbY + mbScreenH);
const mbBaseTopW = $derived(mbScreenW);
const mbBaseBottomW = $derived(mbW);
const mbBaseTopX = $derived(cx - mbBaseTopW / 2);
const mbBaseBottomX = $derived(cx - mbBaseBottomW / 2);
// Keyboard
const mbKbX = $derived(mbBaseTopX + 6);
const mbKbY = $derived(mbBaseY + 3);
const mbKbW = $derived(mbBaseTopW - 12);
const mbKbH = $derived(mbBaseH * 0.55);
// Trackpad
const mbTpW = $derived(mbBaseTopW * 0.4);
const mbTpX = $derived(cx - mbTpW / 2);
const mbTpY = $derived(mbBaseY + mbKbH + 5);
const mbTpH = $derived(mbBaseH * 0.3);
// Clip IDs
const screenClipId = $derived(`di-screen-${uid}`);
const studioClipId = $derived(`di-studio-${uid}`);
</script>
{#if modelLower === "mac studio" || modelLower === "mac mini"}
<!-- Mac Studio / Mac Mini -->
<defs>
<clipPath id={studioClipId}>
<rect
x={studioX}
y={studioY + studioTopH}
width={studioW}
height={studioH - studioTopH}
rx={studioCorner - 1}
/>
</clipPath>
</defs>
<!-- Main body -->
<rect
x={studioX}
y={studioY}
width={studioW}
height={studioH}
rx={studioCorner}
fill="#1a1a1a"
stroke={wireColor}
stroke-width={strokeWidth}
/>
<!-- Memory fill -->
{#if ramPercent > 0}
<rect
x={studioX}
y={studioY + studioTopH + (studioMemTotalH - studioMemH)}
width={studioW}
height={studioMemH}
fill="rgba(255,215,0,0.75)"
clip-path="url(#{studioClipId})"
/>
{/if}
<!-- Top surface divider -->
<line
x1={studioX}
y1={studioY + studioTopH}
x2={studioX + studioW}
y2={studioY + studioTopH}
stroke="rgba(179,179,179,0.3)"
stroke-width="0.5"
/>
<!-- Front panel: vertical slots -->
<rect
x={studioVSlot1X - studioVSlotW / 2}
y={studioVSlotY}
width={studioVSlotW}
height={studioSlotH}
fill="rgba(0,0,0,0.35)"
rx="1.5"
/>
<rect
x={studioVSlot2X - studioVSlotW / 2}
y={studioVSlotY}
width={studioVSlotW}
height={studioSlotH}
fill="rgba(0,0,0,0.35)"
rx="1.5"
/>
<!-- Horizontal slot (SD card) -->
<rect
x={studioHSlotX}
y={studioVSlotY}
width={studioHSlotW}
height={studioSlotH * 0.6}
fill="rgba(0,0,0,0.35)"
rx="1"
/>
{:else}
<!-- MacBook Pro -->
<defs>
<clipPath id={screenClipId}>
<rect
x={mbScreenX + mbBezel}
y={mbY + mbBezel}
width={mbScreenW - mbBezel * 2}
height={mbScreenH - mbBezel * 2}
rx="2"
/>
</clipPath>
</defs>
<!-- Screen outer frame -->
<rect
x={mbScreenX}
y={mbY}
width={mbScreenW}
height={mbScreenH}
rx="3"
fill="#1a1a1a"
stroke={wireColor}
stroke-width={strokeWidth}
/>
<!-- Screen inner (dark) -->
<rect
x={mbScreenX + mbBezel}
y={mbY + mbBezel}
width={mbScreenW - mbBezel * 2}
height={mbScreenH - mbBezel * 2}
rx="2"
fill="#0a0a12"
/>
<!-- Memory fill on screen -->
{#if ramPercent > 0}
<rect
x={mbScreenX + mbBezel}
y={mbY + mbBezel + (mbMemTotalH - mbMemH)}
width={mbScreenW - mbBezel * 2}
height={mbMemH}
fill="rgba(255,215,0,0.85)"
clip-path="url(#{screenClipId})"
/>
{/if}
<!-- Apple logo -->
<path
d={APPLE_LOGO_PATH}
transform="translate({mbLogoX}, {mbLogoY}) scale({mbLogoScale})"
fill="#FFFFFF"
opacity="0.9"
/>
<!-- Keyboard base (trapezoidal) -->
<path
d="M {mbBaseTopX} {mbBaseY} L {mbBaseTopX +
mbBaseTopW} {mbBaseY} L {mbBaseBottomX + mbBaseBottomW} {mbBaseY +
mbBaseH} L {mbBaseBottomX} {mbBaseY + mbBaseH} Z"
fill="#2c2c2c"
stroke={wireColor}
stroke-width="1"
/>
<!-- Keyboard area -->
<rect
x={mbKbX}
y={mbKbY}
width={mbKbW}
height={mbKbH}
fill="rgba(0,0,0,0.2)"
rx="2"
/>
<!-- Trackpad -->
<rect
x={mbTpX}
y={mbTpY}
width={mbTpW}
height={mbTpH}
fill="rgba(255,255,255,0.08)"
rx="2"
/>
{/if}
+63 -18
View File
@@ -6,6 +6,10 @@
export let showSidebarToggle = false;
export let sidebarVisible = true;
export let onToggleSidebar: (() => void) | null = null;
export let downloadProgress: {
count: number;
percentage: number;
} | null = null;
function handleHome(): void {
if (onHome) {
@@ -33,13 +37,17 @@
<div class="absolute left-6 top-1/2 -translate-y-1/2">
<button
onclick={handleToggleSidebar}
class="p-2 rounded border border-exo-medium-gray/40 hover:border-exo-yellow/50 transition-colors cursor-pointer"
class="p-2 rounded border border-exo-light-gray/30 hover:border-exo-yellow/50 hover:bg-exo-medium-gray/30 transition-colors cursor-pointer"
title={sidebarVisible ? "Hide sidebar" : "Show sidebar"}
aria-label={sidebarVisible
? "Hide conversation sidebar"
: "Show conversation sidebar"}
aria-pressed={sidebarVisible}
>
<svg
class="w-5 h-5 {sidebarVisible
? 'text-exo-yellow'
: 'text-exo-medium-gray'}"
: 'text-exo-light-gray'}"
fill="none"
viewBox="0 0 24 24"
stroke="currentColor"
@@ -75,13 +83,14 @@
<img
src="/exo-logo.png"
alt="EXO"
class="h-18 drop-shadow-[0_0_20px_rgba(255,215,0,0.5)]"
class="h-18 drop-shadow-[0_0_4px_rgba(255,215,0,0.3)]"
/>
</button>
<!-- Right: Home + Downloads -->
<div
<nav
class="absolute right-6 top-1/2 -translate-y-1/2 flex items-center gap-4"
aria-label="Main navigation"
>
{#if showHome}
<button
@@ -110,20 +119,56 @@
class="text-sm text-exo-light-gray hover:text-exo-yellow transition-colors tracking-wider uppercase flex items-center gap-2 cursor-pointer"
title="View downloads overview"
>
<svg
class="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<path d="M12 3v12" />
<path d="M7 12l5 5 5-5" />
<path d="M5 21h14" />
</svg>
{#if downloadProgress}
<!-- Compact download progress indicator -->
<div class="relative w-4 h-4 flex-shrink-0">
<svg class="w-4 h-4 -rotate-90" viewBox="0 0 20 20">
<circle
cx="10"
cy="10"
r="8"
fill="none"
stroke="currentColor"
stroke-width="2"
opacity="0.2"
/>
<circle
cx="10"
cy="10"
r="8"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-dasharray={2 * Math.PI * 8}
stroke-dashoffset={2 *
Math.PI *
8 *
(1 - downloadProgress.percentage / 100)}
class="text-blue-400 transition-all duration-300"
/>
</svg>
<div
class="absolute inset-0 flex items-center justify-center text-[6px] font-mono text-blue-400"
>
{downloadProgress.count}
</div>
</div>
{:else}
<svg
class="w-4 h-4"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<path d="M12 3v12" />
<path d="M7 12l5 5 5-5" />
<path d="M5 21h14" />
</svg>
{/if}
Downloads
</a>
</div>
</nav>
</header>
+29 -3
View File
@@ -21,7 +21,7 @@
} | null;
nodes?: Record<string, NodeInfo>;
sharding?: "Pipeline" | "Tensor";
runtime?: "MlxRing" | "MlxIbv" | "MlxJaccl";
runtime?: "MlxRing" | "MlxJaccl";
onLaunch?: () => void;
tags?: string[];
apiPreview?: PlacementPreview | null;
@@ -348,7 +348,7 @@
// Debug mode state
const isDebugMode = $derived(debugMode());
const topology = $derived(topologyData());
const isRdma = $derived(runtime === "MlxIbv" || runtime === "MlxJaccl");
const isRdma = $derived(runtime === "MlxJaccl");
// Get interface name for an IP from node data
function getInterfaceForIp(nodeId: string, ip?: string): string | null {
@@ -567,20 +567,46 @@
<div class="flex items-center gap-1.5 mb-2">
<span
class="px-1.5 py-0.5 text-xs font-mono tracking-wider uppercase bg-exo-medium-gray/30 text-exo-light-gray border border-exo-medium-gray/40"
title={sharding === "Pipeline"
? "Pipeline: splits model into sequential stages across devices. Lower network overhead."
: "Tensor: splits each layer across devices. Best with high-bandwidth connections (Thunderbolt)."}
>
{sharding}
</span>
<span
class="px-1.5 py-0.5 text-xs font-mono tracking-wider uppercase bg-exo-medium-gray/30 text-exo-light-gray border border-exo-medium-gray/40"
title={runtime === "MlxRing"
? "Ring: standard networking. Works over any connection (Wi-Fi, Ethernet, Thunderbolt)."
: "RDMA: direct memory access over Thunderbolt. Significantly faster for multi-device inference."}
>
{runtime === "MlxRing"
? "MLX Ring"
: runtime === "MlxIbv" || runtime === "MlxJaccl"
: runtime === "MlxJaccl"
? "MLX RDMA"
: runtime}
</span>
</div>
<!-- Download Status -->
{#if isDownloading && progress}
<div class="mb-2 space-y-1">
<div class="flex items-center justify-between text-xs font-mono">
<span class="text-blue-400 tracking-wider uppercase">Downloading</span
>
<span class="text-white/60"
>{percentage.toFixed(1)}% &middot; {formatSpeed(progress.speed)}
&middot; {formatEta(progress.etaMs)}</span
>
</div>
<div class="h-1 bg-exo-medium-gray/30 rounded overflow-hidden">
<div
class="h-full bg-blue-500/70 transition-all duration-300"
style="width: {percentage}%"
></div>
</div>
</div>
{/if}
<!-- Mini Topology Preview -->
{#if placementPreview().nodes.length > 0}
{@const preview = placementPreview()}
@@ -512,6 +512,18 @@
);
});
// Split filtered groups into recommended (fits_now) and others for visual separation
const recommendedGroups = $derived(
filteredGroups.filter((g) =>
g.variants.some((v) => getModelFitStatus(v.id) === "fits_now"),
),
);
const otherGroups = $derived(
filteredGroups.filter(
(g) => !g.variants.some((v) => getModelFitStatus(v.id) === "fits_now"),
),
);
function toggleGroupExpanded(groupId: string) {
const next = new Set(expandedGroups);
if (next.has(groupId)) {
@@ -840,7 +852,60 @@
{/if}
</div>
{:else}
{#each filteredGroups as group}
<!-- Recommended for your cluster -->
{#if recommendedGroups.length > 0 && otherGroups.length > 0 && !searchQuery.trim()}
<div
class="sticky top-0 z-10 flex items-center gap-2 px-3 py-2 bg-green-950/60 border-b border-green-500/20 backdrop-blur-sm"
>
<svg
class="w-3.5 h-3.5 text-green-400 flex-shrink-0"
fill="none"
viewBox="0 0 24 24"
stroke="currentColor"
stroke-width="2"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M9 12.75L11.25 15 15 9.75M21 12a9 9 0 11-18 0 9 9 0 0118 0z"
/>
</svg>
<span
class="text-xs font-mono text-green-400 tracking-wider uppercase"
>Recommended for your cluster</span
>
<span class="text-xs font-mono text-green-400/50"
>— fits in available memory</span
>
</div>
{/if}
{#each recommendedGroups as group}
<ModelPickerGroup
{group}
isExpanded={expandedGroups.has(group.id)}
isFavorite={favorites.has(group.id)}
{selectedModelId}
{canModelFit}
{getModelFitStatus}
onToggleExpand={() => toggleGroupExpanded(group.id)}
onSelectModel={handleSelect}
{onToggleFavorite}
onShowInfo={(g) => (infoGroup = g)}
downloadStatusMap={getVariantDownloadMap(group)}
/>
{/each}
<!-- Other models -->
{#if otherGroups.length > 0 && recommendedGroups.length > 0 && !searchQuery.trim()}
<div
class="sticky top-0 z-10 flex items-center gap-2 px-3 py-2 bg-exo-dark-gray/80 border-y border-exo-medium-gray/20 backdrop-blur-sm"
>
<span
class="text-xs font-mono text-white/40 tracking-wider uppercase"
>Other models</span
>
</div>
{/if}
{#each otherGroups as group}
<ModelPickerGroup
{group}
isExpanded={expandedGroups.has(group.id)}
@@ -0,0 +1,117 @@
<script lang="ts">
import { toasts, dismissToast, type Toast } from "$lib/stores/toast.svelte";
import { fly, fade } from "svelte/transition";
import { flip } from "svelte/animate";
const items = $derived(toasts());
const typeStyles: Record<
Toast["type"],
{ border: string; icon: string; iconColor: string }
> = {
success: {
border: "border-l-green-500",
icon: "M9 12.75L11.25 15 15 9.75M21 12a9 9 0 11-18 0 9 9 0 0118 0z",
iconColor: "text-green-400",
},
error: {
border: "border-l-red-500",
icon: "M12 9v3.75m9-.75a9 9 0 11-18 0 9 9 0 0118 0zm-9 3.75h.008v.008H12v-.008z",
iconColor: "text-red-400",
},
warning: {
border: "border-l-yellow-500",
icon: "M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0L2.697 16.126z",
iconColor: "text-yellow-400",
},
info: {
border: "border-l-blue-500",
icon: "M11.25 11.25l.041-.02a.75.75 0 011.063.852l-.708 2.836a.75.75 0 001.063.853l.041-.021M21 12a9 9 0 11-18 0 9 9 0 0118 0zm-9-3.75h.008v.008H12V8.25z",
iconColor: "text-blue-400",
},
};
</script>
{#if items.length > 0}
<div
class="fixed bottom-6 right-6 z-[9999] flex flex-col gap-2 pointer-events-none"
role="log"
aria-live="polite"
aria-label="Notifications"
>
{#each items as toast (toast.id)}
{@const style = typeStyles[toast.type]}
<div
class="pointer-events-auto max-w-sm w-80 bg-exo-dark-gray/95 backdrop-blur-sm border border-exo-medium-gray/60 border-l-[3px] {style.border} rounded shadow-lg shadow-black/40"
in:fly={{ x: 80, duration: 250 }}
out:fade={{ duration: 150 }}
animate:flip={{ duration: 200 }}
role="alert"
>
<div class="flex items-start gap-3 px-4 py-3">
<!-- Icon -->
<svg
class="w-5 h-5 flex-shrink-0 mt-0.5 {style.iconColor}"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d={style.icon}
/>
</svg>
<!-- Message -->
<p class="flex-1 text-sm text-white/90 font-mono leading-snug">
{toast.message}
</p>
<!-- Dismiss button -->
<button
onclick={() => dismissToast(toast.id)}
class="flex-shrink-0 p-0.5 text-white/40 hover:text-white/80 transition-colors cursor-pointer"
aria-label="Dismiss notification"
>
<svg
class="w-4 h-4"
fill="none"
viewBox="0 0 24 24"
stroke-width="2"
stroke="currentColor"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M6 18L18 6M6 6l12 12"
/>
</svg>
</button>
</div>
<!-- Auto-dismiss progress bar -->
{#if toast.duration > 0}
<div class="h-0.5 bg-white/5 rounded-b overflow-hidden">
<div
class="h-full {style.border.replace('border-l-', 'bg-')}/60"
style="animation: shrink {toast.duration}ms linear forwards"
></div>
</div>
{/if}
</div>
{/each}
</div>
{/if}
<style>
@keyframes shrink {
from {
width: 100%;
}
to {
width: 0%;
}
}
</style>
+25 -9
View File
@@ -168,7 +168,7 @@ export interface ModelDownloadStatus {
export interface PlacementPreview {
model_id: string;
sharding: "Pipeline" | "Tensor";
instance_meta: "MlxRing" | "MlxIbv" | "MlxJaccl";
instance_meta: "MlxRing" | "MlxJaccl";
instance: unknown | null;
memory_delta_by_node: Record<string, number> | null;
error: string | null;
@@ -219,7 +219,6 @@ interface RawStateResponse {
string,
{
MlxRingInstance?: Instance;
MlxIbvInstance?: Instance;
MlxJacclInstance?: Instance;
}
>;
@@ -590,6 +589,12 @@ class AppStore {
// Image editing state
editingImage = $state<EditingImage | null>(null);
/** True when the backend is reachable. */
isConnected = $state<boolean>(true);
/** Number of consecutive fetch failures. */
private consecutiveFailures = 0;
private static readonly CONNECTION_LOST_THRESHOLD = 3;
private fetchInterval: ReturnType<typeof setInterval> | null = null;
private previewsInterval: ReturnType<typeof setInterval> | null = null;
private lastConversationPersistTs = 0;
@@ -912,11 +917,7 @@ class AppStore {
let instanceType: string | null = null;
if (instanceTag === "MlxRingInstance") instanceType = "MLX Ring";
else if (
instanceTag === "MlxIbvInstance" ||
instanceTag === "MlxJacclInstance"
)
instanceType = "MLX RDMA";
else if (instanceTag === "MlxJacclInstance") instanceType = "MLX RDMA";
let sharding: string | null = null;
const inst = instance as {
@@ -1295,7 +1296,19 @@ class AppStore {
// Thunderbolt bridge status per node
this.nodeThunderboltBridge = data.nodeThunderboltBridge ?? {};
this.lastUpdate = Date.now();
// Connection recovered
if (!this.isConnected) {
this.isConnected = true;
}
this.consecutiveFailures = 0;
} catch (error) {
this.consecutiveFailures++;
if (
this.consecutiveFailures >= AppStore.CONNECTION_LOST_THRESHOLD &&
this.isConnected
) {
this.isConnected = false;
}
console.error("Error fetching state:", error);
}
}
@@ -1836,7 +1849,7 @@ class AppStore {
assistantMessage.id,
(msg) => {
msg.content =
"Error: No model available. Please launch an instance first.";
"No model is loaded yet. Select a model from the sidebar to get started — it will download and load automatically.";
},
);
this.syncActiveMessagesIfNeeded(targetConversationId);
@@ -2307,7 +2320,7 @@ class AppStore {
const modelToUse = this.getModelForRequest();
if (!modelToUse) {
throw new Error(
"No model selected and no running instances available. Please launch an instance first.",
"No model is loaded yet. Select a model from the sidebar to get started — it will download and load automatically.",
);
}
@@ -3250,6 +3263,9 @@ export const setChatSidebarVisible = (visible: boolean) =>
appStore.setChatSidebarVisible(visible);
export const refreshState = () => appStore.fetchState();
// Connection status
export const isConnected = () => appStore.isConnected;
// Node identities (for OS version mismatch detection)
export const nodeIdentities = () => appStore.nodeIdentities;
+87
View File
@@ -0,0 +1,87 @@
/**
* Toast notification store - Global notification system for the EXO dashboard.
*
* Usage:
* import { addToast, dismissToast, toasts } from "$lib/stores/toast.svelte";
* addToast({ type: "success", message: "Model launched" });
* addToast({ type: "error", message: "Connection lost", persistent: true });
*/
type ToastType = "success" | "error" | "warning" | "info";
export interface Toast {
id: string;
type: ToastType;
message: string;
/** Auto-dismiss after this many ms. 0 = persistent (must be dismissed manually). */
duration: number;
createdAt: number;
}
interface ToastInput {
type: ToastType;
message: string;
/** If true, toast stays until manually dismissed. Default: false. */
persistent?: boolean;
/** Auto-dismiss duration in ms. Default: 4000 for success/info, 6000 for error/warning. */
duration?: number;
}
const DEFAULT_DURATIONS: Record<ToastType, number> = {
success: 4000,
info: 4000,
warning: 6000,
error: 6000,
};
let toastList = $state<Toast[]>([]);
const timers = new Map<string, ReturnType<typeof setTimeout>>();
function generateId(): string {
return `toast-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
}
export function addToast(input: ToastInput): string {
const id = generateId();
const duration = input.persistent
? 0
: (input.duration ?? DEFAULT_DURATIONS[input.type]);
const toast: Toast = {
id,
type: input.type,
message: input.message,
duration,
createdAt: Date.now(),
};
toastList = [...toastList, toast];
if (duration > 0) {
const timer = setTimeout(() => dismissToast(id), duration);
timers.set(id, timer);
}
return id;
}
export function dismissToast(id: string): void {
const timer = timers.get(id);
if (timer) {
clearTimeout(timer);
timers.delete(id);
}
toastList = toastList.filter((t) => t.id !== id);
}
/** Dismiss all toasts matching a message (useful for dedup). */
export function dismissByMessage(message: string): void {
const matching = toastList.filter((t) => t.message === message);
for (const t of matching) {
dismissToast(t.id);
}
}
export function toasts(): Toast[] {
return toastList;
}
+4
View File
@@ -1,5 +1,7 @@
<script lang="ts">
import "../app.css";
import ToastContainer from "$lib/components/ToastContainer.svelte";
import ConnectionBanner from "$lib/components/ConnectionBanner.svelte";
let { children } = $props();
</script>
@@ -10,5 +12,7 @@
</svelte:head>
<div class="min-h-screen bg-background text-foreground">
<ConnectionBanner />
{@render children?.()}
<ToastContainer />
</div>
File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.8 KiB

+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env bash
# create-dmg.sh — Build a polished macOS DMG installer for EXO
#
# Usage:
# ./packaging/dmg/create-dmg.sh <app-path> <output-dmg> [volume-name]
#
# Example:
# ./packaging/dmg/create-dmg.sh output/EXO.app EXO-1.0.0.dmg "EXO"
#
# Creates a DMG with:
# - Custom background image with drag-to-Applications arrow
# - App icon on left, Applications alias on right
# - Proper window size and icon positioning
set -euo pipefail
APP_PATH="${1:?Usage: create-dmg.sh <app-path> <output-dmg> [volume-name]}"
OUTPUT_DMG="${2:?Usage: create-dmg.sh <app-path> <output-dmg> [volume-name]}"
VOLUME_NAME="${3:-EXO}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
BACKGROUND_SCRIPT="${SCRIPT_DIR}/generate-background.py"
TEMP_DIR="$(mktemp -d)"
DMG_STAGING="${TEMP_DIR}/dmg-root"
TEMP_DMG="${TEMP_DIR}/temp.dmg"
BACKGROUND_PNG="${TEMP_DIR}/background.png"
cleanup() { rm -rf "$TEMP_DIR"; }
trap cleanup EXIT
echo "==> Creating DMG installer for ${VOLUME_NAME}"
# ── Step 1: Generate background image ────────────────────────────────────────
if command -v python3 &>/dev/null; then
python3 "$BACKGROUND_SCRIPT" "$BACKGROUND_PNG"
echo " Background image generated"
else
echo " Warning: python3 not found, skipping custom background"
BACKGROUND_PNG=""
fi
# ── Step 2: Prepare staging directory ─────────────────────────────────────────
mkdir -p "$DMG_STAGING"
cp -R "$APP_PATH" "$DMG_STAGING/"
ln -s /Applications "$DMG_STAGING/Applications"
# ── Step 3: Create writable DMG ──────────────────────────────────────────────
# Calculate required size (app size + 20MB headroom)
APP_SIZE_KB=$(du -sk "$APP_PATH" | cut -f1)
DMG_SIZE_KB=$((APP_SIZE_KB + 20480))
hdiutil create \
-volname "$VOLUME_NAME" \
-size "${DMG_SIZE_KB}k" \
-fs HFS+ \
-layout SPUD \
"$TEMP_DMG"
# ── Step 4: Mount and configure ──────────────────────────────────────────────
MOUNT_DIR=$(hdiutil attach "$TEMP_DMG" -readwrite -noverify | awk -F'\t' '/Apple_HFS/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $NF); print $NF}')
echo " Mounted at: $MOUNT_DIR"
# Copy contents
cp -R "$DMG_STAGING/"* "$MOUNT_DIR/"
# Add background image
if [[ -n $BACKGROUND_PNG && -f $BACKGROUND_PNG ]]; then
mkdir -p "$MOUNT_DIR/.background"
cp "$BACKGROUND_PNG" "$MOUNT_DIR/.background/background.png"
fi
# ── Step 5: Configure window appearance via AppleScript ──────────────────────
# Window: 800×400, app icon on left, Applications on right (matches Ollama layout)
# Background image is 1600×740 (2× retina for 800×400 logical window).
APP_NAME="$(basename "$APP_PATH")"
osascript <<APPLESCRIPT
tell application "Finder"
tell disk "$VOLUME_NAME"
open
set current view of container window to icon view
set toolbar visible of container window to false
set statusbar visible of container window to false
set bounds of container window to {200, 120, 1000, 520}
set opts to icon view options of container window
set icon size of opts to 128
set text size of opts to 12
set arrangement of opts to not arranged
if exists file ".background:background.png" then
set background picture of opts to file ".background:background.png"
end if
set position of item "$APP_NAME" of container window to {200, 190}
set position of item "Applications" of container window to {600, 190}
close
open
update without registering applications
delay 1
close
end tell
end tell
APPLESCRIPT
echo " Window layout configured"
# Ensure Finder updates are flushed
sync
# ── Step 6: Finalise ─────────────────────────────────────────────────────────
hdiutil detach "$MOUNT_DIR" -quiet
hdiutil convert "$TEMP_DMG" -format UDZO -imagekey zlib-level=9 -o "$OUTPUT_DMG"
echo "==> DMG created: $OUTPUT_DMG"
echo " Size: $(du -h "$OUTPUT_DMG" | cut -f1)"
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""Generate the DMG background image with a centered drag-to-Applications arrow.
The output is a 1600×740 retina PNG (2× for 800×400 logical window).
Icons are positioned at (200, 190) and (600, 190) in logical coordinates;
the arrow is drawn centered between them.
Usage:
python3 generate-background.py [output.png]
If no output path is given, overwrites the bundled background.png in-place.
"""
from __future__ import annotations
import math
import sys
from pathlib import Path
from PIL import Image, ImageDraw
# Retina dimensions (2× logical 800×400)
WIDTH = 1600
HEIGHT = 740
# Icon positions in logical coords → retina coords
# App icon at (200, 190), Applications at (600, 190)
APP_X = 200 * 2 # 400
APPS_X = 600 * 2 # 1200
ICON_Y = 190 * 2 # 380
# Arrow drawn between icons, slightly above icon center
ARROW_START_X = APP_X + 160 # past the icon
ARROW_END_X = APPS_X - 160 # before the Applications icon
ARROW_Y = ICON_Y # same height as icons
ARROW_RISE = 120 # upward arc height
def draw_arrow(draw: ImageDraw.ImageDraw) -> None:
"""Draw a hand-drawn-style curved arrow from app icon toward Applications."""
color = (30, 30, 30)
line_width = 8
# Compute bezier curve points for a gentle upward arc
points: list[tuple[float, float]] = []
steps = 80
for i in range(steps + 1):
t = i / steps
# Quadratic bezier: start → control → end
cx = (ARROW_START_X + ARROW_END_X) / 2
cy = ARROW_Y - ARROW_RISE
x = (1 - t) ** 2 * ARROW_START_X + 2 * (1 - t) * t * cx + t**2 * ARROW_END_X
y = (1 - t) ** 2 * ARROW_Y + 2 * (1 - t) * t * cy + t**2 * ARROW_Y
points.append((x, y))
# Draw the curve as connected line segments
for i in range(len(points) - 1):
draw.line([points[i], points[i + 1]], fill=color, width=line_width)
# Arrowhead at the end
end_x, end_y = points[-1]
# Direction from second-to-last to last point
prev_x, prev_y = points[-3]
angle = math.atan2(end_y - prev_y, end_x - prev_x)
head_len = 36
head_angle = math.radians(25)
left_x = end_x - head_len * math.cos(angle - head_angle)
left_y = end_y - head_len * math.sin(angle - head_angle)
right_x = end_x - head_len * math.cos(angle + head_angle)
right_y = end_y - head_len * math.sin(angle + head_angle)
draw.polygon(
[(end_x, end_y), (left_x, left_y), (right_x, right_y)],
fill=color,
)
def generate_background(output_path: str) -> None:
"""Generate a white DMG background with a centered arrow."""
img = Image.new("RGBA", (WIDTH, HEIGHT), (255, 255, 255, 255))
draw = ImageDraw.Draw(img)
draw_arrow(draw)
img.save(output_path, "PNG")
if __name__ == "__main__":
default_output = str(Path(__file__).parent / "background.png")
out = sys.argv[1] if len(sys.argv) >= 2 else default_output
generate_background(out)
print(f"Background image written to {out}")
+37 -3
View File
@@ -110,20 +110,54 @@ def map_repo_download_progress_to_download_progress_data(
)
def _resolve_hf_hub_model(search_dir: Path, normalized: str) -> Path | None:
"""Try to find a model in HuggingFace Hub cache format.
HF Hub stores models as ``models--<org>--<name>/snapshots/<commit>/``
with symlinks to ``../../blobs/``. The active commit is read from
``refs/main``.
"""
hf_model_dir = search_dir / f"models--{normalized}"
if not hf_model_dir.is_dir():
return None
# Resolve ref → snapshot
ref_file = hf_model_dir / "refs" / "main"
if ref_file.is_file():
commit_hash = ref_file.read_text().strip()
snapshot = hf_model_dir / "snapshots" / commit_hash
if snapshot.is_dir():
return snapshot
# Fallback: use latest snapshot by mtime
snapshots_dir = hf_model_dir / "snapshots"
if snapshots_dir.is_dir():
snapshots = sorted(
snapshots_dir.iterdir(), key=lambda p: p.stat().st_mtime, reverse=True
)
if snapshots:
return snapshots[0]
return None
def resolve_model_in_path(model_id: ModelId) -> Path | None:
"""Search EXO_MODELS_PATH directories for a pre-existing model.
Checks each directory for the normalized name (org--model). A candidate
is only returned if ``is_model_directory_complete`` confirms all weight
files are present.
Checks each directory for the normalized name (org--model) and the
HuggingFace Hub cache format (models--org--model/snapshots/<ref>/).
A candidate is only returned if ``is_model_directory_complete``
confirms all weight files are present.
"""
if EXO_MODELS_PATH is None:
return None
normalized = model_id.normalize()
for search_dir in EXO_MODELS_PATH:
# Try direct format: <dir>/<org--name>/
candidate = search_dir / normalized
if candidate.is_dir() and is_model_directory_complete(candidate):
return candidate
# Try HF Hub cache format: <dir>/models--<org--name>/snapshots/<ref>/
hf_candidate = _resolve_hf_hub_model(search_dir, normalized)
if hf_candidate is not None and is_model_directory_complete(hf_candidate):
return hf_candidate
return None
+12
View File
@@ -51,6 +51,7 @@ from exo.master.placement import place_instance as get_instance_placements
from exo.shared.apply import apply
from exo.shared.constants import (
DASHBOARD_DIR,
EXO_CACHE_HOME,
EXO_EVENT_LOG_DIR,
EXO_IMAGE_CACHE_DIR,
EXO_MAX_CHUNK_SIZE,
@@ -175,6 +176,7 @@ from exo.utils.channels import Receiver, Sender, channel
from exo.utils.event_buffer import OrderedBuffer
_API_EVENT_LOG_DIR = EXO_EVENT_LOG_DIR / "api"
ONBOARDING_COMPLETE_FILE = EXO_CACHE_HOME / "onboarding_complete"
def _format_to_content_type(image_format: Literal["png", "jpeg", "webp"] | None) -> str:
@@ -345,6 +347,8 @@ class API:
self.app.get("/v1/traces/{task_id}")(self.get_trace)
self.app.get("/v1/traces/{task_id}/stats")(self.get_trace_stats)
self.app.get("/v1/traces/{task_id}/raw")(self.get_trace_raw)
self.app.get("/onboarding")(self.get_onboarding)
self.app.post("/onboarding")(self.complete_onboarding)
async def place_instance(self, payload: PlaceInstanceParams):
command = PlaceInstance(
@@ -1814,3 +1818,11 @@ class API:
media_type="application/json",
filename=f"trace_{task_id}.json",
)
async def get_onboarding(self) -> JSONResponse:
return JSONResponse({"completed": ONBOARDING_COMPLETE_FILE.exists()})
async def complete_onboarding(self) -> JSONResponse:
ONBOARDING_COMPLETE_FILE.parent.mkdir(parents=True, exist_ok=True)
ONBOARDING_COMPLETE_FILE.write_text("true")
return JSONResponse({"completed": True})
+18 -4
View File
@@ -36,12 +36,26 @@ EXO_MODELS_DIR = (
# Read-only search path for pre-downloaded models (colon-separated directories)
_EXO_MODELS_PATH_ENV = os.environ.get("EXO_MODELS_PATH", None)
EXO_MODELS_PATH: tuple[Path, ...] | None = (
tuple(Path(p).expanduser() for p in _EXO_MODELS_PATH_ENV.split(":") if p)
if _EXO_MODELS_PATH_ENV is not None
else None
# Well-known model cache directories from other inference engines
_WELL_KNOWN_MODEL_PATHS: tuple[Path, ...] = tuple(
p for p in (Path.home() / ".cache" / "huggingface" / "hub",) if p.is_dir()
)
def _build_models_path() -> tuple[Path, ...] | None:
if _EXO_MODELS_PATH_ENV is not None:
user_paths = tuple(
Path(p).expanduser() for p in _EXO_MODELS_PATH_ENV.split(":") if p
)
return user_paths + tuple(
p for p in _WELL_KNOWN_MODEL_PATHS if p not in user_paths
)
return _WELL_KNOWN_MODEL_PATHS if _WELL_KNOWN_MODEL_PATHS else None
EXO_MODELS_PATH: tuple[Path, ...] | None = _build_models_path()
_RESOURCES_DIR_ENV = os.environ.get("EXO_RESOURCES_DIR", None)
RESOURCES_DIR = (
find_resources() if _RESOURCES_DIR_ENV is None else Path.home() / _RESOURCES_DIR_ENV
+2 -1
View File
@@ -34,7 +34,8 @@ class RunnerConnected(BaseRunnerStatus):
class RunnerLoading(BaseRunnerStatus):
pass
layers_loaded: int = 0
total_layers: int = 0
class RunnerLoaded(BaseRunnerStatus):
+30
View File
@@ -1,8 +1,27 @@
import logging
import os
import sys
import webbrowser
from exo.shared.constants import EXO_CONFIG_HOME
logger = logging.getLogger(__name__)
_FIRST_RUN_MARKER = EXO_CONFIG_HOME / ".dashboard_opened"
def _is_first_run() -> bool:
return not _FIRST_RUN_MARKER.exists()
def _mark_first_run_done() -> None:
_FIRST_RUN_MARKER.parent.mkdir(parents=True, exist_ok=True)
_FIRST_RUN_MARKER.touch()
def print_startup_banner(port: int) -> None:
dashboard_url = f"http://localhost:{port}"
first_run = _is_first_run()
banner = f"""
@@ -30,3 +49,14 @@ def print_startup_banner(port: int) -> None:
"""
print(banner, file=sys.stderr)
if first_run:
# Skip browser open when running inside the native macOS app —
# FirstLaunchPopout.swift handles the auto-open with a countdown.
if not os.environ.get("EXO_RUNTIME_DIR"):
try:
webbrowser.open(dashboard_url)
logger.info("First run detected — opening dashboard in browser")
except Exception:
logger.debug("Could not auto-open browser", exc_info=True)
_mark_first_run_done()
+64 -39
View File
@@ -47,6 +47,7 @@ if TYPE_CHECKING:
from mlx_lm.models.cache import Cache
TimeoutCallback = Callable[[], None]
LayerLoadedCallback = Callable[[int, int], None] # (layers_loaded, total_layers)
def eval_with_timeout(
@@ -186,7 +187,7 @@ def set_pipeline_prefill(model: nn.Module, is_prefill: bool) -> None:
layer.is_prefill = is_prefill
def _inner_model(model: nn.Module) -> nn.Module:
def get_inner_model(model: nn.Module) -> nn.Module:
inner = getattr(model, "model", None)
if isinstance(inner, nn.Module):
return inner
@@ -204,7 +205,7 @@ def _inner_model(model: nn.Module) -> nn.Module:
raise ValueError("Model must either have a 'model' or 'transformer' attribute")
def _get_layers(inner_model_instance: nn.Module) -> list[_LayerCallable]:
def get_layers(inner_model_instance: nn.Module) -> list[_LayerCallable]:
# Handle both model.layers and model.h cases
layers: list[_LayerCallable]
if hasattr(inner_model_instance, "layers"):
@@ -221,6 +222,7 @@ def pipeline_auto_parallel(
model: nn.Module,
group: mx.distributed.Group,
model_shard_meta: PipelineShardMetadata,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
"""
Automatically parallelize a model across multiple devices.
@@ -230,16 +232,19 @@ def pipeline_auto_parallel(
Returns:
The parallelized model
"""
inner_model_instance: nn.Module = _inner_model(model)
inner_model_instance: nn.Module = get_inner_model(model)
layers = _get_layers(inner_model_instance)
layers = get_layers(inner_model_instance)
start_layer, end_layer = model_shard_meta.start_layer, model_shard_meta.end_layer
device_rank, world_size = model_shard_meta.device_rank, model_shard_meta.world_size
layers = layers[start_layer:end_layer]
for layer in layers:
total = len(layers)
for i, layer in enumerate(layers):
mx.eval(layer) # type: ignore
if on_layer_loaded is not None:
on_layer_loaded(i, total)
layers[0] = PipelineFirstLayer(layers[0], device_rank, group=group)
layers[-1] = PipelineLastLayer(
@@ -351,8 +356,9 @@ def patch_tensor_model[T](model: T) -> T:
def tensor_auto_parallel(
model: nn.Module,
group: mx.distributed.Group,
timeout_seconds: float = 60.0,
on_timeout: TimeoutCallback | None = None,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
all_to_sharded_linear = partial(
shard_linear,
@@ -462,7 +468,7 @@ def tensor_auto_parallel(
raise ValueError(f"Unsupported model type: {type(model)}")
model = tensor_parallel_sharding_strategy.shard_model(
model, timeout_seconds, on_timeout
model, timeout_seconds, on_timeout, on_layer_loaded
)
return patch_tensor_model(model)
@@ -489,6 +495,7 @@ class TensorParallelShardingStrategy(ABC):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module: ...
@@ -498,13 +505,13 @@ class LlamaShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(LlamaModel, model)
for layer in model.layers:
total = len(model.layers)
for i, layer in enumerate(model.layers):
# Force load weights before sharding to avoid FAST_SYNCH deadlock
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj)
layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj)
layer.self_attn.v_proj = self.all_to_sharded_linear(layer.self_attn.v_proj)
@@ -517,11 +524,13 @@ class LlamaShardingStrategy(TensorParallelShardingStrategy):
layer.mlp.down_proj = self.sharded_to_all_linear(layer.mlp.down_proj)
layer.mlp.up_proj = self.all_to_sharded_linear(layer.mlp.up_proj)
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
def _set_layers(model: nn.Module, layers: list[_LayerCallable]) -> None:
inner_model_instance = _inner_model(model)
inner_model_instance = get_inner_model(model)
if hasattr(inner_model_instance, "layers"):
inner_model_instance.layers = layers
@@ -552,13 +561,13 @@ class DeepSeekShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(DeepseekV3Model, model)
total = len(model.layers)
for layer in model.layers:
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
for i, layer in enumerate(model.layers):
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
# Shard the self attention
if layer.self_attn.q_lora_rank is None:
@@ -609,6 +618,8 @@ class DeepSeekShardingStrategy(TensorParallelShardingStrategy):
layer.mlp.sharding_group = self.group
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
@@ -635,13 +646,15 @@ class GLM4MoeLiteShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(GLM4MoeLiteModel, model)
for layer in model.layers: # type: ignore
total = len(model.layers) # type: ignore
for i, layer in enumerate(model.layers): # type: ignore
layer = cast(Glm4MoeLiteDecoderLayer, layer)
eval_with_timeout(
layer.parameters(),
timeout_seconds / len(model.layers), # type: ignore
timeout_seconds / total,
on_timeout,
)
if layer.self_attn.q_lora_rank is None: # type: ignore
@@ -689,6 +702,8 @@ class GLM4MoeLiteShardingStrategy(TensorParallelShardingStrategy):
layer.mlp = ShardedMoE(layer.mlp) # type: ignore
layer.mlp.sharding_group = self.group # type: ignore
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
@@ -777,12 +792,12 @@ class MiniMaxShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(MiniMaxModel, model)
for layer in model.layers:
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
total = len(model.layers)
for i, layer in enumerate(model.layers):
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
# Shard the self attention
layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj)
layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj)
@@ -807,6 +822,8 @@ class MiniMaxShardingStrategy(TensorParallelShardingStrategy):
layer.block_sparse_moe = ShardedMoE(layer.block_sparse_moe) # pyright: ignore[reportAttributeAccessIssue, reportArgumentType]
layer.block_sparse_moe.sharding_group = self.group # pyright: ignore[reportAttributeAccessIssue]
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
@@ -816,12 +833,12 @@ class QwenShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(Qwen3MoeModel | Qwen3NextModel, model)
for layer in model.layers:
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
total = len(model.layers)
for i, layer in enumerate(model.layers):
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
# Shard the self attention
if isinstance(layer, Qwen3DecoderLayer):
layer.self_attn.q_proj = self.all_to_sharded_linear(
@@ -930,6 +947,8 @@ class QwenShardingStrategy(TensorParallelShardingStrategy):
layer.mlp.up_proj = self.all_to_sharded_linear(layer.mlp.up_proj)
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
@@ -939,12 +958,12 @@ class Glm4MoeShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(Glm4MoeModel, model)
for layer in model.layers:
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
total = len(model.layers)
for i, layer in enumerate(model.layers):
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj)
layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj)
@@ -976,6 +995,8 @@ class Glm4MoeShardingStrategy(TensorParallelShardingStrategy):
layer.mlp.up_proj = self.all_to_sharded_linear(layer.mlp.up_proj)
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
@@ -985,13 +1006,13 @@ class GptOssShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(GptOssMoeModel, model)
total = len(model.layers)
for layer in model.layers:
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
for i, layer in enumerate(model.layers):
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj)
layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj)
layer.self_attn.v_proj = self.all_to_sharded_linear(layer.self_attn.v_proj)
@@ -1017,6 +1038,8 @@ class GptOssShardingStrategy(TensorParallelShardingStrategy):
layer.mlp = ShardedMoE(layer.mlp) # type: ignore
layer.mlp.sharding_group = self.group # pyright: ignore[reportAttributeAccessIssue]
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
@@ -1026,13 +1049,13 @@ class Step35ShardingStrategy(TensorParallelShardingStrategy):
model: nn.Module,
timeout_seconds: float,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> nn.Module:
model = cast(Step35Model, model)
total = len(model.layers)
for layer in model.layers:
eval_with_timeout(
layer.parameters(), timeout_seconds / len(model.layers), on_timeout
)
for i, layer in enumerate(model.layers):
eval_with_timeout(layer.parameters(), timeout_seconds / total, on_timeout)
layer.self_attn.q_proj = self.all_to_sharded_linear(layer.self_attn.q_proj)
layer.self_attn.k_proj = self.all_to_sharded_linear(layer.self_attn.k_proj)
layer.self_attn.v_proj = self.all_to_sharded_linear(layer.self_attn.v_proj)
@@ -1060,4 +1083,6 @@ class Step35ShardingStrategy(TensorParallelShardingStrategy):
self.sharded_to_all_linear_in_place(layer.mlp.switch_mlp.down_proj)
mx.eval(layer)
if on_layer_loaded is not None:
on_layer_loaded(i, total)
return model
+33 -6
View File
@@ -55,8 +55,11 @@ from exo.shared.types.worker.shards import (
)
from exo.worker.engines.mlx import Model
from exo.worker.engines.mlx.auto_parallel import (
LayerLoadedCallback,
TimeoutCallback,
eval_with_timeout,
get_inner_model,
get_layers,
pipeline_auto_parallel,
tensor_auto_parallel,
)
@@ -171,13 +174,29 @@ def initialize_mlx(
def load_mlx_items(
bound_instance: BoundInstance,
group: Group | None,
on_timeout: TimeoutCallback | None = None,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> tuple[Model, TokenizerWrapper]:
if group is None:
logger.info(f"Single device used for {bound_instance.instance}")
model_path = build_model_path(bound_instance.bound_shard.model_card.model_id)
start_time = time.perf_counter()
model, _ = load_model(model_path, strict=True)
model, _ = load_model(model_path, lazy=True, strict=False)
# Eval layers one by one for progress reporting
try:
inner = get_inner_model(model)
layers = get_layers(inner)
total = len(layers)
for i, layer in enumerate(layers):
mx.eval(layer) # type: ignore
if on_layer_loaded is not None:
on_layer_loaded(i, total)
except ValueError:
logger.debug(
"Model architecture doesn't support layer-by-layer progress tracking",
exc_info=True,
)
mx.eval(model)
end_time = time.perf_counter()
logger.info(f"Time taken to load model: {(end_time - start_time):.2f}s")
tokenizer = get_tokenizer(model_path, bound_instance.bound_shard)
@@ -186,7 +205,10 @@ def load_mlx_items(
logger.info("Starting distributed init")
start_time = time.perf_counter()
model, tokenizer = shard_and_load(
bound_instance.bound_shard, group=group, on_timeout=on_timeout
bound_instance.bound_shard,
group=group,
on_timeout=on_timeout,
on_layer_loaded=on_layer_loaded,
)
end_time = time.perf_counter()
logger.info(
@@ -201,7 +223,8 @@ def load_mlx_items(
def shard_and_load(
shard_metadata: ShardMetadata,
group: Group,
on_timeout: TimeoutCallback | None = None,
on_timeout: TimeoutCallback | None,
on_layer_loaded: LayerLoadedCallback | None,
) -> tuple[nn.Module, TokenizerWrapper]:
model_path = build_model_path(shard_metadata.model_card.model_id)
@@ -242,10 +265,14 @@ def shard_and_load(
match shard_metadata:
case TensorShardMetadata():
logger.info(f"loading model from {model_path} with tensor parallelism")
model = tensor_auto_parallel(model, group, timeout_seconds, on_timeout)
model = tensor_auto_parallel(
model, group, timeout_seconds, on_timeout, on_layer_loaded
)
case PipelineShardMetadata():
logger.info(f"loading model from {model_path} with pipeline parallelism")
model = pipeline_auto_parallel(model, group, shard_metadata)
model = pipeline_auto_parallel(
model, group, shard_metadata, on_layer_loaded=on_layer_loaded
)
eval_with_timeout(model.parameters(), timeout_seconds, on_timeout)
case CfgShardMetadata():
raise ValueError(
+20 -2
View File
@@ -150,7 +150,10 @@ def main(
case LoadModel() if (
isinstance(current_status, RunnerConnected) and group is not None
) or (isinstance(current_status, RunnerIdle) and group is None):
current_status = RunnerLoading()
total_layers = shard_metadata.end_layer - shard_metadata.start_layer
current_status = RunnerLoading(
layers_loaded=0, total_layers=total_layers
)
logger.info("runner loading")
event_sender.send(
RunnerStatusUpdated(
@@ -170,11 +173,26 @@ def main(
)
time.sleep(0.5)
def on_layer_loaded(layers_loaded: int, total: int) -> None:
nonlocal current_status
current_status = RunnerLoading(
layers_loaded=layers_loaded, total_layers=total
)
event_sender.send(
RunnerStatusUpdated(
runner_id=runner_id,
runner_status=current_status,
)
)
assert (
ModelTask.TextGeneration in shard_metadata.model_card.tasks
), f"Incorrect model task(s): {shard_metadata.model_card.tasks}"
inference_model, tokenizer = load_mlx_items(
bound_instance, group, on_timeout=on_model_load_timeout
bound_instance,
group,
on_timeout=on_model_load_timeout,
on_layer_loaded=on_layer_loaded,
)
logger.info(
f"model has_tool_calling={tokenizer.has_tool_calling} using tokens {tokenizer.tool_call_start}, {tokenizer.tool_call_end}"
@@ -96,7 +96,9 @@ def run_gpt_oss_pipeline_device(
n_layers=24,
)
model, tokenizer = shard_and_load(shard_meta, group)
model, tokenizer = shard_and_load(
shard_meta, group, on_timeout=None, on_layer_loaded=None
)
model = cast(Model, model)
# Generate a prompt of exact token length
@@ -172,7 +174,9 @@ def run_gpt_oss_tensor_parallel_device(
n_layers=24,
)
model, tokenizer = shard_and_load(shard_meta, group)
model, tokenizer = shard_and_load(
shard_meta, group, on_timeout=None, on_layer_loaded=None
)
model = cast(Model, model)
base_text = "The quick brown fox jumps over the lazy dog. "
@@ -224,7 +224,10 @@ def test_events_processed_in_correct_order(patch_out_mlx: pytest.MonkeyPatch):
),
RunnerStatusUpdated(runner_id=RUNNER_1_ID, runner_status=RunnerConnected()),
TaskStatusUpdated(task_id=LOAD_TASK_ID, task_status=TaskStatus.Running),
RunnerStatusUpdated(runner_id=RUNNER_1_ID, runner_status=RunnerLoading()),
RunnerStatusUpdated(
runner_id=RUNNER_1_ID,
runner_status=RunnerLoading(layers_loaded=0, total_layers=32),
),
TaskAcknowledged(task_id=LOAD_TASK_ID),
TaskStatusUpdated(task_id=LOAD_TASK_ID, task_status=TaskStatus.Complete),
RunnerStatusUpdated(runner_id=RUNNER_1_ID, runner_status=RunnerLoaded()),