diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..f879c5e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,242 @@ +# Agents, Sous-agents, Competences + +> "L'infrastructure est une decision politique deployee." -- electron rare + +## Orchestration + +- Agent racine: **Coordinateur** — planifie, arbitre, synchronise PLAN/TODO/docs +- Sous-agents specialises: analyse code, veille OSS, audit securite, optimisation +- Cadence: synchroniser PLAN.md + TODO.md + docs apres chaque lot + +## Matrice des agents (lot 17+) + +| Agent | Competences | Perimetre | Etat | +|---|---|---|---| +| Coordinateur | planification, arbitrage, docs de pilotage | PLAN.md, TODO.md, AGENTS.md, README.md | actif | +| Securite | validation input, hardening, rate-limit, RBAC | apps/api, ws-chat, packages/auth | veille | +| Backend API | Express, WS, Ollama, RAG, multimodal pipeline | apps/api/src/ | actif | +| Node Engine | DAG, queue, runs, sandbox, training adapters | packages/node-engine, apps/worker | actif | +| Personas | source/feedback/proposals/pharmacius, memoire | packages/persona-domain, ws-chat | actif | +| Frontend | React/Vite, UX Minitel, React Flow, chat, voice | apps/web/src/ | actif | +| Ops/TUI | scripts, logs, rotate/purge, health, audit | ops/v2/, scripts/ | actif | +| Training | DPO, SFT, Unsloth, eval, autoresearch, Ollama import | scripts/, packages/node-engine | actif | +| Multimodal | STT, TTS, vision, PDF, RAG, recherche web | apps/api/src/ws-chat.ts | actif | +| Veille OSS | recherche projets, libs, modeles, benchmarks | docs/OSS_WATCH, docs/HF_MODEL_RESEARCH | periodique | + +## Sous-agents et skill routing + +```mermaid +flowchart TD + Coord[Coordinateur] + + Coord --> SecAgent[Securite] + Coord --> BackAgent[Backend API] + Coord --> EngAgent[Node Engine] + Coord --> PersAgent[Personas] + Coord --> FrontAgent[Frontend] + Coord --> OpsAgent[Ops/TUI] + Coord --> TrainAgent[Training] + Coord --> MultiAgent[Multimodal] + Coord --> OSSAgent[Veille OSS] + + SecAgent --> |audit| SecScan[Pattern scan P0/P1/P2] + SecAgent --> |fix| SecFix[Correctifs chirurgicaux] + + BackAgent --> |analyse| APIAudit[Deep analyse app.ts, ws-chat.ts] + BackAgent --> |refactor| APISplit[Extraction modules] + + EngAgent --> |test| EngTest[Tests unitaires node-engine] + EngAgent --> |extend| EngNew[Nouveaux node types] + + PersAgent --> |pipeline| PersPipe[Editorial pipeline] + PersAgent --> |finetune| PersDPO[DPO + PCL methodology] + + FrontAgent --> |ui| FrontUI[Minitel theme CSS] + FrontAgent --> |perf| FrontPerf[Memoization, lazy load] + + OpsAgent --> |monitor| OpsHealth[health-check, deep-audit] + OpsAgent --> |deploy| OpsDeploy[Docker, kxkm-ai] + + TrainAgent --> |train| TrainRun[Unsloth/TRL runs] + TrainAgent --> |eval| TrainEval[Scoring, registry] + + MultiAgent --> |voice| MultiVoice[XTTS-v2, WebRTC] + MultiAgent --> |search| MultiSearch[SearXNG] + + OSSAgent --> |web| OSSWeb[Recherche web] + OSSAgent --> |hf| OSSHF[HuggingFace models] +``` + +## Todo agents (lot 17+ — mis a jour 2026-03-20) + +### Coordinateur + +- [x] Consolider PLAN.md avec etat reel (lots 0-94 complets) +- [x] Synchroniser FEATURE_MAP.md matrice +- [x] Mettre a jour TODO.md avec backlog Phase session 2026-03-19/20 +- [x] Documenter actions dans ops/v2/logs/ +- [ ] lot-95: Coordonner E2E Playwright test plan +- [ ] lot-100: Design public demo mode access control + +### Backend API + +- [x] Extraire app-bootstrap.ts et app-middleware.ts de app.ts +- [x] Extraire ws-conversation-router.ts de ws-chat.ts +- [x] ws-chat.ts modularized (425 to 335 LOC, 3 modules extracted) +- [x] app.ts extraction (540 to 131 LOC, create-repos.ts extracted) +- [x] Zod validation on all 19 API route schemas +- [x] Error telemetry (16 labels) +- [x] Perf instrumentation (6 labels, p50/p95/p99), TTFC 284ms +- [x] Smart routing (5 topic domains) +- [x] Dynamic context window (4k-32k) +- [x] NLP auto-detect generation intent (compose vs imagine) +- [x] /speed command for latency diagnostics +- [ ] lot-178: ACE-Step API direct integration (duration fix) +- [ ] lot-180: Timeline data model +- [ ] lot-97: Multi-channel support (create/join channels) +- [ ] lot-100: Public demo mode read-only routes + +### Node Engine + +- [x] Extraire registry.ts du hotspot node-engine +- [ ] Ajouter node type `music_generation` (ACE-Step 1.5) +- [ ] Ajouter node type `voice_clone` (Chatterbox) +- [ ] Ajouter node type `audio_mix` (multi-track composition) +- [ ] Ajouter node type `audio_effects` (FX chain) +- [ ] lot-96: Automated DPO pipeline (feedback → pairs → training trigger) + +### Multimodal (composition pipeline) + +- [x] 35 music styles ACE-Step +- [x] ComfyUI smart checkpoint selection (32 checkpoints + 24 LoRAs) +- [ ] lot-178: ACE-Step API direct (duration fix) +- [ ] lot-181: TTS voiceover mix into timeline +- [ ] lot-182: Audio effects pipeline (reverb, delay, EQ, compression) +- [ ] lot-183: DAW export (stems, markers, project file) +- [ ] lot-187: Auto-mastering (loudness normalization, limiting) +- [ ] lot-188: Stem separation (vocals, drums, bass, other) + +### Personas + +- [ ] Evaluer PCL (Persona-Aware Contrastive Learning) pour coherence +- [ ] Evaluer OpenCharacter pour generation profils synthetiques +- [x] Ajouter `/compose` command (generation musicale) + +### Frontend + +- [x] Implementer lot 16 UI Minitel rose (phosphore, VIDEOTEX) +- [x] VoiceChat push-to-talk + level meter + silence auto +- [x] Player audio + viewer image plein ecran +- [x] Mediatheque gallery/playlist +- [x] Progress bars animees Compose/Imagine +- [x] React.memo + useCallback on ChatSidebar, ChatInput, ChatHistory +- [x] 17 lazy-loaded routes (-53% initial JS) +- [x] CRT CSS-only effect (scanlines, vignette, phosphor glow, boot 0.8s) +- [x] Chat virtualization (react-window, variable row heights) +- [x] Markdown rendering (marked + DOMPurify) +- [x] CRT boot animation (modem dial, scanline reveal) +- [x] 5 CSS themes (minitel, crt, hacker, synthwave, default) +- [x] Mobile responsive pass (touch, bottom nav, viewport units) +- [x] Guest mode read-only UI +- [ ] lot-185: Composition timeline UI (waveform view, track lanes) +- [ ] lot-194: FX rack UI (drag-drop effects chain) +- [ ] lot-95: E2E Playwright tests (login, chat, upload, admin) +- [ ] lot-98: File sharing UI (upload → gallery) + +### Ops/TUI + +- [x] Deployer deep-audit.js sur kxkm-ai +- [x] Ajouter SearXNG au docker-compose +- [x] TTS sidecar HTTP (tts-server.py :9100, dual Chatterbox/Piper) +- [x] deploy.sh migrated tmux → systemd +- [x] Systemd services (kxkm-tts + kxkm-lightrag, auto-restart) +- [x] health-check.sh TUI (19 checks) +- [x] Docker compose 12 services with health checks +- [ ] Fix Docker transformers (rebuild propre avec torch) + +### Training + +- [x] Spike BGE-M3 (resultat negatif sur Apple/Metal, baseline maintenue) +- [x] TTS dual backend Chatterbox/Piper valide +- [x] Tool-calling benchmark (llama3.1 vs qwen3 vs mistral) +- [x] Sherlock migrated to llama3.1:8b-instruct-q4_0 +- [ ] lot-96: Persona DPO automation pipeline +- [ ] Tester ACE-Step 1.5 sur RTX 4090 + +### Veille OSS + +- [x] Veille mars 2026 complete (40+ projets analyses, top 10 recommandations) +- [ ] Suivre LLMRTC (WebRTC voice TypeScript) +- [ ] Suivre A2A Protocol (interop agents) +- [ ] Suivre MCP SDK updates +- [ ] Evaluer Kokoro TTS (82M params, ultra-leger) + +## Pipeline d'intervention + +```mermaid +stateDiagram-v2 + [*] --> Analyse: agent lance + Analyse --> Findings: scan code/docs/web + Findings --> Triage: P0/P1/P2 classification + Triage --> Fix_P0: P0 critique + Triage --> Plan_P1: P1 important + Triage --> Backlog_P2: P2 mineur + Fix_P0 --> Test: correction chirurgicale + Plan_P1 --> Test: correction planifiee + Test --> Deploy: tests OK + Deploy --> Log: log + purge + Log --> [*]: cycle termine + Backlog_P2 --> [*]: ajoute au TODO +``` + +## Affectations en cours (2026-03-20) + +### Mission globale +- Deep analyse continue du code, optimisation chirurgicale, et synchronisation documentaire apres chaque lot. +- Priorite execution: P1 fiabilite, puis dette perf/complexite, puis features lot 18-19. + +### Assignations agents -> sous-agents -> competences + +| Agent | Sous-agent | Competences principales | Taches assignees immediates | +|---|---|---|---| +| Coordinateur | Planner/Docs | triage, synchronisation, runbook | Maintenir PLAN/TODO, chainer les lots, tracer actions | +| Backend API | WS/HTTP surgeon | websocket, express, validation input | Extraire `ws-chat.ts` en modules, reduire logs, limiter hot paths | +| Node Engine | DAG runtime | graph validation, queue, state machine | Ajouter nodes `music_generation`, `voice_clone`, `document_extraction` | +| Ops/TUI | Audit operator | TUI scripts, logs, rotation, cron | Rendre `deep-audit` zero faux positif critique, pipeline logs + purge | +| Veille OSS | Scout | benchmark OSS, licences, interop | Evaluer Open WebUI, LibreChat, LangGraph, SearXNG, Docling | + +### Workflow d'enchainement +1. Executer audit + tests +2. Corriger de maniere chirurgicale +3. Re-executer audit + tests +4. Mettre a jour docs de pilotage +5. Alimenter TODO suivant avec ordre d'execution + +### Regles d'operation +- Interroger le user uniquement en cas de blocage reel (acces, choix irreversibles, secrets). +- Privilegier TUI et scripts avec logs lisibles, puis purge des logs obsoletes. +- Conserver la V1 comme reference comportementale, V2 comme cible active. + +### Etat de cycle (2026-03-20 18:00) + +- 130+ lots termines (lot-24 a lot-177). +- 425 tests, 0 failures. +- 13 services en production. +- 43 chat commands, 33 personas. +- 35 music styles (ACE-Step), 5 CSS themes. +- 32 ComfyUI checkpoints + 24 LoRAs, smart selection NLP. +- TTFC 284ms. +- Guest mode, mobile responsive. +- Structured logging complet (pino JSON, 0 console.log). +- Systemd services (TTS + LightRAG). +- Frontend: lazy routes (-53%), React.memo, CRT boot, chat virtualization. + +### Prochains lots (178-200) — Composition Pipeline + +1. lot-178: Compose duration fix (ACE-Step API direct) +2. lot-179: SPEC_COMPOSE_ADVANCED plan +3. lot-180: Timeline data model (tracks, clips, markers) +4. lot-181: TTS voiceover mix into timeline +5. lot-182: Audio effects pipeline (reverb, delay, EQ) +6. lot-183: DAW export (stems, markers, project) +7. lot-184-200: Multi-track, arrangement, mastering, stem separation, MIDI, templates, collab, lyrics, FX rack, automation, samples, spectral view, history, render queue, sharing diff --git a/FEATURE_MAP.md b/FEATURE_MAP.md new file mode 100644 index 0000000..0176a45 --- /dev/null +++ b/FEATURE_MAP.md @@ -0,0 +1,315 @@ +# Carte Fonctionnelle + +> "Nous sommes les saboteurs du big daddy mainframe." -- VNS Matrix, detourne par electron rare +> +> Ce document cartographie un systeme de chat IA multimodal local -- +> IRC dans l'ame, crypto-anarchiste dans l'infrastructure, musique concrete dans le traitement du signal. + +## Carte synthetique + +```mermaid +flowchart TD + Product[KXKM_Clown] + Product --> Chat[Chat Multimodal] + Product --> Admin[Admin] + Product --> Personas[Personas] + Product --> Engine[Node Engine] + Product --> Multimodal[Multimodal Pipeline] + Product --> Training[Training & DPO] + Product --> Ops[Ops] + Product --> Data[Donnees] + Product --> Security[Securite] + + Chat --> Channels[Canaux] + Chat --> Sessions[Sessions] + Chat --> Attachments[Pieces jointes] + Chat --> Routing[Routing IA] + Chat --> Streaming[Streaming LLM] + Chat --> WebSearch[Recherche web] + Chat --> Memory[Memoire persona] + Chat --> History[Historique chat] + + Admin --> Auth[Session admin] + Admin --> Modules[Modules switcher] + Admin --> Status[Status strip] + Admin --> Runtime[Runtime] + Admin --> Exports[Exports] + + Personas --> Sources[Sources web] + Personas --> Feedback[Feedback] + Personas --> Proposals[Proposals] + Personas --> Pharmacius[Pharmacius] + Personas --> Drawflow[React Flow nodal] + + Engine --> Graphs[Graphes DAG] + Engine --> Runs[Runs lifecycle] + Engine --> Artifacts[Artifacts] + Engine --> Models[Registry modeles] + Engine --> Queue[Queue async] + + Multimodal --> RAG[RAG local] + Multimodal --> STT[STT / Whisper] + Multimodal --> TTS[TTS / Piper] + Multimodal --> Vision[Vision / minicpm-v] + Multimodal --> PDF[Extraction PDF] + + Training --> DPO[Pipeline DPO] + Training --> TrainDash[Training dashboard] + Training --> Autoresearch[Autoresearch] + Training --> OllamaImport[Ollama import adapter] + + Ops --> Check[check / smoke] + Ops --> Build[build V1+V2] + Ops --> Batch[Batch orchestrator] + Ops --> Retention[Retention sweep] + + Data --> Persist[JSON/JSONL flat-file + Postgres] + Data --> DPOData[DPO pairs] + Data --> Export[HTML / training export] + Data --> ConvMemory[Conversation memory] + + Security --> NetGate[Subnet gate] + Security --> Cookie[HttpOnly cookies] + Security --> RBAC[RBAC V2] +``` + +--- + +## Chat + +| Fonctionnalite | Detail | +| --- | --- | +| WebSocket temps reel | Connexion persistante `/ws`, events bidirectionnels JSON | +| Multi-canaux | `#general`, canaux par persona, routage automatique | +| Streaming LLM | Reponses en streaming via Ollama avec indicateur d'ecriture | +| Tab completion | Completion nicks et commandes via Tab | +| Historique messages | ArrowUp/Down, 100 items en memoire client | +| DOM pruning | Elagage automatique a 500 messages max dans le DOM | +| Couleurs par bot | Mapping couleur unique par persona (palette deterministe) | +| Messages prives (PM) | Conversation directe utilisateur-persona via @mention | +| Upload fichiers | Pipeline multimodal: texte, image, audio, PDF | +| Commandes slash | `/help`, `/web`, `/nick`, `/who`, `/personas`, `/clear` | +| Recherche web | `/web ` — DuckDuckGo Lite ou API custom | +| Memoire persona | Contexte persistant par persona (faits, resume) | +| Historique chat | Logs JSONL par jour, API de consultation paginee | + +## Pipeline Multimodal + +| Fonctionnalite | Detail | +| --- | --- | +| RAG local | Embeddings via Ollama (`nomic-embed-text`), cosine similarity, contexte manifeste injecte | +| STT (Speech-to-Text) | `faster-whisper` (CTranslate2, int8) ou fallback `openai-whisper`, modeles tiny/base/small/medium/large | +| TTS (Text-to-Speech) | Dual backend: `piper-tts` (CPU) + `Chatterbox` (GPU, emotion tags, MIT) via TTS sidecar HTTP :9100 | +| Vision | `qwen3-vl:8b` via Ollama, analyse d'images uploadees, description en francais | +| VoiceChat | Push-to-talk (maintenir), level meter temps reel, silence auto 2s, 26 voix piper | +| Mediatheque | Gallery images + playlist audio, persistance disk, API REST | +| Extraction PDF | Docling (tables, layout, OCR) + fallback PyMuPDF | +| Texte/CSV/JSON | Lecture directe des fichiers texte, CSV, JSONL uploades | +| Generation musicale | `/compose` — ACE-Step 1.5 (GPU, <4GB VRAM) + fallback MusicGen | +| Generation images | `/imagine` — ComfyUI (SDXL Lightning + Flux 2 support) | +| Recherche web | SearXNG self-hosted + DuckDuckGo fallback | + +## Discord + +| Fonctionnalite | Detail | +| --- | --- | +| Bot texte Pharmacius | Bridge 2 salons Discord ↔ KXKM chat (WebSocket + Discord Gateway, sans deps) | +| Bot vocal | STT → personas → TTS en salon vocal Discord (@discordjs/voice) | +| Commandes | `!help`, `!personas`, `!status`, `@PersonaName message` | + +## MCP (Model Context Protocol) + +| Fonctionnalite | Detail | +| --- | --- | +| MCP Server | stdio transport, protocole 2024-11-05, compatible Claude Desktop | +| kxkm_chat | Envoyer un message aux personas | +| kxkm_personas | Lister les personas actives | +| kxkm_web_search | Recherche web via SearXNG | +| kxkm_status | Statut systeme (health + perf) | + +## Admin Dashboard + +| Fonctionnalite | Detail | +| --- | --- | +| Auth session | Cookie HttpOnly + fallback header legacy | +| Module switcher | dashboard, personas, runtime, channels, data, node-engine | +| Status strip | Connexion, clients connectes, sessions, personas actives, modeles | +| Endpoint status public | `/api/v2/status` sans authentification | +| Gestion runtime | Demarrage/arret personas, overrides modele | + +## Personas + +| Fonctionnalite | Detail | +| --- | --- | +| Personas seed | Schaeffer, Batty, Radigue, Oliveros, Lessig, etc. (catalogue initial) | +| Creation custom | Nouvelle persona depuis source editoriale | +| Overrides runtime | Nom, modele, style modifiables a chaud | +| Gestion sources | Subject, query, tone, themes, lexicon par persona | +| Pipeline feedback | Votes, signaux, edits utilisateur | +| Pharmacius | Orchestrateur editorial automatique | +| Systeme proposals | Suggestions d'amelioration auto-generees | +| Apply/revert | Application et annulation des proposals | +| Visualisation React Flow | Interface nodale pour le graphe de persona | +| Enable/disable | Activation/desactivation a chaud | +| Memoire persistante | Faits et resume par persona, mis a jour toutes les 5 interactions | + +## Node Engine + +| Fonctionnalite | Detail | +| --- | --- | +| Graphes DAG | Definition par noeuds et aretes, validation acyclique | +| 7 familles de noeuds | dataset_source, data_processing, dataset_builder, training, evaluation, registry, deployment | +| 16+ types de noeuds | Types specialises par famille | +| Cycle de vie run | queued -> running -> completed / failed / cancelled | +| Queue async | Concurrence controlee, execution asynchrone (Postgres) | +| Artifacts par etape | Tracking et stockage d'artifacts a chaque step | +| Registry modeles | Versionnage et catalogue des modeles produits | +| Templates seed | Graphes pre-configures comme point de depart | +| Recovery on crash | Reprise des runs interrompus au redemarrage | +| Cancel support | Annulation propre d'un run en cours | + +## Training & DPO + +| Fonctionnalite | Detail | +| --- | --- | +| Pipeline DPO | Extraction paires chosen/rejected depuis feedback personas, export JSONL | +| Training adapters | TRL (Hugging Face) + Unsloth, execution via Python venv | +| Autoresearch | Boucle d'experimentation automatisee: mutations, scoring, keep/discard | +| Evaluation | Score artifacts (accuracy, f1, bleu, perplexity), scoring deterministe | +| Registry | Enregistrement automatique du meilleur modele par session | +| Ollama import adapter | Import LoRA adapter dans Ollama via Modelfile (bash + node wrapper) | +| Training dashboard | React Flow dans le frontend V2, visualisation graphes et runs | +| Sandboxing runtimes | local_cpu, local_gpu, cloud_api | + +## Stockage & Donnees + +| Fonctionnalite | Detail | +| --- | --- | +| Persistance double | Postgres (production) ou flat-file JSON/JSONL (dev/demo) | +| Stats utilisateur | Compteurs et metriques par utilisateur | +| Memoire conversation | Contexte borne par persona/session | +| Memoire persona | Faits et resume persistants par persona (`data/persona-memory/`) | +| Logging DPO | Paires chosen/rejected pour entrainement RLHF | +| Chat logs | JSONL quotidien (`data/chat-logs/v2-YYYY-MM-DD.jsonl`) | +| Export training data | Extraction formatee pour fine-tuning | +| Recherche historique | Recherche dans les conversations passees | +| Export HTML | Export conversations en HTML lisible | +| Snapshots session | Capture d'etat de session a un instant T | +| Retention sweep | Nettoyage automatique sessions et logs anciens | + +## Ops & Scripts + +| Fonctionnalite | Detail | +| --- | --- | +| `npm run check` | Validation syntaxe V1 + TypeScript V2 | +| `npm run smoke` | 30+ tests d'integration automatises | +| `npm run build` | Build dist V1 + compilation V2 | +| `npm run v2:autoresearch` | Boucle autoresearch continue | +| Batch orchestrator | DAG Python pour orchestration de taches batch | +| TUI monitoring | health-check, queue-viewer, persona-manager, log-rotate | + +## Securite + +| Fonctionnalite | Detail | +| --- | --- | +| Subnet gate | Restriction d'acces admin par sous-reseau (`ADMIN_SUBNET`, `ADMIN_ALLOWED_SUBNETS`) | +| Cookies HttpOnly | Sessions admin non accessibles depuis JS client | +| Same-origin check | Verification origine sur les mutations | +| RBAC (V2) | Roles admin, editor, operator, viewer | + +## Fonctionnalites session 2026-03-20 (lots 174-177) + +| Fonctionnalite | Detail | +| --- | --- | +| **35 music styles** | ACE-Step: ambient, techno, house, drum-n-bass, dubstep, trance, IDM, noise, drone, classical, jazz, blues, rock, metal, hip-hop, trap, lo-fi, chillwave, synthwave, vaporwave, industrial, EBM, darkwave, post-punk, shoegaze, krautrock, musique concrete, electroacoustique, minimal, glitch, breakcore, gabber, footwork, grime, UK garage | +| **5 CSS themes** | minitel (phosphore vert), crt (amber), hacker (matrix green), synthwave (neon pink/blue), default (neutral) | +| **ComfyUI smart selection** | 32 checkpoints + 24 LoRAs, auto-selection based on prompt NLP analysis (style, subject, medium) | +| **NLP auto-detect generation** | Automatic detection of generation intent from natural language (compose vs imagine routing) | +| **Guest mode** | Read-only access without login, rate-limited, no admin functions | +| **Mobile responsive** | Touch gestures, bottom nav, viewport units, responsive layout pass | + +## Fonctionnalites session 2026-03-19/20 + +| Fonctionnalite | Detail | +| --- | --- | +| **43 chat commands** | /help, /nick, /who, /personas, /web, /clear, /status, /model, /persona, /reload, /export, /compose, /imagine, /voice, /memory, /context, /rag, /stats, /uptime, /changelog, /version, /dice, /roll, /flip, /ban, /unban, /mute, /unmute, /whisper, /w, /history, /search, /react, /invite, /time, /date, /session, /speed, + more | +| **Markdown rendering** | marked + DOMPurify, bold, italic, code blocks, links, lists in chat messages | +| **Smart routing** | 5 topic domains (music, philosophy, tech, arts, science), keyword + embedding scoring | +| **Dynamic context window** | Adaptive 4k-32k tokens based on conversation length and complexity | +| **Perf instrumentation** | 6 labels (http, ollama_ttfb, ollama_total, rag_search, rag_rerank, ws_message), p50/p95/p99 endpoint | +| **Error telemetry** | 16 error labels (validation, auth, ws, ollama, rag, tts, comfy, vision, stt, pdf, upload, memory, context, persona, mcp, search) | +| **Zod validation** | 19 schemas covering personas, sessions, node-engine, chat, media, admin, analytics routes | +| **MIME magic bytes** | file-type library magic bytes detection, SAFE_MIMES allowlist, blocks disguised executables | +| **WS reconnect + seq numbers** | Auto-reconnect (1s-30s exponential backoff), seq numbers for gap detection, missed message replay | +| **CRT boot animation** | Modem dial sound, progressive scanline reveal, phosphor glow warmup, 0.8s boot sequence | + +--- + +## Matrice de statut + +| Fonctionnalite | V1 | V2 | Priorite | +| --- | --- | --- | --- | +| **Chat** | | | | +| WebSocket temps reel | OK | OK | haute | +| Multi-canaux | OK | OK | haute | +| Streaming LLM | OK | OK | haute | +| Tab completion | OK | OK | moyenne | +| Historique messages | OK | OK | basse | +| DOM pruning | OK | OK | basse | +| Upload fichiers | OK | OK | moyenne | +| Commandes slash | OK | OK | moyenne | +| Recherche web (`/web`) | OK | OK | moyenne | +| **Multimodal** | | | | +| RAG (embeddings locaux) | -- | OK | haute | +| STT (faster-whisper) | -- | OK | haute | +| TTS (piper + Chatterbox) | -- | OK | moyenne | +| VoiceChat push-to-talk | -- | OK | haute | +| Mediatheque gallery/playlist | -- | OK | moyenne | +| Vision (qwen3-vl:8b) | -- | OK | haute | +| Extraction PDF | -- | OK | moyenne | +| Memoire persona | -- | OK | haute | +| **Admin** | | | | +| Auth session cookie | OK | OK | haute | +| Module switcher | OK | OK | haute | +| Status strip | OK | OK | moyenne | +| Status public | OK | OK | basse | +| **Personas** | | | | +| Personas seed | OK | OK | haute | +| Creation custom | OK | OK | haute | +| Overrides runtime | OK | OK | haute | +| Pipeline feedback | OK | OK | haute | +| Pharmacius | OK | OK | haute | +| Proposals apply/revert | OK | OK | haute | +| Drawflow nodal | OK | OK (React Flow) | moyenne | +| **Node Engine** | | | | +| Graphes DAG | OK | OK | critique | +| Run lifecycle | OK | OK | critique | +| Queue async | OK | OK (Postgres) | critique | +| Artifacts | OK | OK | critique | +| Registry modeles | OK | OK | critique | +| Recovery on crash | OK | OK | haute | +| Cancel support | OK | OK | haute | +| **Training & DPO** | | | | +| DPO export (JSONL) | OK | OK | haute | +| Training adapters (TRL/Unsloth) | -- | OK | haute | +| Autoresearch loop | -- | OK | haute | +| Ollama import adapter | -- | OK | haute | +| Training dashboard (React) | -- | OK | haute | +| Sandboxing runtimes | -- | OK | haute | +| **Stockage** | | | | +| Flat-file JSON/JSONL | OK | OK (+ Postgres) | haute | +| Memoire conversation | OK | OK | haute | +| Chat history (JSONL logs) | -- | OK | haute | +| Export HTML | OK | OK | basse | +| Retention sweep | OK | OK | moyenne | +| **Securite** | | | | +| Subnet gate | OK | OK | haute | +| Cookies HttpOnly | OK | OK | haute | +| RBAC roles | -- | OK | haute | +| **Tests** | | | | +| Backend unit tests | -- | OK (425) | haute | +| React component tests | -- | OK | haute | +| Smoke tests | OK | OK | haute | +| UI Minitel VIDEOTEX | -- | OK | haute | +| Deploy tmux (deploy.sh) | -- | OK | moyenne | +| CI/CD GitHub Actions | -- | OK | haute | diff --git a/PLAN.md b/PLAN.md index 53642f4..11c5f89 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1,15 +1,17 @@ # PLAN.md — KXKM_Clown -Updated: 2026-03-20T09:00:00Z +Updated: 2026-03-20T18:00:00Z ## Summary -- **120 lots executed** (24-143) in session 2026-03-19/20 +- **130+ lots executed** (24-177) in session 2026-03-19/20 - **425 tests**, 0 fail -- **40 commands**, 12 services, 9 spec docs +- **43 commands**, 13 services, 9 spec docs - **All SEC-01-05 resolved** - Ollama v0.18.2, qwen3.5:9b (256K ctx, adaptive thinking) -- Next: lots 144+ (E2E Playwright, DPO automation, mobile responsive) +- 35 music styles, 5 CSS themes, 32 ComfyUI checkpoints + 24 LoRAs +- TTFC 284ms +- Next: lots 178+ (Compose pipeline, DAW integration, composition advanced) --- @@ -592,68 +594,141 @@ Updated: 2026-03-20T12:00:00Z ## lot-149-file-sharing [planned] - Description: Upload files visible in MediaExplorer gallery -## lot-150-reload-markdown [done] -- Summary: /reload, markdown rendering, /help + tab-complete +## lot-174-perf-ttfc [done] +- Description: Performance optimization, TTFC 284ms +- Summary: Time-to-first-chunk optimized to 284ms, connection pooling, lazy service init. -## lot-151-export-memory-models [done] -- Summary: /export, /memory, /models commands +## lot-175-speed-command [done] +- Description: /speed command for latency diagnostics +- Summary: /speed returns TTFC, p50/p95 latencies, Ollama/RAG/TTS response times. -## lot-152-comfyui-integration [done] -- Summary: ComfyUI :8188 live, systemd unit +## lot-176-auto-gen-fix [done] +- Description: Auto-generation fix (NLP detect, ComfyUI smart selection) +- Summary: NLP auto-detect generation intent (/compose vs /imagine), ComfyUI smart checkpoint selection based on prompt analysis. -## lot-153-rag-perf [done] -- Summary: RAG timeout 2s, TTFC 284ms (27x faster) +## lot-177-music-styles [done] +- Description: 35 music styles + 5 CSS themes + guest mode + mobile responsive +- Summary: 35 ACE-Step music styles, 5 CSS themes (minitel/crt/hacker/synthwave/default), guest mode read-only, mobile responsive pass. -## lot-154-progress-bars [done] -- Summary: Broadcast progress for /imagine and /compose +## lot-178-compose-duration-fix [planned] +- Description: Compose duration fix (ACE-Step API direct integration) +- Owner: Multimodal +- Priority: P1 +- Tasks: + - [ ] Direct ACE-Step API call (bypass shell script) + - [ ] Duration parameter passthrough (5-300s) + - [ ] Progress callback from ACE-Step process -## lot-155-comfyui-smart-models [done] -- Summary: Smart model selection, 32 checkpoints, 24 LoRAs +## lot-179-spec-compose-advanced [planned] +- Description: SPEC_COMPOSE_ADVANCED plan (multi-track, effects, mastering) +- Owner: Coordinateur +- Priority: P1 +- Tasks: + - [ ] Write SPEC_COMPOSE_ADVANCED.md + - [ ] Define timeline data model + - [ ] Define mix/master pipeline stages -## lot-156-imagine-end2end [done] -- Summary: /imagine end-to-end verified +## lot-180-timeline-model [planned] +- Description: Composition timeline data model (tracks, clips, markers) +- Owner: Backend API +- Priority: P2 -## lot-157-compose-end2end [done] -- Summary: /compose end-to-end verified +## lot-181-tts-mix [planned] +- Description: TTS voiceover mix into composition timeline +- Owner: Multimodal +- Priority: P2 -## lot-158-e2e-tests [done] -- Summary: E2E 10/10 pass +## lot-182-audio-effects [planned] +- Description: Audio effects pipeline (reverb, delay, EQ, compression) +- Owner: Multimodal +- Priority: P2 -## lot-159-45-commands [done] -- Summary: 45 commands, tab-complete +## lot-183-daw-export [planned] +- Description: DAW export (stems, markers, project file) +- Owner: Multimodal +- Priority: P3 -## lot-160-nlp-autodetect [done] -- Summary: NL auto-triggers /imagine + /compose +## lot-184-multi-track [planned] +- Description: Multi-track composition (layer multiple ACE-Step generations) +- Owner: Multimodal +- Priority: P2 -## lot-161-admin [done] -- Summary: Admin panel, services, users +## lot-185-composition-ui [planned] +- Description: Composition timeline UI (waveform view, track lanes) +- Owner: Frontend +- Priority: P2 -## lot-162-media-explorer [done] -- Summary: Gallery for generated images/audio +## lot-186-arrangement [planned] +- Description: Arrangement tools (intro/verse/chorus structure) +- Owner: Backend API + Frontend +- Priority: P2 -## lot-163-persona-voices [done] -- Summary: Per-persona voice config +## lot-187-mastering [planned] +- Description: Auto-mastering pipeline (loudness normalization, limiting) +- Owner: Multimodal +- Priority: P3 -## lot-164-context-stats [done] -- Summary: /context stats +## lot-188-stem-separation [planned] +- Description: Stem separation (vocals, drums, bass, other) +- Owner: Multimodal +- Priority: P3 -## lot-165-security [done] -- Summary: SEC-01 to SEC-03 +## lot-189-midi-export [planned] +- Description: MIDI export from generated compositions +- Owner: Multimodal +- Priority: P3 -## lot-166-docs [done] -- Summary: Full spec docs +## lot-190-composition-templates [planned] +- Description: Composition templates (ambient, techno, orchestral presets) +- Owner: Backend API +- Priority: P2 -## lot-167-oss-veille [done] -- Summary: Qwen3-TTS, NexusRAG research +## lot-191-collab-composition [planned] +- Description: Collaborative composition (multi-user timeline editing) +- Owner: Backend API + Frontend +- Priority: P3 -## lot-168-nlp-verified [done] -- Summary: NLP auto-detect E2E verified +## lot-192-lyrics-gen [planned] +- Description: Lyrics generation (LLM-driven, synced to composition) +- Owner: Backend API +- Priority: P3 -## lot-169-plan-update [done] -- Summary: PLAN.md lots 150-169 +## lot-193-voice-composition [planned] +- Description: Voice-driven composition (hum/sing to generate) +- Owner: Multimodal +- Priority: P3 -## lot-170-final-health [done] -- Summary: 8/8 services, 45 cmds, TTFC 284ms +## lot-194-fx-rack [planned] +- Description: FX rack UI (drag-drop effects chain) +- Owner: Frontend +- Priority: P3 -## lot-171-final [done] -- Summary: Final session commit +## lot-195-automation [planned] +- Description: Parameter automation (volume, pan, effects over time) +- Owner: Frontend + Backend API +- Priority: P3 + +## lot-196-sample-library [planned] +- Description: Sample library integration (one-shots, loops, foley) +- Owner: Multimodal +- Priority: P3 + +## lot-197-spectral-view [planned] +- Description: Spectral analysis view for compositions +- Owner: Frontend +- Priority: P3 + +## lot-198-composition-history [planned] +- Description: Composition version history (undo/redo, snapshots) +- Owner: Backend API +- Priority: P3 + +## lot-199-render-queue [planned] +- Description: Render queue (batch export, format options) +- Owner: Backend API + Multimodal +- Priority: P3 + +## lot-200-composition-sharing [planned] +- Description: Composition sharing (public links, embeds) +- Owner: Backend API + Frontend +- Priority: P3 diff --git a/README.md b/README.md index 17c8ece..b7ffad5 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ docker compose --profile v2 --profile ollama up -d Par defaut, Ollama est attendu en natif sur le host (port 11434). -## Services (12) +## Services (13) | Service | Port | Description | | --- | --- | --- | @@ -54,9 +54,10 @@ Par defaut, Ollama est attendu en natif sur le host (port 11434). | PostgreSQL | 5432 | Persistence (personas, runs, logs) | | SearXNG | 8080 | Recherche web self-hosted | | TTS Sidecar | 9100 | Piper + Chatterbox (dual backend) | +| Qwen3-TTS | 9300 | Qwen3-TTS 0.6B CustomVoice (9 speakers) | | Reranker | 8787 | BGE/Jina reranking | | Docling | 5001 | Extraction PDF (tables, OCR) | -| ComfyUI | 8188 | Generation images (SDXL + Flux 2) | +| ComfyUI | 8188 | Generation images (32 checkpoints + 24 LoRAs) | | Worker | --- | Node Engine job processor (GPU) | | Discord Bot | --- | Pharmacius bridge (2 salons) | @@ -72,18 +73,21 @@ Par defaut, Ollama est attendu en natif sur le host (port 11434). - **TTS** — Piper-tts + Chatterbox (dual backend via TTS sidecar HTTP :9100) - **PDF** — Extraction via Docling/PyMuPDF (tables, layout, OCR) - **Recherche web** — SearXNG self-hosted + DuckDuckGo fallback -- **Generation musicale** — `/compose` via ACE-Step 1.5 / MusicGen -- **Generation images** — `/imagine` via ComfyUI (SDXL Lightning + Flux 2) +- **Generation musicale** — `/compose` via ACE-Step 1.5 / MusicGen, 35 music styles +- **Generation images** — `/imagine` via ComfyUI (32 checkpoints + 24 LoRAs, smart NLP selection) - **Memoire persona** — Faits et resume persistants, compaction LLM auto (750 MB) - **Inter-persona** — @mention directe, dialogue depth 3 -- **19 commandes slash** — /help, /nick, /who, /personas, /web, /clear, /status, /compose, /imagine, /voice, /memory, /context, /rag, /stats, /uptime, /model, /persona, /reload, /export +- **43 commandes slash** — /help, /nick, /who, /personas, /web, /clear, /status, /compose, /imagine, /voice, /memory, /context, /rag, /stats, /uptime, /model, /persona, /reload, /export, /changelog, /version, /dice, /roll, /flip, /ban, /unban, /mute, /unmute, /whisper, /history, /search, /react, /invite, /time, /date, /session, /speed, + more - **Markdown chat** — Rendu Markdown (marked + DOMPurify) dans les messages - **Smart routing** — 5 domaines thematiques (musique, philosophie, tech, arts, science) - **Validation Zod** — 19 schemas sur toutes les routes API - **Pino logging** — Logs structures JSON, rotation automatique - **Dynamic ctx** — Contexte LLM adaptatif 4k-32k selon la conversation -- **Perf instrumentation** — 6 labels (http, ollama, rag, ws), p50/p95/p99 +- **Perf instrumentation** — 6 labels (http, ollama, rag, ws), p50/p95/p99, TTFC 284ms - **Error telemetry** — 16 labels d'erreur, logging structure +- **5 CSS themes** — minitel (phosphore), crt (amber), hacker (matrix), synthwave (neon), default +- **Guest mode** — Acces lecture seule sans login, rate-limited +- **Mobile responsive** — Touch gestures, bottom nav, viewport units - **MIME magic bytes** — Validation magic bytes sur uploads, allowlist SAFE_MIMES - **WS reconnect** — Reconnexion auto (backoff 1s-30s), seq numbers, detection de gaps - **CRT effect** — Phosphore vert, scanlines, flicker, boot animation modem @@ -141,7 +145,7 @@ Par defaut, Ollama est attendu en natif sur le host (port 11434). | `PYTHON_BIN` | `python3` | Python avec libs ML (PyTorch, faster-whisper, piper-tts) | | `SCRIPTS_DIR` | `./scripts` | Chemin vers les scripts Python (TTS, STT, training) | -## Commandes slash (19) +## Commandes slash (43) | Commande | Description | Admin | | --- | --- | --- | @@ -152,18 +156,36 @@ Par defaut, Ollama est attendu en natif sur le host (port 11434). | `/web ` | Recherche web + commentaire personas | non | | `/clear` | Effacer le chat | non | | `/status` | Statut systeme | non | -| `/compose ` | Generation musicale (ACE-Step) | non | -| `/imagine ` | Generation image (ComfyUI) | non | +| `/compose ` | Generation musicale (ACE-Step, 35 styles) | non | +| `/imagine ` | Generation image (ComfyUI, smart selection) | non | | `/voice` | Toggle TTS voix | non | | `/memory` | Afficher memoire persona | non | | `/context` | Afficher contexte conversation | non | | `/rag ` | Recherche RAG directe | non | | `/stats` | Statistiques chat | non | | `/uptime` | Temps de fonctionnement | non | +| `/changelog` | Historique git recent | non | +| `/version` | Version et info app | non | +| `/dice NdS` | Lancer des des | non | +| `/roll` | Lancer 1d6 | non | +| `/flip` | Pile ou face | non | +| `/whisper ` | Message prive a une persona | non | +| `/history N` | Historique N derniers messages | non | +| `/search ` | Recherche dans le contexte | non | +| `/react ` | Reaction emoji broadcast | non | +| `/invite ` | Inviter persona dans le canal | non | +| `/time` | Heure (Europe/Paris) | non | +| `/date` | Date (FR locale) | non | +| `/session` | Info session courante | non | +| `/speed` | Diagnostics latence (TTFC, p50/p95) | non | | `/model` | Changer modele | oui | | `/persona` | Gerer personas | oui | | `/reload` | Recharger config | oui | | `/export` | Exporter donnees | oui | +| `/ban ` | Bannir un utilisateur | oui | +| `/unban ` | Debannir un utilisateur | oui | +| `/mute ` | Muter une persona | non | +| `/unmute ` | Demuter une persona | non | Mention directe: `@Schaeffer ta question` pour s'adresser a une persona specifique. @@ -304,7 +326,7 @@ kxkm_clown/ | RBAC | n/a | operationnel | | Frontend React | n/a | operationnel | | Training (TRL/Unsloth) | n/a | operationnel | -| Tests (425) | smoke | unit + component + smoke (425 pass) | +| Tests (425+) | smoke | unit + component + smoke (425+ pass) | | VoiceChat push-to-talk | n/a | operationnel | | Mediatheque gallery/playlist | n/a | operationnel | | UI Minitel VIDEOTEX | n/a | operationnel | diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 5d99a90..f879c5e 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -86,9 +86,13 @@ flowchart TD - [x] app.ts extraction (540 to 131 LOC, create-repos.ts extracted) - [x] Zod validation on all 19 API route schemas - [x] Error telemetry (16 labels) -- [x] Perf instrumentation (6 labels, p50/p95/p99) +- [x] Perf instrumentation (6 labels, p50/p95/p99), TTFC 284ms - [x] Smart routing (5 topic domains) - [x] Dynamic context window (4k-32k) +- [x] NLP auto-detect generation intent (compose vs imagine) +- [x] /speed command for latency diagnostics +- [ ] lot-178: ACE-Step API direct integration (duration fix) +- [ ] lot-180: Timeline data model - [ ] lot-97: Multi-channel support (create/join channels) - [ ] lot-100: Public demo mode read-only routes @@ -97,8 +101,21 @@ flowchart TD - [x] Extraire registry.ts du hotspot node-engine - [ ] Ajouter node type `music_generation` (ACE-Step 1.5) - [ ] Ajouter node type `voice_clone` (Chatterbox) +- [ ] Ajouter node type `audio_mix` (multi-track composition) +- [ ] Ajouter node type `audio_effects` (FX chain) - [ ] lot-96: Automated DPO pipeline (feedback → pairs → training trigger) +### Multimodal (composition pipeline) + +- [x] 35 music styles ACE-Step +- [x] ComfyUI smart checkpoint selection (32 checkpoints + 24 LoRAs) +- [ ] lot-178: ACE-Step API direct (duration fix) +- [ ] lot-181: TTS voiceover mix into timeline +- [ ] lot-182: Audio effects pipeline (reverb, delay, EQ, compression) +- [ ] lot-183: DAW export (stems, markers, project file) +- [ ] lot-187: Auto-mastering (loudness normalization, limiting) +- [ ] lot-188: Stem separation (vocals, drums, bass, other) + ### Personas - [ ] Evaluer PCL (Persona-Aware Contrastive Learning) pour coherence @@ -118,9 +135,13 @@ flowchart TD - [x] Chat virtualization (react-window, variable row heights) - [x] Markdown rendering (marked + DOMPurify) - [x] CRT boot animation (modem dial, scanline reveal) +- [x] 5 CSS themes (minitel, crt, hacker, synthwave, default) +- [x] Mobile responsive pass (touch, bottom nav, viewport units) +- [x] Guest mode read-only UI +- [ ] lot-185: Composition timeline UI (waveform view, track lanes) +- [ ] lot-194: FX rack UI (drag-drop effects chain) - [ ] lot-95: E2E Playwright tests (login, chat, upload, admin) - [ ] lot-98: File sharing UI (upload → gallery) -- [ ] lot-99: Mobile responsive deep pass (touch, bottom nav, viewport) ### Ops/TUI @@ -196,21 +217,26 @@ stateDiagram-v2 - Privilegier TUI et scripts avec logs lisibles, puis purge des logs obsoletes. - Conserver la V1 comme reference comportementale, V2 comme cible active. -### Etat de cycle (2026-03-20) +### Etat de cycle (2026-03-20 18:00) -- 71 lots termines (lot-24 a lot-94). +- 130+ lots termines (lot-24 a lot-177). - 425 tests, 0 failures. -- 12 services en production. -- 19 chat commands, 33 personas. +- 13 services en production. +- 43 chat commands, 33 personas. +- 35 music styles (ACE-Step), 5 CSS themes. +- 32 ComfyUI checkpoints + 24 LoRAs, smart selection NLP. +- TTFC 284ms. +- Guest mode, mobile responsive. - Structured logging complet (pino JSON, 0 console.log). - Systemd services (TTS + LightRAG). - Frontend: lazy routes (-53%), React.memo, CRT boot, chat virtualization. -### Prochains lots (95-100) +### Prochains lots (178-200) — Composition Pipeline -1. lot-95: E2E Playwright tests -2. lot-96: Persona DPO automation pipeline -3. lot-97: Multi-channel support -4. lot-98: File sharing between users -5. lot-99: Mobile responsive deep pass -6. lot-100: Public demo mode +1. lot-178: Compose duration fix (ACE-Step API direct) +2. lot-179: SPEC_COMPOSE_ADVANCED plan +3. lot-180: Timeline data model (tracks, clips, markers) +4. lot-181: TTS voiceover mix into timeline +5. lot-182: Audio effects pipeline (reverb, delay, EQ) +6. lot-183: DAW export (stems, markers, project) +7. lot-184-200: Multi-track, arrangement, mastering, stem separation, MIDI, templates, collab, lyrics, FX rack, automation, samples, spectral view, history, render queue, sharing diff --git a/docs/FEATURE_MAP.md b/docs/FEATURE_MAP.md index 0dc34fd..0176a45 100644 --- a/docs/FEATURE_MAP.md +++ b/docs/FEATURE_MAP.md @@ -217,11 +217,22 @@ flowchart TD | Same-origin check | Verification origine sur les mutations | | RBAC (V2) | Roles admin, editor, operator, viewer | +## Fonctionnalites session 2026-03-20 (lots 174-177) + +| Fonctionnalite | Detail | +| --- | --- | +| **35 music styles** | ACE-Step: ambient, techno, house, drum-n-bass, dubstep, trance, IDM, noise, drone, classical, jazz, blues, rock, metal, hip-hop, trap, lo-fi, chillwave, synthwave, vaporwave, industrial, EBM, darkwave, post-punk, shoegaze, krautrock, musique concrete, electroacoustique, minimal, glitch, breakcore, gabber, footwork, grime, UK garage | +| **5 CSS themes** | minitel (phosphore vert), crt (amber), hacker (matrix green), synthwave (neon pink/blue), default (neutral) | +| **ComfyUI smart selection** | 32 checkpoints + 24 LoRAs, auto-selection based on prompt NLP analysis (style, subject, medium) | +| **NLP auto-detect generation** | Automatic detection of generation intent from natural language (compose vs imagine routing) | +| **Guest mode** | Read-only access without login, rate-limited, no admin functions | +| **Mobile responsive** | Touch gestures, bottom nav, viewport units, responsive layout pass | + ## Fonctionnalites session 2026-03-19/20 | Fonctionnalite | Detail | | --- | --- | -| **19 chat commands** | /help, /nick, /who, /personas, /web, /clear, /status, /model, /persona, /reload, /export, /compose, /imagine, /voice, /memory, /context, /rag, /stats, /uptime | +| **43 chat commands** | /help, /nick, /who, /personas, /web, /clear, /status, /model, /persona, /reload, /export, /compose, /imagine, /voice, /memory, /context, /rag, /stats, /uptime, /changelog, /version, /dice, /roll, /flip, /ban, /unban, /mute, /unmute, /whisper, /w, /history, /search, /react, /invite, /time, /date, /session, /speed, + more | | **Markdown rendering** | marked + DOMPurify, bold, italic, code blocks, links, lists in chat messages | | **Smart routing** | 5 topic domains (music, philosophy, tech, arts, science), keyword + embedding scoring | | **Dynamic context window** | Adaptive 4k-32k tokens based on conversation length and complexity | diff --git a/docs/SPEC_COMPOSE_ADVANCED.md b/docs/SPEC_COMPOSE_ADVANCED.md new file mode 100644 index 0000000..080eda4 --- /dev/null +++ b/docs/SPEC_COMPOSE_ADVANCED.md @@ -0,0 +1,719 @@ +# SPEC: Advanced Music Composition Features + +> Date: 2026-03-20 +> Status: Planned (Phase 1 target: lots 180-185) +> Depends on: SPEC_COMPOSE.md (current single-generation pipeline) +> Stack: TypeScript (API) / Python (audio processing, GPU worker) / React (timeline UI) + +--- + +## 0. Current State Summary + +The existing `/compose` command (SPEC_COMPOSE.md) provides single-shot music generation: +- ACE-Step 1.5 (primary) or MusicGen-small (fallback) on RTX 4090 +- Single prompt -> single WAV file -> base64 broadcast via WebSocket +- No multi-track, no mixing, no post-processing +- ComposePage.tsx: form + result list with inline `