docs(ops): track lot-204 persona memory benchmark and harden log purge

2026-03-25 06:58:09 +01:00
parent 65d4cad55a
commit 9d687dda5e
8 changed files with 628 additions and 0 deletions
@@ -0,0 +1,168 @@
+# OSS Veille — Personas Runtime & Memory (2026-03-25)
+
+## Objectif
+
+Identifier des briques OSS directement exploitables pour renforcer le runtime personas sur 3 axes:
+
+- memoire runtime et persistence
+- coherence/persona drift
+- benchmark produit et training offline
+
+## Shortlist operationnel
+
+| Projet | Type | Fit principal | Decision |
+|---|---|---|---|
+| Letta | runtime agents stateful | separation memory working vs archival, blocs explicites | adopter les patterns |
+| LangGraph + LangMem | patterns memory | profil, store, semantic recall, policies d ecriture | adopter les patterns |
+| Mem0 + OpenMemory | memory layer | benchmark retrieval/write-back contre store local | bench court terme |
+| Graphiti (Zep) | temporal knowledge graph | relations temporelles et faits relies | spike optionnel |
+| LibreChat / Open WebUI | reference produit | benchmark UX/admin/persona tooling | benchmark uniquement |
+| OpenCharacter + PCL | R&D training | generation profils synthetiques et coherence | spike offline |
+
+## Resultats detailles
+
+### 1. Letta
+
+- Apport:
+  architecture memory explicite, stateful, avec blocs de memoire durables plutot qu un simple append de contexte.
+- Ce qui nous interesse:
+  modeliser par persona une `working_memory` courte et une `archival_memory` stabilisee.
+- Reuse concret:
+  reprendre la separation entre blocs visibles en permanence et memoire archivee interrogee a la demande.
+- Cout integration:
+  moyen.
+- Sources:
+  - https://docs.letta.com/guides/agents/memory
+  - https://github.com/letta-ai/letta
+
+### 2. LangGraph + LangMem
+
+- Apport:
+  patterns concrets pour profil utilisateur, memoire longue duree, extraction structuree et semantic search.
+- Ce qui nous interesse:
+  un write-back explicite depuis les echanges vers des facts structurés ou des summaries.
+- Reuse concret:
+  re-implementer en TypeScript des stores `profile`, `episodes`, `facts`, sans adopter tout le framework.
+- Cout integration:
+  faible a moyen.
+- Sources:
+  - https://langchain-ai.github.io/langgraph/how-tos/memory/add-memory/
+  - https://github.com/langchain-ai/langmem
+
+### 3. Mem0 + OpenMemory
+
+- Apport:
+  couche memory orientee extraction et recall, deja pensee pour usage production.
+- Ce qui nous interesse:
+  comparaison directe de recall et precision vs notre store local per-file.
+- Reuse concret:
+  benchmarker un adaptateur minimal avant toute integration profonde.
+- Cout integration:
+  moyen.
+- Sources:
+  - https://docs.mem0.ai/open-source/openmemory/overview
+  - https://github.com/mem0ai/mem0
+
+### 4. Graphiti / Zep
+
+- Apport:
+  knowledge graph temporel pour memories relationnelles, utile si les personas doivent conserver des faits lies dans le temps.
+- Ce qui nous interesse:
+  episodes, entites, relations, resolution des contradictions et faits datees.
+- Reuse concret:
+  pas pour le chemin critique immediat, mais pertinent si la memoire persona sort du simple KV/profile.
+- Cout integration:
+  moyen a eleve.
+- Sources:
+  - https://github.com/getzep/graphiti
+  - https://docs.getzep.com/graphiti
+
+### 5. LibreChat / Open WebUI
+
+- Apport:
+  references produit solides pour agents, prompts, knowledge, outils, administration self-hosted.
+- Ce qui nous interesse:
+  benchmark des flows admin/editor pour `persona-source-feedback-proposal`.
+- Reuse concret:
+  benchmark UX/admin, pas une dependance runtime cible.
+- Cout integration:
+  faible en veille, eleve si adoption produit.
+- Sources:
+  - https://github.com/danny-avila/LibreChat
+  - https://docs.openwebui.com/
+  - https://github.com/open-webui/open-webui
+
+### 6. OpenCharacter + PCL
+
+- Apport:
+  generation de profils synthetiques et amelioration de coherence persona sans gros volume d annotations manuelles.
+- Ce qui nous interesse:
+  pipeline offline pour enrichir ou evaluer les personas existantes, pas le runtime temps reel.
+- Reuse concret:
+  spike training/eval sur 5 personas avec score de coherence et drift.
+- Cout integration:
+  moyen a eleve.
+- Sources:
+  - https://arxiv.org/abs/2501.15427
+  - https://arxiv.org/abs/2503.17662
+
+## Decision engineering
+
+### A adopter maintenant
+
+- schema `working_memory` / `archival_memory`
+- policies d ecriture explicites depuis chat/feedback/proposals
+- separation entre source editoriale, memoire runtime, et evaluations
+
+### A benchmarker tout de suite
+
+- Mem0/OpenMemory contre le store local per-file
+- patterns Letta vs LangGraph sur 3 cas:
+  - fait stable
+  - preference volatile
+  - resume de session
+
+### A garder hors chemin critique
+
+- Graphiti tant que la memoire n exige pas de raisonnement relationnel temporel
+- OpenCharacter + PCL tant que le runtime n est pas stabilise
+- adoption produit complete LibreChat/Open WebUI
+
+## Plan benchmark minimal
+
+### Baseline actuelle
+
+- store local per-file sous `data/v2-local`
+- sources separees de feedback et proposals
+- migration legacy lecture seule encore presente
+
+### Mesures
+
+- TTFC avant et apres recall memory
+- precision factuelle sur 20 prompts controles
+- taux de drift persona sur 20 prompts controles
+- write-rate memory et taille moyenne des objets persistés
+
+### Harness propose
+
+- 5 personas fixes
+- 20 prompts par persona:
+  - faits biographiques
+  - preferences stylistiques
+  - contradiction volontaire
+  - rappel a long terme
+- 3 strategies:
+  - baseline locale
+  - baseline + policies type LangGraph
+  - baseline + adaptateur Mem0/OpenMemory
+
+## Recommandation execution
+
+- lot-202:
+  schema memory v2 avec `working_memory`, `archival_memory`, `policy_state`
+- lot-203:
+  policy engine `extract -> summarize -> prune -> persist`
+- lot-204:
+  benchmark Letta/LangGraph/Mem0 documente et harness local
+- lot-205:
+  spike offline OpenCharacter/PCL sur 5 personas
@@ -0,0 +1,35 @@
+# PLAN (kxkm-personas-runtime-20260325)
+
+Updated: 2026-03-25T06:35:00Z
+
+## lot-201-runtime-hardening [done]
+- Description: Durcir le runtime personas local et basculer le store actif en per-file v2-local
+- Depends on: none
+- Owner: Personas
+- Execution: manual
+- Checks: npm run check, npm run test:v2
+- Summary: Done: persistence per-file, fallback legacy-read, clones defensifs, tests de non-regression, scripts smoke/build alignes.
+
+## lot-202-memory-schema [pending]
+- Description: Formaliser un schema memory persona v2 avec working memory et archival memory
+- Depends on: lot-201-runtime-hardening
+- Owner: Personas
+- Execution: manual
+- Checks: npm run check, npm run test:v2
+- Summary: A lancer apres hardening du store local.
+
+## lot-203-memory-policy [pending]
+- Description: Ajouter un moteur de policies pour extraction, summarization, pruning et evaluation memory
+- Depends on: lot-202-memory-schema
+- Owner: Backend API
+- Execution: manual
+- Checks: npm run check, npm run test:v2
+- Summary: Pending.
+
+## lot-204-oss-benchmark [in_progress]
+- Description: Comparer patterns Letta LangGraph Mem0 et preparer un spike OpenCharacter PCL
+- Depends on: lot-201-runtime-hardening
+- Owner: Veille OSS
+- Execution: manual
+- Checks: docs-reviewed
+- Summary: Veille documentee et plan benchmark poses; spike OpenCharacter/PCL encore pending.
@@ -0,0 +1,40 @@
+# EXECUTION STATUS (kxkm-personas-runtime-20260325)
+
+Updated: 2026-03-25T06:35:00Z
+
+## lot-201-runtime-hardening
+- Status: done
+- Owner: Personas
+- Execution: manual
+- Checks: npm run check, npm run test:v2
+- Open tasks: none
+
+## lot-202-memory-schema
+- Status: pending
+- Owner: Personas
+- Execution: manual
+- Checks: npm run check, npm run test:v2
+- Open tasks:
+  - schema-v2 [pending] (P1, Personas)
+  - storage-layout [pending] (P1, Backend API)
+  - migration-soft [pending] (P2, Coordinateur)
+
+## lot-203-memory-policy
+- Status: pending
+- Owner: Backend API
+- Execution: manual
+- Checks: npm run check, npm run test:v2
+- Open tasks:
+  - policy-engine [pending] (P1, Backend API)
+  - telemetry [pending] (P2, Ops/TUI)
+  - eval-harness [pending] (P2, Personas)
+
+## lot-204-oss-benchmark
+- Status: in_progress
+- Owner: Veille OSS
+- Execution: manual
+- Checks: docs-reviewed
+- Open tasks:
+  - letta-langgraph-patterns [done] (P2, Veille OSS)
+  - mem0-benchmark-plan [done] (P2, Veille OSS)
+  - opencharacter-pcl-spike [pending] (P3, Training)
@@ -0,0 +1,25 @@
+# TODO (kxkm-personas-runtime-20260325)
+
+Updated: 2026-03-25T06:35:00Z
+
+## lot-201-runtime-hardening
+- [x] per-file-store (done) | owner: Personas | severity: P1
+- [x] defensive-snapshots (done) | owner: Backend API | severity: P1
+- [x] regression-tests (done) | owner: Backend API | severity: P1
+- [x] script-alignment (done) | owner: Ops/TUI | severity: P1
+- [x] docs-sync (done) | owner: Coordinateur | severity: P2
+
+## lot-202-memory-schema
+- [ ] schema-v2 (pending) | owner: Personas | severity: P1
+- [ ] storage-layout (pending) | owner: Backend API | severity: P1
+- [ ] migration-soft (pending) | owner: Coordinateur | severity: P2
+
+## lot-203-memory-policy
+- [ ] policy-engine (pending) | owner: Backend API | severity: P1
+- [ ] telemetry (pending) | owner: Ops/TUI | severity: P2
+- [ ] eval-harness (pending) | owner: Personas | severity: P2
+
+## lot-204-oss-benchmark
+- [x] letta-langgraph-patterns (done) | owner: Veille OSS | severity: P2
+- [x] mem0-benchmark-plan (done) | owner: Veille OSS | severity: P2
+- [ ] opencharacter-pcl-spike (pending) | owner: Training | severity: P3
@@ -0,0 +1,19 @@
+# lot-204-oss-benchmark
+
+## Status
+
+in_progress
+
+## Artifacts
+
+- `docs/OSS_VEILLE_2026-03-25_PERSONAS.md`
+
+## Completed
+
+- comparative shortlist for Letta, LangGraph/LangMem, Mem0/OpenMemory, Graphiti, LibreChat/Open WebUI
+- benchmark plan for memory recall vs current local per-file runtime
+
+## Remaining
+
+- define the concrete harness implementation for the Mem0/OpenMemory adapter
+- scope the offline OpenCharacter/PCL spike on 5 personas
@@ -0,0 +1,69 @@
+{
+  "project": "kxkm-personas-runtime-20260325",
+  "max_parallel_tasks": 3,
+  "task_command_template": "bash -lc \"PATH=\\\"/tmp/node-local/bin:$PATH\\\" node scripts/v2-agent-task.js --batch {batch} --task {task} --query {query} --out {out}\"",
+  "batches": [
+    {
+      "id": "lot-201-runtime-hardening",
+      "description": "Durcir le runtime personas local et basculer le store actif en per-file v2-local",
+      "summary": "Done: persistence per-file, fallback legacy-read, clones defensifs, tests de non-regression, scripts smoke/build alignes.",
+      "status": "done",
+      "owner": "Personas",
+      "execution": "manual",
+      "checks": ["npm run check", "npm run test:v2"],
+      "depends_on": [],
+      "tasks": [
+        { "id": "per-file-store", "query": "Migrer le store runtime personas vers data/v2-local fichier par persona", "status": "done", "owner": "Personas", "severity": "P1" },
+        { "id": "defensive-snapshots", "query": "Supprimer les fuites de references mutables dans les repos in-memory personas", "status": "done", "owner": "Backend API", "severity": "P1" },
+        { "id": "regression-tests", "query": "Ajouter des tests pour migration legacy et immutabilite des snapshots", "status": "done", "owner": "Backend API", "severity": "P1" },
+        { "id": "script-alignment", "query": "Aligner smoke/build/log purge sur data/v2-local et compat explicite", "status": "done", "owner": "Ops/TUI", "severity": "P1" },
+        { "id": "docs-sync", "query": "Synchroniser PLAN TODO et runbooks avec l etat reel du runtime personas", "status": "done", "owner": "Coordinateur", "severity": "P2" }
+      ]
+    },
+    {
+      "id": "lot-202-memory-schema",
+      "description": "Formaliser un schema memory persona v2 avec working memory et archival memory",
+      "summary": "A lancer apres hardening du store local.",
+      "status": "pending",
+      "owner": "Personas",
+      "execution": "manual",
+      "checks": ["npm run check", "npm run test:v2"],
+      "depends_on": ["lot-201-runtime-hardening"],
+      "tasks": [
+        { "id": "schema-v2", "query": "Definir les blocs working memory et archival memory par persona", "status": "pending", "owner": "Personas", "severity": "P1" },
+        { "id": "storage-layout", "query": "Etablir le layout de persistance memory sans recroiser la source editoriale", "status": "pending", "owner": "Backend API", "severity": "P1" },
+        { "id": "migration-soft", "query": "Preparer une migration douce depuis les structures runtime actuelles", "status": "pending", "owner": "Coordinateur", "severity": "P2" }
+      ]
+    },
+    {
+      "id": "lot-203-memory-policy",
+      "description": "Ajouter un moteur de policies pour extraction, summarization, pruning et evaluation memory",
+      "summary": "Pending.",
+      "status": "pending",
+      "owner": "Backend API",
+      "execution": "manual",
+      "checks": ["npm run check", "npm run test:v2"],
+      "depends_on": ["lot-202-memory-schema"],
+      "tasks": [
+        { "id": "policy-engine", "query": "Implementer des policies configurables extraction summarize prune", "status": "pending", "owner": "Backend API", "severity": "P1" },
+        { "id": "telemetry", "query": "Mesurer drift, recall et write-rate des memories personas", "status": "pending", "owner": "Ops/TUI", "severity": "P2" },
+        { "id": "eval-harness", "query": "Construire un harness de non-regression pour coherence memory", "status": "pending", "owner": "Personas", "severity": "P2" }
+      ]
+    },
+    {
+      "id": "lot-204-oss-benchmark",
+      "description": "Comparer patterns Letta LangGraph Mem0 et preparer un spike OpenCharacter PCL",
+      "summary": "Veille documentee et plan benchmark poses; spike offline restant a planifier.",
+      "status": "in_progress",
+      "owner": "Veille OSS",
+      "execution": "manual",
+      "checks": ["docs-reviewed"],
+      "depends_on": ["lot-201-runtime-hardening"],
+      "tasks": [
+        { "id": "letta-langgraph-patterns", "query": "Comparer working memory archival memory et stores profils sur sources officielles", "status": "done", "owner": "Veille OSS", "severity": "P2" },
+        { "id": "mem0-benchmark-plan", "query": "Preparer un benchmark Mem0 contre le runtime local sur latence et recall", "status": "done", "owner": "Veille OSS", "severity": "P2" },
+        { "id": "opencharacter-pcl-spike", "query": "Definir un spike training offline sur 5 personas avec OpenCharacter et PCL", "status": "pending", "owner": "Training", "severity": "P3" }
+      ]
+    }
+  ]
+}
@@ -0,0 +1,124 @@
+{
+  "project": "kxkm-personas-runtime-20260325",
+  "updated_at": "2026-03-25T06:35:00Z",
+  "batches": {
+    "lot-201-runtime-hardening": {
+      "status": "done",
+      "depends_on": [],
+      "owner": "Personas",
+      "tasks": {
+        "per-file-store": {
+          "status": "done",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "defensive-snapshots": {
+          "status": "done",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "regression-tests": {
+          "status": "done",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "script-alignment": {
+          "status": "done",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "docs-sync": {
+          "status": "done",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        }
+      }
+    },
+    "lot-202-memory-schema": {
+      "status": "pending",
+      "depends_on": [
+        "lot-201-runtime-hardening"
+      ],
+      "owner": "Personas",
+      "tasks": {
+        "schema-v2": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "storage-layout": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "migration-soft": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        }
+      }
+    },
+    "lot-203-memory-policy": {
+      "status": "pending",
+      "depends_on": [
+        "lot-202-memory-schema"
+      ],
+      "owner": "Backend API",
+      "tasks": {
+        "policy-engine": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "telemetry": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        },
+        "eval-harness": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        }
+      }
+    },
+    "lot-204-oss-benchmark": {
+      "status": "in_progress",
+      "depends_on": [
+        "lot-201-runtime-hardening"
+      ],
+      "owner": "Veille OSS",
+      "tasks": {
+        "letta-langgraph-patterns": {
+          "status": "done",
+          "attempts": 1,
+          "output": "docs/OSS_VEILLE_2026-03-25_PERSONAS.md",
+          "last_error": ""
+        },
+        "mem0-benchmark-plan": {
+          "status": "done",
+          "attempts": 1,
+          "output": "ops/v2/personas-runtime-20260325/outputs/lot-204-oss-benchmark/summary.md",
+          "last_error": ""
+        },
+        "opencharacter-pcl-spike": {
+          "status": "pending",
+          "attempts": 0,
+          "output": "",
+          "last_error": ""
+        }
+      }
+    }
+  }
+}
@@ -0,0 +1,148 @@
+#!/usr/bin/env bash
+set -Eeuo pipefail
+umask 027
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+OPS_DIR="${ROOT_DIR}/ops/v2"
+LOG_DIR="${OPS_DIR}/logs"
+SUMMARY_LOG="${LOG_DIR}/purge-ops-logs.jsonl"
+MAX_AGE_DAYS="${MAX_AGE_DAYS:-3}"
+DRY_RUN=1
+RUN_ID="$(date -u +%Y%m%dT%H%M%SZ)-$$"
+CANDIDATE_COUNT=0
+TOTAL_BYTES=0
+DELETED_COUNT=0
+RESULT="ok"
+
+json_escape() {
+  local value="$1"
+  value="${value//\\/\\\\}"
+  value="${value//\"/\\\"}"
+  printf '%s' "$value"
+}
+
+write_summary() {
+  local exit_code="$1"
+  local mode="dry-run"
+  local finished_at
+  finished_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
+  if [[ "$DRY_RUN" -eq 0 ]]; then
+    mode="delete"
+  fi
+  if [[ -d "$LOG_DIR" ]]; then
+    printf '{"ts":"%s","run_id":"%s","mode":"%s","max_age_days":%s,"candidates":%s,"bytes":%s,"deleted":%s,"result":"%s","exit_code":%s}\n' \
+      "$finished_at" \
+      "$(json_escape "$RUN_ID")" \
+      "$mode" \
+      "$MAX_AGE_DAYS" \
+      "$CANDIDATE_COUNT" \
+      "$TOTAL_BYTES" \
+      "$DELETED_COUNT" \
+      "$(json_escape "$RESULT")" \
+      "$exit_code" >> "$SUMMARY_LOG" || true
+  fi
+}
+
+on_exit() {
+  local exit_code="$1"
+  trap - EXIT
+  write_summary "$exit_code"
+  exit "$exit_code"
+}
+
+trap 'on_exit $?' EXIT
+
+for arg in "$@"; do
+  case "$arg" in
+    --yes) DRY_RUN=0 ;;
+    --days=*) MAX_AGE_DAYS="${arg#*=}" ;;
+    --help|-h)
+      cat <<'EOF'
+Usage: ops/v2/purge-ops-logs.sh [--days=N] [--yes]
+
+Options:
+  --days=N   Maximum age in days (default: 3)
+  --yes      Delete files (default is dry-run)
+EOF
+      exit 0
+      ;;
+  esac
+done
+
+if [[ ! "$MAX_AGE_DAYS" =~ ^[0-9]+$ ]]; then
+  echo "Invalid --days value: $MAX_AGE_DAYS" >&2
+  RESULT="invalid-args"
+  exit 2
+fi
+
+if [[ -z "$LOG_DIR" || "$LOG_DIR" == "/" || "$LOG_DIR" != "${OPS_DIR}/logs" ]]; then
+  echo "Refusing to purge unsafe log dir: $LOG_DIR" >&2
+  RESULT="unsafe-log-dir"
+  exit 2
+fi
+
+if [[ ! -d "$LOG_DIR" ]]; then
+  echo "[purge-ops-logs] log dir missing: $LOG_DIR"
+  RESULT="log-dir-missing"
+  exit 0
+fi
+
+if command -v flock >/dev/null 2>&1; then
+  exec 9>"${LOG_DIR}/.purge.lock"
+  if ! flock -n 9; then
+    echo "[purge-ops-logs] another purge run is already in progress"
+    RESULT="locked"
+    exit 0
+  fi
+fi
+
+echo "=== OPS Logs Purge ==="
+echo "Run:      $RUN_ID"
+echo "Dir:      $LOG_DIR"
+echo "Max age:  ${MAX_AGE_DAYS}d"
+if [[ "$DRY_RUN" -eq 1 ]]; then
+  echo "Mode:     DRY-RUN"
+else
+  echo "Mode:     DELETE"
+fi
+echo
+
+mapfile -t CANDIDATES < <(
+  find "$LOG_DIR" \
+    -maxdepth 1 \
+    -type f \
+    -mtime +"$MAX_AGE_DAYS" \
+    ! -name "$(basename "$SUMMARY_LOG")" \
+    ! -name ".purge.lock" | sort
+)
+
+if [[ "${#CANDIDATES[@]}" -eq 0 ]]; then
+  echo "No files to purge."
+  RESULT="no-candidates"
+  exit 0
+fi
+
+CANDIDATE_COUNT="${#CANDIDATES[@]}"
+for f in "${CANDIDATES[@]}"; do
+  size="$(stat -c%s "$f" 2>/dev/null || echo 0)"
+  TOTAL_BYTES=$((TOTAL_BYTES + size))
+  printf "  %8s  %s\n" "$size" "$(basename "$f")"
+done
+
+echo
+echo "Candidates: ${CANDIDATE_COUNT} files"
+echo "Bytes:      ${TOTAL_BYTES}"
+
+if [[ "$DRY_RUN" -eq 1 ]]; then
+  echo "Dry-run complete. Re-run with --yes to delete."
+  RESULT="dry-run-complete"
+  exit 0
+fi
+
+for f in "${CANDIDATES[@]}"; do
+  rm -f "$f"
+  DELETED_COUNT=$((DELETED_COUNT + 1))
+done
+
+echo "Purge complete."
+RESULT="purge-complete"