electron-rare/lisael-box
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
4bca57e514a2e45f3ab0f37da4c03cfb3b32c3af
lisael-box/docs/superpowers/plans/2026-06-21-merlin-studio-v2.md
T
L'électron rare 4bca57e514 docs(plans): firmware routine + merlin studio v2 
Two implementation plans for the routine mode (firmware) and Merlin Studio v2 (Tower/web: routines tab + transfers viz).
2026-06-21 09:00:21 +02:00

58 KiB
Raw Blame History

Merlin Studio v2 — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Côté Tower (lisael-content) + UI web (merlin.saillant.cc), livrer (A) le pipeline de génération des routines illustrées (TTS voix fr + Twemoji→.bin RGB565A8 + routines.json SD + push delta), (B) l'onglet web « 🗓️ Routines » + son API, et (C) le journal des transferts (durée/débit/retry) + l'onglet web « 📤 Transferts » + son API. Le firmware est traité dans un plan séparé ; ce plan respecte le contrat d'interface SD/push partagé.

Architecture : tower/lisael_content.py (module partagé, Python stdlib + Pillow) gère feeds/podcasts + désormais routines + journal transferts. tower/lisael_server.py (:9555, ThreadingHTTPServer, classe H) expose /register (machine) et l'API web sous Basic Auth. tower/merlin_ui.html (vanilla JS, onglets) est l'UI. La box annonce dans have (POST /register) l'union des basenames de /sdcard/podcasts ET /sdcard/routines ; Tower calcule le delta routines = (basenames staging routines) − have et pousse vers ?name=routines/<file>. Vérité éditoriale = routines.json côté Tower (/home/clems/lisael-content/routines.json), régénéré en assets par build_routine_manifest.

Tech Stack : Python 3 stdlib (http.server, urllib, json, subprocess, tempfile, threading, hashlib, time) + Pillow (PIL.Image) + LVGLImage.py (script LVGL, --ofmt BIN) + ffmpeg. UI = HTML/CSS/JS vanilla, pas de framework. Tests hôte = asserts purs if __name__ == "__main__" (pattern tower/test_delta.py / tower/test_content.py), pas de pytest.

Global Constraints

  • Python stdlib only (+ Pillow déjà utilisé pour les covers) ; aucune nouvelle dépendance pip.
  • TTS via la gateway origine tailnet http://100.78.6.122:9300/v1/audio/speech (model=tts-1, voice=fr, response_format=mp3) — JAMAIS le domaine public (Cloudflare bloque Python-urllib).
  • Twemoji raw via https://raw.githubusercontent.com/jdecked/twemoji/main/assets/72x72/<cp>.png avec User-Agent: curl/8 ; cp() strip le VS16 (U+FE0F) sinon 404.
  • Vérif des URLs web publiques (merlin.saillant.cc) toujours avec User-Agent: Mozilla/5.0 (Cloudflare 403 sinon).
  • Pictos étape = LVGLImage.py --ofmt BIN --cf RGB565A8 sur PNG 160×160 .convert("RGBA") (garde l'alpha de l'emoji) — PAS RGB565/.convert("RGB") (réservé aux covers podcasts).
  • Audio = TTS mono mp3 ; si re-transcode nécessaire, ffmpeg -ac 1 -b:a 48k -c:a libmp3lame (la sortie gateway est déjà mono mp3, transcode optionnel).
  • Noms de fichiers SD strictement ASCII, préfixés par id de routine (matin_/soir_) ; les libellés accentués vivent dans le JSON UTF-8. Pas de collision avec les podcasts (dossier SD distinct routines/).
  • Push delta : le manifest (routines.json) part TOUJOURS en dernier (réutiliser la sémantique compute_delta).
  • Toute écriture de fichier d'état (routines.json, transfers.json) via _atomic_write_json.
  • API web sous Basic Auth MERLIN_AUTH (court-circuit /register machine sans auth, restreint IP privée via _is_private).
  • Déploiement manuel : rsync vers /home/clems/lisael-content sur Tower (clems@192.168.0.120) puis sudo systemctl restart lisael-content.
  • Commits : sujet ≤ 50 chars, scope sans underscore, PAS d'attribution AI, pas de --no-verify.

Task 1: Helper TTS tts_speak() dans lisael_content.py

Files:

  • tower/lisael_content.py (ajout)

Interfaces:

  • Produces : def tts_speak(text: str) -> bytes — POST gateway /v1/audio/speech, renvoie les octets mp3 (lève en cas d'échec réseau/HTTP).

  • Consumes : env LISAEL_TTS_URL (défaut http://100.78.6.122:9300/v1/audio/speech), env LISAEL_TTS_VOICE (défaut fr).

  • Step 1: Ajouter en tête de tower/lisael_content.py (près des autres constantes, après BOX_PORT) :

    TTS_URL   = os.environ.get("LISAEL_TTS_URL", "http://100.78.6.122:9300/v1/audio/speech")
TTS_VOICE = os.environ.get("LISAEL_TTS_VOICE", "fr")

  • Step 2: Ajouter la fonction (après transcode, avant make_cover) :

    def tts_speak(text):
    """Generate a mono mp3 announcement via the tailnet gateway. Returns bytes.
    Raises on network/HTTP failure (caller decides degraded behaviour)."""
    body = json.dumps({"model": "tts-1", "input": text, "voice": TTS_VOICE,
                       "response_format": "mp3"}).encode()
    req = urllib.request.Request(TTS_URL, data=body,
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=30) as r:
        return r.read()

  • Step 3: Vérif manuelle (réseau, non unit-testable). Depuis grosmac via Tower : ssh clems@192.168.0.120 'cd /home/clems/lisael-content && python3 -c "import lisael_content as C; b=C.tts_speak(\"Habille-toi\"); print(len(b)); open(\"/tmp/tts_test.mp3\",\"wb\").write(b)"' puis vérifier que le fichier fait > 1 Ko et est lisible (file /tmp/tts_test.mp3 → MPEG ADTS / MP3). Si la gateway n'est pas joignable depuis grosmac, exécuter directement sur Tower. Commit : feat(tower): add gateway TTS helper.

Task 2: Helper Twemoji→.bin emoji_cp() + emoji_bin() (TDD pour emoji_cp)

Files:

  • tower/lisael_content.py (ajout)
  • tower/test_routines.py (nouveau fichier de test)

Interfaces:

  • Produces : def emoji_cp(emoji: str) -> str — codepoints Twemoji joints par -, VS16 stripé (pure, testable).

  • Produces : def emoji_bin(emoji: str, key: str) -> str — télécharge le Twemoji, resize 160×160 RGBA, génère <key>.bin RGB565A8 dans STAGING_ROUTINES, renvoie "<key>.bin" ou "" (réseau, non unit-testable).

  • Consumes : constante TWEMOJI, LVGLIMG, STAGING_ROUTINES (Task 5 ; pour cette task on peut écrire dans un dossier passé/par défaut — voir Step 4).

  • Step 1 (RED): Créer tower/test_routines.py avec en tête le pattern d'isolation de test_content.py :

    import os, tempfile
os.environ["LISAEL_BASE"] = tempfile.mkdtemp(prefix="routines_test_")
os.environ["LISAEL_STAGING"] = os.path.join(os.environ["LISAEL_BASE"], "podcasts")
os.environ["LISAEL_STAGING_ROUTINES"] = os.path.join(os.environ["LISAEL_BASE"], "routines")
import lisael_content as C

def test_emoji_cp_strips_vs16():
    # plain emoji: single codepoint
    assert C.emoji_cp("\U0001F600") == "1f600"           # 😀
    # emoji with VS16 (U+FE0F) must be stripped (twemoji 404s otherwise)
    assert C.emoji_cp("☀️") == "2600"          # ☀️ -> 2600
    # multi-codepoint (no VS16): joined by '-'
    assert C.emoji_cp("\U0001F468‍\U0001F4BB").split("-")[0] == "1f468"

    Et le runner en bas (calqué sur test_content.py) :

    if __name__ == "__main__":
    for name, fn in sorted(globals().items()):
        if name.startswith("test_"):
            fn(); print("ok", name)
    print("ALL OK")

  • Step 2 (RED run): cd tower && python3 test_routines.py → DOIT échouer (AttributeError: module ... has no attribute 'emoji_cp').

  • Step 3 (GREEN): Ajouter dans lisael_content.py (la constante près des autres, la fonction après transcode/tts_speak) :

    TWEMOJI = os.environ.get("LISAEL_TWEMOJI",
                         "https://raw.githubusercontent.com/jdecked/twemoji/main/assets/72x72")

def emoji_cp(emoji):
    """Twemoji filename codepoints, VS16 (U+FE0F) stripped or twemoji 404s."""
    return "-".join(f"{ord(c):x}" for c in emoji if c != "️")

  • Step 4 (GREEN run): cd tower && python3 test_routines.py → ok test_emoji_cp_strips_vs16 puis ALL OK.

  • Step 5: Ajouter emoji_bin (après emoji_cp). Écrit dans STAGING_ROUTINES (défini en Task 5 ; si Task 5 pas encore mergée, ce step suppose la constante présente — l'ordre du plan place Task 5 avant l'usage réel ; ici on référence la constante) :

    def emoji_bin(emoji, key):
    """Download the Twemoji for `emoji`, resize 160x160 RGBA, write `<key>.bin`
    (RGB565A8, keeps alpha) into STAGING_ROUTINES. Returns '<key>.bin' or ''."""
    try:
        url = f"{TWEMOJI}/{emoji_cp(emoji)}.png"
        raw = urllib.request.urlopen(
            urllib.request.Request(url, headers={"User-Agent": "curl/8"}),
            timeout=15, context=ssl.create_default_context()).read()
        im = Image.open(io.BytesIO(raw)).convert("RGBA").resize((160, 160), Image.LANCZOS)
        os.makedirs(STAGING_ROUTINES, exist_ok=True)
        with tempfile.TemporaryDirectory() as td:
            png = os.path.join(td, f"{key}.png"); im.save(png)
            subprocess.run([sys.executable, LVGLIMG, "--ofmt", "BIN", "--cf", "RGB565A8",
                            "-o", td, png], check=True, capture_output=True)
            binp = os.path.join(td, f"{key}.bin")
            if os.path.exists(binp):
                os.replace(binp, os.path.join(STAGING_ROUTINES, f"{key}.bin"))
                return f"{key}.bin"
    except Exception as e:
        print("emoji_bin fail", key, emoji, e, flush=True)
    return ""

  • Step 6: Vérif manuelle (réseau + LVGLImage). Sur Tower (vérifier d'abord que LVGLImage.py existe : ls -l $LISAEL_LVGLIMG ou ls -l /home/clems/lisael-content/LVGLImage.py) : cd /home/clems/lisael-content && python3 -c "import lisael_content as C; print(C.emoji_bin('☀️','matin'))" → doit imprimer matin.bin et le fichier doit exister dans le staging routines, taille > 1 Ko. Commit : feat(tower): twemoji to RGB565A8 bin helper.

Task 3: Modèle routines — routines_load() / routines_save() + seed défaut (TDD)

Files:

  • tower/lisael_content.py (ajout)
  • tower/test_routines.py (ajout de tests)

Interfaces:

  • Produces : def routines_load() -> list — lit ROUTINES_JSON (vérité éditoriale Tower), seed défaut Matin/Soir si absent.
  • Produces : def routines_save(routines: list) -> None — _atomic_write_json vers ROUTINES_JSON.
  • Consumes : env LISAEL_BASE (chemin ROUTINES_JSON = BASE/routines.json).

Format de la vérité éditoriale Tower (routines.json à la racine BASE, distinct du manifest SD STAGING_ROUTINES/routines.json) — porte l'emoji source par étape, qui n'apparaît PAS dans le manifest SD :

{"routines":[
  {"id":"matin","title":"Le matin","emoji":"☀️","steps":[
     {"text":"Habille-toi","emoji":"\U0001F455"},
     {"text":"Petit-déjeuner","emoji":"\U0001F950"}]},
  {"id":"soir","title":"Le soir","emoji":"\U0001F319","steps":[
     {"text":"Brosse tes dents","emoji":"\U0001FAA5"}]}]}
  • Step 1 (RED): Ajouter dans tower/test_routines.py : 
    def test_routines_seed_and_roundtrip():
    r = C.routines_load()                       # seeds default if missing
    ids = [x["id"] for x in r]
    assert ids == ["matin", "soir"]
    assert os.path.exists(C.ROUTINES_JSON)
    assert all("emoji" in x and "title" in x and "steps" in x for x in r)
    r[0]["steps"].append({"text": "Cartable", "emoji": "\U0001F392"})
    C.routines_save(r)
    r2 = C.routines_load()
    assert r2[0]["steps"][-1]["text"] == "Cartable"
  • Step 2 (RED run): cd tower && python3 test_routines.py → échoue (ROUTINES_JSON / routines_load absents).
  • Step 3 (GREEN): Ajouter dans lisael_content.py (constante près de FEEDS_JSON, fonctions dans une section # ---- routines (truth) ----) : 
    ROUTINES_JSON = os.path.join(BASE, "routines.json")

DEFAULT_ROUTINES = [
    {"id": "matin", "title": "Le matin", "emoji": "☀️", "steps": [
        {"text": "Habille-toi",          "emoji": "\U0001F455"},
        {"text": "Petit-déjeuner",  "emoji": "\U0001F950"},
        {"text": "Brosse tes dents",     "emoji": "\U0001FAA5"},
        {"text": "Prépare ton cartable", "emoji": "\U0001F392"}]},
    {"id": "soir", "title": "Le soir", "emoji": "\U0001F319", "steps": [
        {"text": "Mets ton pyjama",      "emoji": "\U0001F476"},
        {"text": "Brosse tes dents",     "emoji": "\U0001FAA5"},
        {"text": "Lis une histoire",     "emoji": "\U0001F4D6"}]},
]

def routines_load():
    if os.path.exists(ROUTINES_JSON):
        try:
            return json.load(open(ROUTINES_JSON, encoding="utf-8")).get("routines", [])
        except Exception:
            pass
    routines_save([dict(r) for r in DEFAULT_ROUTINES])
    return [dict(r) for r in DEFAULT_ROUTINES]

def routines_save(routines):
    _atomic_write_json(ROUTINES_JSON, {"routines": routines})
  • Step 4 (GREEN run): cd tower && python3 test_routines.py → ok test_emoji_cp_strips_vs16, ok test_routines_seed_and_roundtrip, ALL OK. Commit : feat(tower): routines truth model load and save.

Task 4: build_routine_manifest() — assets en cache + manifest SD (TDD avec assets factices)

Files:

  • tower/lisael_content.py (ajout)
  • tower/test_routines.py (ajout de tests)

Interfaces:

  • Produces : def asset_key(rid: str, idx: int) -> str — base ASCII d'une étape : f"{rid}_{idx:02d}" (idx 1-based). Pure, testable.
  • Produces : def build_routine_manifest(gen_bin=None, gen_audio=None) -> dict — pour chaque routine/étape régénère picto .bin (si emoji changé) + audio .mp3 (si texte changé), écrit le manifest SD STAGING_ROUTINES/routines.json au format firmware, renvoie {"generated": n_assets, "skipped": n_cached, "routines": <manifest>}. gen_bin/gen_audio injectables (TDD : factices ; défaut = emoji_bin / TTS). Cache via index STAGING_ROUTINES/.cache.json (clé asset → {"emoji":..., "text":...}).
  • Consumes : routines_load(), emoji_bin, tts_speak, STAGING_ROUTINES, _atomic_write_json.

Le manifest SD produit (format que le firmware parse — chemins .bin/.mp3 relatifs, PAS d'emoji) :

{"routines":[{"id":"matin","title":"Le matin","icon":"matin.bin","steps":[
   {"text":"Habille-toi","icon":"matin_01.bin","audio":"matin_01.mp3"}, ...]}, ...]}
  • Step 1 (RED): Ajouter dans tower/test_routines.py : 
    def test_asset_key():
    assert C.asset_key("matin", 1) == "matin_01"
    assert C.asset_key("soir", 12) == "soir_12"

def test_build_routine_manifest_caches():
    # deterministic content, fake generators that count calls and "write" files
    C.routines_save([{"id": "matin", "title": "Le matin", "emoji": "☀️",
                      "steps": [{"text": "Habille-toi", "emoji": "\U0001F455"}]}])
    os.makedirs(C.STAGING_ROUTINES, exist_ok=True)
    calls = {"bin": 0, "audio": 0}
    def fake_bin(emoji, key):
        calls["bin"] += 1
        open(os.path.join(C.STAGING_ROUTINES, key + ".bin"), "wb").close()
        return key + ".bin"
    def fake_audio(text, key):
        calls["audio"] += 1
        open(os.path.join(C.STAGING_ROUTINES, key + ".mp3"), "wb").close()
        return key + ".mp3"
    r1 = C.build_routine_manifest(gen_bin=fake_bin, gen_audio=fake_audio)
    # 1 step => 1 step bin + 1 audio + 1 routine icon bin = 2 bin, 1 audio
    assert calls == {"bin": 2, "audio": 1}
    man = r1["routines"]
    assert man[0]["icon"] == "matin.bin"
    assert man[0]["steps"][0] == {"text": "Habille-toi", "icon": "matin_01.bin", "audio": "matin_01.mp3"}
    assert os.path.exists(os.path.join(C.STAGING_ROUTINES, "routines.json"))
    # second build, unchanged content => everything cached, no regeneration
    calls2 = {"bin": 0, "audio": 0}
    def fb(e, k): calls2["bin"] += 1; return k + ".bin"
    def fa(t, k): calls2["audio"] += 1; return k + ".mp3"
    C.build_routine_manifest(gen_bin=fb, gen_audio=fa)
    assert calls2 == {"bin": 0, "audio": 0}
  • Step 2 (RED run): cd tower && python3 test_routines.py → échoue (asset_key / build_routine_manifest absents).
  • Step 3 (GREEN): Ajouter dans lisael_content.py : 
    ROUTINES_CACHE = lambda: os.path.join(STAGING_ROUTINES, ".cache.json")

def asset_key(rid, idx):
    return f"{rid}_{idx:02d}"

def _routine_cache_load():
    p = ROUTINES_CACHE()
    if os.path.exists(p):
        try:
            return json.load(open(p, encoding="utf-8"))
        except Exception:
            pass
    return {}

def build_routine_manifest(gen_bin=None, gen_audio=None):
    """Regenerate per-step .bin (emoji) + .mp3 (TTS) into STAGING_ROUTINES,
    skipping assets whose (emoji|text) is unchanged, then write the SD manifest.
    gen_bin(emoji, key)->bin_name ; gen_audio(text, key)->mp3_name (injectable)."""
    gen_bin = gen_bin or emoji_bin
    gen_audio = gen_audio or _gen_step_audio
    os.makedirs(STAGING_ROUTINES, exist_ok=True)
    cache = _routine_cache_load()
    new_cache = {}
    generated = skipped = 0
    manifest = []
    for r in routines_load():
        rid = r["id"]
        # routine icon (emoji -> .bin), key = rid
        icon_key = rid
        c = cache.get(icon_key)
        if c and c.get("emoji") == r.get("emoji") and os.path.exists(
                os.path.join(STAGING_ROUTINES, icon_key + ".bin")):
            icon = icon_key + ".bin"; skipped += 1
        else:
            icon = gen_bin(r.get("emoji", ""), icon_key) or (icon_key + ".bin"); generated += 1
        new_cache[icon_key] = {"emoji": r.get("emoji")}
        steps = []
        for i, st in enumerate(r.get("steps", []), start=1):
            k = asset_key(rid, i)
            prev = cache.get(k) or {}
            bin_name = k + ".bin"; mp3_name = k + ".mp3"
            # picto
            if prev.get("emoji") == st.get("emoji") and os.path.exists(
                    os.path.join(STAGING_ROUTINES, bin_name)):
                skipped += 1
            else:
                gen_bin(st.get("emoji", ""), k); generated += 1
            # audio
            if prev.get("text") == st.get("text") and os.path.exists(
                    os.path.join(STAGING_ROUTINES, mp3_name)):
                skipped += 1
            else:
                gen_audio(st.get("text", ""), k); generated += 1
            new_cache[k] = {"emoji": st.get("emoji"), "text": st.get("text")}
            steps.append({"text": st.get("text", ""), "icon": bin_name, "audio": mp3_name})
        manifest.append({"id": rid, "title": r.get("title", rid),
                         "icon": icon, "steps": steps})
    _atomic_write_json(os.path.join(STAGING_ROUTINES, "routines.json"),
                       {"routines": manifest})
    _atomic_write_json(ROUTINES_CACHE(), new_cache)
    return {"generated": generated, "skipped": skipped, "routines": manifest}

def _gen_step_audio(text, key):
    """Default audio generator: TTS -> mono mp3 in STAGING_ROUTINES. '' on failure."""
    try:
        au = tts_speak(text)
        _atomic_write_bytes(os.path.join(STAGING_ROUTINES, key + ".mp3"), au)
        return key + ".mp3"
    except Exception as e:
        print("step audio fail", key, e, flush=True)
        return ""
    Note : la sortie gateway est déjà mono mp3 ; pas de re-transcode ffmpeg nécessaire ici (le contrat « mono 48k » est satisfait par la voix fr de la gateway). Si un jour la gateway renvoie du stéréo, brancher transcode dans _gen_step_audio.
  • Step 4 (GREEN run): cd tower && python3 test_routines.py → tous les ok ... + ALL OK. Commit : feat(tower): build routine manifest with asset cache.

Task 5: Staging routines + push_file(subdir) généralisé + delta routines (TDD)

Files:

  • tower/lisael_content.py (modif + ajout)
  • tower/test_routines.py (ajout de tests)

Interfaces:

  • Produces : constante STAGING_ROUTINES (env LISAEL_STAGING_ROUTINES, défaut BASE/routines).

  • Produces : def routines_staging_files() -> list — basenames triés de STAGING_ROUTINES (hors cachés, hors .cache.json).

  • Modifie : def push_file(ip, name, subdir="podcasts") -> int — POST vers ?name=<subdir>/<name>, lit depuis le bon staging (podcasts → STAGING, routines → STAGING_ROUTINES). Défaut "podcasts" = compat totale.

  • Produces : def compute_routines_delta(have) -> list — compute_delta(routines_staging_files(), have) (le manifest SD routines.json part en dernier).

  • Consumes : compute_delta (réutilisé tel quel : il traite manifest.json comme nom de manifest — voir Step 2 pour l'adaptation au nom routines.json).

  • Step 1 (RED): Ajouter dans tower/test_routines.py :

    def test_routines_delta_manifest_last():
    os.makedirs(C.STAGING_ROUTINES, exist_ok=True)
    for f in ("matin.bin", "matin_01.bin", "matin_01.mp3", "routines.json"):
        open(os.path.join(C.STAGING_ROUTINES, f), "wb").close()
    d = C.compute_routines_delta([])
    assert d[-1] == "routines.json"            # manifest always last
    assert set(d[:-1]) == {"matin.bin", "matin_01.bin", "matin_01.mp3"}
    # box already has assets -> only manifest moves if assets changed; here all present
    assert C.compute_routines_delta(["matin.bin", "matin_01.bin", "matin_01.mp3"]) == []

  • Step 2 (RED run): cd tower && python3 test_routines.py → échoue (STAGING_ROUTINES/compute_routines_delta absents).

  • Step 3 (GREEN — constante + staging): Ajouter la constante près de STAGING :

    STAGING_ROUTINES = os.environ.get("LISAEL_STAGING_ROUTINES", os.path.join(BASE, "routines"))

    Et la fonction (section routines) :

    def routines_staging_files():
    if not os.path.isdir(STAGING_ROUTINES):
        return []
    return sorted(f for f in os.listdir(STAGING_ROUTINES)
                  if not f.startswith("."))

  • Step 4 (GREEN — delta): Le compute_delta existant traite littéralement "manifest.json" comme manifest. Pour réutiliser la logique « manifest en dernier » avec un nom paramétrable, ajouter :

    def compute_delta_named(staging, have, manifest_name):
    haveset = set(have)
    files = [f for f in staging if f != manifest_name and f not in haveset]
    if files and manifest_name in staging:
        files.append(manifest_name)
    return files

def compute_routines_delta(have):
    return compute_delta_named(routines_staging_files(), have, "routines.json")

    (Ne PAS toucher compute_delta — il reste la version podcasts manifest.json pour la compat et test_delta.py.)

  • Step 5 (GREEN — push_file subdir): Modifier push_file :

    def push_file(ip, name, subdir="podcasts"):
    src_dir = STAGING_ROUTINES if subdir == "routines" else STAGING
    data = open(os.path.join(src_dir, name), "rb").read()
    url = f"http://{ip}:{BOX_PORT}/put?name={subdir}/{name}"
    with urllib.request.urlopen(urllib.request.Request(url, data=data, method="POST"),
                                timeout=300) as r:
        return r.status

  • Step 6 (GREEN run): cd tower && python3 test_routines.py (delta) ET cd tower && python3 test_delta.py (régression podcasts → doit toujours afficher OK). Commit : feat(tower): routines staging push and delta.

Task 6: Journal des transferts — transfers_begin/mark/end + persistance (TDD)

Files:

  • tower/lisael_content.py (ajout)
  • tower/test_transfers.py (nouveau fichier de test)

Interfaces:

  • Produces : transfers_begin(box_ip, files) — files = [(name, subdir, size)] ou [(name, subdir)] (taille via os.path.getsize si absente) ; initialise le lot actif (tous pending).
  • Produces : transfers_mark(name, status, error=None) — status ∈ {"sending","done","failed"} ; au passage done|failed calcule duration_s (monotone, depuis le passage sending) et rate_bps = size/duration_s.
  • Produces : transfers_end() — vide le lot actif, flush les enregistrements done|failed dans TRANSFERS_JSON (borné aux ~200 derniers, _atomic_write_json).
  • Produces : transfers_active() -> dict|None, transfers_history(limit=200) -> list.
  • Consumes : env LISAEL_TRANSFERS (défaut BASE/transfers.json), _atomic_write_json, time.monotonic.

Enregistrement par fichier :

{"name":"matin_01.mp3","dir":"routines","size":428111,"status":"done",
 "started":"2026-06-21T09:12:03","ended":"2026-06-21T09:12:07",
 "duration_s":4.1,"rate_bps":104417,"error":null,"box_ip":"192.168.0.250"}
  • Step 1 (RED): Créer tower/test_transfers.py : 
    import os, tempfile, time, json
os.environ["LISAEL_BASE"] = tempfile.mkdtemp(prefix="transfers_test_")
os.environ["LISAEL_STAGING"] = os.path.join(os.environ["LISAEL_BASE"], "podcasts")
os.environ["LISAEL_STAGING_ROUTINES"] = os.path.join(os.environ["LISAEL_BASE"], "routines")
os.environ["LISAEL_TRANSFERS"] = os.path.join(os.environ["LISAEL_BASE"], "transfers.json")
import lisael_content as C

def test_begin_mark_end_cycle():
    C.transfers_begin("192.168.0.250", [("a.mp3", "routines", 1000),
                                        ("routines.json", "routines", 50)])
    a = C.transfers_active()
    assert a["box_ip"] == "192.168.0.250" and a["total"] == 2 and a["current"] == 0
    assert all(f["status"] == "pending" for f in a["files"])
    C.transfers_mark("a.mp3", "sending")
    assert C.transfers_active()["current"] == 1
    time.sleep(0.05)
    C.transfers_mark("a.mp3", "done")
    f = next(x for x in C.transfers_active()["files"] if x["name"] == "a.mp3")
    assert f["status"] == "done" and f["duration_s"] > 0 and f["rate_bps"] > 0
    C.transfers_mark("routines.json", "sending")
    C.transfers_mark("routines.json", "failed", "boom")
    C.transfers_end()
    assert C.transfers_active() is None
    hist = C.transfers_history()
    names = {h["name"]: h for h in hist}
    assert names["a.mp3"]["status"] == "done"
    assert names["routines.json"]["status"] == "failed" and names["routines.json"]["error"] == "boom"

def test_history_truncates_200():
    for i in range(250):
        C.transfers_begin("1.2.3.4", [(f"f{i}.mp3", "routines", 10)])
        C.transfers_mark(f"f{i}.mp3", "sending"); C.transfers_mark(f"f{i}.mp3", "done")
        C.transfers_end()
    assert len(C.transfers_history()) == 200
    # the most recent survive
    assert any(h["name"] == "f249.mp3" for h in C.transfers_history())

if __name__ == "__main__":
    for name, fn in sorted(globals().items()):
        if name.startswith("test_"):
            fn(); print("ok", name)
    print("ALL OK")
  • Step 2 (RED run): cd tower && python3 test_transfers.py → échoue (transfers_* absents).
  • Step 3 (GREEN): Ajouter import time en tête de lisael_content.py (sur la ligne d'imports stdlib existante) puis une section # ---- transfers journal ---- : 
    TRANSFERS_JSON = os.environ.get("LISAEL_TRANSFERS", os.path.join(BASE, "transfers.json"))
TRANSFERS_MAX  = 200

_active = None                 # active batch dict or None
_active_lock = threading.Lock()

def _now_iso():
    return time.strftime("%Y-%m-%dT%H:%M:%S")

def transfers_begin(box_ip, files):
    """files: list of (name, subdir[, size]); size resolved via getsize if missing."""
    global _active
    recs = []
    for it in files:
        name, subdir = it[0], it[1]
        size = it[2] if len(it) > 2 else None
        if size is None:
            src = STAGING_ROUTINES if subdir == "routines" else STAGING
            try:
                size = os.path.getsize(os.path.join(src, name))
            except OSError:
                size = 0
        recs.append({"name": name, "dir": subdir, "size": size, "status": "pending",
                     "started": None, "ended": None, "duration_s": None,
                     "rate_bps": None, "error": None, "box_ip": box_ip,
                     "_t0": None})
    with _active_lock:
        _active = {"box_ip": box_ip, "started": _now_iso(),
                   "total": len(recs), "current": 0, "files": recs}

def transfers_mark(name, status, error=None):
    with _active_lock:
        if not _active:
            return
        for f in _active["files"]:
            if f["name"] != name:
                continue
            if status == "sending":
                f["status"] = "sending"; f["started"] = _now_iso(); f["_t0"] = time.monotonic()
                _active["current"] += 1
            elif status in ("done", "failed"):
                f["status"] = status; f["ended"] = _now_iso(); f["error"] = error
                if f.get("_t0") is not None:
                    dur = max(time.monotonic() - f["_t0"], 0.001)
                    f["duration_s"] = round(dur, 3)
                    f["rate_bps"] = int((f["size"] or 0) / dur)
            return

def transfers_active():
    with _active_lock:
        if not _active:
            return None
        # strip internal _t0 for the API view
        files = [{k: v for k, v in f.items() if k != "_t0"} for f in _active["files"]]
        return {**_active, "files": files}

def _transfers_history_load():
    if os.path.exists(TRANSFERS_JSON):
        try:
            return json.load(open(TRANSFERS_JSON, encoding="utf-8"))
        except Exception:
            pass
    return []

def transfers_history(limit=TRANSFERS_MAX):
    return _transfers_history_load()[-limit:]

def transfers_end():
    global _active
    with _active_lock:
        if not _active:
            return
        done = [{k: v for k, v in f.items() if k != "_t0"}
                for f in _active["files"] if f["status"] in ("done", "failed")]
        _active = None
    hist = _transfers_history_load()
    hist.extend(done)
    hist = hist[-TRANSFERS_MAX:]
    _atomic_write_json(TRANSFERS_JSON, hist)
  • Step 4 (GREEN run): cd tower && python3 test_transfers.py → ok test_begin_mark_end_cycle, ok test_history_truncates_200, ALL OK. Commit : feat(tower): transfers journal begin mark end.

Task 7: Instrumenter push_batch + try_push(subdir) + retry_failed (TDD pour la sélection)

Files:

  • tower/lisael_content.py (modif + ajout)
  • tower/test_transfers.py (ajout de tests)

Interfaces:

  • Modifie : def push_batch(ip, delta, subdir="podcasts") — transfers_begin au début (chaque fichier (f, subdir)), transfers_mark(f,"sending") avant push_file, transfers_mark(f,"done"/"failed",err) après, transfers_end à la fin ; passe subdir à push_file.

  • Modifie : def try_push(ip, delta, subdir="podcasts") — passe subdir au thread push_batch.

  • Produces : def failed_names(box_ip) -> list — derniers fichiers failed connus pour box_ip (lot actif sinon historique récent), dédupliqués, en gardant le dir. Pure-ish (lit l'état), testable.

  • Produces : def retry_failed(box_ip) -> dict — collecte failed_names, relance try_push par sous-dossier ; renvoie {"started": bool, "count": n}. 409-able côté API si count == 0.

  • Step 1 (RED): Ajouter dans tower/test_transfers.py (test de sélection des échecs sans réseau) :

    def test_failed_names_from_history():
    # seed a finished batch with one failed file for a box
    C.transfers_begin("9.9.9.9", [("ok.mp3", "routines", 10), ("bad.mp3", "routines", 10)])
    C.transfers_mark("ok.mp3", "sending"); C.transfers_mark("ok.mp3", "done")
    C.transfers_mark("bad.mp3", "sending"); C.transfers_mark("bad.mp3", "failed", "net")
    C.transfers_end()
    fn = C.failed_names("9.9.9.9")
    assert ("bad.mp3", "routines") in fn
    assert ("ok.mp3", "routines") not in fn
    # unknown box -> nothing
    assert C.failed_names("0.0.0.0") == []

  • Step 2 (RED run): cd tower && python3 test_transfers.py → échoue (failed_names absent).

  • Step 3 (GREEN — failed_names + retry_failed): Ajouter (section transfers) :

    def failed_names(box_ip):
    """Failed (name, dir) for box_ip: active batch first, else recent history.
    Skips names already re-sent successfully later in history."""
    seen, failed = {}, []
    a = transfers_active()
    pool = (a["files"] if a and a.get("box_ip") == box_ip else []) \
        + [h for h in transfers_history() if h.get("box_ip") == box_ip]
    # later success overrides earlier failure: iterate chronological, keep last status
    status_by = {}
    for h in pool:
        status_by[h["name"]] = (h["status"], h.get("dir", "podcasts"))
    out = []
    for name, (st, d) in status_by.items():
        if st == "failed":
            out.append((name, d))
    return out

def retry_failed(box_ip):
    fails = failed_names(box_ip)
    if not fails:
        return {"started": False, "count": 0}
    by_dir = {}
    for name, d in fails:
        by_dir.setdefault(d, []).append(name)
    started = False
    for d, names in by_dir.items():
        if try_push(box_ip, names, subdir=d):
            started = True
    return {"started": started, "count": len(fails)}

    Note : transfers_active()["files"] est ordonné, et l'historique est chronologique (append), donc status_by garde bien le dernier statut connu par nom.

  • Step 4 (GREEN — instrument push): Modifier push_batch + try_push :

    def push_batch(ip, delta, subdir="podcasts"):
    transfers_begin(ip, [(f, subdir) for f in delta])
    ok = 0
    for f in delta:
        transfers_mark(f, "sending")
        try:
            push_file(ip, f, subdir=subdir); ok += 1
            transfers_mark(f, "done")
        except Exception as e:
            transfers_mark(f, "failed", str(e)[:200])
            print("push fail", f, e, flush=True)
    transfers_end()
    print(f"push done {ip} {subdir} {ok}/{len(delta)}", flush=True)
    with _busy_lock:
        _busy.discard(ip)

def try_push(ip, delta, subdir="podcasts"):
    """Start a background push unless one already runs for ip. Returns started?"""
    if not delta:
        return False
    with _busy_lock:
        if ip in _busy:
            return False
        _busy.add(ip)
    threading.Thread(target=push_batch, args=(ip, delta, subdir), daemon=True).start()
    return True

  • Step 5 (GREEN run): cd tower && python3 test_transfers.py → tous ok + ALL OK. Régression : cd tower && python3 test_delta.py → OK. Commit : feat(tower): instrument push and retry failed.

Task 8: API /api/routines (GET/POST) + /api/routines/push dans lisael_server.py

Files:

  • tower/lisael_server.py (modif do_GET / do_POST)

Interfaces:

  • Produces : GET /api/routines → routines_load() (vérité éditoriale, avec emojis).

  • Produces : POST /api/routines body {"routines":[...]} → routines_save + build_routine_manifest en thread daemon → 202 {"queued": n_steps}.

  • Produces : POST /api/routines/push → compute_routines_delta(have) + try_push(ip, delta, subdir="routines") → {"ip","delta","started"} ou 409 si pas d'IP.

  • Consumes : box_state_load(), routines_load/save, build_routine_manifest, compute_routines_delta, try_push.

  • Step 1: Dans do_GET, après la branche if path == "/api/box": (avant self.send_error(404)), ajouter :

            if path == "/api/routines":
            self._json({"routines": C.routines_load()}); return

  • Step 2: Dans do_POST, après la branche /api/push (avant self.send_error(404)), ajouter la sauvegarde + régénération en thread (pattern /api/select) :

            if path == "/api/routines":
            try:
                b = json.loads(self._body() or b"{}")
                routines = b["routines"]
                assert isinstance(routines, list)
            except Exception as e:
                self._json({"error": str(e)}, 400); return
            C.routines_save(routines)
            n_steps = sum(len(r.get("steps", [])) for r in routines)
            def work():
                try:
                    res = C.build_routine_manifest()
                    print("routines built", res["generated"], "gen",
                          res["skipped"], "cached", flush=True)
                except Exception as e:
                    print("routines build fail", e, flush=True)
            threading.Thread(target=work, daemon=True).start()
            self._json({"queued": n_steps}, 202); return
        if path == "/api/routines/push":
            st = C.box_state_load()
            ip = st.get("ip")
            if not ip:
                self._json({"error": "box never registered"}, 409); return
            delta = C.compute_routines_delta(st.get("have", []))
            started = C.try_push(ip, delta, subdir="routines")
            self._json({"ip": ip, "delta": len(delta), "started": started}); return

  • Step 3: Mettre à jour le handler /register pour que le delta routines soit aussi poussé au boot. Dans do_POST branche /register, après started = C.try_push(ip, delta) ajouter le push routines (la box annonce déjà l'union des basenames dans have) :

                r_delta = C.compute_routines_delta(have)
            if r_delta:
                C.try_push(ip, r_delta, subdir="routines")
            self._json({"queued": len(delta) + len(r_delta), "started": started}); return

    (Remplace la ligne self._json({"queued": len(delta), "started": started}); return existante.)

  • Step 4: Vérif manuelle locale (pas de box requise). Lancer le serveur en local avec auth désactivée et staging temporaire :

    cd tower && LISAEL_BASE=/tmp/merlin_v2 LISAEL_STAGING=/tmp/merlin_v2/podcasts \
  LISAEL_STAGING_ROUTINES=/tmp/merlin_v2/routines LISAEL_PORT=9556 \
  python3 lisael_server.py &
sleep 1
curl -s localhost:9556/api/routines | head -c 400          # -> seed matin/soir
curl -s -X POST localhost:9556/api/routines -H 'Content-Type: application/json' \
  -d '{"routines":[{"id":"matin","title":"Le matin","emoji":"☀️","steps":[{"text":"Test","emoji":"👕"}]}]}' -i | head -1   # -> 202
curl -s -X POST localhost:9556/api/routines/push -i | head -1   # -> 409 (no box) JSON
kill %1

    (Le build manifest tentera un vrai TTS/Twemoji ; sur grosmac sans accès gateway il échouera proprement — vérifier que l'API répond bien 202 quoi qu'il en soit, le build est en thread.) Commit : feat(tower): routines api save build push.

Task 9: API /api/transfers (GET) + /api/transfers/retry (POST)

Files:

  • tower/lisael_server.py (modif do_GET / do_POST)

Interfaces:

  • Produces : GET /api/transfers → {"active": transfers_active(), "history": transfers_history()}.

  • Produces : POST /api/transfers/retry → retry_failed(box_ip) ; 202 {"started","count"} si count>0, sinon 409 {"error": "..."} (pas d'IP ou rien à relancer).

  • Consumes : box_state_load(), transfers_active/history, retry_failed.

  • Step 1: Dans do_GET, après la branche /api/routines (Task 8), ajouter :

            if path == "/api/transfers":
            self._json({"active": C.transfers_active(),
                        "history": C.transfers_history()}); return

  • Step 2: Dans do_POST, après la branche /api/routines/push (Task 8), ajouter :

            if path == "/api/transfers/retry":
            st = C.box_state_load()
            ip = st.get("ip")
            if not ip:
                self._json({"error": "box never registered"}, 409); return
            res = C.retry_failed(ip)
            if not res["count"]:
                self._json({"error": "rien a relancer"}, 409); return
            self._json(res, 202); return

  • Step 3: Vérif manuelle locale (reprend le serveur de la Task 8) :

    cd tower && LISAEL_BASE=/tmp/merlin_v2 LISAEL_STAGING=/tmp/merlin_v2/podcasts \
  LISAEL_STAGING_ROUTINES=/tmp/merlin_v2/routines LISAEL_PORT=9556 \
  python3 lisael_server.py &
sleep 1
curl -s localhost:9556/api/transfers   # -> {"active":null,"history":[]}
curl -s -X POST localhost:9556/api/transfers/retry -i | head -1   # -> 409 JSON
kill %1

    Commit : feat(tower): transfers api and retry endpoint.

Task 10: UI onglet « 🗓️ Routines » dans merlin_ui.html

Files:

  • tower/merlin_ui.html (modif : tabs, panneau, JS)

Interfaces:

  • Consumes : GET /api/routines, POST /api/routines, POST /api/routines/push (via le helper api() existant).

  • Produces : édition en mémoire de la liste routines + rendu (2 sections Matin/Soir, étapes texte+emoji, +/−, ↑↓, Enregistrer & pousser).

  • Step 1: Ajouter le bouton d'onglet dans .tabs (après le bouton tab-box) :

        <button id="tab-routines" onclick="showTab('routines')">🗓️ Routines</button>

  • Step 2: Ajouter le panneau (après <section id="panel-box" ...>...</section>, avant la fermeture </div> de .wrap) :

      <!-- ===== ROUTINES ===== -->
  <section id="panel-routines" class="panel">
    <div id="routines"><div class="loading"><span class="spinner"></span> chargement…</div></div>
    <div class="row" style="margin-top:14px">
      <button id="routines-save" class="btn orange" onclick="saveRoutines()">💾 Enregistrer &amp; pousser</button>
    </div>
  </section>

  • Step 3: Étendre TABS et showTab :

    const TABS = ["podcasts", "fichiers", "box", "routines", "transfers"];

    et dans showTab, ajouter :

      if (name === "routines") loadRoutines();

    (la branche transfers sera câblée en Task 11 ; on déclare déjà l'entrée TABS ici pour éviter un second diff sur la même ligne.)

  • Step 4: Ajouter une palette d'emojis enfant + l'état en mémoire (dans le <script>, après les helpers) :

    // ---------- Routines ----------
const EMOJI_PALETTE = ["☀️","🌙","👕","🥣","🦷","🎒","🧸","🛁","👖","🧦","🪥","📖","🚽","🍽️","💤","🧼","👟","🧥"];
let ROUTINES = [];   // editable in-memory copy of /api/routines

function routineCard(r, ri) {
  const card = document.createElement("div");
  card.className = "card";
  card.innerHTML = '<h3>' + esc(r.emoji || "") + ' ' + esc(r.title || r.id) + '</h3>'
    + '<div class="steps-host"></div>'
    + '<div class="row" style="margin-top:10px"><button class="btn ghost add-step">+ Ajouter une étape</button></div>';
  const host = card.querySelector(".steps-host");
  (r.steps || []).forEach(function (st, si) { host.appendChild(stepRow(ri, si)); });
  card.querySelector(".add-step").onclick = function () {
    ROUTINES[ri].steps.push({ text: "", emoji: "🧸" });
    loadRoutines(true);
  };
  return card;
}

function stepRow(ri, si) {
  const st = ROUTINES[ri].steps[si];
  const li = document.createElement("div");
  li.className = "row";
  li.style.borderTop = "1px solid var(--line)";
  li.style.padding = "8px 0";
  // emoji select
  let opts = "";
  const inPalette = EMOJI_PALETTE.indexOf(st.emoji) !== -1;
  EMOJI_PALETTE.forEach(function (e) {
    opts += '<option value="' + esc(e) + '"' + (e === st.emoji ? " selected" : "") + '>' + esc(e) + '</option>';
  });
  if (!inPalette && st.emoji) opts += '<option value="' + esc(st.emoji) + '" selected>' + esc(st.emoji) + '</option>';
  li.innerHTML =
    '<select class="st-emoji" style="flex:none;font-size:1.4rem;padding:8px;border-radius:12px;border:2px solid var(--line);background:#fbf8ff">' + opts + '</select>'
    + '<input type="text" class="st-text grow" value="' + esc(st.text) + '" placeholder="Texte de l\'étape">'
    + '<button class="icon st-up" title="Monter">⬆️</button>'
    + '<button class="icon st-down" title="Descendre">⬇️</button>'
    + '<button class="icon st-del" title="Supprimer">🗑</button>';
  li.querySelector(".st-text").oninput = function (e) { ROUTINES[ri].steps[si].text = e.target.value; };
  li.querySelector(".st-emoji").onchange = function (e) { ROUTINES[ri].steps[si].emoji = e.target.value; };
  li.querySelector(".st-up").onclick = function () { moveStep(ri, si, -1); };
  li.querySelector(".st-down").onclick = function () { moveStep(ri, si, 1); };
  li.querySelector(".st-del").onclick = function () { ROUTINES[ri].steps.splice(si, 1); loadRoutines(true); };
  return li;
}

function moveStep(ri, si, d) {
  const s = ROUTINES[ri].steps;
  const j = si + d;
  if (j < 0 || j >= s.length) return;
  const t = s[si]; s[si] = s[j]; s[j] = t;
  loadRoutines(true);
}

async function loadRoutines(fromMemory) {
  const box = $("routines");
  if (!fromMemory) {
    box.innerHTML = '<div class="loading"><span class="spinner"></span> chargement…</div>';
    try {
      clearBanner();
      const data = await api("/api/routines");
      ROUTINES = (data && data.routines) || [];
    } catch (e) {
      box.innerHTML = '<div class="empty">Impossible de charger les routines.</div>';
      showBanner("Routines: " + e.message);
      return;
    }
  }
  box.innerHTML = "";
  ROUTINES.forEach(function (r, ri) { box.appendChild(routineCard(r, ri)); });
}

async function saveRoutines() {
  const btn = $("routines-save");
  btn.disabled = true;
  const old = btn.textContent;
  btn.textContent = "Enregistrement…";
  try {
    await api("/api/routines", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ routines: ROUTINES }),
    });
    toast("Routines enregistrées — génération en cours…");
    // give the server a moment to regenerate assets, then push
    setTimeout(async function () {
      try {
        const r = await api("/api/routines/push", { method: "POST" });
        if (r && r.started) toast("Envoi des routines démarré (" + (r.delta || 0) + " fichier(s))");
        else toast("Routines prêtes (rien à pousser ou box absente)");
      } catch (e) {
        if (e.status === 409) toast("La box ne s'est jamais annoncée", true);
        else { toast("Erreur push: " + e.message, true); showBanner("Push routines: " + e.message); }
      }
    }, 6000);
  } catch (e) {
    toast("Erreur: " + e.message, true);
    showBanner("Enregistrer routines: " + e.message);
  } finally {
    btn.disabled = false;
    btn.textContent = old;
  }
}

  • Step 5: Vérif manuelle. Reprendre le serveur local (Task 8) avec un merlin_ui.html servi, ouvrir http://localhost:9556/ (auth OFF en local), cliquer l'onglet « 🗓️ Routines » → voir 2 cartes Matin/Soir avec étapes, tester +ajouter / supprimer / ↑↓ / changer texte+emoji, puis « Enregistrer & pousser » → toast « enregistrées », et /api/routines reflète les changements (curl après). Vérifier dans la console navigateur l'absence d'erreur JS. Commit : feat(ui): routines tab editor.

Task 11: UI onglet « 📤 Transferts » avec auto-refresh

Files:

  • tower/merlin_ui.html (modif : tabs, panneau, JS)

Interfaces:

  • Consumes : GET /api/transfers, POST /api/transfers/retry.

  • Produces : carte « en cours » (X/N + fichier courant), table effectués (fichier/dossier/taille/durée/débit/statut/heure), bouton « Relancer les échecs », auto-refresh ~2 s tant que active != null.

  • Step 1: Ajouter le bouton d'onglet dans .tabs (après tab-routines) :

        <button id="tab-transfers" onclick="showTab('transfers')">📤 Transferts</button>

  • Step 2: Ajouter le panneau (après le panneau Routines) :

      <!-- ===== TRANSFERTS ===== -->
  <section id="panel-transfers" class="panel">
    <div class="card">
      <h3>En cours</h3>
      <div id="tr-active"><div class="empty">Aucun transfert en cours.</div></div>
    </div>
    <div class="card">
      <div class="row between">
        <h3 style="margin:0">Effectués</h3>
        <button id="tr-retry" class="btn ghost" onclick="retryTransfers()" disabled>🔁 Relancer les échecs</button>
      </div>
      <div id="tr-history" style="margin-top:10px"><div class="empty">Aucun transfert.</div></div>
    </div>
  </section>

  • Step 3: Câbler showTab (la branche, l'entrée TABS ayant été ajoutée en Task 10) et gérer l'arrêt de l'auto-refresh en quittant l'onglet. Dans showTab, ajouter :

      if (name === "transfers") loadTransfers(); else stopTransfersTimer();

  • Step 4: Ajouter le JS Transferts (après le bloc Routines) :

    // ---------- Transferts ----------
let trTimer = null;
function stopTransfersTimer() { if (trTimer) { clearInterval(trTimer); trTimer = null; } }

function fmtRate(bps) {
  bps = Number(bps) || 0;
  if (bps >= 1048576) return (bps / 1048576).toFixed(1) + " Mo/s";
  if (bps >= 1024) return (bps / 1024).toFixed(0) + " Ko/s";
  return bps + " o/s";
}
function fmtTime(iso) { return iso ? esc(String(iso).replace("T", " ").slice(0, 19)) : "—"; }

function renderActive(active) {
  const host = $("tr-active");
  if (!active) { host.innerHTML = '<div class="empty">Aucun transfert en cours.</div>'; return; }
  const cur = (active.files || []).find(function (f) { return f.status === "sending"; });
  const pct = active.total ? Math.round(100 * active.current / active.total) : 0;
  host.innerHTML =
    '<div class="stat"><span class="k">Vers</span><span class="v">' + esc(active.box_ip || "?") + '</span></div>' +
    '<div class="stat"><span class="k">Avancement</span><span class="v">' + active.current + '/' + active.total + ' (' + pct + '%)</span></div>' +
    '<div class="stat"><span class="k">Fichier en cours</span><span class="v fname">' + (cur ? esc(cur.name) : "—") + '</span></div>';
}

function renderHistory(history) {
  const host = $("tr-history");
  const rows = (history || []).slice().reverse();   // most recent first
  if (!rows.length) { host.innerHTML = '<div class="empty">Aucun transfert.</div>'; $("tr-retry").disabled = true; return; }
  let hasFailed = false;
  let html = '<table><thead><tr><th>Fichier</th><th>Dossier</th><th>Taille</th><th>Durée</th><th>Débit</th><th>Statut</th><th>Heure</th></tr></thead><tbody>';
  rows.forEach(function (h) {
    const failed = h.status === "failed";
    if (failed) hasFailed = true;
    const stCell = failed
      ? '<span style="color:var(--err)">✗ ' + esc(h.error || "échec") + '</span>'
      : '<span style="color:var(--ok)">✓</span>';
    html +=
      '<tr' + (failed ? ' style="background:#fff0f2"' : '') + '>' +
        '<td class="fname">' + esc(h.name) + '</td>' +
        '<td>' + esc(h.dir || "") + '</td>' +
        '<td class="num">' + humanSize(h.size) + '</td>' +
        '<td class="num">' + (h.duration_s != null ? h.duration_s + ' s' : '—') + '</td>' +
        '<td class="num">' + (h.rate_bps != null ? fmtRate(h.rate_bps) : '—') + '</td>' +
        '<td>' + stCell + '</td>' +
        '<td class="num">' + fmtTime(h.ended || h.started) + '</td>' +
      '</tr>';
  });
  html += '</tbody></table>';
  host.innerHTML = html;
  $("tr-retry").disabled = !hasFailed;
}

async function loadTransfers() {
  try {
    clearBanner();
    const data = await api("/api/transfers");
    const active = (data && data.active) || null;
    renderActive(active);
    renderHistory((data && data.history) || []);
    if (active) {
      if (!trTimer) trTimer = setInterval(loadTransfers, 2000);
    } else {
      stopTransfersTimer();   // one last refresh already froze the history
    }
  } catch (e) {
    showBanner("Transferts: " + e.message);
    stopTransfersTimer();
  }
}

async function retryTransfers() {
  const btn = $("tr-retry");
  btn.disabled = true;
  try {
    const r = await api("/api/transfers/retry", { method: "POST" });
    toast("Relance de " + (r.count || 0) + " fichier(s)…");
    loadTransfers();   // restart the auto-refresh as a batch becomes active
  } catch (e) {
    if (e.status === 409) toast("Rien à relancer ou box absente", true);
    else { toast("Erreur: " + e.message, true); showBanner("Retry: " + e.message); }
    btn.disabled = false;
  }
}

  • Step 5: Vérif manuelle. Sur le serveur local, ouvrir l'onglet « 📤 Transferts » → « Aucun transfert en cours » + « Aucun transfert » (historique vide), bouton « Relancer » désactivé. Pour tester le rendu avec données, écrire un transfers.json factice dans /tmp/merlin_v2/transfers.json (un done + un failed) puis recharger l'onglet → table avec ligne rouge + bouton « Relancer » actif. Vérifier qu'aucun setInterval ne reste actif quand active est null (Network tab : pas de requête /api/transfers répétée). Commit : feat(ui): transfers tab with autorefresh.

Task 12: Déploiement Tower + vérification e2e

Files:

  • (aucun fichier modifié — déploiement)

Interfaces:

  • Consumes : tous les livrables Tasks 1-11.

  • Step 1: Vérifier que LVGLImage.py est présent sur Tower au chemin attendu par LISAEL_LVGLIMG (sinon emoji_bin échouera). ssh clems@192.168.0.120 'ls -l /home/clems/lisael-content/LVGLImage.py || echo MISSING'. S'il manque, le copier depuis le repo box (managed_components/lvgl__lvgl/scripts/LVGLImage.py, présent après idf.py build) vers /home/clems/lisael-content/ et noter le chemin réel.

  • Step 2: Lancer la suite de tests hôte en local avant déploiement (preuve verte) :

    cd tower && for t in test_delta.py test_content.py test_routines.py test_transfers.py; do
  echo "== $t =="; python3 "$t" || { echo "FAIL $t"; break; }
done

    Tous doivent finir par OK / ALL OK.

  • Step 3: Déployer sur Tower (déploiement manuel, pattern repo) :

    rsync -av tower/lisael_content.py tower/lisael_server.py tower/merlin_ui.html \
  tower/test_routines.py tower/test_transfers.py clems@192.168.0.120:/home/clems/lisael-content/
ssh clems@192.168.0.120 'sudo systemctl restart lisael-content && sleep 1 && systemctl is-active lisael-content'

    Doit afficher active.

  • Step 4: Vérif API publique via merlin.saillant.cc (Basic Auth + UA Mozilla anti-Cloudflare). Récupérer les creds (MERLIN_AUTH côté service) puis :

    curl -s -u "$MERLIN_USER:$MERLIN_PASS" -A "Mozilla/5.0" https://merlin.saillant.cc/api/routines | head -c 400
curl -s -u "$MERLIN_USER:$MERLIN_PASS" -A "Mozilla/5.0" https://merlin.saillant.cc/api/transfers

    /api/routines doit renvoyer le seed Matin/Soir ; /api/transfers {"active":null,"history":[...]}.

  • Step 5: Test e2e routines : depuis l'UI (https://merlin.saillant.cc/, onglet Routines) modifier une étape Matin (texte+emoji), « Enregistrer & pousser ». Vérifier sur Tower que les assets ont été générés et que le manifest SD existe :

    ssh clems@192.168.0.120 'ls -l /home/clems/lisael-content/routines/ && cat /home/clems/lisael-content/routines/routines.json'

    Doit lister matin.bin, matin_NN.bin, matin_NN.mp3, soir*.{bin,mp3} + routines.json au format firmware (champs icon/audio).

  • Step 6: Test e2e transferts : si la box est joignable et s'annonce (/register), le delta routines part ; ouvrir l'onglet « 📤 Transferts » et observer le lot actif (X/N) puis son basculement en historique avec durée/débit. Sinon, déclencher « Enregistrer & pousser » avec box absente → vérifier que l'historique reste cohérent et que 409 est géré côté UI (toast). Vérifier la persistance : ssh clems@192.168.0.120 'cat /home/clems/lisael-content/transfers.json' survit à sudo systemctl restart lisael-content. Commit : chore(tower): deploy merlin studio v2.

Self-review

Couverture spec Routines (2026-06-21-lisael-box-routines-design.md, sections 5/7/8) :

  • §5 manifest SD /sdcard/routines/routines.json format firmware (id/title/icon/steps[text,icon,audio]), noms ASCII matin_NN, pictos RGB565A8, audio mp3 → Task 4 (build_routine_manifest) + Task 2 (RGB565A8) + Task 1 (TTS).
  • §7 Tower : vérité routines.json (Task 3), Twemoji→.bin (Task 2), TTS voix fr gateway tailnet (Task 1), manifest SD (Task 4), push routines/ dans have/delta (Task 5 + Task 8 Step 3), cache par emoji+texte (Task 4).
  • §8 Merlin onglet Routines : GET/POST/push API (Task 8), UI 2 sections + texte+emoji + +/− + ↑↓ + Enregistrer&pousser + palette emoji (Task 10).

Couverture spec Transferts (2026-06-21-merlin-transfers-viz-design.md) :

  • §5 modèle (record par fichier + lot actif) → Task 6. §6 begin/mark/end + instrumentation push + retry_failed → Task 6/7. §7 API GET/retry (202/409) → Task 9. §8 UI en cours + table + relancer + auto-refresh 2 s → Task 11. §10 tests hôte (statuts, duration/rate, troncature 200, sélection failed) → Task 6/7.

Cohérence noms/types : routines.json Tower (avec emoji) ≠ manifest SD STAGING_ROUTINES/routines.json (sans emoji, avec icon/audio) — explicité Task 3/4. compute_delta (podcasts, manifest.json) intouché ; routines via compute_delta_named/compute_routines_delta (manifest routines.json en dernier). push_file(subdir="podcasts") rétrocompatible. try_push/push_batch gagnent subdir avec défaut compat. Pas de placeholder, code complet dans chaque step.

Risque identifié (non bloquant) : Task 8 Step 2 lance build_routine_manifest en thread puis l'UI (Task 10) attend 6 s avant le push — si la régénération TTS est plus longue (plusieurs étapes × ~25 s TTS), le premier push peut partir sur un manifest incomplet ; le prochain /register de la box ou un second « Enregistrer & pousser » corrige via le delta. Acceptable pour un usage parent occasionnel.

Reference in New Issue View Git Blame Copy Permalink