diff --git a/docs/superpowers/plans/2026-09-13-cross-chat-memory.md b/docs/superpowers/plans/2026-09-13-cross-chat-memory.md new file mode 100644 index 0000000..32d474f --- /dev/null +++ b/docs/superpowers/plans/2026-09-13-cross-chat-memory.md @@ -0,0 +1,921 @@ +# Chatuebergreifendes Gedaechtnis Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** JARVIS soll sich Dinge chatuebergreifend merken - explizite Fakten ("merke dir...") und automatische Zusammenfassungen frueherer Conversations, die per Aehnlichkeitssuche in neue Chats einfliessen. + +**Architecture:** Zwei neue Postgres-Tabellen (`memory_facts`, `conversation_summaries`, letztere mit `pgvector`-Embedding wie die bestehende Knowledge-Base). Zwei neue Claude-Tools (`remember_fact`, `forget_fact`) analog zu den bestehenden Kalender/E-Mail-Tools. Nach jedem Chat-Turn laeuft im Hintergrund ein Claude-Call, der die Conversation zusammenfasst und embedded. Vor jedem Chat-Turn wird der System-Prompt um alle gespeicherten Fakten (vollstaendig) und die Top-3 aehnlichsten frueheren Zusammenfassungen (per Vector-Suche) ergaenzt. + +**Tech Stack:** Python 3.11, FastAPI, psycopg2 (Postgres), Anthropic SDK (`claude_client.messages.create`), aiohttp (Ollama-Embeddings), pytest + pytest-asyncio + unittest.mock. Alles in der bestehenden einzigen Backend-Datei `Claude outputs/main.py` (etabliertes Pattern dieses Projekts - kein Split in mehrere Module). + +**Spec:** `docs/superpowers/specs/2026-09-13-cross-chat-memory-design.md` + +## Global Constraints + +- Kein `ivfflat`/`hnsw`-Index auf `conversation_summaries.embedding` in dieser Phase (Sequential Scan reicht bei der erwarteten Datenmenge - gleiche Begruendung wie bei `document_chunks`, Phase 3a). +- `remember_fact` wird **ohne** Rueckfrage aufgerufen, `forget_fact` erst **nach** expliziter Bestaetigung im Chat (gleiche Konvention wie `create_calendar_event` vs. `delete_calendar_event`). +- Der Hintergrund-Task fuer die Zusammenfassung darf die Chat-Antwort an den Nutzer **niemals** verzoegern oder durch eine Exception beeintraechtigen - jeder Fehler wird geloggt, nie weitergeworfen. +- Es gibt weiterhin nur einen Nutzer (`DEFAULT_USER_ID`) - kein Multi-User-Scoping fuer `memory_facts` noetig. +- Alles landet in `Claude outputs/main.py`; Tests in `Claude outputs/tests/`; neue Migration in `Claude outputs/migrations/006_memory.sql` (dort liegen bereits 002-005, nicht im JARVIS-Root). +- Tests laufen lokal nicht per `pytest` direkt (kein `psycopg2-binary`-Wheel fuer Python 3.14 lokal) - Ausfuehrung ueber den Docker-Container-Weg aus `JARVIS_HANDOFF.md` ("Backend-Tests lokal ausfuehren"). + +--- + +## Task 1: Migration `006_memory.sql` + +**Files:** +- Create: `Claude outputs/migrations/006_memory.sql` + +**Interfaces:** +- Produces: Tabellen `memory_facts (id, user_id, content, created_at)` und `conversation_summaries (conversation_id PK, summary, embedding vector(768), updated_at)`, auf die alle folgenden Tasks per SQL zugreifen. + +- [ ] **Step 1: Migration schreiben** + +```sql +CREATE TABLE memory_facts ( + id SERIAL PRIMARY KEY, + user_id INTEGER NOT NULL REFERENCES users(id), + content TEXT NOT NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +); + +CREATE TABLE conversation_summaries ( + conversation_id INTEGER PRIMARY KEY REFERENCES conversations(id), + summary TEXT NOT NULL, + embedding vector(768) NOT NULL, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +); +``` + +- [ ] **Step 2: Kein automatisierter Test moeglich** + +Reines SQL-DDL ohne lokale Postgres-Instanz (siehe Global Constraints) - +kann nicht per pytest verifiziert werden. Die tatsaechliche Anwendung und +Verifikation (`\d memory_facts`, `\d conversation_summaries`) passiert in +Task 8 gegen die echte VPS-Datenbank. + +- [ ] **Step 3: Commit** + +```bash +git add "Claude outputs/migrations/006_memory.sql" +git commit -m "feat: add memory_facts and conversation_summaries tables" +``` + +--- + +## Task 2: `memory_facts` DB-Helper + +**Files:** +- Modify: `Claude outputs/main.py` (neue Funktionen direkt unter `get_latest_emails`, vor `_caldav_calendar`) +- Test: `Claude outputs/tests/test_memory.py` (neu) + +**Interfaces:** +- Consumes: `db_query(query, params, fetch)` (bestehend, `main.py:143`) +- Produces: `insert_memory_fact(user_id: int, content: str) -> int`, `search_memory_facts(query: str) -> list[dict]` (Zeilen mit `id`, `content`), `delete_memory_fact(fact_id: int) -> None`, `get_all_memory_facts() -> list[dict]` (Zeilen mit `id`, `content`) - werden von Task 4 (Tools) und Task 5 (Prompt-Context) konsumiert. + +- [ ] **Step 1: Failing tests schreiben** + +```python +# Claude outputs/tests/test_memory.py +import os +import sys +from unittest.mock import AsyncMock, patch + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) + +import pytest + +import main + + +@pytest.mark.asyncio +async def test_insert_memory_fact_returns_new_id(): + with patch.object(main, "db_query", new=AsyncMock(return_value={"id": 42})) as mock_query: + fact_id = await main.insert_memory_fact(1, "Mag lieber Tee als Kaffee") + + assert fact_id == 42 + args, kwargs = mock_query.call_args + assert "INSERT INTO memory_facts" in args[0] + assert args[1] == (1, "Mag lieber Tee als Kaffee") + + +@pytest.mark.asyncio +async def test_search_memory_facts_uses_ilike(): + rows = [{"id": 1, "content": "Hund heisst Bruno"}] + with patch.object(main, "db_query", new=AsyncMock(return_value=rows)) as mock_query: + result = await main.search_memory_facts("Bruno") + + assert result == rows + args, kwargs = mock_query.call_args + assert "ILIKE" in args[0] + assert args[1] == ("%Bruno%",) + + +@pytest.mark.asyncio +async def test_delete_memory_fact_deletes_by_id(): + with patch.object(main, "db_query", new=AsyncMock(return_value=None)) as mock_query: + await main.delete_memory_fact(7) + + args, kwargs = mock_query.call_args + assert "DELETE FROM memory_facts" in args[0] + assert args[1] == (7,) + + +@pytest.mark.asyncio +async def test_get_all_memory_facts_returns_rows(): + rows = [{"id": 1, "content": "A"}, {"id": 2, "content": "B"}] + with patch.object(main, "db_query", new=AsyncMock(return_value=rows)): + result = await main.get_all_memory_facts() + + assert result == rows +``` + +- [ ] **Step 2: Tests laufen lassen, Fehlschlag pruefen** + +Run: `python -m pytest tests/test_memory.py -v` (im Docker-Testcontainer, +siehe `JARVIS_HANDOFF.md`) +Expected: FAIL mit `AttributeError: module 'main' has no attribute +'insert_memory_fact'` (und analog fuer die anderen drei Funktionen) + +- [ ] **Step 3: Implementierung** + +In `main.py`, direkt nach `get_latest_emails`: + +```python +async def insert_memory_fact(user_id: int, content: str) -> int: + row = await db_query( + "INSERT INTO memory_facts (user_id, content) VALUES (%s, %s) RETURNING id", + (user_id, content), + fetch="one", + ) + return row["id"] + + +async def search_memory_facts(query: str) -> list: + return await db_query( + "SELECT id, content FROM memory_facts WHERE content ILIKE %s ORDER BY created_at ASC", + (f"%{query}%",), + fetch="all", + ) + + +async def delete_memory_fact(fact_id: int): + await db_query("DELETE FROM memory_facts WHERE id = %s", (fact_id,)) + + +async def get_all_memory_facts() -> list: + return await db_query( + "SELECT id, content FROM memory_facts ORDER BY created_at ASC", + fetch="all", + ) +``` + +- [ ] **Step 4: Tests laufen lassen, Erfolg pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 4 PASS + +- [ ] **Step 5: Commit** + +```bash +git add "Claude outputs/main.py" "Claude outputs/tests/test_memory.py" +git commit -m "feat: add memory_facts DB helpers" +``` + +--- + +## Task 3: `conversation_summaries` DB-Helper + +**Files:** +- Modify: `Claude outputs/main.py` (neue Funktionen direkt nach den in Task 2 hinzugefuegten) +- Test: `Claude outputs/tests/test_memory.py` (erweitern) + +**Interfaces:** +- Consumes: `db_query` (bestehend), `_vector_literal(embedding: list) -> str` (bestehend, `main.py:220`) +- Produces: `upsert_conversation_summary(conversation_id: int, summary: str, embedding: list) -> None`, `search_similar_conversation_summaries(query_embedding: list, exclude_conversation_id: int, limit: int = 3) -> list[dict]` (Zeilen mit `conversation_id`, `summary`, `distance`) - werden von Task 5 (Retrieval) und Task 7 (Hintergrund-Summary) konsumiert. + +- [ ] **Step 1: Failing tests schreiben** + +An `Claude outputs/tests/test_memory.py` anhaengen: + +```python +@pytest.mark.asyncio +async def test_upsert_conversation_summary_inserts_with_vector_literal(): + with patch.object(main, "db_query", new=AsyncMock(return_value=None)) as mock_query: + await main.upsert_conversation_summary(5, "Kurze Zusammenfassung", [0.1, 0.2]) + + args, kwargs = mock_query.call_args + assert "INSERT INTO conversation_summaries" in args[0] + assert "ON CONFLICT (conversation_id) DO UPDATE" in args[0] + assert args[1] == (5, "Kurze Zusammenfassung", "[0.1,0.2]") + + +@pytest.mark.asyncio +async def test_search_similar_conversation_summaries_excludes_current_conversation(): + rows = [{"conversation_id": 3, "summary": "Ueber Kalender", "distance": 0.1}] + with patch.object(main, "db_query", new=AsyncMock(return_value=rows)) as mock_query: + result = await main.search_similar_conversation_summaries([0.1, 0.2], exclude_conversation_id=9, limit=3) + + assert result == rows + args, kwargs = mock_query.call_args + assert "conversation_id != %s" in args[0] + assert args[1] == ("[0.1,0.2]", 9, 3) +``` + +- [ ] **Step 2: Tests laufen lassen, Fehlschlag pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 2 neue FAIL mit `AttributeError` + +- [ ] **Step 3: Implementierung** + +```python +async def upsert_conversation_summary(conversation_id: int, summary: str, embedding: list): + await db_query( + """ + INSERT INTO conversation_summaries (conversation_id, summary, embedding) + VALUES (%s, %s, %s::vector) + ON CONFLICT (conversation_id) DO UPDATE + SET summary = EXCLUDED.summary, embedding = EXCLUDED.embedding, updated_at = CURRENT_TIMESTAMP + """, + (conversation_id, summary, _vector_literal(embedding)), + ) + + +async def search_similar_conversation_summaries(query_embedding: list, exclude_conversation_id: int, limit: int = 3) -> list: + return await db_query( + """ + SELECT conversation_id, summary, embedding <=> %s::vector AS distance + FROM conversation_summaries + WHERE conversation_id != %s + ORDER BY distance ASC + LIMIT %s + """, + (_vector_literal(query_embedding), exclude_conversation_id, limit), + fetch="all", + ) +``` + +Beachte: `upsert_conversation_summary` ruft `db_query` mit drei Positional- +Parametern auf (query, params) - die Test-Assertion auf `args[1]` prueft das +Tupel `(conversation_id, summary, vector_literal)`, passend zu den drei +`%s`-Platzhaltern in der Query. + +- [ ] **Step 4: Tests laufen lassen, Erfolg pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 6 PASS insgesamt + +- [ ] **Step 5: Commit** + +```bash +git add "Claude outputs/main.py" "Claude outputs/tests/test_memory.py" +git commit -m "feat: add conversation_summaries DB helpers" +``` + +--- + +## Task 4: `remember_fact` / `forget_fact` Tools + +**Files:** +- Modify: `Claude outputs/main.py` (Business-Logik nach `send_email`/`async def send_email`, Tool-Liste nach `CALENDAR_TOOLS`, Instructions-Text nach `CALENDAR_ASSISTANT_INSTRUCTIONS`, `execute_tool` erweitern) +- Test: `Claude outputs/tests/test_memory.py` (erweitern) + +**Interfaces:** +- Consumes: `insert_memory_fact`, `search_memory_facts`, `delete_memory_fact` (Task 2), `DEFAULT_USER_ID` (bestehende globale Variable) +- Produces: `remember_fact(fact: str) -> dict`, `forget_fact(query: str) -> dict`, Liste `MEMORY_TOOLS`, String `MEMORY_ASSISTANT_INSTRUCTIONS`, `execute_tool` kennt `"remember_fact"`/`"forget_fact"` - werden von Task 6 (Wiring in `run_chat_completion`) konsumiert. + +- [ ] **Step 1: Failing tests schreiben** + +An `Claude outputs/tests/test_memory.py` anhaengen: + +```python +@pytest.mark.asyncio +async def test_remember_fact_inserts_and_returns_content(): + main.DEFAULT_USER_ID = 1 + with patch.object(main, "insert_memory_fact", new=AsyncMock(return_value=99)): + result = await main.remember_fact("Hund heisst Bruno") + + assert result == {"id": 99, "content": "Hund heisst Bruno"} + + +@pytest.mark.asyncio +async def test_forget_fact_no_match_returns_message(): + with patch.object(main, "search_memory_facts", new=AsyncMock(return_value=[])): + result = await main.forget_fact("Katze") + + assert result["deleted"] is False + assert result["matches"] == [] + + +@pytest.mark.asyncio +async def test_forget_fact_single_match_deletes(): + matches = [{"id": 5, "content": "Hund heisst Bruno"}] + with patch.object(main, "search_memory_facts", new=AsyncMock(return_value=matches)), patch.object( + main, "delete_memory_fact", new=AsyncMock() + ) as mock_delete: + result = await main.forget_fact("Bruno") + + mock_delete.assert_called_once_with(5) + assert result == {"deleted": True, "content": "Hund heisst Bruno"} + + +@pytest.mark.asyncio +async def test_forget_fact_multiple_matches_does_not_delete(): + matches = [ + {"id": 5, "content": "Hund heisst Bruno"}, + {"id": 6, "content": "Bruno ist der Nachbar"}, + ] + with patch.object(main, "search_memory_facts", new=AsyncMock(return_value=matches)), patch.object( + main, "delete_memory_fact", new=AsyncMock() + ) as mock_delete: + result = await main.forget_fact("Bruno") + + mock_delete.assert_not_called() + assert result["deleted"] is False + assert len(result["matches"]) == 2 + + +@pytest.mark.asyncio +async def test_execute_tool_dispatches_remember_and_forget_fact(): + import json + + with patch.object(main, "remember_fact", new=AsyncMock(return_value={"id": 1, "content": "X"})): + result = await main.execute_tool("remember_fact", {"fact": "X"}) + assert json.loads(result) == {"id": 1, "content": "X"} + + with patch.object(main, "forget_fact", new=AsyncMock(return_value={"deleted": True, "content": "X"})): + result = await main.execute_tool("forget_fact", {"query": "X"}) + assert json.loads(result) == {"deleted": True, "content": "X"} +``` + +- [ ] **Step 2: Tests laufen lassen, Fehlschlag pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 5 neue FAIL (`AttributeError` bzw. `ValueError: Unknown tool`) + +- [ ] **Step 3: Implementierung** + +In `main.py` nach `async def send_email(...)`: + +```python +async def remember_fact(fact: str) -> dict: + fact_id = await insert_memory_fact(DEFAULT_USER_ID, fact) + return {"id": fact_id, "content": fact} + + +async def forget_fact(query: str) -> dict: + matches = await search_memory_facts(query) + if not matches: + return {"deleted": False, "matches": [], "message": "Kein passender Fakt gefunden."} + if len(matches) > 1: + return {"deleted": False, "matches": [{"id": m["id"], "content": m["content"]} for m in matches]} + await delete_memory_fact(matches[0]["id"]) + return {"deleted": True, "content": matches[0]["content"]} +``` + +Nach der `CALENDAR_TOOLS`-Liste (vor `CALENDAR_ASSISTANT_INSTRUCTIONS`): + +```python +MEMORY_TOOLS = [ + { + "name": "remember_fact", + "description": "Store a fact the user explicitly asked to remember, so it is available in future chats.", + "input_schema": { + "type": "object", + "properties": { + "fact": {"type": "string", "description": "The fact to remember, phrased as a standalone statement"}, + }, + "required": ["fact"], + }, + }, + { + "name": "forget_fact", + "description": ( + "Search remembered facts matching a query and delete it if exactly " + "one matches. Only call after the user has explicitly confirmed " + "which fact to forget in the conversation." + ), + "input_schema": { + "type": "object", + "properties": { + "query": {"type": "string", "description": "Text to search for among remembered facts"}, + }, + "required": ["query"], + }, + }, +] +``` + +Nach `CALENDAR_ASSISTANT_INSTRUCTIONS`: + +```python +MEMORY_ASSISTANT_INSTRUCTIONS = ( + "Du hast ausserdem Zugriff auf ein chatuebergreifendes Gedaechtnis ueber " + "die Tools remember_fact und forget_fact. Wenn der Nutzer dich explizit " + "bittet, dir etwas zu merken (z.B. 'merke dir, dass...'), rufe " + "remember_fact direkt auf - keine Rueckfrage noetig. Wenn der Nutzer " + "dich bittet, einen gemerkten Fakt zu vergessen, frage zuerst explizit " + "im Klartext nach Bestaetigung, welcher Fakt gemeint ist, und rufe " + "forget_fact erst auf, nachdem der Nutzer zugestimmt hat. Gibt " + "forget_fact mehrere moegliche Treffer zurueck, liste sie im Chat auf " + "und frage nach, welcher gemeint ist, statt den falschen zu loeschen." +) +``` + +In `execute_tool`, vor der abschliessenden `raise ValueError`-Zeile: + +```python + if name == "remember_fact": + result = await remember_fact(tool_input["fact"]) + return json.dumps(result) + if name == "forget_fact": + result = await forget_fact(tool_input["query"]) + return json.dumps(result) +``` + +- [ ] **Step 4: Tests laufen lassen, Erfolg pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 11 PASS insgesamt + +- [ ] **Step 5: Commit** + +```bash +git add "Claude outputs/main.py" "Claude outputs/tests/test_memory.py" +git commit -m "feat: add remember_fact and forget_fact chat tools" +``` + +--- + +## Task 5: `build_memory_context` (Prompt-Zusammensetzung) + +**Files:** +- Modify: `Claude outputs/main.py` (neue Funktion nach `search_similar_conversation_summaries`) +- Test: `Claude outputs/tests/test_memory.py` (erweitern) + +**Interfaces:** +- Consumes: `get_all_memory_facts()` (Task 2), `get_embedding(text: str) -> list` (bestehend, `main.py:128`), `search_similar_conversation_summaries` (Task 3) +- Produces: `build_memory_context(user_message: str, conversation_id: int) -> str` (leerer String, wenn nichts zu ergaenzen ist) - wird von Task 6 in `run_chat_completion` konsumiert. + +- [ ] **Step 1: Failing tests schreiben** + +An `Claude outputs/tests/test_memory.py` anhaengen: + +```python +@pytest.mark.asyncio +async def test_build_memory_context_empty_when_nothing_stored(): + with patch.object(main, "get_all_memory_facts", new=AsyncMock(return_value=[])), patch.object( + main, "get_embedding", new=AsyncMock(return_value=[0.1]) + ), patch.object(main, "search_similar_conversation_summaries", new=AsyncMock(return_value=[])): + context = await main.build_memory_context("Hallo", conversation_id=1) + + assert context == "" + + +@pytest.mark.asyncio +async def test_build_memory_context_includes_facts_section(): + facts = [{"id": 1, "content": "Hund heisst Bruno"}] + with patch.object(main, "get_all_memory_facts", new=AsyncMock(return_value=facts)), patch.object( + main, "get_embedding", new=AsyncMock(return_value=[0.1]) + ), patch.object(main, "search_similar_conversation_summaries", new=AsyncMock(return_value=[])): + context = await main.build_memory_context("Wie geht es meinem Hund?", conversation_id=1) + + assert "Bekannte Fakten ueber den Nutzer" in context + assert "Hund heisst Bruno" in context + assert "Relevante fruehere Gespraeche" not in context + + +@pytest.mark.asyncio +async def test_build_memory_context_includes_summaries_section(): + summaries = [{"conversation_id": 2, "summary": "Ging um den Kalender", "distance": 0.05}] + with patch.object(main, "get_all_memory_facts", new=AsyncMock(return_value=[])), patch.object( + main, "get_embedding", new=AsyncMock(return_value=[0.1]) + ), patch.object(main, "search_similar_conversation_summaries", new=AsyncMock(return_value=summaries)): + context = await main.build_memory_context("Was war nochmal mit dem Termin?", conversation_id=1) + + assert "Relevante fruehere Gespraeche" in context + assert "Ging um den Kalender" in context + + +@pytest.mark.asyncio +async def test_build_memory_context_skips_summaries_when_embedding_fails(): + with patch.object(main, "get_all_memory_facts", new=AsyncMock(return_value=[])), patch.object( + main, "get_embedding", new=AsyncMock(side_effect=RuntimeError("Ollama down")) + ): + context = await main.build_memory_context("Hallo", conversation_id=1) + + assert context == "" +``` + +- [ ] **Step 2: Tests laufen lassen, Fehlschlag pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 4 neue FAIL mit `AttributeError` + +- [ ] **Step 3: Implementierung** + +```python +async def build_memory_context(user_message: str, conversation_id: int) -> str: + sections = [] + + facts = await get_all_memory_facts() + if facts: + facts_lines = "\n".join(f"- {f['content']}" for f in facts) + sections.append(f"Bekannte Fakten ueber den Nutzer:\n{facts_lines}") + + try: + query_embedding = await get_embedding(user_message) + summaries = await search_similar_conversation_summaries(query_embedding, conversation_id, limit=3) + except Exception as e: + logger.warning(f"Memory retrieval skipped, embedding/search failed: {e}") + summaries = [] + + if summaries: + summary_lines = "\n".join(f"- {s['summary']}" for s in summaries) + sections.append(f"Relevante fruehere Gespraeche:\n{summary_lines}") + + return "\n\n".join(sections) +``` + +- [ ] **Step 4: Tests laufen lassen, Erfolg pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 15 PASS insgesamt + +- [ ] **Step 5: Commit** + +```bash +git add "Claude outputs/main.py" "Claude outputs/tests/test_memory.py" +git commit -m "feat: assemble memory context from facts and similar summaries" +``` + +--- + +## Task 6: Memory in `run_chat_completion` einbinden + +**Files:** +- Modify: `Claude outputs/main.py` (`run_chat_completion`-Signatur und System-Prompt-Aufbau, Aufrufstelle in `chat()`) +- Test: `Claude outputs/tests/test_chat_tools.py` (bestehende 4 Tests anpassen), `Claude outputs/tests/test_memory.py` (neuer Test) + +**Interfaces:** +- Consumes: `build_memory_context(user_message, conversation_id)` (Task 5), `MEMORY_TOOLS`, `MEMORY_ASSISTANT_INSTRUCTIONS` (Task 4) +- Produces: `run_chat_completion(claude_messages: list, conversation_id: int)` (neue Signatur, **breaking change** gegenueber vorher `run_chat_completion(claude_messages)`) - Aufrufstelle in `chat()` (main.py:765) wird in diesem Task mit angepasst. + +- [ ] **Step 1: Bestehende Tests an neue Signatur anpassen + neuen Test schreiben** + +In `Claude outputs/tests/test_chat_tools.py`: bei allen 4 bestehenden +Aufrufen von `main.run_chat_completion(...)` das zweite Argument +`conversation_id=1` ergaenzen und `build_memory_context` patchen, damit +kein echter Ollama-/DB-Zugriff noetig ist. Beispiel fuer die erste +Testfunktion (die anderen 3 analog): + +```python +@pytest.mark.asyncio +async def test_run_chat_completion_without_tool_use(): + completion = MagicMock() + completion.stop_reason = "end_turn" + completion.content = [_text_block("Hallo!")] + completion.usage = _usage(10, 5) + + main.claude_client = MagicMock() + main.claude_client.messages.create.return_value = completion + + with patch.object(main, "build_memory_context", new=AsyncMock(return_value="")): + text, output_tokens, total_tokens = await main.run_chat_completion( + [{"role": "user", "content": "Hi"}], conversation_id=1 + ) + + assert text == "Hallo!" + assert output_tokens == 5 + assert total_tokens == 15 +``` + +Wende dasselbe Muster (zusaetzliches `conversation_id=1` Argument, +`with patch.object(main, "build_memory_context", new=AsyncMock(return_value=""))` +um den bestehenden `with patch.object(...)`-Block herum bzw. als +zusaetzlichen `patch.object`-Parameter) auf die restlichen 3 Tests in +dieser Datei an: +`test_run_chat_completion_executes_tool_and_returns_followup`, +`test_run_chat_completion_handles_two_sequential_tool_calls`, +`test_run_chat_completion_lists_recent_emails`. + +Neuer Test in `Claude outputs/tests/test_memory.py` - dafuer zuerst den +Import am Dateianfang um `MagicMock` erweitern (wird ab hier und in Task 7 +gebraucht): + +```python +from unittest.mock import AsyncMock, MagicMock, patch +``` + +```python +@pytest.mark.asyncio +async def test_run_chat_completion_includes_memory_context_in_system_prompt(): + completion = MagicMock() + completion.stop_reason = "end_turn" + text_block = MagicMock() + text_block.type = "text" + text_block.text = "Bruno geht es gut." + completion.content = [text_block] + usage = MagicMock() + usage.input_tokens = 10 + usage.output_tokens = 5 + completion.usage = usage + + main.claude_client = MagicMock() + main.claude_client.messages.create.return_value = completion + + with patch.object( + main, "build_memory_context", new=AsyncMock(return_value="Bekannte Fakten ueber den Nutzer:\n- Hund heisst Bruno") + ): + await main.run_chat_completion([{"role": "user", "content": "Wie geht es meinem Hund?"}], conversation_id=1) + + _, kwargs = main.claude_client.messages.create.call_args + assert "Hund heisst Bruno" in kwargs["system"] + assert kwargs["tools"] == main.CALENDAR_TOOLS + main.MEMORY_TOOLS +``` + +- [ ] **Step 2: Tests laufen lassen, Fehlschlag pruefen** + +Run: `python -m pytest tests/test_chat_tools.py tests/test_memory.py -v` +Expected: die 4 angepassten Tests FAILEN mit `TypeError: +run_chat_completion() takes 1 positional argument but 2 were given`; der +neue Test FAILT ebenso. + +- [ ] **Step 3: Implementierung** + +`run_chat_completion` in `main.py` ersetzen durch: + +```python +async def run_chat_completion(claude_messages: list, conversation_id: int): + latest_user_message = claude_messages[-1]["content"] + memory_context = await build_memory_context(latest_user_message, conversation_id) + + system_prompt = f"{CLAUDE_SYSTEM_PROMPT}\n\n{CALENDAR_ASSISTANT_INSTRUCTIONS}\n\n{MEMORY_ASSISTANT_INSTRUCTIONS}" + if memory_context: + system_prompt = f"{system_prompt}\n\n{memory_context}" + + messages = list(claude_messages) + total_input = 0 + total_output = 0 + + for _ in range(MAX_TOOL_ROUNDS): + completion = await asyncio.to_thread( + claude_client.messages.create, + model=CLAUDE_MODEL, + max_tokens=1024, + system=system_prompt, + tools=CALENDAR_TOOLS + MEMORY_TOOLS, + messages=messages, + ) + total_input += completion.usage.input_tokens + total_output += completion.usage.output_tokens + + if completion.stop_reason != "tool_use": + response_text = "".join(b.text for b in completion.content if b.type == "text") + return response_text, total_output, total_input + total_output + + tool_results = [] + for block in completion.content: + if block.type != "tool_use": + continue + try: + result_text = await execute_tool(block.name, block.input) + tool_results.append({"type": "tool_result", "tool_use_id": block.id, "content": result_text}) + except Exception as e: + tool_results.append( + {"type": "tool_result", "tool_use_id": block.id, "content": str(e), "is_error": True} + ) + + messages = messages + [ + {"role": "assistant", "content": completion.content}, + {"role": "user", "content": tool_results}, + ] + + return ( + "Entschuldigung, das hat zu viele Zwischenschritte gebraucht. Bitte formuliere die Anfrage neu.", + total_output, + total_input + total_output, + ) +``` + +(Einzige Aenderungen gegenueber vorher: neuer Parameter +`conversation_id`, die zwei neuen Zeilen fuer `memory_context`, der um +`MEMORY_ASSISTANT_INSTRUCTIONS` erweiterte `system_prompt` und +`tools=CALENDAR_TOOLS + MEMORY_TOOLS` statt `tools=CALENDAR_TOOLS`.) + +In `chat()` die Aufrufstelle anpassen (`main.py:765`): + +```python + response_text, output_tokens, tokens_used = await run_chat_completion(claude_messages, conversation_id) +``` + +- [ ] **Step 4: Tests laufen lassen, Erfolg pruefen** + +Run: `python -m pytest tests/test_chat_tools.py tests/test_memory.py -v` +Expected: alle PASS (4 angepasste + 16 aus test_memory.py) + +- [ ] **Step 5: Commit** + +```bash +git add "Claude outputs/main.py" "Claude outputs/tests/test_chat_tools.py" "Claude outputs/tests/test_memory.py" +git commit -m "feat: inject remembered facts and similar summaries into chat system prompt" +``` + +--- + +## Task 7: Automatische Zusammenfassung im Hintergrund + +**Files:** +- Modify: `Claude outputs/main.py` (neue Funktionen nach `run_chat_completion`, Aufruf in `chat()`) +- Test: `Claude outputs/tests/test_memory.py` (erweitern) + +**Interfaces:** +- Consumes: `get_messages(conversation_id)` (bestehend), `claude_client.messages.create` (bestehend), `get_embedding` (bestehend), `upsert_conversation_summary` (Task 3) +- Produces: `update_conversation_summary(conversation_id: int) -> None` (faengt alle Exceptions intern ab), `_spawn_background_task(coro) -> asyncio.Task` - wird in `chat()` nach dem Speichern der Assistant-Antwort aufgerufen. + +- [ ] **Step 1: Failing tests schreiben** + +An `Claude outputs/tests/test_memory.py` anhaengen: + +```python +@pytest.mark.asyncio +async def test_update_conversation_summary_upserts_summary_and_embedding(): + history = [ + {"role": "user", "content": "Wie geht es meinem Hund?", "tokens_used": None, "created_at": None}, + {"role": "assistant", "content": "Bruno geht es gut.", "tokens_used": 5, "created_at": None}, + ] + completion = MagicMock() + text_block = MagicMock() + text_block.type = "text" + text_block.text = "Nutzer fragte nach seinem Hund Bruno." + completion.content = [text_block] + + main.claude_client = MagicMock() + main.claude_client.messages.create.return_value = completion + + with patch.object(main, "get_messages", new=AsyncMock(return_value=history)), patch.object( + main, "get_embedding", new=AsyncMock(return_value=[0.1, 0.2]) + ), patch.object(main, "upsert_conversation_summary", new=AsyncMock()) as mock_upsert: + await main.update_conversation_summary(conversation_id=3) + + mock_upsert.assert_called_once_with(3, "Nutzer fragte nach seinem Hund Bruno.", [0.1, 0.2]) + + +@pytest.mark.asyncio +async def test_update_conversation_summary_swallows_exceptions(): + with patch.object(main, "get_messages", new=AsyncMock(side_effect=RuntimeError("DB down"))): + await main.update_conversation_summary(conversation_id=3) + # kein Raise - das ist der Test + + +def test_spawn_background_task_tracks_and_releases_task(): + import asyncio as aio + + async def _run(): + async def noop(): + return "done" + + task = main._spawn_background_task(noop()) + assert task in main._background_tasks + result = await task + assert result == "done" + assert task not in main._background_tasks + + aio.run(_run()) +``` + +- [ ] **Step 2: Tests laufen lassen, Fehlschlag pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: 3 neue FAIL mit `AttributeError` + +- [ ] **Step 3: Implementierung** + +In `main.py` nach `run_chat_completion`: + +```python +SUMMARY_SYSTEM_PROMPT = ( + "Fasse das folgende Gespraech in 2-3 Saetzen auf Deutsch zusammen, damit " + "ein spaeterer Chat den Kontext wiedererkennt. Gib nur die Zusammenfassung " + "aus, ohne Einleitung." +) + + +async def update_conversation_summary(conversation_id: int): + try: + history = await get_messages(conversation_id) + claude_messages = [{"role": m["role"], "content": m["content"]} for m in history] + completion = await asyncio.to_thread( + claude_client.messages.create, + model=CLAUDE_MODEL, + max_tokens=200, + system=SUMMARY_SYSTEM_PROMPT, + messages=claude_messages, + ) + summary_text = "".join(b.text for b in completion.content if b.type == "text") + embedding = await get_embedding(summary_text) + await upsert_conversation_summary(conversation_id, summary_text, embedding) + except Exception as e: + logger.error(f"Conversation summary update failed for conversation {conversation_id}: {e}") + + +_background_tasks: set = set() + + +def _spawn_background_task(coro): + task = asyncio.create_task(coro) + _background_tasks.add(task) + task.add_done_callback(_background_tasks.discard) + return task +``` + +In `chat()`, direkt nach +`await save_message(conversation_id, DEFAULT_USER_ID, "assistant", response_text, output_tokens)`: + +```python + _spawn_background_task(update_conversation_summary(conversation_id)) +``` + +- [ ] **Step 4: Tests laufen lassen, Erfolg pruefen** + +Run: `python -m pytest tests/test_memory.py -v` +Expected: alle PASS (19 insgesamt) + +- [ ] **Step 5: Gesamten Testlauf pruefen** + +Run: `python -m pytest tests/ -v` (kompletter Testcontainer-Durchlauf aus +`JARVIS_HANDOFF.md`) +Expected: alle Tests aller Dateien PASS, keine Regression + +- [ ] **Step 6: Commit** + +```bash +git add "Claude outputs/main.py" "Claude outputs/tests/test_memory.py" +git commit -m "feat: summarize conversations in the background after each turn" +``` + +--- + +## Task 8: Migration anwenden & Deployment + +**Files:** +- Keine Code-Aenderungen - reine Deployment-Aktion gegen die echte VPS-Infrastruktur (Befehle aus `JARVIS_HANDOFF.md`, Abschnitte "Database Management" und "Backend-Code aktualisieren"). + +**Interfaces:** +- Consumes: alle vorherigen Tasks (fertiger, getesteter Code + Migration) +- Produces: laufendes Feature auf `https://jarvis.mbo-tech-it.de` + +- [ ] **Step 1: Migration auf die VPS kopieren und anwenden** + +```bash +scp -F /dev/null -o IdentitiesOnly=yes -i ~/.ssh/jarvis_core_key \ + "Claude outputs/migrations/006_memory.sql" jarvis-core@72.61.186.98:/home/jarvis-core/jarvis/migrations/ +ssh -F /dev/null -o IdentitiesOnly=yes -i ~/.ssh/jarvis_core_key jarvis-core@72.61.186.98 \ + "docker exec -i jarvis-postgres psql -U jarvis -d jarvis < /home/jarvis-core/jarvis/migrations/006_memory.sql" +``` + +- [ ] **Step 2: Migration verifizieren** + +```bash +ssh -F /dev/null -o IdentitiesOnly=yes -i ~/.ssh/jarvis_core_key jarvis-core@72.61.186.98 \ + "docker exec jarvis-postgres psql -U jarvis -d jarvis -c '\d memory_facts' -c '\d conversation_summaries'" +``` + +Expected: beide Tabellen mit den erwarteten Spalten werden angezeigt. + +- [ ] **Step 3: Backend deployen** + +```bash +scp -F /dev/null -o IdentitiesOnly=yes -i ~/.ssh/jarvis_core_key \ + "Claude outputs/main.py" jarvis-core@72.61.186.98:/home/jarvis-core/jarvis/api/main.py +ssh -F /dev/null -o IdentitiesOnly=yes -i ~/.ssh/jarvis_core_key jarvis-core@72.61.186.98 \ + "cd /home/jarvis-core/jarvis && docker compose restart jarvis-api" +``` + +- [ ] **Step 4: Manueller End-to-End-Test** + +Im Chat (Web-UI): +1. "Merke dir, dass mein Hund Bruno heisst" - Antwort sollte die + Speicherung bestaetigen. +2. Ueber den "Neuer Chat"-Button einen neuen Chat starten. +3. "Wie heisst mein Hund?" fragen - Antwort sollte "Bruno" korrekt + verwenden (Fakten-Pfad). +4. Ein thematisch verwandtes, aber nicht identisches Thema aus dem ersten + Chat ansprechen und pruefen, dass JARVIS darauf Bezug nehmen kann + (Zusammenfassungs-Retrieval-Pfad - ggf. kurz warten, da die + Zusammenfassung erst nach der ersten Antwort im Hintergrund entsteht). +5. In einem dritten Chat "Vergiss, dass mein Hund Bruno heisst" sagen, + die Rueckfrage bestaetigen, und in einem vierten Chat verifizieren, + dass der Fakt nicht mehr bekannt ist. + +- [ ] **Step 5: Logs pruefen** + +```bash +ssh -F /dev/null -o IdentitiesOnly=yes -i ~/.ssh/jarvis_core_key jarvis-core@72.61.186.98 \ + "docker compose -f /home/jarvis-core/jarvis/docker-compose.yml logs --tail 100 jarvis-api" +``` + +Expected: keine unerwarteten Exceptions, insbesondere keine +"Conversation summary update failed" ausser bei absichtlich simulierten +Fehlern. diff --git a/docs/superpowers/specs/2026-09-13-cross-chat-memory-design.md b/docs/superpowers/specs/2026-09-13-cross-chat-memory-design.md index e47a0da..e2f9684 100644 --- a/docs/superpowers/specs/2026-09-13-cross-chat-memory-design.md +++ b/docs/superpowers/specs/2026-09-13-cross-chat-memory-design.md @@ -22,7 +22,7 @@ Die vorhandene Knowledge-Base-Infrastruktur (Phase 3a: `pgvector`, Ollama `nomic-embed-text` Embeddings) wird fuer die Zusammenfassungen wiederverwendet statt eine zweite Vector-Search-Loesung einzufuehren. -## Datenmodell (`migrations/006_memory.sql`) +## Datenmodell (`Claude outputs/migrations/006_memory.sql`) ```sql CREATE TABLE memory_facts (