diff --git a/.taskmaster/tasks/tasks.json b/.taskmaster/tasks/tasks.json index 3468ede..09e10ba 100644 --- a/.taskmaster/tasks/tasks.json +++ b/.taskmaster/tasks/tasks.json @@ -2,7 +2,7 @@ "master": { "tasks": [ { - "id": 1, + "id": "1", "title": "feat: search-mode split-view with PDF + page jump", "description": "Render /kb/search results as a two-column split view: left column is the stack of ranked hit cards (kind, title, section, page label), right column is an iframe viewing the selected hit's PDF scrolled to page_number. Clicking any hit swaps the iframe. Text-only sources (law from Wikisource) fall back to a chunk/text panel with a Wikisource link so the right column never 404s.", "status": "done", @@ -14,7 +14,7 @@ "createdAt": "2026-04-24T15:47:00Z" }, { - "id": 2, + "id": "2", "title": "fix(kb/chunker): page_number propagation through split + packing (shira-hermes)", "description": "Applied in shira-hermes commits e534709 + 1ca1cc0: (1) _split_oversized now re-derives page_number per piece from markers embedded in parent content + a virtual (offset 0, parent.page_number) anchor, (2) chunk_circular tracks absolute paragraph offsets in the original text so each packed sub-chunk reports its real starting page, (3) _as_dicts strips stray \\x00 bytes left behind when _split_oversized slices through a marker sentinel. Reference data: ספר הליקויים now 211 chunks × 210 pages (1-413) vs 209 × 1 before. Not work for this repo, but the search split-view in task #1 depends on it.", "status": "done", @@ -26,7 +26,7 @@ "createdAt": "2026-04-24T15:47:00Z" }, { - "id": 3, + "id": "3", "title": "feat(kb/search): LLM query expansion so ranked-by-title docs don't crowd everything else out (shira-hermes)", "description": "User reports searching 'מהי תקנה 37' returns only the תקנה 37 circular and the law — never ספר הליקויים, even though that's where the underlying medical content lives. Root cause is a single-shot retrieval where any document whose title contains the query terms wins by a huge margin. Fix: ask the LLM (via ai-gateway) for up to 3 alternative phrasings, retrieve candidates for the original + each variant in parallel, merge by chunk_id keeping best RRF score, then rerank the union against the ORIGINAL query. After fix: src 29 (ספר הליקויים) shows up with 2 hits in the same query.", "status": "done", @@ -38,7 +38,7 @@ "createdAt": "2026-04-25T10:00:00Z" }, { - "id": 8, + "id": "8", "title": "feat(kb/ask): SSE streaming with live agent progress", "description": "User wanted Shira's progress to be visible while she thinks — like Claude does — instead of just an incrementing seconds counter. Implementation: AgentRunner.run accepts an optional sync on_event callback that emits {type, label, ...} dicts at each agent step. New POST /kb/ask/stream returns text/event-stream, with the runner running in an asyncio task whose events feed an asyncio.Queue that the response generator drains as SSE. Client opens EventSource against a new EspoCRM EntryPoint KnowledgeBaseAskStream that proxies the SSE bytes through cURL with output buffering disabled (echo + flush + exit, bypassing the framework Response). The progress UI now stacks one line per event (thinking, tool_start with the query Shira chose, tool_done with the chunk count, writing) instead of just a seconds counter. v0.1.9 _activeAsk persistence pattern preserved — events accumulate at module scope so a view switch + return replays the live log.", "status": "done", @@ -50,7 +50,7 @@ "createdAt": "2026-04-25T13:00:00Z" }, { - "id": 7, + "id": "7", "title": "fix: search preview duplicates left card for text sources", "description": "User report: searching the law (Wikisource source 5) shows the same chunk content in both the left list-card and the right preview pane. The preview adds zero value when the matched source has no PDF — both panes are visually identical (modulo the left's 14em truncation), which reads as a bug. Fixed by fetching the matched chunk plus K=2 surrounding sections from /kb/source/{id}/chunks?around=N&ctx=2 and rendering them stacked: matched section is highlighted with a yellow header (#fff3cd), neighbors render as muted context blocks above and below. Adds a prominent 'פתח ב-Wikisource' button in the panel header, scrolls the matched section into view on render. Race-safe via _previewToken so a quick second click doesn't render stale results.", "status": "done", @@ -62,7 +62,7 @@ "createdAt": "2026-04-25T12:30:00Z" }, { - "id": 5, + "id": "5", "title": "feat(kb): drag-to-resize splitter (both ask + search)", "description": "Replace the fixed Bootstrap col-md-5/7 split with a flex layout containing a 6px drag handle. mousedown on the handle starts the drag; while dragging an invisible full-screen overlay covers any iframe so the PDF.js viewer doesn't swallow mouse events. The chosen ratio (15-85%) persists to localStorage as kb-split-pct so the layout sticks across sessions. Both modes share the same _buildSplitShell helper so the look stays consistent.", "status": "done", @@ -74,7 +74,7 @@ "createdAt": "2026-04-25T11:00:00Z" }, { - "id": 6, + "id": "6", "title": "fix: search survives view switch (parity with ask in v0.1.9)", "description": "Search mode had the same view-bound promise problem as ask did before v0.1.9: navigating to another EspoCRM screen and back blanked the results. Hoisted {query, kind, promise} to a module-level _activeSearch with the same {query, kind, hits, selectedIdx} sessionStorage replay we already built for ask. Selected hit index also persists so the right pane comes back to the same source/page the user had open.", "status": "done", @@ -86,7 +86,7 @@ "createdAt": "2026-04-25T11:00:00Z" }, { - "id": 4, + "id": "4", "title": "fix: ask survives view switch via module-level promise + sessionStorage", "description": "User reports asking Shira a question and switching to a different EspoCRM view loses the in-flight request — coming back shows a blank screen as if nothing was asked. Cause: the Espo.Ajax promise was view-bound, so handlers fired against a detached DOM after re-mount. Fix: hoist {question, promise, startedAt} to a module-level _activeAsk; afterRender re-attaches handlers and resumes the progress UI with the original startedAt; completed answers persist to sessionStorage and replay on remount or a hard reload.", "status": "done", @@ -96,10 +96,9 @@ "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T10:00:00Z" - } - , + }, { - "id": 9, + "id": "9", "title": "feat(kb/ask): rejoin in-flight SSE stream after browser reload", "description": "When the user F5s or closes+reopens the tab while Shira is thinking, the asyncio task in shira-hermes keeps running (tokens burn) but the EventSource is gone — there's no way to reattach. After reload, the v0.1.9 sessionStorage replay only shows the LAST completed answer, not the live in-flight stream. Build a rejoinable SSE channel: POST /kb/ask/stream allocates a request_id and starts the runner; the SSE response writes events into an in-memory store keyed by request_id AND streams them. New GET /kb/ask/stream/{request_id} reconnects: replays accumulated events first, then continues with new ones as they arrive. Garbage-collect after 5 minutes past the answer event.", "status": "pending", @@ -111,7 +110,7 @@ "createdAt": "2026-04-25T13:30:00Z" }, { - "id": 10, + "id": "10", "title": "ops: investigate n8n release-asset auto-attach failures", "description": "The n8n workflow `Git: Gitea Push - Auto-Release + Mattermost` (id IcF30v1Pw3wbucbu, per ~/CLAUDE.md) is supposed to attach the built .zip as a release asset within ~12s of a tag push. Across every KnowledgeBase release in this period (v0.1.8 through v0.2.1) the workflow did NOT attach the zip — every release was published with zero assets and I had to upload manually via curl + the Gitea API. Investigate the workflow's recent execution history, find the failing step, and either fix the n8n workflow or document a better fallback.", "status": "pending", @@ -123,7 +122,7 @@ "createdAt": "2026-04-25T13:30:00Z" }, { - "id": 11, + "id": "11", "title": "feat(kb/ask): show text-source citations in the ask mode source picker", "description": "/kb/ask currently filters sources_used through _filter_pdf_sources (kb_public.py) before returning, dropping any source whose original_path doesn't end with .pdf. The Wikisource law (source 5) is therefore invisible to the user even when Shira explicitly cites a section from it. With the v0.1.11 'section in context' preview that already exists for search mode, ask mode should be able to show the same. Drop the filter, and when the user clicks a non-PDF source pill, route to the same _renderSectionContext flow that search uses (factor it into a shared helper).", "status": "pending", @@ -135,19 +134,20 @@ "createdAt": "2026-04-25T13:30:00Z" }, { - "id": 12, + "id": "12", "title": "Phase 1 — Multi-topic KB foundation (DB schema, topic-aware endpoints, topic selector)", "description": "Generalize the KB from being insurance-only to supporting multiple legal domains within one CRM (e.g. ביטוח לאומי, דיני עבודה, דין פלילי). A 'topic' is the firm-level grouping; each kb_source belongs to exactly one topic. Search/ask are scoped to a single topic at a time, chosen from a topic dropdown the user controls. The system_prompt for /kb/ask becomes per-topic so Shira's domain context is correct (today the prompt hardcodes 'Israeli National Insurance'). Backwards-compatible migration: all 6 existing sources move to topic_id=1 named 'ביטוח לאומי' with the existing system-prompt addendum, and the UI defaults to that topic so nothing visible changes for existing users until they choose another topic.", - "status": "pending", + "status": "in-progress", "priority": "high", "details": "DB migration (shira-hermes side, against insurance_kb DB):\n- New table kb_topic: id PK, slug TEXT UNIQUE, name TEXT NOT NULL, description TEXT, system_prompt_addendum TEXT, is_active BOOLEAN DEFAULT true, created_at TIMESTAMPTZ DEFAULT now(), updated_at TIMESTAMPTZ DEFAULT now().\n- Seed: INSERT (id=1, slug='national-insurance', name='ביטוח לאומי', system_prompt_addendum=, is_active=true). RENAME the DB itself? Probably leave as 'insurance_kb' — costly to rename, no real value. Document in memory.\n- ALTER TABLE kb_source ADD COLUMN topic_id INTEGER REFERENCES kb_topic(id). Backfill: UPDATE kb_source SET topic_id = 1. Then ALTER COLUMN SET NOT NULL.\n\nshira-hermes endpoints:\n- New GET /kb/topics — public, returns active topics with id, slug, name, description (no prompt).\n- /kb/search: accepts optional topic_id (single int). When present, the SQL adds AND s.topic_id = $N. When absent (back-compat), still returns all (might want to deprecate this and require topic_id later).\n- /kb/sources, /kb/source/{id}/chunks, /kb/source/{id}/pdf: same — optional topic filter on listings, no change to single-source endpoints.\n- /kb/ask + /kb/ask/stream: accept topic_id param; load topic.system_prompt_addendum and inject into system_prompt instead of the hardcoded insurance text in _build_ask_runner_context. Pin the legal_kb tool to topic_id too (so search_insurance_kb queries are constrained — RENAME the tool to search_legal_kb and pass topic_id at registration time).\n- _expand_query in kb_search.py: parametrize the prompt by topic name (currently hardcoded 'הביטוח הלאומי בישראל'). Pass topic.name in.\n\nKnowledgeBase extension (EspoCRM):\n- New route+action GET /KnowledgeBase/action/topics → proxies /kb/topics.\n- Service.search/ask gain topic_id, controller passes through.\n- Template gets a topic