{ "master": { "tasks": [ { "id": "1", "title": "feat: search-mode split-view with PDF + page jump", "description": "Render /kb/search results as a two-column split view: left column is the stack of ranked hit cards (kind, title, section, page label), right column is an iframe viewing the selected hit's PDF scrolled to page_number. Clicking any hit swaps the iframe. Text-only sources (law from Wikisource) fall back to a chunk/text panel with a Wikisource link so the right column never 404s.", "status": "done", "priority": "high", "details": "Before this change renderSearchResults produced plain vertical text cards — users got the chunk body and had to open the Wikisource link (or track down the PDF manually) to verify context. With 5 of 5 PDFs now carrying accurate page_number (shira-hermes e534709 + 1ca1cc0), the search UI can finally deep-link to the right page. Mirrors the ask-mode split view (v0.1.7 uncommitted).", "testStrategy": "Search 'תקנה 37' → first hit should be a regulation/circular → right pane loads the PDF at page ~N where the section appears. Click a lower-ranked law hit → right pane swaps to a Wikisource link view. Search 'הגדרות' → hits span multiple sources → each click swaps iframe source.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-24T15:47:00Z" }, { "id": "2", "title": "fix(kb/chunker): page_number propagation through split + packing (shira-hermes)", "description": "Applied in shira-hermes commits e534709 + 1ca1cc0: (1) _split_oversized now re-derives page_number per piece from markers embedded in parent content + a virtual (offset 0, parent.page_number) anchor, (2) chunk_circular tracks absolute paragraph offsets in the original text so each packed sub-chunk reports its real starting page, (3) _as_dicts strips stray \\x00 bytes left behind when _split_oversized slices through a marker sentinel. Reference data: ספר הליקויים now 211 chunks × 210 pages (1-413) vs 209 × 1 before. Not work for this repo, but the search split-view in task #1 depends on it.", "status": "done", "priority": "high", "details": "Lives in espocrm-extensions/shira-hermes. Logged here for traceability because task #1 can't demonstrate correct page jumps without it.", "testStrategy": "Query DB: SELECT source_id, MIN/MAX(page_number), COUNT(DISTINCT page_number) FROM kb_chunk GROUP BY source_id — every PDF source should span multiple distinct pages.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-24T15:47:00Z" }, { "id": "3", "title": "feat(kb/search): LLM query expansion so ranked-by-title docs don't crowd everything else out (shira-hermes)", "description": "User reports searching 'מהי תקנה 37' returns only the תקנה 37 circular and the law — never ספר הליקויים, even though that's where the underlying medical content lives. Root cause is a single-shot retrieval where any document whose title contains the query terms wins by a huge margin. Fix: ask the LLM (via ai-gateway) for up to 3 alternative phrasings, retrieve candidates for the original + each variant in parallel, merge by chunk_id keeping best RRF score, then rerank the union against the ORIGINAL query. After fix: src 29 (ספר הליקויים) shows up with 2 hits in the same query.", "status": "done", "priority": "high", "details": "Lives in espocrm-extensions/shira-hermes commit 1441a41. /kb/search now has expand=true by default, agent path (search_insurance_kb tool) keeps expand=false because the agent already does multi-query via tool calls.", "testStrategy": "POST /kb/search {query:'מהי תקנה 37', top_k:8} should return hits from at least 2 distinct source_ids, including src 29 (ספר הליקויים) and src 26 (תקנה 37 חוזר).", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T10:00:00Z" }, { "id": "8", "title": "feat(kb/ask): SSE streaming with live agent progress", "description": "User wanted Shira's progress to be visible while she thinks — like Claude does — instead of just an incrementing seconds counter. Implementation: AgentRunner.run accepts an optional sync on_event callback that emits {type, label, ...} dicts at each agent step. New POST /kb/ask/stream returns text/event-stream, with the runner running in an asyncio task whose events feed an asyncio.Queue that the response generator drains as SSE. Client opens EventSource against a new EspoCRM EntryPoint KnowledgeBaseAskStream that proxies the SSE bytes through cURL with output buffering disabled (echo + flush + exit, bypassing the framework Response). The progress UI now stacks one line per event (thinking, tool_start with the query Shira chose, tool_done with the chunk count, writing) instead of just a seconds counter. v0.1.9 _activeAsk persistence pattern preserved — events accumulate at module scope so a view switch + return replays the live log.", "status": "done", "priority": "high", "details": "shira-hermes 76fd77f. EspoCRM EntryPoint registered in metadata/app/entryPoints.json. Hebrew labels live in agent_runner._tool_label / _tool_done_label so the wire format already carries display-ready strings (no client-side mapping). 15s heartbeats keep the connection open through proxy idle-timeouts.", "testStrategy": "curl -N POST /kb/ask/stream {message:'מהי תקנה 36'} should produce data: lines with thinking → tool_start (with query) → tool_done (with chunk count) → ... → writing → answer. Browser EventSource should drive the same flow into the UI.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T13:00:00Z" }, { "id": "7", "title": "fix: search preview duplicates left card for text sources", "description": "User report: searching the law (Wikisource source 5) shows the same chunk content in both the left list-card and the right preview pane. The preview adds zero value when the matched source has no PDF — both panes are visually identical (modulo the left's 14em truncation), which reads as a bug. Fixed by fetching the matched chunk plus K=2 surrounding sections from /kb/source/{id}/chunks?around=N&ctx=2 and rendering them stacked: matched section is highlighted with a yellow header (#fff3cd), neighbors render as muted context blocks above and below. Adds a prominent 'פתח ב-Wikisource' button in the panel header, scrolls the matched section into view on render. Race-safe via _previewToken so a quick second click doesn't render stale results.", "status": "done", "priority": "high", "details": "shira-hermes b127caf added chunk_index to /kb/search hits + an `around`/`ctx` query param to /source/{id}/chunks. Espo controller passes them through. Client uses the new fields in showSearchPreview when the hit is a text source.", "testStrategy": "Search 'מהי תקנה 36'; click the law-kind hit. Right pane: yellow-highlighted ס' 36 in the middle, ס' 35 above (muted), ס' 37 below (muted), Wikisource button in header. Click the תקנה 37 חוזר hit: right pane swaps to PDF iframe at the cited page (PDF path unchanged).", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T12:30:00Z" }, { "id": "5", "title": "feat(kb): drag-to-resize splitter (both ask + search)", "description": "Replace the fixed Bootstrap col-md-5/7 split with a flex layout containing a 6px drag handle. mousedown on the handle starts the drag; while dragging an invisible full-screen overlay covers any iframe so the PDF.js viewer doesn't swallow mouse events. The chosen ratio (15-85%) persists to localStorage as kb-split-pct so the layout sticks across sessions. Both modes share the same _buildSplitShell helper so the look stays consistent.", "status": "done", "priority": "normal", "details": "Default ratio = 42% left (matches the previous col-md-5 ≈ 41.66%). Applied to renderSearchResults, renderAskAnswer, and showSearchPreview (which now uses 100%-height containers via flex column).", "testStrategy": "Open ask mode, drag the divider; verify PDF iframe doesn't lose mouse focus mid-drag; reload — divider stays at last position. Repeat in search mode.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T11:00:00Z" }, { "id": "6", "title": "fix: search survives view switch (parity with ask in v0.1.9)", "description": "Search mode had the same view-bound promise problem as ask did before v0.1.9: navigating to another EspoCRM screen and back blanked the results. Hoisted {query, kind, promise} to a module-level _activeSearch with the same {query, kind, hits, selectedIdx} sessionStorage replay we already built for ask. Selected hit index also persists so the right pane comes back to the same source/page the user had open.", "status": "done", "priority": "high", "details": "Mirrors the v0.1.9 ask fix in _attachSearchHandlers + afterRender. Click handler updates _lastSearch.selectedIdx so a return-trip lands on the user's last-clicked hit, not always the top-ranked one.", "testStrategy": "Search 'מהי תקנה 37'; click a result other than the first; navigate to Contacts; navigate back — same hit selected, same PDF page on the right.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T11:00:00Z" }, { "id": "4", "title": "fix: ask survives view switch via module-level promise + sessionStorage", "description": "User reports asking Shira a question and switching to a different EspoCRM view loses the in-flight request — coming back shows a blank screen as if nothing was asked. Cause: the Espo.Ajax promise was view-bound, so handlers fired against a detached DOM after re-mount. Fix: hoist {question, promise, startedAt} to a module-level _activeAsk; afterRender re-attaches handlers and resumes the progress UI with the original startedAt; completed answers persist to sessionStorage and replay on remount or a hard reload.", "status": "done", "priority": "high", "details": "All client-side, files/client/custom/modules/knowledge-base/src/views/kb/index.js. State is intentionally session-scoped — kept in sessionStorage so the next browser session starts clean.", "testStrategy": "Ask a question; while the spinner shows, click Contacts (or any other navbar item); wait until the request would have finished; click back to בסיס ידע — the answer (or active spinner with correct elapsed-time) should be there.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T10:00:00Z" }, { "id": "9", "title": "feat(kb/ask): rejoin in-flight SSE stream after browser reload", "description": "When the user F5s or closes+reopens the tab while Shira is thinking, the asyncio task in shira-hermes keeps running (tokens burn) but the EventSource is gone — there's no way to reattach. After reload, the v0.1.9 sessionStorage replay only shows the LAST completed answer, not the live in-flight stream. Build a rejoinable SSE channel: POST /kb/ask/stream allocates a request_id and starts the runner; the SSE response writes events into an in-memory store keyed by request_id AND streams them. New GET /kb/ask/stream/{request_id} reconnects: replays accumulated events first, then continues with new ones as they arrive. Garbage-collect after 5 minutes past the answer event.", "status": "pending", "priority": "normal", "details": "Architecture: in-process LRU dict {request_id -> {events:list, queue:asyncio.Queue, done:bool, last_used:ts}}. POST returns Set-Cookie or response header X-Request-Id; client stores it in _activeAsk.requestId + sessionStorage. afterRender: if _activeAsk has requestId and no live es, open EventSource to /kb/ask/stream/{requestId}. Server: re-emit all stored events on reconnect, then drain queue. Two-process safety: this works only because shira-hermes runs as a single FastAPI process — if we ever scale horizontally we'll need Redis pub/sub or sticky sessions. Note that for now it is single-process.", "testStrategy": "1) Ask Shira a long question. 2) F5 after 10s. 3) After reload, KB tab should resume showing live events including those that arrived during the reload window. 4) Answer event arrives — refresh again still shows it (sessionStorage path, unchanged from v0.1.9). 5) After 5 minutes, GET /kb/ask/stream/{old_request_id} returns 410 Gone.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T13:30:00Z" }, { "id": "10", "title": "ops: investigate n8n release-asset auto-attach failures", "description": "The n8n workflow `Git: Gitea Push - Auto-Release + Mattermost` (id IcF30v1Pw3wbucbu, per ~/CLAUDE.md) is supposed to attach the built .zip as a release asset within ~12s of a tag push. Across every KnowledgeBase release in this period (v0.1.8 through v0.2.1) the workflow did NOT attach the zip — every release was published with zero assets and I had to upload manually via curl + the Gitea API. Investigate the workflow's recent execution history, find the failing step, and either fix the n8n workflow or document a better fallback.", "status": "pending", "priority": "low", "details": "Test pattern: after `mcp__gitea-dev__create_release`, poll GET /api/v1/repos/.../releases/{id}/assets every 10s for ~60s. Currently always returns 0. The 'Check Existing Release → Release Exists?' branch works (skips creating a duplicate release). The asset-upload branch is what's silently failing. Possible causes: (a) the n8n workflow doesn't have the zip artifact (zip is built ad-hoc with `bash build.sh` from the workspace, not in CI), (b) bad credential to Gitea API, (c) the release-creation event doesn't trigger that node. Note this workflow is for OTHER extensions too — if it's broken for all of them, fixing helps the whole release pipeline.", "testStrategy": "After fix: push v0.X.Y tag to KnowledgeBase via mcp__gitea-dev__create_release; verify the zip appears at GET /releases/{id}/assets within 30s without manual upload.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T13:30:00Z" }, { "id": "11", "title": "feat(kb/ask): show text-source citations in the ask mode source picker", "description": "/kb/ask currently filters sources_used through _filter_pdf_sources (kb_public.py) before returning, dropping any source whose original_path doesn't end with .pdf. The Wikisource law (source 5) is therefore invisible to the user even when Shira explicitly cites a section from it. With the v0.1.11 'section in context' preview that already exists for search mode, ask mode should be able to show the same. Drop the filter, and when the user clicks a non-PDF source pill, route to the same _renderSectionContext flow that search uses (factor it into a shared helper).", "status": "pending", "priority": "normal", "details": "Server (shira-hermes): drop _filter_pdf_sources in both /kb/ask and /kb/ask/stream answer-event payload. Either return all sources, or re-introduce a different filter that keeps text sources but strips ones with no source_id. Client: in renderAskAnswer the source-pill click handler currently always rebuilds the iframe; route through showSearchPreview when the source is text-only. Probably easiest to extract _renderSectionContext / _renderSectionContextPlaceholder into a shared helper that both modes call. Need to add chunk_index to the sources returned from /kb/ask too — currently only on /kb/search hits.", "testStrategy": "Ask 'מהי תקנה 36' — Shira's answer cites both תקנה 37 (PDF) and ס׳ 36 of the law (text). Source picker shows both. Click PDF pill → iframe at the cited page. Click law pill → section in context (matched section + 2 neighbors + Wikisource button) — same look as search mode.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T13:30:00Z" }, { "id": "12", "title": "Phase 1 — Multi-topic KB foundation (DB schema, topic-aware endpoints, topic selector)", "description": "Generalize the KB from being insurance-only to supporting multiple legal domains within one CRM (e.g. ביטוח לאומי, דיני עבודה, דין פלילי). A 'topic' is the firm-level grouping; each kb_source belongs to exactly one topic. Search/ask are scoped to a single topic at a time, chosen from a topic dropdown the user controls. The system_prompt for /kb/ask becomes per-topic so Shira's domain context is correct (today the prompt hardcodes 'Israeli National Insurance'). Backwards-compatible migration: all 6 existing sources move to topic_id=1 named 'ביטוח לאומי' with the existing system-prompt addendum, and the UI defaults to that topic so nothing visible changes for existing users until they choose another topic.", "status": "done", "priority": "high", "details": "DB migration (shira-hermes side, against insurance_kb DB):\n- New table kb_topic: id PK, slug TEXT UNIQUE, name TEXT NOT NULL, description TEXT, system_prompt_addendum TEXT, is_active BOOLEAN DEFAULT true, created_at TIMESTAMPTZ DEFAULT now(), updated_at TIMESTAMPTZ DEFAULT now().\n- Seed: INSERT (id=1, slug='national-insurance', name='ביטוח לאומי', system_prompt_addendum=, is_active=true). RENAME the DB itself? Probably leave as 'insurance_kb' — costly to rename, no real value. Document in memory.\n- ALTER TABLE kb_source ADD COLUMN topic_id INTEGER REFERENCES kb_topic(id). Backfill: UPDATE kb_source SET topic_id = 1. Then ALTER COLUMN SET NOT NULL.\n\nshira-hermes endpoints:\n- New GET /kb/topics — public, returns active topics with id, slug, name, description (no prompt).\n- /kb/search: accepts optional topic_id (single int). When present, the SQL adds AND s.topic_id = $N. When absent (back-compat), still returns all (might want to deprecate this and require topic_id later).\n- /kb/sources, /kb/source/{id}/chunks, /kb/source/{id}/pdf: same — optional topic filter on listings, no change to single-source endpoints.\n- /kb/ask + /kb/ask/stream: accept topic_id param; load topic.system_prompt_addendum and inject into system_prompt instead of the hardcoded insurance text in _build_ask_runner_context. Pin the legal_kb tool to topic_id too (so search_insurance_kb queries are constrained — RENAME the tool to search_legal_kb and pass topic_id at registration time).\n- _expand_query in kb_search.py: parametrize the prompt by topic name (currently hardcoded 'הביטוח הלאומי בישראל'). Pass topic.name in.\n\nKnowledgeBase extension (EspoCRM):\n- New route+action GET /KnowledgeBase/action/topics → proxies /kb/topics.\n- Service.search/ask gain topic_id, controller passes through.\n- Template gets a topic in index.tpl (search box + upload form); update kindHe in index.js.", "testStrategy": "", "status": "done", "dependencies": [], "priority": "medium", "subtasks": [], "updatedAt": "2026-04-25T17:02:14.241Z" }, { "id": "19", "title": "feat(kb): tag/label sub-topics within a topic (e.g. 'ילד נכה')", "description": "Group multiple sources within a topic by sub-subject so users can find e.g. all 'ילד נכה' circulars together. Recommendation: many-to-many kb_label + kb_source_label so a source can carry multiple tags reflecting that legal docs commonly touch several subjects.", "status": "pending", "priority": "medium", "details": "Three approaches considered:\n\n1. Free-text 'subject' column on kb_source (~1d): cheapest, typo-prone (ילד נכה / ילדים נכים → separate buckets).\n\n2. Many-to-many labels (~3d, RECOMMENDED): kb_label(id, topic_id, slug UNIQUE per topic, name) + kb_source_label(source_id, label_id, PK both). UI shows label chips on sources; click to filter. Naturally handles cross-cutting subjects (a circular tagged both 'ילד נכה' AND 'אזרח ותיק'). Needs admin UI to manage labels.\n\n3. Hierarchical sub-topics under kb_topic (~5d): clean tree, but a source can only live in one branch — wrong fit for the legal world. Also clashes with planned Phase 4 topic-CRUD UI.\n\nPragmatic path: ship #1 first (subject TEXT field on kb_source + dropdown of seen values + browse-view grouping); upgrade to #2 once we see how labels are used in practice.", "testStrategy": "Upload 3 circulars all in 'ילד נכה' subject; verify they appear grouped together in browse view; verify search results show the subject chip; verify topic switch resets the subject filter.", "subtasks": [], "dependencies": [], "createdAt": "2026-04-25T16:54:54.478470Z" }, { "id": "20", "title": "v0.8.0 — kind 'tool' + labels + bulk upload with AI classification", "description": "Phase 6. Adds 'tool' kind for academic assessment instruments (GMFCS, MACS, etc), label-based sub-topic grouping (kb_label many-to-many), multi-file drag-drop upload, and AI-extracted metadata via Claude/ai-gateway. Two-phase ingest: classify → await_review → embed. Replaces single-file upload with batch-review screen. Also creates 'תוויות' admin tab. Subsumes task #19 (labels). Plan: ~/.claude/plans/hidden-tickling-ullman.md", "details": "See /home/chaim/.claude/plans/hidden-tickling-ullman.md for the full architecture, migration scripts, API surface, classifier prompt, UX, and file-level breakdown. ~7 days estimated.", "testStrategy": "", "status": "in-progress", "dependencies": [], "priority": "medium", "subtasks": [], "updatedAt": "2026-04-25T18:21:44.731Z" } ], "metadata": { "version": "1.0.0", "lastModified": "2026-04-25T18:21:44.731Z", "taskCount": 20, "completedCount": 13, "tags": [ "master" ] } } }