chaim
a00fc3729c
User wanted to see what Shira is doing while she thinks — like Claude does — instead of staring at an incrementing seconds counter. Now the ask UI shows one line per agent step as it happens: 3s 🧠 קוראת את השאלה 4s 🔍 מחפשת בבסיס הידע: "תקנה 36" 9s ✓ נמצאו 5 קטעים רלוונטיים 10s 🔍 מחפשת בבסיס הידע: "תקנה 36 ביטוח לאומי" 14s ✓ נמצאו 5 קטעים רלוונטיים 15s 🧠 ממשיכה לחקור 22s ✍️ מנסחת תשובה → תשובה Client wiring: - runAsk now opens an EventSource at ?entryPoint=KnowledgeBaseAskStream instead of POSTing JSON. Each SSE event of type thinking/tool_start/ tool_done/tool_error/writing appends a row to a stacked progress log. The final {type:answer,text,sources} event triggers renderAskAnswer with the existing split view. {type:error} surfaces via showError. - Module-level state from v0.1.9 unchanged in spirit but now stores the EventSource + accumulated events. afterRender on view re-mount replays the entire event list and re-binds onmessage/onerror — the SSE connection itself doesn't drop because it lives at module scope. - Elapsed-time pill stays for cadence but is decoration; the real signal is the per-event log. EspoCRM proxy: - New EntryPoint KnowledgeBaseAskStream — required because EventSource is GET-only and ?message= must come via the URL. PHP buffering is disabled (zlib + ob + apache_setenv no-gzip), Apache headers are set for SSE (no-cache, X-Accel-Buffering: no), and we cURL into shira-hermes /kb/ask/stream with CURLOPT_WRITEFUNCTION echoing each byte and flushing immediately. The handler ends with exit; to bypass EspoCRM's Response wrapper which would otherwise emit headers/body on top of our raw stream. - KnowledgeBaseService gained two public accessors (getStreamingUrl, getStreamingApiKey) so the EntryPoint can build the cURL without going through request() (which buffers the whole body). - entryPoints.json registers the new class. Server (shira-hermes 76fd77f): - AgentRunner.run accepts an on_event sync callback fired before each LLM call (thinking) and around each tool call (tool_start, tool_done, tool_error). Hebrew labels in agent_runner._tool_label / _tool_done_label ship over the wire ready-to-display — keeps the client free of translation concerns. - POST /kb/ask/stream wraps the existing agent runner in an asyncio task feeding an asyncio.Queue, returns StreamingResponse with proper SSE headers (X-Accel-Buffering, Cache-Control). 15s comment-line heartbeats keep the connection open through proxy idle-timeouts. - Original POST /kb/ask still works — additive change. Refs Task Master #8
108 lines
11 KiB
JSON
108 lines
11 KiB
JSON
{
|
||
"master": {
|
||
"tasks": [
|
||
{
|
||
"id": 1,
|
||
"title": "feat: search-mode split-view with PDF + page jump",
|
||
"description": "Render /kb/search results as a two-column split view: left column is the stack of ranked hit cards (kind, title, section, page label), right column is an iframe viewing the selected hit's PDF scrolled to page_number. Clicking any hit swaps the iframe. Text-only sources (law from Wikisource) fall back to a chunk/text panel with a Wikisource link so the right column never 404s.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "Before this change renderSearchResults produced plain vertical text cards — users got the chunk body and had to open the Wikisource link (or track down the PDF manually) to verify context. With 5 of 5 PDFs now carrying accurate page_number (shira-hermes e534709 + 1ca1cc0), the search UI can finally deep-link to the right page. Mirrors the ask-mode split view (v0.1.7 uncommitted).",
|
||
"testStrategy": "Search 'תקנה 37' → first hit should be a regulation/circular → right pane loads the PDF at page ~N where the section appears. Click a lower-ranked law hit → right pane swaps to a Wikisource link view. Search 'הגדרות' → hits span multiple sources → each click swaps iframe source.",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-24T15:47:00Z"
|
||
},
|
||
{
|
||
"id": 2,
|
||
"title": "fix(kb/chunker): page_number propagation through split + packing (shira-hermes)",
|
||
"description": "Applied in shira-hermes commits e534709 + 1ca1cc0: (1) _split_oversized now re-derives page_number per piece from markers embedded in parent content + a virtual (offset 0, parent.page_number) anchor, (2) chunk_circular tracks absolute paragraph offsets in the original text so each packed sub-chunk reports its real starting page, (3) _as_dicts strips stray \\x00 bytes left behind when _split_oversized slices through a marker sentinel. Reference data: ספר הליקויים now 211 chunks × 210 pages (1-413) vs 209 × 1 before. Not work for this repo, but the search split-view in task #1 depends on it.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "Lives in espocrm-extensions/shira-hermes. Logged here for traceability because task #1 can't demonstrate correct page jumps without it.",
|
||
"testStrategy": "Query DB: SELECT source_id, MIN/MAX(page_number), COUNT(DISTINCT page_number) FROM kb_chunk GROUP BY source_id — every PDF source should span multiple distinct pages.",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-24T15:47:00Z"
|
||
},
|
||
{
|
||
"id": 3,
|
||
"title": "feat(kb/search): LLM query expansion so ranked-by-title docs don't crowd everything else out (shira-hermes)",
|
||
"description": "User reports searching 'מהי תקנה 37' returns only the תקנה 37 circular and the law — never ספר הליקויים, even though that's where the underlying medical content lives. Root cause is a single-shot retrieval where any document whose title contains the query terms wins by a huge margin. Fix: ask the LLM (via ai-gateway) for up to 3 alternative phrasings, retrieve candidates for the original + each variant in parallel, merge by chunk_id keeping best RRF score, then rerank the union against the ORIGINAL query. After fix: src 29 (ספר הליקויים) shows up with 2 hits in the same query.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "Lives in espocrm-extensions/shira-hermes commit 1441a41. /kb/search now has expand=true by default, agent path (search_insurance_kb tool) keeps expand=false because the agent already does multi-query via tool calls.",
|
||
"testStrategy": "POST /kb/search {query:'מהי תקנה 37', top_k:8} should return hits from at least 2 distinct source_ids, including src 29 (ספר הליקויים) and src 26 (תקנה 37 חוזר).",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-25T10:00:00Z"
|
||
},
|
||
{
|
||
"id": 8,
|
||
"title": "feat(kb/ask): SSE streaming with live agent progress",
|
||
"description": "User wanted Shira's progress to be visible while she thinks — like Claude does — instead of just an incrementing seconds counter. Implementation: AgentRunner.run accepts an optional sync on_event callback that emits {type, label, ...} dicts at each agent step. New POST /kb/ask/stream returns text/event-stream, with the runner running in an asyncio task whose events feed an asyncio.Queue that the response generator drains as SSE. Client opens EventSource against a new EspoCRM EntryPoint KnowledgeBaseAskStream that proxies the SSE bytes through cURL with output buffering disabled (echo + flush + exit, bypassing the framework Response). The progress UI now stacks one line per event (thinking, tool_start with the query Shira chose, tool_done with the chunk count, writing) instead of just a seconds counter. v0.1.9 _activeAsk persistence pattern preserved — events accumulate at module scope so a view switch + return replays the live log.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "shira-hermes 76fd77f. EspoCRM EntryPoint registered in metadata/app/entryPoints.json. Hebrew labels live in agent_runner._tool_label / _tool_done_label so the wire format already carries display-ready strings (no client-side mapping). 15s heartbeats keep the connection open through proxy idle-timeouts.",
|
||
"testStrategy": "curl -N POST /kb/ask/stream {message:'מהי תקנה 36'} should produce data: lines with thinking → tool_start (with query) → tool_done (with chunk count) → ... → writing → answer. Browser EventSource should drive the same flow into the UI.",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-25T13:00:00Z"
|
||
},
|
||
{
|
||
"id": 7,
|
||
"title": "fix: search preview duplicates left card for text sources",
|
||
"description": "User report: searching the law (Wikisource source 5) shows the same chunk content in both the left list-card and the right preview pane. The preview adds zero value when the matched source has no PDF — both panes are visually identical (modulo the left's 14em truncation), which reads as a bug. Fixed by fetching the matched chunk plus K=2 surrounding sections from /kb/source/{id}/chunks?around=N&ctx=2 and rendering them stacked: matched section is highlighted with a yellow header (#fff3cd), neighbors render as muted context blocks above and below. Adds a prominent 'פתח ב-Wikisource' button in the panel header, scrolls the matched section into view on render. Race-safe via _previewToken so a quick second click doesn't render stale results.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "shira-hermes b127caf added chunk_index to /kb/search hits + an `around`/`ctx` query param to /source/{id}/chunks. Espo controller passes them through. Client uses the new fields in showSearchPreview when the hit is a text source.",
|
||
"testStrategy": "Search 'מהי תקנה 36'; click the law-kind hit. Right pane: yellow-highlighted ס' 36 in the middle, ס' 35 above (muted), ס' 37 below (muted), Wikisource button in header. Click the תקנה 37 חוזר hit: right pane swaps to PDF iframe at the cited page (PDF path unchanged).",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-25T12:30:00Z"
|
||
},
|
||
{
|
||
"id": 5,
|
||
"title": "feat(kb): drag-to-resize splitter (both ask + search)",
|
||
"description": "Replace the fixed Bootstrap col-md-5/7 split with a flex layout containing a 6px drag handle. mousedown on the handle starts the drag; while dragging an invisible full-screen overlay covers any iframe so the PDF.js viewer doesn't swallow mouse events. The chosen ratio (15-85%) persists to localStorage as kb-split-pct so the layout sticks across sessions. Both modes share the same _buildSplitShell helper so the look stays consistent.",
|
||
"status": "done",
|
||
"priority": "normal",
|
||
"details": "Default ratio = 42% left (matches the previous col-md-5 ≈ 41.66%). Applied to renderSearchResults, renderAskAnswer, and showSearchPreview (which now uses 100%-height containers via flex column).",
|
||
"testStrategy": "Open ask mode, drag the divider; verify PDF iframe doesn't lose mouse focus mid-drag; reload — divider stays at last position. Repeat in search mode.",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-25T11:00:00Z"
|
||
},
|
||
{
|
||
"id": 6,
|
||
"title": "fix: search survives view switch (parity with ask in v0.1.9)",
|
||
"description": "Search mode had the same view-bound promise problem as ask did before v0.1.9: navigating to another EspoCRM screen and back blanked the results. Hoisted {query, kind, promise} to a module-level _activeSearch with the same {query, kind, hits, selectedIdx} sessionStorage replay we already built for ask. Selected hit index also persists so the right pane comes back to the same source/page the user had open.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "Mirrors the v0.1.9 ask fix in _attachSearchHandlers + afterRender. Click handler updates _lastSearch.selectedIdx so a return-trip lands on the user's last-clicked hit, not always the top-ranked one.",
|
||
"testStrategy": "Search 'מהי תקנה 37'; click a result other than the first; navigate to Contacts; navigate back — same hit selected, same PDF page on the right.",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-25T11:00:00Z"
|
||
},
|
||
{
|
||
"id": 4,
|
||
"title": "fix: ask survives view switch via module-level promise + sessionStorage",
|
||
"description": "User reports asking Shira a question and switching to a different EspoCRM view loses the in-flight request — coming back shows a blank screen as if nothing was asked. Cause: the Espo.Ajax promise was view-bound, so handlers fired against a detached DOM after re-mount. Fix: hoist {question, promise, startedAt} to a module-level _activeAsk; afterRender re-attaches handlers and resumes the progress UI with the original startedAt; completed answers persist to sessionStorage and replay on remount or a hard reload.",
|
||
"status": "done",
|
||
"priority": "high",
|
||
"details": "All client-side, files/client/custom/modules/knowledge-base/src/views/kb/index.js. State is intentionally session-scoped — kept in sessionStorage so the next browser session starts clean.",
|
||
"testStrategy": "Ask a question; while the spinner shows, click Contacts (or any other navbar item); wait until the request would have finished; click back to בסיס ידע — the answer (or active spinner with correct elapsed-time) should be there.",
|
||
"subtasks": [],
|
||
"dependencies": [],
|
||
"createdAt": "2026-04-25T10:00:00Z"
|
||
}
|
||
],
|
||
"metadata": {
|
||
"created": "2026-04-24T15:47:00Z",
|
||
"updated": "2026-04-25T10:30:00Z",
|
||
"description": "KnowledgeBase extension tasks"
|
||
}
|
||
}
|
||
}
|