5 Commits

Author SHA1 Message Date
chaim a00fc3729c feat(kb/ask): live SSE streaming of Shira's progress (v0.2.0)
User wanted to see what Shira is doing while she thinks — like Claude
does — instead of staring at an incrementing seconds counter. Now the
ask UI shows one line per agent step as it happens:

  3s  🧠  קוראת את השאלה
  4s  🔍  מחפשת בבסיס הידע: "תקנה 36"
  9s  ✓   נמצאו 5 קטעים רלוונטיים
 10s  🔍  מחפשת בבסיס הידע: "תקנה 36 ביטוח לאומי"
 14s  ✓   נמצאו 5 קטעים רלוונטיים
 15s  🧠  ממשיכה לחקור
 22s  ✍️  מנסחת תשובה
 → תשובה

Client wiring:
- runAsk now opens an EventSource at ?entryPoint=KnowledgeBaseAskStream
  instead of POSTing JSON. Each SSE event of type thinking/tool_start/
  tool_done/tool_error/writing appends a row to a stacked progress log.
  The final {type:answer,text,sources} event triggers renderAskAnswer
  with the existing split view. {type:error} surfaces via showError.
- Module-level state from v0.1.9 unchanged in spirit but now stores the
  EventSource + accumulated events. afterRender on view re-mount replays
  the entire event list and re-binds onmessage/onerror — the SSE
  connection itself doesn't drop because it lives at module scope.
- Elapsed-time pill stays for cadence but is decoration; the real
  signal is the per-event log.

EspoCRM proxy:
- New EntryPoint KnowledgeBaseAskStream — required because EventSource
  is GET-only and ?message= must come via the URL. PHP buffering is
  disabled (zlib + ob + apache_setenv no-gzip), Apache headers are set
  for SSE (no-cache, X-Accel-Buffering: no), and we cURL into
  shira-hermes /kb/ask/stream with CURLOPT_WRITEFUNCTION echoing each
  byte and flushing immediately. The handler ends with exit; to bypass
  EspoCRM's Response wrapper which would otherwise emit headers/body
  on top of our raw stream.
- KnowledgeBaseService gained two public accessors (getStreamingUrl,
  getStreamingApiKey) so the EntryPoint can build the cURL without
  going through request() (which buffers the whole body).
- entryPoints.json registers the new class.

Server (shira-hermes 76fd77f):
- AgentRunner.run accepts an on_event sync callback fired before each
  LLM call (thinking) and around each tool call (tool_start, tool_done,
  tool_error). Hebrew labels in agent_runner._tool_label / _tool_done_label
  ship over the wire ready-to-display — keeps the client free of
  translation concerns.
- POST /kb/ask/stream wraps the existing agent runner in an asyncio
  task feeding an asyncio.Queue, returns StreamingResponse with proper
  SSE headers (X-Accel-Buffering, Cache-Control). 15s comment-line
  heartbeats keep the connection open through proxy idle-timeouts.
- Original POST /kb/ask still works — additive change.

Refs Task Master #8
2026-04-25 13:00:58 +00:00
chaim 2fe10727eb fix(kb): search preview shows section-in-context for text sources
User reported the search card "looks broken" when the selected hit is the
law (Wikisource source 5) — the right preview pane just duplicates the
chunk text already visible in the left list card. For PDF sources the
right pane shows the PDF page, but for text-only sources it had nothing
useful to add and fell back to a copy.

Fix:
- /kb/search hits gained chunk_index (shira-hermes b127caf), and
  /kb/source/{id}/chunks gained an `around=<chunk_index>&ctx=<k>` window.
- Espo proxy plumbing: getActionChunks reads `around`/`ctx` query
  params and forwards them; KnowledgeBaseService::sourceChunks accepts
  optional $around + $ctx and appends them to the upstream URL.
- showSearchPreview for non-PDF hits now fetches the matched chunk plus
  2 sections before and 2 after, renders them as stacked panels: the
  matched section gets a yellow #fff3cd highlight, neighbors render as
  muted context blocks. The panel header carries a prominent
  "פתח ב-Wikisource" button so users can jump to the public source page.
  After render the matched section auto-scrolls into view.
- Race protection via _previewToken: a click on a different hit while a
  fetch is in flight cancels the stale render so the user always sees
  the section that matches the currently-selected card.

PDF path unchanged.

Refs Task Master #7
2026-04-25 12:28:50 +00:00
chaim ac5823ffe5 feat(kb): drag-to-resize splitter + search survives view switch
Two requests on top of v0.1.9:

1. Drag-to-resize splitter
   The previous fixed Bootstrap col-md-5/col-md-7 split locked the
   user into 41.66% answer / 58.33% PDF. Replaced with a flex-row
   layout containing a 6px drag handle. mousedown spawns a fixed
   transparent overlay so the PDF.js iframe doesn't swallow mouse
   events while dragging; mouseup persists the chosen ratio (clamped
   15-85%) to localStorage as kb-split-pct, so the next page-load and
   the next session both come back to the same layout. Both the
   ask split view and the search split view share _buildSplitShell +
   _wireSplitter — single source of truth for the geometry.

2. Search survives a view switch
   Search had the same view-bound promise problem ask had before
   v0.1.9. Hoisted the in-flight request to a module-level
   _activeSearch and the most recent {query, kind, hits, selectedIdx}
   to _lastSearch + sessionStorage. afterRender restores the input,
   the kind filter, the hit list (with the same hit highlighted), and
   the right-pane preview. Click handler updates selectedIdx so a
   round-trip lands on the user's last-clicked hit, not always on the
   top-ranked one.

The persistence helpers grew a tiny refactor: _loadJson/_saveJson/_clear
replace the ask-only SS_KEY shim and now back both ask + search keys.

Refs Task Master #5, #6
2026-04-25 10:46:39 +00:00
chaim 50a60c13b3 feat(kb): ask survives view switch + search query expansion (server-side)
Two recurring frustrations from the v0.1.8 demo:

1. Searching "מהי תקנה 37" only returned the תקנה 37 חוזר and the law —
   ספר הליקויים, the canonical medical reference, was crowded out by the
   document whose title literally contained the query terms. Fixed in
   shira-hermes 1441a41 by adding an LLM-driven query-expansion path to
   /kb/search (default expand=true). Same query now also returns 2 hits
   from src 29 (ספר הליקויים).
2. Asking Shira a question and clicking another EspoCRM view dropped the
   in-flight request — coming back rendered a blank kb-results pane, even
   though the request had completed in the background. Fixed here:
   {question, promise, startedAt} are now stored at module scope (closure
   shared across remounts of this view). afterRender re-binds handlers
   for any active ask, resumes the progress panel with the original start
   time so the elapsed counter doesn't reset, and replays the most recent
   completed answer from sessionStorage so the user sees their result
   even after a hard reload. State is intentionally session-scoped — no
   answers persist across browser sessions.

Refs Task Master #3, #4
2026-04-25 10:23:49 +00:00
chaim eb1b0fc7d4 chore: mark task #1 done after v0.1.8 release 2026-04-24 16:01:03 +00:00
7 changed files with 726 additions and 107 deletions
+74 -2
View File
@@ -5,7 +5,7 @@
"id": 1,
"title": "feat: search-mode split-view with PDF + page jump",
"description": "Render /kb/search results as a two-column split view: left column is the stack of ranked hit cards (kind, title, section, page label), right column is an iframe viewing the selected hit's PDF scrolled to page_number. Clicking any hit swaps the iframe. Text-only sources (law from Wikisource) fall back to a chunk/text panel with a Wikisource link so the right column never 404s.",
"status": "in-progress",
"status": "done",
"priority": "high",
"details": "Before this change renderSearchResults produced plain vertical text cards — users got the chunk body and had to open the Wikisource link (or track down the PDF manually) to verify context. With 5 of 5 PDFs now carrying accurate page_number (shira-hermes e534709 + 1ca1cc0), the search UI can finally deep-link to the right page. Mirrors the ask-mode split view (v0.1.7 uncommitted).",
"testStrategy": "Search 'תקנה 37' → first hit should be a regulation/circular → right pane loads the PDF at page ~N where the section appears. Click a lower-ranked law hit → right pane swaps to a Wikisource link view. Search 'הגדרות' → hits span multiple sources → each click swaps iframe source.",
@@ -24,11 +24,83 @@
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-24T15:47:00Z"
},
{
"id": 3,
"title": "feat(kb/search): LLM query expansion so ranked-by-title docs don't crowd everything else out (shira-hermes)",
"description": "User reports searching 'מהי תקנה 37' returns only the תקנה 37 circular and the law — never ספר הליקויים, even though that's where the underlying medical content lives. Root cause is a single-shot retrieval where any document whose title contains the query terms wins by a huge margin. Fix: ask the LLM (via ai-gateway) for up to 3 alternative phrasings, retrieve candidates for the original + each variant in parallel, merge by chunk_id keeping best RRF score, then rerank the union against the ORIGINAL query. After fix: src 29 (ספר הליקויים) shows up with 2 hits in the same query.",
"status": "done",
"priority": "high",
"details": "Lives in espocrm-extensions/shira-hermes commit 1441a41. /kb/search now has expand=true by default, agent path (search_insurance_kb tool) keeps expand=false because the agent already does multi-query via tool calls.",
"testStrategy": "POST /kb/search {query:'מהי תקנה 37', top_k:8} should return hits from at least 2 distinct source_ids, including src 29 (ספר הליקויים) and src 26 (תקנה 37 חוזר).",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T10:00:00Z"
},
{
"id": 8,
"title": "feat(kb/ask): SSE streaming with live agent progress",
"description": "User wanted Shira's progress to be visible while she thinks — like Claude does — instead of just an incrementing seconds counter. Implementation: AgentRunner.run accepts an optional sync on_event callback that emits {type, label, ...} dicts at each agent step. New POST /kb/ask/stream returns text/event-stream, with the runner running in an asyncio task whose events feed an asyncio.Queue that the response generator drains as SSE. Client opens EventSource against a new EspoCRM EntryPoint KnowledgeBaseAskStream that proxies the SSE bytes through cURL with output buffering disabled (echo + flush + exit, bypassing the framework Response). The progress UI now stacks one line per event (thinking, tool_start with the query Shira chose, tool_done with the chunk count, writing) instead of just a seconds counter. v0.1.9 _activeAsk persistence pattern preserved — events accumulate at module scope so a view switch + return replays the live log.",
"status": "done",
"priority": "high",
"details": "shira-hermes 76fd77f. EspoCRM EntryPoint registered in metadata/app/entryPoints.json. Hebrew labels live in agent_runner._tool_label / _tool_done_label so the wire format already carries display-ready strings (no client-side mapping). 15s heartbeats keep the connection open through proxy idle-timeouts.",
"testStrategy": "curl -N POST /kb/ask/stream {message:'מהי תקנה 36'} should produce data: lines with thinking → tool_start (with query) → tool_done (with chunk count) → ... → writing → answer. Browser EventSource should drive the same flow into the UI.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T13:00:00Z"
},
{
"id": 7,
"title": "fix: search preview duplicates left card for text sources",
"description": "User report: searching the law (Wikisource source 5) shows the same chunk content in both the left list-card and the right preview pane. The preview adds zero value when the matched source has no PDF — both panes are visually identical (modulo the left's 14em truncation), which reads as a bug. Fixed by fetching the matched chunk plus K=2 surrounding sections from /kb/source/{id}/chunks?around=N&ctx=2 and rendering them stacked: matched section is highlighted with a yellow header (#fff3cd), neighbors render as muted context blocks above and below. Adds a prominent 'פתח ב-Wikisource' button in the panel header, scrolls the matched section into view on render. Race-safe via _previewToken so a quick second click doesn't render stale results.",
"status": "done",
"priority": "high",
"details": "shira-hermes b127caf added chunk_index to /kb/search hits + an `around`/`ctx` query param to /source/{id}/chunks. Espo controller passes them through. Client uses the new fields in showSearchPreview when the hit is a text source.",
"testStrategy": "Search 'מהי תקנה 36'; click the law-kind hit. Right pane: yellow-highlighted ס' 36 in the middle, ס' 35 above (muted), ס' 37 below (muted), Wikisource button in header. Click the תקנה 37 חוזר hit: right pane swaps to PDF iframe at the cited page (PDF path unchanged).",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T12:30:00Z"
},
{
"id": 5,
"title": "feat(kb): drag-to-resize splitter (both ask + search)",
"description": "Replace the fixed Bootstrap col-md-5/7 split with a flex layout containing a 6px drag handle. mousedown on the handle starts the drag; while dragging an invisible full-screen overlay covers any iframe so the PDF.js viewer doesn't swallow mouse events. The chosen ratio (15-85%) persists to localStorage as kb-split-pct so the layout sticks across sessions. Both modes share the same _buildSplitShell helper so the look stays consistent.",
"status": "done",
"priority": "normal",
"details": "Default ratio = 42% left (matches the previous col-md-5 ≈ 41.66%). Applied to renderSearchResults, renderAskAnswer, and showSearchPreview (which now uses 100%-height containers via flex column).",
"testStrategy": "Open ask mode, drag the divider; verify PDF iframe doesn't lose mouse focus mid-drag; reload — divider stays at last position. Repeat in search mode.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T11:00:00Z"
},
{
"id": 6,
"title": "fix: search survives view switch (parity with ask in v0.1.9)",
"description": "Search mode had the same view-bound promise problem as ask did before v0.1.9: navigating to another EspoCRM screen and back blanked the results. Hoisted {query, kind, promise} to a module-level _activeSearch with the same {query, kind, hits, selectedIdx} sessionStorage replay we already built for ask. Selected hit index also persists so the right pane comes back to the same source/page the user had open.",
"status": "done",
"priority": "high",
"details": "Mirrors the v0.1.9 ask fix in _attachSearchHandlers + afterRender. Click handler updates _lastSearch.selectedIdx so a return-trip lands on the user's last-clicked hit, not always the top-ranked one.",
"testStrategy": "Search 'מהי תקנה 37'; click a result other than the first; navigate to Contacts; navigate back — same hit selected, same PDF page on the right.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T11:00:00Z"
},
{
"id": 4,
"title": "fix: ask survives view switch via module-level promise + sessionStorage",
"description": "User reports asking Shira a question and switching to a different EspoCRM view loses the in-flight request — coming back shows a blank screen as if nothing was asked. Cause: the Espo.Ajax promise was view-bound, so handlers fired against a detached DOM after re-mount. Fix: hoist {question, promise, startedAt} to a module-level _activeAsk; afterRender re-attaches handlers and resumes the progress UI with the original startedAt; completed answers persist to sessionStorage and replay on remount or a hard reload.",
"status": "done",
"priority": "high",
"details": "All client-side, files/client/custom/modules/knowledge-base/src/views/kb/index.js. State is intentionally session-scoped — kept in sessionStorage so the next browser session starts clean.",
"testStrategy": "Ask a question; while the spinner shows, click Contacts (or any other navbar item); wait until the request would have finished; click back to בסיס ידע — the answer (or active spinner with correct elapsed-time) should be there.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T10:00:00Z"
}
],
"metadata": {
"created": "2026-04-24T15:47:00Z",
"updated": "2026-04-24T15:47:00Z",
"updated": "2026-04-25T10:30:00Z",
"description": "KnowledgeBase extension tasks"
}
}
@@ -1,5 +1,42 @@
define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
// Module-level state survives view unmount/remount within a single
// SPA session so a user can fire an ask, navigate elsewhere, and come
// back to the answer (or to the still-spinning request) without losing
// it. sessionStorage backs the last completed answer so a hard reload
// also recovers it. None of this leaks to the next browser session —
// ask history is intentionally ephemeral.
let _activeAsk = null; // {question, promise, startedAt} while in flight
let _lastAsk = null; // {question, text, sources, completedAt} after success
let _activeSearch = null; // {query, kind, promise} while in flight
let _lastSearch = null; // {query, kind, hits, selectedIdx, completedAt}
const SS_ASK = 'kb-last-ask';
const SS_SEARCH = 'kb-last-search';
const LS_SPLIT = 'kb-split-pct'; // long-lived UI preference (not session)
function _loadJson(storage, key, validator) {
try {
const raw = storage.getItem(key);
if (!raw) return null;
const parsed = JSON.parse(raw);
return validator(parsed) ? parsed : null;
} catch (e) { return null; }
}
function _saveJson(storage, key, value) {
try { storage.setItem(key, JSON.stringify(value)); }
catch (e) { /* quota or disabled, fine */ }
}
function _clear(storage, key) {
try { storage.removeItem(key); } catch (e) { /* ignore */ }
}
if (!_lastAsk) {
_lastAsk = _loadJson(sessionStorage, SS_ASK, x => x && typeof x.question === 'string');
}
if (!_lastSearch) {
_lastSearch = _loadJson(sessionStorage, SS_SEARCH, x => x && typeof x.query === 'string' && Array.isArray(x.hits));
}
return Dep.extend({
template: 'knowledge-base:kb/index',
@@ -68,6 +105,40 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
} else if (this._sources) {
this.renderSourcesList();
}
// Recover prior state across view remounts (user navigated away and
// back). Active in-flight wins; otherwise replay the last result
// from sessionStorage so the work the user already did doesn't
// disappear behind a tab switch.
if (this.mode === 'ask') {
if (_activeAsk) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_activeAsk.question);
this.setLoading(true);
// Pass the whole ask so startAskProgress can replay the
// accumulated event log into the freshly-rendered DOM.
this.startAskProgress(_activeAsk);
this._attachAskHandlers(_activeAsk);
} else if (_lastAsk) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_lastAsk.question);
this.renderAskAnswer(_lastAsk.text, _lastAsk.sources);
}
} else if (this.mode === 'search') {
if (_activeSearch) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_activeSearch.query);
this.setLoading(true);
this._attachSearchHandlers(_activeSearch);
} else if (_lastSearch && _lastSearch.hits.length) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_lastSearch.query);
const $kind = this.$el.find('select[data-name="kind"]');
if ($kind.length && _lastSearch.kind) $kind.val(_lastSearch.kind);
this.renderSearchResults(_lastSearch.hits, _lastSearch.selectedIdx || 0);
}
}
// Focus the input on every render so users can start typing right away.
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.trigger('focus');
@@ -87,75 +158,220 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
},
runSearch: function (query, kind) {
// A new search invalidates the prior cached one.
_lastSearch = null;
_clear(sessionStorage, SS_SEARCH);
this.setLoading(true);
Espo.Ajax.postRequest('KnowledgeBase/action/search', {
const promise = Espo.Ajax.postRequest('KnowledgeBase/action/search', {
query: query,
kind: kind,
topK: 8,
}).then(res => {
this.setLoading(false);
this.renderSearchResults(res.hits || []);
});
_activeSearch = {query: query, kind: kind, promise: promise};
this._attachSearchHandlers(_activeSearch);
},
_attachSearchHandlers: function (search) {
const self = this;
search.promise.then(res => {
const hits = res.hits || [];
const entry = {
query: search.query,
kind: search.kind,
hits: hits,
selectedIdx: 0,
completedAt: Date.now(),
};
_lastSearch = entry;
_saveJson(sessionStorage, SS_SEARCH, entry);
if (_activeSearch === search) _activeSearch = null;
if (self.$el && self.$el.length) {
self.setLoading(false);
if (self.mode === 'search') {
self.renderSearchResults(hits);
}
}
}).catch(err => {
this.setLoading(false);
this.showError(err);
if (_activeSearch === search) _activeSearch = null;
if (self.$el && self.$el.length) {
self.setLoading(false);
self.showError(err);
}
});
},
runAsk: function (message) {
// A new ask invalidates whatever last-result was on screen.
_lastAsk = null;
_clear(sessionStorage, SS_ASK);
this.setLoading(true);
this.startAskProgress();
// Ask goes through the full agent loop (search + rerank + LLM),
// which routinely takes 30-90s. The default jQuery Ajax timeout
// is too short; override it here so the browser doesn't give up
// before shira-hermes responds.
Espo.Ajax.postRequest('KnowledgeBase/action/ask', {
message: message,
}, {timeout: 240000}).then(res => {
this.stopAskProgress();
this.setLoading(false);
this.renderAskAnswer(res.text || '', res.sources || []);
}).catch(err => {
this.stopAskProgress();
this.setLoading(false);
this.showError(err);
});
// Streaming via Server-Sent Events. The browser opens an
// EventSource against KnowledgeBaseAskStream which proxies
// /kb/ask/stream on shira-hermes; one SSE event per agent step
// (thinking, tool_start, tool_done) plus a final answer event.
const url = '?entryPoint=KnowledgeBaseAskStream&message='
+ encodeURIComponent(message);
const es = new EventSource(url);
_activeAsk = {
question: message,
es: es,
events: [],
startedAt: Date.now(),
};
this.startAskProgress(_activeAsk);
this._attachAskHandlers(_activeAsk);
},
startAskProgress: function () {
// Show a live elapsed-time indicator so users know the request
// is still running rather than stuck. Typical answers arrive in
// 30-90s; we warn after 2 minutes.
// Wire SSE handlers on a (possibly module-level) ask. Called from
// runAsk for the initial bind and from afterRender when the user
// navigates back to the KB view while an earlier stream is still
// open. The EventSource itself lives at module scope (in _activeAsk)
// so handlers can be re-bound without dropping the connection.
_attachAskHandlers: function (ask) {
const self = this;
ask.es.onmessage = function (msgEvent) {
let data;
try { data = JSON.parse(msgEvent.data); }
catch (e) {
console.warn('KB: malformed SSE payload', e);
return;
}
if (data.type === 'answer') {
self._finishAsk(ask, {
question: ask.question,
text: data.text || '',
sources: data.sources || [],
completedAt: Date.now(),
});
} else if (data.type === 'error') {
self._failAsk(ask, data.message || 'שגיאה בשרת');
} else {
// Status event — record it and append to the live log.
ask.events.push(Object.assign({_clientTs: Date.now()}, data));
if (self.$el && self.$el.length && self.mode === 'ask') {
self._appendAskEvent(ask, ask.events[ask.events.length - 1]);
}
}
};
ask.es.onerror = function () {
// Don't tear down on transient blips while the answer is
// already arriving; only treat it as a hard failure if no
// answer has been delivered.
if (ask._completed) return;
self._failAsk(ask, 'הזרם נסגר לפני שהגיעה תשובה');
};
},
_finishAsk: function (ask, entry) {
ask._completed = true;
try { ask.es.close(); } catch (e) { /* ignore */ }
_lastAsk = entry;
_saveJson(sessionStorage, SS_ASK, entry);
if (_activeAsk === ask) _activeAsk = null;
if (this.$el && this.$el.length) {
this.stopAskProgress();
this.setLoading(false);
if (this.mode === 'ask') {
this.renderAskAnswer(entry.text, entry.sources);
}
}
},
_failAsk: function (ask, message) {
try { ask.es.close(); } catch (e) { /* ignore */ }
if (_activeAsk === ask) _activeAsk = null;
if (this.$el && this.$el.length) {
this.stopAskProgress();
this.setLoading(false);
this.showError(new Error(message));
}
},
startAskProgress: function (askOrStartedAt) {
// Render a live progress panel: a "thinking" header with elapsed
// time, plus a stacked log of the events Shira's emitted so far
// (one line per LLM step / tool call). `askOrStartedAt` is
// either the active-ask object (from runAsk and afterRender) or
// a bare timestamp (legacy call sites). Re-mounting on view
// switch replays the accumulated event list from ask.events.
if (this.mode !== 'ask') return;
this.stopAskProgress();
let ask = null;
let startedAt;
if (askOrStartedAt && typeof askOrStartedAt === 'object') {
ask = askOrStartedAt;
startedAt = ask.startedAt || Date.now();
} else {
startedAt = askOrStartedAt || Date.now();
}
const $results = this.$el.find('.kb-results');
$results.html(
'<div class="kb-ask-progress panel panel-info">' +
' <div class="panel-body" style="direction:rtl;text-align:right;">' +
' <i class="fas fa-circle-notch fa-spin" style="margin-left:8px;"></i>' +
' <strong>שירה חושבת…</strong>' +
' <div class="small text-muted" style="margin-top:6px;">' +
' שירה מחפשת בבסיס הידע, מסכמת ומציגה ציטוטים מדויקים.' +
' זה יכול לקחת בין 30 ל-90 שניות.' +
' </div>' +
' <div class="small" style="margin-top:10px;">' +
' <span>זמן שעבר: </span>' +
' <span class="kb-elapsed">0</span>' +
' <span> שניות</span>' +
' <div style="display:flex;align-items:center;gap:8px;margin-bottom:10px;">' +
' <i class="fas fa-circle-notch fa-spin"></i>' +
' <strong>שירה חושבת</strong>' +
' <span class="text-muted small kb-elapsed-pill" ' +
'style="margin-right:auto;font-variant-numeric:tabular-nums;">0 שניות</span>' +
' </div>' +
' <div class="kb-events" style="' +
'border-top:1px solid #d8e0e8;padding-top:8px;' +
'max-height:60vh;overflow-y:auto;font-size:0.95em;"></div>' +
' </div>' +
'</div>'
);
const startedAt = Date.now();
this._askTimer = setInterval(() => {
// Replay any events already collected (matters when the user
// navigates away mid-think and comes back to a still-streaming
// ask — the stream kept going at module scope, but the DOM was
// freshly re-rendered).
if (ask && Array.isArray(ask.events)) {
ask.events.forEach(ev => this._appendAskEvent(ask, ev));
}
const tick = () => {
const sec = Math.round((Date.now() - startedAt) / 1000);
const $elapsed = this.$el.find('.kb-elapsed');
if ($elapsed.length) $elapsed.text(sec);
if (sec === 120) {
this.$el.find('.kb-ask-progress .panel-body').append(
'<div class="alert alert-warning" style="margin-top:10px;">' +
'לוקח יותר זמן מהרגיל. עדיין ממתינים…</div>'
);
}
}, 1000);
this.$el.find('.kb-elapsed-pill').text(sec + ' שניות');
};
tick();
this._askTimer = setInterval(tick, 1000);
},
_appendAskEvent: function (ask, ev) {
const $events = this.$el.find('.kb-events');
if (!$events.length) return;
const startedAt = (ask && ask.startedAt) || Date.now();
const ts = ev._clientTs || Date.now();
const sec = Math.max(0, Math.round((ts - startedAt) / 1000));
const icon = {
thinking: '🧠',
tool_start: '🔍',
tool_done: '✓',
tool_error: '⚠',
writing: '✍️',
}[ev.type] || '·';
const colour = {
thinking: '#475569',
tool_start: '#0f5298',
tool_done: '#1e6f40',
tool_error: '#a02622',
writing: '#5a3a91',
}[ev.type] || '#475569';
const label = this.escape(ev.label || ev.type);
const row = $(
'<div style="padding:3px 0;color:' + colour + ';display:flex;gap:8px;align-items:flex-start;">' +
' <span style="display:inline-block;min-width:40px;color:#94a3b8;font-variant-numeric:tabular-nums;">' +
sec + 'ש</span>' +
' <span style="margin:0 2px;">' + icon + '</span>' +
' <span style="flex:1 1 auto;word-break:break-word;">' + label + '</span>' +
'</div>'
);
$events.append(row);
// Always keep the latest line visible.
$events.scrollTop($events[0].scrollHeight);
},
stopAskProgress: function () {
@@ -165,12 +381,13 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
}
},
renderSearchResults: function (hits) {
renderSearchResults: function (hits, selectedIdx) {
const $results = this.$el.find('.kb-results');
if (!hits.length) {
$results.html('<div class="text-muted">לא נמצאו תוצאות.</div>');
return;
}
const initialIdx = (typeof selectedIdx === 'number' && hits[selectedIdx]) ? selectedIdx : 0;
const kindHe = {law: 'חוק', regulation: 'תקנה', circular: 'חוזר'};
const self = this;
@@ -188,7 +405,7 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
const pub = h.published_at ? ' · פורסם ' + h.published_at : '';
const pageLabel = (isPdfBacked(h) && h.page_number)
? ` · <span class="label label-default">עמ׳ ${h.page_number}</span>` : '';
const activeClass = i === 0 ? 'panel-primary' : 'panel-default';
const activeClass = i === initialIdx ? 'panel-primary' : 'panel-default';
return `
<div class="kb-hit panel ${activeClass}" data-hit-idx="${i}"
style="margin-bottom:8px;cursor:pointer;">
@@ -201,20 +418,15 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
</div>`;
}).join('');
$results.html(`
<div class="row" style="margin:0;">
<div class="col-md-5" style="padding-right:0;max-height:86vh;overflow-y:auto;">
${listHtml}
</div>
<div class="col-md-7 kb-search-preview" style="padding-left:0;">
<div class="kb-preview-slot"></div>
</div>
</div>
`);
$results.html(this._buildSplitShell({
leftHtml: `<div class="kb-search-list" style="height:100%;overflow-y:auto;padding-left:6px;">${listHtml}</div>`,
rightHtml: '<div class="kb-preview-slot" style="height:100%;"></div>',
}));
this._wireSplitter($results.find('.kb-split'));
// Cache hits on the view so click handlers can reach them.
this._searchHits = hits;
this.showSearchPreview(0);
this.showSearchPreview(initialIdx);
this.$el.find('.kb-hit').off('click').on('click', function () {
const idx = parseInt($(this).data('hit-idx'), 10);
@@ -223,6 +435,90 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
.removeClass('panel-primary').addClass('panel-default');
$(this).removeClass('panel-default').addClass('panel-primary');
self.showSearchPreview(idx);
if (_lastSearch && _lastSearch.hits === self._searchHits) {
_lastSearch.selectedIdx = idx;
_saveJson(sessionStorage, SS_SEARCH, _lastSearch);
}
});
},
// Returns the HTML for a flex-row split layout with a draggable
// 6px handle. The container's flex direction is forced LTR so the
// splitter math (clientX) doesn't get inverted by RTL — content
// inside each pane keeps its own direction.
_buildSplitShell: function ({leftHtml, rightHtml}) {
const pct = this._loadSplitPct();
return (
'<div class="kb-split" style="' +
'display:flex;flex-direction:row;direction:ltr;' +
'gap:0;height:86vh;align-items:stretch;">' +
' <div class="kb-split-left" style="' +
'flex:0 0 ' + pct + '%;min-width:200px;overflow:hidden;">' +
leftHtml +
' </div>' +
' <div class="kb-split-handle" title="גרור כדי לשנות חלוקה" style="' +
'flex:0 0 6px;background:#dde1e7;cursor:col-resize;' +
'border-left:1px solid #cbd0d8;border-right:1px solid #cbd0d8;' +
'transition:background 0.15s;"></div>' +
' <div class="kb-split-right" style="' +
'flex:1 1 auto;min-width:200px;overflow:hidden;">' +
rightHtml +
' </div>' +
'</div>'
);
},
_loadSplitPct: function () {
try {
const v = parseFloat(localStorage.getItem(LS_SPLIT));
if (!isNaN(v) && v >= 15 && v <= 85) return v;
} catch (e) { /* ignore */ }
return 42;
},
_saveSplitPct: function (pct) {
try { localStorage.setItem(LS_SPLIT, String(pct)); }
catch (e) { /* ignore */ }
},
// Wires mousedown/move/up on the handle to resize the left pane.
// While dragging we cover any iframe with a transparent overlay so
// mouse events don't disappear into PDF.js. On mouseup the chosen
// ratio persists to localStorage.
_wireSplitter: function ($split) {
if (!$split.length) return;
const $handle = $split.find('.kb-split-handle');
const $left = $split.find('.kb-split-left');
const self = this;
$handle.on('mouseenter', () => $handle.css('background', '#b9c0cb'));
$handle.on('mouseleave', () => $handle.css('background', '#dde1e7'));
$handle.on('mousedown', function (e) {
e.preventDefault();
const rect = $split[0].getBoundingClientRect();
const $overlay = $('<div class="kb-drag-overlay">').css({
position: 'fixed',
inset: '0',
cursor: 'col-resize',
zIndex: 9999,
background: 'transparent',
});
$('body').append($overlay).css('user-select', 'none');
const onMove = function (ev) {
const x = ev.clientX - rect.left;
let pct = (x / rect.width) * 100;
pct = Math.max(15, Math.min(85, pct));
$left.css('flex', '0 0 ' + pct + '%');
self._dragPct = pct;
};
const onUp = function () {
$(document).off('mousemove', onMove).off('mouseup', onUp);
$overlay.remove();
$('body').css('user-select', '');
if (typeof self._dragPct === 'number') {
self._saveSplitPct(self._dragPct);
}
};
$(document).on('mousemove', onMove).on('mouseup', onUp);
});
},
@@ -239,34 +535,126 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
+ encodeURIComponent(h.source_id)
+ '#page=' + encodeURIComponent(page);
$slot.html(`
<div style="direction:rtl;text-align:right;margin-bottom:6px;">
<span class="text-muted small">מציג עמ׳ ${page} של המסמך</span>
<div style="display:flex;flex-direction:column;height:100%;">
<div style="direction:rtl;text-align:right;margin-bottom:6px;flex:0 0 auto;">
<span class="text-muted small">מציג עמ׳ ${page} של המסמך</span>
</div>
<iframe class="kb-pdf-iframe" src="${url}"
style="flex:1 1 auto;width:100%;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
</div>
<iframe class="kb-pdf-iframe"
src="${url}"
style="width:100%;height:82vh;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
`);
return;
}
// Text source (law from Wikisource) — show the full chunk content
// and a link to the public source page.
// Text source (law from Wikisource): fetch the matched chunk +
// surrounding sections so the preview pane shows context, not a
// copy of what's already in the left card. While loading, show a
// placeholder; on failure fall back to the chunk-only view.
this._renderSectionContextPlaceholder(h);
const token = this._previewToken;
const sourceId = h.source_id;
const around = (typeof h.chunk_index === 'number') ? h.chunk_index : null;
if (sourceId == null || around == null) {
this._renderSectionContextSingle(h);
return;
}
const self = this;
Espo.Ajax.getRequest('KnowledgeBase/action/chunks', {
sourceId: sourceId,
around: around,
ctx: 2,
}).then(res => {
// Bail if user has clicked a different hit while we were waiting.
if (self._previewToken !== token) return;
self._renderSectionContext(h, res && res.chunks ? res.chunks : []);
}).catch(err => {
console.warn('KB: section-window fetch failed', err);
if (self._previewToken === token) self._renderSectionContextSingle(h);
});
},
_renderSectionContextPlaceholder: function (h) {
this._previewToken = (this._previewToken || 0) + 1;
const $slot = this.$el.find('.kb-preview-slot');
const title = this.escape(h.title || '');
$slot.html(
'<div class="panel panel-default" style="direction:rtl;text-align:right;height:100%;display:flex;flex-direction:column;">' +
' <div class="panel-heading" style="flex:0 0 auto;">' + title +
' <span class="text-muted small" style="margin-right:8px;">— טוען הקשר…</span>' +
' </div>' +
' <div class="panel-body" style="flex:1 1 auto;overflow-y:auto;">' +
' <i class="fas fa-circle-notch fa-spin"></i>' +
' </div>' +
'</div>'
);
},
_renderSectionContextSingle: function (h) {
// Fallback when we can't fetch the window — render just the
// matched chunk, which is at least the full section for the
// statute chunker (1 section = 1 chunk).
this._renderSectionContext(h, [{
chunk_index: h.chunk_index,
section_ref: h.section_ref || '',
heading_path: h.heading_path || '',
content: h.content || '',
}]);
},
_renderSectionContext: function (h, chunks) {
const $slot = this.$el.find('.kb-preview-slot');
const bodyStyle =
'direction:rtl;unicode-bidi:plaintext;white-space:pre-wrap;' +
'font-family:"Segoe UI",Arial,sans-serif;line-height:1.7;text-align:right;';
const srcLink = h.source_url
? `<a href="${this.escape(h.source_url)}" target="_blank" rel="noopener">מקור ב-Wikisource</a>`
? '<a href="' + this.escape(h.source_url) + '" target="_blank" rel="noopener" ' +
'class="btn btn-xs btn-default" style="margin-right:6px;">' +
'<i class="fas fa-external-link-alt"></i> פתח ב-Wikisource</a>'
: '';
$slot.html(`
<div class="panel panel-default" style="direction:rtl;text-align:right;">
<div class="panel-heading">${this.escape(h.title || '')}</div>
<div class="panel-body" style="${bodyStyle};max-height:80vh;overflow-y:auto;">
${this.escape(h.content || '')}
</div>
${srcLink ? '<div class="panel-footer small">' + srcLink + '</div>' : ''}
</div>
`);
// The matched chunk is highlighted; siblings render as muted
// context blocks above and below so the user can read into the
// section without leaving the CRM.
const sectionsHtml = chunks.map(c => {
const isMatch = c.chunk_index === h.chunk_index;
const ref = c.section_ref || c.heading_path || '';
const headerStyle = isMatch
? 'background:#fff3cd;border-bottom:1px solid #ffe69c;color:#664d03;font-weight:600;'
: 'background:#f5f5f7;border-bottom:1px solid #e7e7ea;color:#6c757d;';
return (
'<div style="margin-bottom:12px;border:1px solid ' +
(isMatch ? '#ffe69c' : '#e7e7ea') + ';border-radius:4px;overflow:hidden;">' +
' <div style="' + headerStyle + 'padding:6px 10px;direction:rtl;text-align:right;font-size:0.9em;">' +
(isMatch ? 'התאמה — ' : '') + this.escape(ref) +
' </div>' +
' <div style="' + bodyStyle + ';padding:10px;">' +
this.escape(c.content || '') +
' </div>' +
'</div>'
);
}).join('');
$slot.html(
'<div class="panel panel-default" style="direction:rtl;text-align:right;height:100%;display:flex;flex-direction:column;">' +
' <div class="panel-heading" style="flex:0 0 auto;display:flex;justify-content:space-between;align-items:center;">' +
' <div><strong>' + this.escape(h.title || '') + '</strong>' +
' <span class="text-muted small" style="margin-right:6px;">— הסעיף בהקשרו</span>' +
' </div>' +
srcLink +
' </div>' +
' <div class="panel-body" style="flex:1 1 auto;overflow-y:auto;background:#fafbfc;">' +
sectionsHtml +
' </div>' +
'</div>'
);
// Scroll the matched chunk into view inside the preview pane.
const $body = $slot.find('.panel-body');
const $match = $body.find('div[style*="#fff3cd"]').closest('div[style*="border-radius:4px"]');
if ($match.length && $body.length) {
const offset = $match.position().top - 20;
$body.scrollTop(Math.max(0, offset));
}
},
renderAskAnswer: function (text, sources) {
@@ -353,32 +741,32 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
+ encodeURIComponent(first.source_id)
+ '#page=' + encodeURIComponent(firstPage);
$results.html(`
<div class="row" style="margin:0;">
<div class="col-md-6" style="padding-right:0;">
<div class="kb-answer panel panel-info">
<div class="panel-body" style="direction:rtl;text-align:right;max-height:80vh;overflow-y:auto;">
${answerHtml}
</div>
</div>
<div class="panel panel-default" style="direction:rtl;text-align:right;">
<div class="panel-heading"><strong>📎 מקורות</strong></div>
<div class="panel-body">
<div style="margin-bottom:6px;">${pickerItems}</div>
<div class="kb-page-links" style="font-size:0.9em;">
<span class="text-muted">עמודים רלוונטיים:</span> ${pageLinks}
</div>
</div>
const leftHtml = `
<div style="height:100%;overflow-y:auto;padding-left:6px;">
<div class="kb-answer panel panel-info">
<div class="panel-body" style="direction:rtl;text-align:right;">
${answerHtml}
</div>
</div>
<div class="col-md-6" style="padding-left:0;">
<iframe class="kb-pdf-iframe"
src="${initialUrl}"
style="width:100%;height:86vh;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
<div class="panel panel-default" style="direction:rtl;text-align:right;">
<div class="panel-heading"><strong>📎 מקורות</strong></div>
<div class="panel-body">
<div style="margin-bottom:6px;">${pickerItems}</div>
<div class="kb-page-links" style="font-size:0.9em;">
<span class="text-muted">עמודים רלוונטיים:</span> ${pageLinks}
</div>
</div>
</div>
</div>
`);
`;
const rightHtml = `
<iframe class="kb-pdf-iframe"
src="${initialUrl}"
style="width:100%;height:100%;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
`;
$results.html(this._buildSplitShell({leftHtml, rightHtml}));
this._wireSplitter($results.find('.kb-split'));
const self = this;
this.$el.find('.kb-pdf-pick').off('click').on('click', function () {
@@ -73,7 +73,12 @@ class KnowledgeBase
throw new BadRequest('sourceId (numeric) is required.');
}
return $this->getService()->sourceChunks((int) $sourceId);
$around = $request->getQueryParam('around');
$ctx = $request->getQueryParam('ctx');
$aroundInt = ($around !== null && is_numeric($around)) ? (int) $around : null;
$ctxInt = ($ctx !== null && is_numeric($ctx)) ? (int) $ctx : 2;
return $this->getService()->sourceChunks((int) $sourceId, $aroundInt, $ctxInt);
}
public function postActionAsk(Request $request, Response $response): array
@@ -0,0 +1,128 @@
<?php
namespace Espo\Modules\KnowledgeBase\EntryPoints;
use Espo\Core\Api\Request;
use Espo\Core\Api\Response;
use Espo\Core\EntryPoint\EntryPoint;
use Espo\Core\Exceptions\BadRequest;
use Espo\Core\Exceptions\Error;
use Espo\Core\Exceptions\Forbidden;
use Espo\Core\InjectableFactory;
use Espo\Entities\User;
use Espo\Modules\KnowledgeBase\Services\KnowledgeBaseService;
/**
* Streams Server-Sent Events from shira-hermes /kb/ask/stream to the
* browser's EventSource. The browser opens this with a GET (EventSource
* has no POST), passes the question via ?message=, and we forward it
* to shira-hermes with the firm-wide API key — the credential never
* leaves the EspoCRM server.
*
* Response is text/event-stream with no buffering. We bypass EspoCRM's
* Response wrapper entirely (echo + flush + exit) because the framework
* is wired for buffered JSON responses, not infinite streams.
*
* Access: `?entryPoint=KnowledgeBaseAskStream&message=<urlencoded>`
*/
class KnowledgeBaseAskStream implements EntryPoint
{
public function __construct(
private InjectableFactory $injectableFactory,
private User $user,
) {}
public function run(Request $request, Response $response): void
{
if ($this->user->isPortal()) {
throw new Forbidden('Portal users have no KB access.');
}
$message = $request->getQueryParam('message');
if (!is_string($message) || trim($message) === '') {
throw new BadRequest('message (string) is required.');
}
if (strlen($message) > 2000) {
throw new BadRequest('message is too long.');
}
$service = $this->injectableFactory->create(KnowledgeBaseService::class);
try {
$upstreamUrl = $service->getStreamingUrl('/kb/ask/stream');
$apiKey = $service->getStreamingApiKey();
} catch (\Throwable $e) {
throw new Error('SmartAssistant integration is not configured: ' . $e->getMessage());
}
// Disable PHP/Apache output buffering and compression so each SSE
// event flushes to the wire as soon as it arrives from upstream.
// Order matters: clean any open buffers BEFORE setting headers.
while (ob_get_level() > 0) {
@ob_end_clean();
}
@ini_set('zlib.output_compression', 'Off');
@ini_set('output_buffering', 'Off');
@ini_set('implicit_flush', '1');
if (function_exists('apache_setenv')) {
@apache_setenv('no-gzip', '1');
}
ignore_user_abort(false);
@set_time_limit(300);
header('Content-Type: text/event-stream; charset=utf-8');
header('Cache-Control: no-cache, no-transform');
header('X-Accel-Buffering: no');
header('Connection: keep-alive');
// Defense in depth: same locked-down CSP we use for the PDF entry.
header("Content-Security-Policy: default-src 'self'");
$payload = json_encode(['message' => $message], JSON_UNESCAPED_UNICODE);
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => $upstreamUrl,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $payload,
CURLOPT_HTTPHEADER => [
'Content-Type: application/json',
'Accept: text/event-stream',
'X-Api-Key: ' . $apiKey,
],
CURLOPT_RETURNTRANSFER => false,
CURLOPT_BUFFERSIZE => 1024,
CURLOPT_TIMEOUT => 300,
CURLOPT_CONNECTTIMEOUT => 10,
CURLOPT_WRITEFUNCTION => function ($ch, $data) {
if (connection_aborted()) {
// Browser closed the EventSource — bail so cURL stops.
return -1;
}
echo $data;
@ob_flush();
@flush();
return strlen($data);
},
]);
$ok = curl_exec($ch);
$errno = curl_errno($ch);
$errmsg = curl_error($ch);
curl_close($ch);
if (!$ok && $errno !== 23 /* aborted by callback */) {
// Best effort: if upstream errored before any data was
// flushed, surface a final SSE error event so the client
// doesn't silently spin.
echo "data: " . json_encode(
['type' => 'error', 'message' => 'upstream: ' . $errmsg],
JSON_UNESCAPED_UNICODE,
) . "\n\n";
@ob_flush();
@flush();
}
// We've fully written the response. Bypass EspoCRM's Response
// emit() to avoid double-headers / extra buffering.
exit;
}
}
@@ -1,5 +1,8 @@
{
"KnowledgeBasePdf": {
"className": "Espo\\Modules\\KnowledgeBase\\EntryPoints\\KnowledgeBasePdf"
},
"KnowledgeBaseAskStream": {
"className": "Espo\\Modules\\KnowledgeBase\\EntryPoints\\KnowledgeBaseAskStream"
}
}
@@ -48,9 +48,13 @@ class KnowledgeBaseService
return $this->get('/kb/sources');
}
public function sourceChunks(int $sourceId): array
public function sourceChunks(int $sourceId, ?int $around = null, int $ctx = 2): array
{
return $this->get('/kb/source/' . $sourceId . '/chunks');
$path = '/kb/source/' . $sourceId . '/chunks';
if ($around !== null) {
$path .= '?around=' . $around . '&ctx=' . $ctx;
}
return $this->get($path);
}
public function ask(string $message, ?string $conversationId, array $history): array
@@ -136,6 +140,25 @@ class KnowledgeBaseService
return $data->apiKey ?? null;
}
/**
* Public accessors so the streaming EntryPoint can build a cURL
* connection to shira-hermes without going through the JSON request()
* helper (which buffers the whole body).
*/
public function getStreamingUrl(string $path): string
{
return rtrim($this->getBaseUrl(), '/') . $path;
}
public function getStreamingApiKey(): string
{
$key = $this->getApiKey();
if (!$key) {
throw new Error('SmartAssistant API key is not configured.');
}
return $key;
}
private function get(string $path): array
{
return $this->request('GET', $path, null);
+2 -2
View File
@@ -1,14 +1,14 @@
{
"name": "KnowledgeBase",
"module": "KnowledgeBase",
"version": "0.1.8",
"version": "0.2.0",
"acceptableVersions": [
">=8.0.0"
],
"php": [
">=8.1"
],
"releaseDate": "2026-04-24",
"releaseDate": "2026-04-25",
"author": "klear",
"description": "Knowledge Base — Israeli National Insurance law, regulations, and circulars. Hybrid search + ask-shira, powered by shira-hermes KB."
}