3 Commits

Author SHA1 Message Date
chaim a00fc3729c feat(kb/ask): live SSE streaming of Shira's progress (v0.2.0)
User wanted to see what Shira is doing while she thinks — like Claude
does — instead of staring at an incrementing seconds counter. Now the
ask UI shows one line per agent step as it happens:

  3s  🧠  קוראת את השאלה
  4s  🔍  מחפשת בבסיס הידע: "תקנה 36"
  9s  ✓   נמצאו 5 קטעים רלוונטיים
 10s  🔍  מחפשת בבסיס הידע: "תקנה 36 ביטוח לאומי"
 14s  ✓   נמצאו 5 קטעים רלוונטיים
 15s  🧠  ממשיכה לחקור
 22s  ✍️  מנסחת תשובה
 → תשובה

Client wiring:
- runAsk now opens an EventSource at ?entryPoint=KnowledgeBaseAskStream
  instead of POSTing JSON. Each SSE event of type thinking/tool_start/
  tool_done/tool_error/writing appends a row to a stacked progress log.
  The final {type:answer,text,sources} event triggers renderAskAnswer
  with the existing split view. {type:error} surfaces via showError.
- Module-level state from v0.1.9 unchanged in spirit but now stores the
  EventSource + accumulated events. afterRender on view re-mount replays
  the entire event list and re-binds onmessage/onerror — the SSE
  connection itself doesn't drop because it lives at module scope.
- Elapsed-time pill stays for cadence but is decoration; the real
  signal is the per-event log.

EspoCRM proxy:
- New EntryPoint KnowledgeBaseAskStream — required because EventSource
  is GET-only and ?message= must come via the URL. PHP buffering is
  disabled (zlib + ob + apache_setenv no-gzip), Apache headers are set
  for SSE (no-cache, X-Accel-Buffering: no), and we cURL into
  shira-hermes /kb/ask/stream with CURLOPT_WRITEFUNCTION echoing each
  byte and flushing immediately. The handler ends with exit; to bypass
  EspoCRM's Response wrapper which would otherwise emit headers/body
  on top of our raw stream.
- KnowledgeBaseService gained two public accessors (getStreamingUrl,
  getStreamingApiKey) so the EntryPoint can build the cURL without
  going through request() (which buffers the whole body).
- entryPoints.json registers the new class.

Server (shira-hermes 76fd77f):
- AgentRunner.run accepts an on_event sync callback fired before each
  LLM call (thinking) and around each tool call (tool_start, tool_done,
  tool_error). Hebrew labels in agent_runner._tool_label / _tool_done_label
  ship over the wire ready-to-display — keeps the client free of
  translation concerns.
- POST /kb/ask/stream wraps the existing agent runner in an asyncio
  task feeding an asyncio.Queue, returns StreamingResponse with proper
  SSE headers (X-Accel-Buffering, Cache-Control). 15s comment-line
  heartbeats keep the connection open through proxy idle-timeouts.
- Original POST /kb/ask still works — additive change.

Refs Task Master #8
2026-04-25 13:00:58 +00:00
chaim 2fe10727eb fix(kb): search preview shows section-in-context for text sources
User reported the search card "looks broken" when the selected hit is the
law (Wikisource source 5) — the right preview pane just duplicates the
chunk text already visible in the left list card. For PDF sources the
right pane shows the PDF page, but for text-only sources it had nothing
useful to add and fell back to a copy.

Fix:
- /kb/search hits gained chunk_index (shira-hermes b127caf), and
  /kb/source/{id}/chunks gained an `around=<chunk_index>&ctx=<k>` window.
- Espo proxy plumbing: getActionChunks reads `around`/`ctx` query
  params and forwards them; KnowledgeBaseService::sourceChunks accepts
  optional $around + $ctx and appends them to the upstream URL.
- showSearchPreview for non-PDF hits now fetches the matched chunk plus
  2 sections before and 2 after, renders them as stacked panels: the
  matched section gets a yellow #fff3cd highlight, neighbors render as
  muted context blocks. The panel header carries a prominent
  "פתח ב-Wikisource" button so users can jump to the public source page.
  After render the matched section auto-scrolls into view.
- Race protection via _previewToken: a click on a different hit while a
  fetch is in flight cancels the stale render so the user always sees
  the section that matches the currently-selected card.

PDF path unchanged.

Refs Task Master #7
2026-04-25 12:28:50 +00:00
chaim ac5823ffe5 feat(kb): drag-to-resize splitter + search survives view switch
Two requests on top of v0.1.9:

1. Drag-to-resize splitter
   The previous fixed Bootstrap col-md-5/col-md-7 split locked the
   user into 41.66% answer / 58.33% PDF. Replaced with a flex-row
   layout containing a 6px drag handle. mousedown spawns a fixed
   transparent overlay so the PDF.js iframe doesn't swallow mouse
   events while dragging; mouseup persists the chosen ratio (clamped
   15-85%) to localStorage as kb-split-pct, so the next page-load and
   the next session both come back to the same layout. Both the
   ask split view and the search split view share _buildSplitShell +
   _wireSplitter — single source of truth for the geometry.

2. Search survives a view switch
   Search had the same view-bound promise problem ask had before
   v0.1.9. Hoisted the in-flight request to a module-level
   _activeSearch and the most recent {query, kind, hits, selectedIdx}
   to _lastSearch + sessionStorage. afterRender restores the input,
   the kind filter, the hit list (with the same hit highlighted), and
   the right-pane preview. Click handler updates selectedIdx so a
   round-trip lands on the user's last-clicked hit, not always on the
   top-ranked one.

The persistence helpers grew a tiny refactor: _loadJson/_saveJson/_clear
replace the ask-only SS_KEY shim and now back both ask + search keys.

Refs Task Master #5, #6
2026-04-25 10:46:39 +00:00
7 changed files with 653 additions and 142 deletions
+48
View File
@@ -37,6 +37,54 @@
"dependencies": [],
"createdAt": "2026-04-25T10:00:00Z"
},
{
"id": 8,
"title": "feat(kb/ask): SSE streaming with live agent progress",
"description": "User wanted Shira's progress to be visible while she thinks — like Claude does — instead of just an incrementing seconds counter. Implementation: AgentRunner.run accepts an optional sync on_event callback that emits {type, label, ...} dicts at each agent step. New POST /kb/ask/stream returns text/event-stream, with the runner running in an asyncio task whose events feed an asyncio.Queue that the response generator drains as SSE. Client opens EventSource against a new EspoCRM EntryPoint KnowledgeBaseAskStream that proxies the SSE bytes through cURL with output buffering disabled (echo + flush + exit, bypassing the framework Response). The progress UI now stacks one line per event (thinking, tool_start with the query Shira chose, tool_done with the chunk count, writing) instead of just a seconds counter. v0.1.9 _activeAsk persistence pattern preserved — events accumulate at module scope so a view switch + return replays the live log.",
"status": "done",
"priority": "high",
"details": "shira-hermes 76fd77f. EspoCRM EntryPoint registered in metadata/app/entryPoints.json. Hebrew labels live in agent_runner._tool_label / _tool_done_label so the wire format already carries display-ready strings (no client-side mapping). 15s heartbeats keep the connection open through proxy idle-timeouts.",
"testStrategy": "curl -N POST /kb/ask/stream {message:'מהי תקנה 36'} should produce data: lines with thinking → tool_start (with query) → tool_done (with chunk count) → ... → writing → answer. Browser EventSource should drive the same flow into the UI.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T13:00:00Z"
},
{
"id": 7,
"title": "fix: search preview duplicates left card for text sources",
"description": "User report: searching the law (Wikisource source 5) shows the same chunk content in both the left list-card and the right preview pane. The preview adds zero value when the matched source has no PDF — both panes are visually identical (modulo the left's 14em truncation), which reads as a bug. Fixed by fetching the matched chunk plus K=2 surrounding sections from /kb/source/{id}/chunks?around=N&ctx=2 and rendering them stacked: matched section is highlighted with a yellow header (#fff3cd), neighbors render as muted context blocks above and below. Adds a prominent 'פתח ב-Wikisource' button in the panel header, scrolls the matched section into view on render. Race-safe via _previewToken so a quick second click doesn't render stale results.",
"status": "done",
"priority": "high",
"details": "shira-hermes b127caf added chunk_index to /kb/search hits + an `around`/`ctx` query param to /source/{id}/chunks. Espo controller passes them through. Client uses the new fields in showSearchPreview when the hit is a text source.",
"testStrategy": "Search 'מהי תקנה 36'; click the law-kind hit. Right pane: yellow-highlighted ס' 36 in the middle, ס' 35 above (muted), ס' 37 below (muted), Wikisource button in header. Click the תקנה 37 חוזר hit: right pane swaps to PDF iframe at the cited page (PDF path unchanged).",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T12:30:00Z"
},
{
"id": 5,
"title": "feat(kb): drag-to-resize splitter (both ask + search)",
"description": "Replace the fixed Bootstrap col-md-5/7 split with a flex layout containing a 6px drag handle. mousedown on the handle starts the drag; while dragging an invisible full-screen overlay covers any iframe so the PDF.js viewer doesn't swallow mouse events. The chosen ratio (15-85%) persists to localStorage as kb-split-pct so the layout sticks across sessions. Both modes share the same _buildSplitShell helper so the look stays consistent.",
"status": "done",
"priority": "normal",
"details": "Default ratio = 42% left (matches the previous col-md-5 ≈ 41.66%). Applied to renderSearchResults, renderAskAnswer, and showSearchPreview (which now uses 100%-height containers via flex column).",
"testStrategy": "Open ask mode, drag the divider; verify PDF iframe doesn't lose mouse focus mid-drag; reload — divider stays at last position. Repeat in search mode.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T11:00:00Z"
},
{
"id": 6,
"title": "fix: search survives view switch (parity with ask in v0.1.9)",
"description": "Search mode had the same view-bound promise problem as ask did before v0.1.9: navigating to another EspoCRM screen and back blanked the results. Hoisted {query, kind, promise} to a module-level _activeSearch with the same {query, kind, hits, selectedIdx} sessionStorage replay we already built for ask. Selected hit index also persists so the right pane comes back to the same source/page the user had open.",
"status": "done",
"priority": "high",
"details": "Mirrors the v0.1.9 ask fix in _attachSearchHandlers + afterRender. Click handler updates _lastSearch.selectedIdx so a return-trip lands on the user's last-clicked hit, not always the top-ranked one.",
"testStrategy": "Search 'מהי תקנה 37'; click a result other than the first; navigate to Contacts; navigate back — same hit selected, same PDF page on the right.",
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T11:00:00Z"
},
{
"id": 4,
"title": "fix: ask survives view switch via module-level promise + sessionStorage",
@@ -6,28 +6,36 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
// it. sessionStorage backs the last completed answer so a hard reload
// also recovers it. None of this leaks to the next browser session —
// ask history is intentionally ephemeral.
let _activeAsk = null; // {question, promise} while a request is in flight
let _lastAsk = null; // {question, text, sources, completedAt} after success
let _activeAsk = null; // {question, promise, startedAt} while in flight
let _lastAsk = null; // {question, text, sources, completedAt} after success
let _activeSearch = null; // {query, kind, promise} while in flight
let _lastSearch = null; // {query, kind, hits, selectedIdx, completedAt}
const SS_KEY = 'kb-last-ask';
const SS_ASK = 'kb-last-ask';
const SS_SEARCH = 'kb-last-search';
const LS_SPLIT = 'kb-split-pct'; // long-lived UI preference (not session)
function _loadLastFromSession() {
function _loadJson(storage, key, validator) {
try {
const raw = sessionStorage.getItem(SS_KEY);
const raw = storage.getItem(key);
if (!raw) return null;
const parsed = JSON.parse(raw);
if (parsed && typeof parsed.question === 'string') return parsed;
} catch (e) { /* corrupted, ignore */ }
return null;
return validator(parsed) ? parsed : null;
} catch (e) { return null; }
}
function _saveLastToSession(entry) {
try { sessionStorage.setItem(SS_KEY, JSON.stringify(entry)); }
function _saveJson(storage, key, value) {
try { storage.setItem(key, JSON.stringify(value)); }
catch (e) { /* quota or disabled, fine */ }
}
function _clearLastInSession() {
try { sessionStorage.removeItem(SS_KEY); } catch (e) { /* ignore */ }
function _clear(storage, key) {
try { storage.removeItem(key); } catch (e) { /* ignore */ }
}
if (!_lastAsk) {
_lastAsk = _loadJson(sessionStorage, SS_ASK, x => x && typeof x.question === 'string');
}
if (!_lastSearch) {
_lastSearch = _loadJson(sessionStorage, SS_SEARCH, x => x && typeof x.query === 'string' && Array.isArray(x.hits));
}
if (!_lastAsk) _lastAsk = _loadLastFromSession();
return Dep.extend({
@@ -98,22 +106,37 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
this.renderSourcesList();
}
// Recover ask state across view remounts (user navigated away and
// back). _activeAsk winsthere's still a request in flight, so
// re-show the progress panel and re-bind handlers. Otherwise
// _lastAsk replays the most recent answer in this browser session.
// Recover prior state across view remounts (user navigated away and
// back). Active in-flight wins; otherwise replay the last result
// from sessionStorage so the work the user already did doesn't
// disappear behind a tab switch.
if (this.mode === 'ask') {
if (_activeAsk) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_activeAsk.question);
this.setLoading(true);
this.startAskProgress(_activeAsk.startedAt);
// Pass the whole ask so startAskProgress can replay the
// accumulated event log into the freshly-rendered DOM.
this.startAskProgress(_activeAsk);
this._attachAskHandlers(_activeAsk);
} else if (_lastAsk) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_lastAsk.question);
this.renderAskAnswer(_lastAsk.text, _lastAsk.sources);
}
} else if (this.mode === 'search') {
if (_activeSearch) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_activeSearch.query);
this.setLoading(true);
this._attachSearchHandlers(_activeSearch);
} else if (_lastSearch && _lastSearch.hits.length) {
const $input = this.$el.find('input[data-name="query"]');
if ($input.length) $input.val(_lastSearch.query);
const $kind = this.$el.find('select[data-name="kind"]');
if ($kind.length && _lastSearch.kind) $kind.val(_lastSearch.kind);
this.renderSearchResults(_lastSearch.hits, _lastSearch.selectedIdx || 0);
}
}
// Focus the input on every render so users can start typing right away.
@@ -135,113 +158,222 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
},
runSearch: function (query, kind) {
// A new search invalidates the prior cached one.
_lastSearch = null;
_clear(sessionStorage, SS_SEARCH);
this.setLoading(true);
Espo.Ajax.postRequest('KnowledgeBase/action/search', {
const promise = Espo.Ajax.postRequest('KnowledgeBase/action/search', {
query: query,
kind: kind,
topK: 8,
}).then(res => {
this.setLoading(false);
this.renderSearchResults(res.hits || []);
}).catch(err => {
this.setLoading(false);
this.showError(err);
});
_activeSearch = {query: query, kind: kind, promise: promise};
this._attachSearchHandlers(_activeSearch);
},
runAsk: function (message) {
// A new ask invalidates whatever last-result was on screen.
_lastAsk = null;
_clearLastInSession();
this.setLoading(true);
// Ask goes through the full agent loop (search + rerank + LLM),
// which routinely takes 30-90s. The default jQuery Ajax timeout
// is too short; override it here so the browser doesn't give up
// before shira-hermes responds.
const promise = Espo.Ajax.postRequest('KnowledgeBase/action/ask', {
message: message,
}, {timeout: 240000});
_activeAsk = {question: message, promise: promise, startedAt: Date.now()};
this.startAskProgress(_activeAsk.startedAt);
this._attachAskHandlers(_activeAsk);
},
// Wire .then/.catch on a (possibly module-level) ask promise to
// whichever view is currently mounted. Called from runAsk for the
// initial bind and from afterRender when the user navigates back to
// the KB view while an earlier request is still in flight.
_attachAskHandlers: function (ask) {
_attachSearchHandlers: function (search) {
const self = this;
ask.promise.then(res => {
search.promise.then(res => {
const hits = res.hits || [];
const entry = {
question: ask.question,
text: res.text || '',
sources: res.sources || [],
query: search.query,
kind: search.kind,
hits: hits,
selectedIdx: 0,
completedAt: Date.now(),
};
_lastAsk = entry;
_saveLastToSession(entry);
if (_activeAsk === ask) _activeAsk = null;
_lastSearch = entry;
_saveJson(sessionStorage, SS_SEARCH, entry);
if (_activeSearch === search) _activeSearch = null;
if (self.$el && self.$el.length) {
self.stopAskProgress();
self.setLoading(false);
if (self.mode === 'ask') {
self.renderAskAnswer(entry.text, entry.sources);
if (self.mode === 'search') {
self.renderSearchResults(hits);
}
}
}).catch(err => {
if (_activeAsk === ask) _activeAsk = null;
if (_activeSearch === search) _activeSearch = null;
if (self.$el && self.$el.length) {
self.stopAskProgress();
self.setLoading(false);
self.showError(err);
}
});
},
startAskProgress: function (startedAt) {
// Show a live elapsed-time indicator so users know the request
// is still running rather than stuck. Typical answers arrive in
// 30-90s; we warn after 2 minutes. `startedAt` is passed in so
// a request that began before a view re-mount keeps showing the
// accumulated elapsed time, not 0.
runAsk: function (message) {
// A new ask invalidates whatever last-result was on screen.
_lastAsk = null;
_clear(sessionStorage, SS_ASK);
this.setLoading(true);
// Streaming via Server-Sent Events. The browser opens an
// EventSource against KnowledgeBaseAskStream which proxies
// /kb/ask/stream on shira-hermes; one SSE event per agent step
// (thinking, tool_start, tool_done) plus a final answer event.
const url = '?entryPoint=KnowledgeBaseAskStream&message='
+ encodeURIComponent(message);
const es = new EventSource(url);
_activeAsk = {
question: message,
es: es,
events: [],
startedAt: Date.now(),
};
this.startAskProgress(_activeAsk);
this._attachAskHandlers(_activeAsk);
},
// Wire SSE handlers on a (possibly module-level) ask. Called from
// runAsk for the initial bind and from afterRender when the user
// navigates back to the KB view while an earlier stream is still
// open. The EventSource itself lives at module scope (in _activeAsk)
// so handlers can be re-bound without dropping the connection.
_attachAskHandlers: function (ask) {
const self = this;
ask.es.onmessage = function (msgEvent) {
let data;
try { data = JSON.parse(msgEvent.data); }
catch (e) {
console.warn('KB: malformed SSE payload', e);
return;
}
if (data.type === 'answer') {
self._finishAsk(ask, {
question: ask.question,
text: data.text || '',
sources: data.sources || [],
completedAt: Date.now(),
});
} else if (data.type === 'error') {
self._failAsk(ask, data.message || 'שגיאה בשרת');
} else {
// Status event — record it and append to the live log.
ask.events.push(Object.assign({_clientTs: Date.now()}, data));
if (self.$el && self.$el.length && self.mode === 'ask') {
self._appendAskEvent(ask, ask.events[ask.events.length - 1]);
}
}
};
ask.es.onerror = function () {
// Don't tear down on transient blips while the answer is
// already arriving; only treat it as a hard failure if no
// answer has been delivered.
if (ask._completed) return;
self._failAsk(ask, 'הזרם נסגר לפני שהגיעה תשובה');
};
},
_finishAsk: function (ask, entry) {
ask._completed = true;
try { ask.es.close(); } catch (e) { /* ignore */ }
_lastAsk = entry;
_saveJson(sessionStorage, SS_ASK, entry);
if (_activeAsk === ask) _activeAsk = null;
if (this.$el && this.$el.length) {
this.stopAskProgress();
this.setLoading(false);
if (this.mode === 'ask') {
this.renderAskAnswer(entry.text, entry.sources);
}
}
},
_failAsk: function (ask, message) {
try { ask.es.close(); } catch (e) { /* ignore */ }
if (_activeAsk === ask) _activeAsk = null;
if (this.$el && this.$el.length) {
this.stopAskProgress();
this.setLoading(false);
this.showError(new Error(message));
}
},
startAskProgress: function (askOrStartedAt) {
// Render a live progress panel: a "thinking" header with elapsed
// time, plus a stacked log of the events Shira's emitted so far
// (one line per LLM step / tool call). `askOrStartedAt` is
// either the active-ask object (from runAsk and afterRender) or
// a bare timestamp (legacy call sites). Re-mounting on view
// switch replays the accumulated event list from ask.events.
if (this.mode !== 'ask') return;
this.stopAskProgress();
startedAt = startedAt || Date.now();
let ask = null;
let startedAt;
if (askOrStartedAt && typeof askOrStartedAt === 'object') {
ask = askOrStartedAt;
startedAt = ask.startedAt || Date.now();
} else {
startedAt = askOrStartedAt || Date.now();
}
const $results = this.$el.find('.kb-results');
$results.html(
'<div class="kb-ask-progress panel panel-info">' +
' <div class="panel-body" style="direction:rtl;text-align:right;">' +
' <i class="fas fa-circle-notch fa-spin" style="margin-left:8px;"></i>' +
' <strong>שירה חושבת…</strong>' +
' <div class="small text-muted" style="margin-top:6px;">' +
' שירה מחפשת בבסיס הידע, מסכמת ומציגה ציטוטים מדויקים.' +
' זה יכול לקחת בין 30 ל-90 שניות.' +
' </div>' +
' <div class="small" style="margin-top:10px;">' +
' <span>זמן שעבר: </span>' +
' <span class="kb-elapsed">0</span>' +
' <span> שניות</span>' +
' <div style="display:flex;align-items:center;gap:8px;margin-bottom:10px;">' +
' <i class="fas fa-circle-notch fa-spin"></i>' +
' <strong>שירה חושבת</strong>' +
' <span class="text-muted small kb-elapsed-pill" ' +
'style="margin-right:auto;font-variant-numeric:tabular-nums;">0 שניות</span>' +
' </div>' +
' <div class="kb-events" style="' +
'border-top:1px solid #d8e0e8;padding-top:8px;' +
'max-height:60vh;overflow-y:auto;font-size:0.95em;"></div>' +
' </div>' +
'</div>'
);
// Replay any events already collected (matters when the user
// navigates away mid-think and comes back to a still-streaming
// ask — the stream kept going at module scope, but the DOM was
// freshly re-rendered).
if (ask && Array.isArray(ask.events)) {
ask.events.forEach(ev => this._appendAskEvent(ask, ev));
}
const tick = () => {
const sec = Math.round((Date.now() - startedAt) / 1000);
const $elapsed = this.$el.find('.kb-elapsed');
if ($elapsed.length) $elapsed.text(sec);
if (sec === 120) {
this.$el.find('.kb-ask-progress .panel-body').append(
'<div class="alert alert-warning" style="margin-top:10px;">' +
'לוקח יותר זמן מהרגיל. עדיין ממתינים…</div>'
);
}
this.$el.find('.kb-elapsed-pill').text(sec + ' שניות');
};
tick();
this._askTimer = setInterval(tick, 1000);
},
_appendAskEvent: function (ask, ev) {
const $events = this.$el.find('.kb-events');
if (!$events.length) return;
const startedAt = (ask && ask.startedAt) || Date.now();
const ts = ev._clientTs || Date.now();
const sec = Math.max(0, Math.round((ts - startedAt) / 1000));
const icon = {
thinking: '🧠',
tool_start: '🔍',
tool_done: '✓',
tool_error: '⚠',
writing: '✍️',
}[ev.type] || '·';
const colour = {
thinking: '#475569',
tool_start: '#0f5298',
tool_done: '#1e6f40',
tool_error: '#a02622',
writing: '#5a3a91',
}[ev.type] || '#475569';
const label = this.escape(ev.label || ev.type);
const row = $(
'<div style="padding:3px 0;color:' + colour + ';display:flex;gap:8px;align-items:flex-start;">' +
' <span style="display:inline-block;min-width:40px;color:#94a3b8;font-variant-numeric:tabular-nums;">' +
sec + 'ש</span>' +
' <span style="margin:0 2px;">' + icon + '</span>' +
' <span style="flex:1 1 auto;word-break:break-word;">' + label + '</span>' +
'</div>'
);
$events.append(row);
// Always keep the latest line visible.
$events.scrollTop($events[0].scrollHeight);
},
stopAskProgress: function () {
if (this._askTimer) {
clearInterval(this._askTimer);
@@ -249,12 +381,13 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
}
},
renderSearchResults: function (hits) {
renderSearchResults: function (hits, selectedIdx) {
const $results = this.$el.find('.kb-results');
if (!hits.length) {
$results.html('<div class="text-muted">לא נמצאו תוצאות.</div>');
return;
}
const initialIdx = (typeof selectedIdx === 'number' && hits[selectedIdx]) ? selectedIdx : 0;
const kindHe = {law: 'חוק', regulation: 'תקנה', circular: 'חוזר'};
const self = this;
@@ -272,7 +405,7 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
const pub = h.published_at ? ' · פורסם ' + h.published_at : '';
const pageLabel = (isPdfBacked(h) && h.page_number)
? ` · <span class="label label-default">עמ׳ ${h.page_number}</span>` : '';
const activeClass = i === 0 ? 'panel-primary' : 'panel-default';
const activeClass = i === initialIdx ? 'panel-primary' : 'panel-default';
return `
<div class="kb-hit panel ${activeClass}" data-hit-idx="${i}"
style="margin-bottom:8px;cursor:pointer;">
@@ -285,20 +418,15 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
</div>`;
}).join('');
$results.html(`
<div class="row" style="margin:0;">
<div class="col-md-5" style="padding-right:0;max-height:86vh;overflow-y:auto;">
${listHtml}
</div>
<div class="col-md-7 kb-search-preview" style="padding-left:0;">
<div class="kb-preview-slot"></div>
</div>
</div>
`);
$results.html(this._buildSplitShell({
leftHtml: `<div class="kb-search-list" style="height:100%;overflow-y:auto;padding-left:6px;">${listHtml}</div>`,
rightHtml: '<div class="kb-preview-slot" style="height:100%;"></div>',
}));
this._wireSplitter($results.find('.kb-split'));
// Cache hits on the view so click handlers can reach them.
this._searchHits = hits;
this.showSearchPreview(0);
this.showSearchPreview(initialIdx);
this.$el.find('.kb-hit').off('click').on('click', function () {
const idx = parseInt($(this).data('hit-idx'), 10);
@@ -307,6 +435,90 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
.removeClass('panel-primary').addClass('panel-default');
$(this).removeClass('panel-default').addClass('panel-primary');
self.showSearchPreview(idx);
if (_lastSearch && _lastSearch.hits === self._searchHits) {
_lastSearch.selectedIdx = idx;
_saveJson(sessionStorage, SS_SEARCH, _lastSearch);
}
});
},
// Returns the HTML for a flex-row split layout with a draggable
// 6px handle. The container's flex direction is forced LTR so the
// splitter math (clientX) doesn't get inverted by RTL — content
// inside each pane keeps its own direction.
_buildSplitShell: function ({leftHtml, rightHtml}) {
const pct = this._loadSplitPct();
return (
'<div class="kb-split" style="' +
'display:flex;flex-direction:row;direction:ltr;' +
'gap:0;height:86vh;align-items:stretch;">' +
' <div class="kb-split-left" style="' +
'flex:0 0 ' + pct + '%;min-width:200px;overflow:hidden;">' +
leftHtml +
' </div>' +
' <div class="kb-split-handle" title="גרור כדי לשנות חלוקה" style="' +
'flex:0 0 6px;background:#dde1e7;cursor:col-resize;' +
'border-left:1px solid #cbd0d8;border-right:1px solid #cbd0d8;' +
'transition:background 0.15s;"></div>' +
' <div class="kb-split-right" style="' +
'flex:1 1 auto;min-width:200px;overflow:hidden;">' +
rightHtml +
' </div>' +
'</div>'
);
},
_loadSplitPct: function () {
try {
const v = parseFloat(localStorage.getItem(LS_SPLIT));
if (!isNaN(v) && v >= 15 && v <= 85) return v;
} catch (e) { /* ignore */ }
return 42;
},
_saveSplitPct: function (pct) {
try { localStorage.setItem(LS_SPLIT, String(pct)); }
catch (e) { /* ignore */ }
},
// Wires mousedown/move/up on the handle to resize the left pane.
// While dragging we cover any iframe with a transparent overlay so
// mouse events don't disappear into PDF.js. On mouseup the chosen
// ratio persists to localStorage.
_wireSplitter: function ($split) {
if (!$split.length) return;
const $handle = $split.find('.kb-split-handle');
const $left = $split.find('.kb-split-left');
const self = this;
$handle.on('mouseenter', () => $handle.css('background', '#b9c0cb'));
$handle.on('mouseleave', () => $handle.css('background', '#dde1e7'));
$handle.on('mousedown', function (e) {
e.preventDefault();
const rect = $split[0].getBoundingClientRect();
const $overlay = $('<div class="kb-drag-overlay">').css({
position: 'fixed',
inset: '0',
cursor: 'col-resize',
zIndex: 9999,
background: 'transparent',
});
$('body').append($overlay).css('user-select', 'none');
const onMove = function (ev) {
const x = ev.clientX - rect.left;
let pct = (x / rect.width) * 100;
pct = Math.max(15, Math.min(85, pct));
$left.css('flex', '0 0 ' + pct + '%');
self._dragPct = pct;
};
const onUp = function () {
$(document).off('mousemove', onMove).off('mouseup', onUp);
$overlay.remove();
$('body').css('user-select', '');
if (typeof self._dragPct === 'number') {
self._saveSplitPct(self._dragPct);
}
};
$(document).on('mousemove', onMove).on('mouseup', onUp);
});
},
@@ -323,34 +535,126 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
+ encodeURIComponent(h.source_id)
+ '#page=' + encodeURIComponent(page);
$slot.html(`
<div style="direction:rtl;text-align:right;margin-bottom:6px;">
<span class="text-muted small">מציג עמ׳ ${page} של המסמך</span>
<div style="display:flex;flex-direction:column;height:100%;">
<div style="direction:rtl;text-align:right;margin-bottom:6px;flex:0 0 auto;">
<span class="text-muted small">מציג עמ׳ ${page} של המסמך</span>
</div>
<iframe class="kb-pdf-iframe" src="${url}"
style="flex:1 1 auto;width:100%;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
</div>
<iframe class="kb-pdf-iframe"
src="${url}"
style="width:100%;height:82vh;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
`);
return;
}
// Text source (law from Wikisource) — show the full chunk content
// and a link to the public source page.
// Text source (law from Wikisource): fetch the matched chunk +
// surrounding sections so the preview pane shows context, not a
// copy of what's already in the left card. While loading, show a
// placeholder; on failure fall back to the chunk-only view.
this._renderSectionContextPlaceholder(h);
const token = this._previewToken;
const sourceId = h.source_id;
const around = (typeof h.chunk_index === 'number') ? h.chunk_index : null;
if (sourceId == null || around == null) {
this._renderSectionContextSingle(h);
return;
}
const self = this;
Espo.Ajax.getRequest('KnowledgeBase/action/chunks', {
sourceId: sourceId,
around: around,
ctx: 2,
}).then(res => {
// Bail if user has clicked a different hit while we were waiting.
if (self._previewToken !== token) return;
self._renderSectionContext(h, res && res.chunks ? res.chunks : []);
}).catch(err => {
console.warn('KB: section-window fetch failed', err);
if (self._previewToken === token) self._renderSectionContextSingle(h);
});
},
_renderSectionContextPlaceholder: function (h) {
this._previewToken = (this._previewToken || 0) + 1;
const $slot = this.$el.find('.kb-preview-slot');
const title = this.escape(h.title || '');
$slot.html(
'<div class="panel panel-default" style="direction:rtl;text-align:right;height:100%;display:flex;flex-direction:column;">' +
' <div class="panel-heading" style="flex:0 0 auto;">' + title +
' <span class="text-muted small" style="margin-right:8px;">— טוען הקשר…</span>' +
' </div>' +
' <div class="panel-body" style="flex:1 1 auto;overflow-y:auto;">' +
' <i class="fas fa-circle-notch fa-spin"></i>' +
' </div>' +
'</div>'
);
},
_renderSectionContextSingle: function (h) {
// Fallback when we can't fetch the window — render just the
// matched chunk, which is at least the full section for the
// statute chunker (1 section = 1 chunk).
this._renderSectionContext(h, [{
chunk_index: h.chunk_index,
section_ref: h.section_ref || '',
heading_path: h.heading_path || '',
content: h.content || '',
}]);
},
_renderSectionContext: function (h, chunks) {
const $slot = this.$el.find('.kb-preview-slot');
const bodyStyle =
'direction:rtl;unicode-bidi:plaintext;white-space:pre-wrap;' +
'font-family:"Segoe UI",Arial,sans-serif;line-height:1.7;text-align:right;';
const srcLink = h.source_url
? `<a href="${this.escape(h.source_url)}" target="_blank" rel="noopener">מקור ב-Wikisource</a>`
? '<a href="' + this.escape(h.source_url) + '" target="_blank" rel="noopener" ' +
'class="btn btn-xs btn-default" style="margin-right:6px;">' +
'<i class="fas fa-external-link-alt"></i> פתח ב-Wikisource</a>'
: '';
$slot.html(`
<div class="panel panel-default" style="direction:rtl;text-align:right;">
<div class="panel-heading">${this.escape(h.title || '')}</div>
<div class="panel-body" style="${bodyStyle};max-height:80vh;overflow-y:auto;">
${this.escape(h.content || '')}
</div>
${srcLink ? '<div class="panel-footer small">' + srcLink + '</div>' : ''}
</div>
`);
// The matched chunk is highlighted; siblings render as muted
// context blocks above and below so the user can read into the
// section without leaving the CRM.
const sectionsHtml = chunks.map(c => {
const isMatch = c.chunk_index === h.chunk_index;
const ref = c.section_ref || c.heading_path || '';
const headerStyle = isMatch
? 'background:#fff3cd;border-bottom:1px solid #ffe69c;color:#664d03;font-weight:600;'
: 'background:#f5f5f7;border-bottom:1px solid #e7e7ea;color:#6c757d;';
return (
'<div style="margin-bottom:12px;border:1px solid ' +
(isMatch ? '#ffe69c' : '#e7e7ea') + ';border-radius:4px;overflow:hidden;">' +
' <div style="' + headerStyle + 'padding:6px 10px;direction:rtl;text-align:right;font-size:0.9em;">' +
(isMatch ? 'התאמה — ' : '') + this.escape(ref) +
' </div>' +
' <div style="' + bodyStyle + ';padding:10px;">' +
this.escape(c.content || '') +
' </div>' +
'</div>'
);
}).join('');
$slot.html(
'<div class="panel panel-default" style="direction:rtl;text-align:right;height:100%;display:flex;flex-direction:column;">' +
' <div class="panel-heading" style="flex:0 0 auto;display:flex;justify-content:space-between;align-items:center;">' +
' <div><strong>' + this.escape(h.title || '') + '</strong>' +
' <span class="text-muted small" style="margin-right:6px;">— הסעיף בהקשרו</span>' +
' </div>' +
srcLink +
' </div>' +
' <div class="panel-body" style="flex:1 1 auto;overflow-y:auto;background:#fafbfc;">' +
sectionsHtml +
' </div>' +
'</div>'
);
// Scroll the matched chunk into view inside the preview pane.
const $body = $slot.find('.panel-body');
const $match = $body.find('div[style*="#fff3cd"]').closest('div[style*="border-radius:4px"]');
if ($match.length && $body.length) {
const offset = $match.position().top - 20;
$body.scrollTop(Math.max(0, offset));
}
},
renderAskAnswer: function (text, sources) {
@@ -437,32 +741,32 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
+ encodeURIComponent(first.source_id)
+ '#page=' + encodeURIComponent(firstPage);
$results.html(`
<div class="row" style="margin:0;">
<div class="col-md-6" style="padding-right:0;">
<div class="kb-answer panel panel-info">
<div class="panel-body" style="direction:rtl;text-align:right;max-height:80vh;overflow-y:auto;">
${answerHtml}
</div>
</div>
<div class="panel panel-default" style="direction:rtl;text-align:right;">
<div class="panel-heading"><strong>📎 מקורות</strong></div>
<div class="panel-body">
<div style="margin-bottom:6px;">${pickerItems}</div>
<div class="kb-page-links" style="font-size:0.9em;">
<span class="text-muted">עמודים רלוונטיים:</span> ${pageLinks}
</div>
</div>
const leftHtml = `
<div style="height:100%;overflow-y:auto;padding-left:6px;">
<div class="kb-answer panel panel-info">
<div class="panel-body" style="direction:rtl;text-align:right;">
${answerHtml}
</div>
</div>
<div class="col-md-6" style="padding-left:0;">
<iframe class="kb-pdf-iframe"
src="${initialUrl}"
style="width:100%;height:86vh;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
<div class="panel panel-default" style="direction:rtl;text-align:right;">
<div class="panel-heading"><strong>📎 מקורות</strong></div>
<div class="panel-body">
<div style="margin-bottom:6px;">${pickerItems}</div>
<div class="kb-page-links" style="font-size:0.9em;">
<span class="text-muted">עמודים רלוונטיים:</span> ${pageLinks}
</div>
</div>
</div>
</div>
`);
`;
const rightHtml = `
<iframe class="kb-pdf-iframe"
src="${initialUrl}"
style="width:100%;height:100%;border:1px solid #ddd;background:#fafafa;"
title="מקור"></iframe>
`;
$results.html(this._buildSplitShell({leftHtml, rightHtml}));
this._wireSplitter($results.find('.kb-split'));
const self = this;
this.$el.find('.kb-pdf-pick').off('click').on('click', function () {
@@ -73,7 +73,12 @@ class KnowledgeBase
throw new BadRequest('sourceId (numeric) is required.');
}
return $this->getService()->sourceChunks((int) $sourceId);
$around = $request->getQueryParam('around');
$ctx = $request->getQueryParam('ctx');
$aroundInt = ($around !== null && is_numeric($around)) ? (int) $around : null;
$ctxInt = ($ctx !== null && is_numeric($ctx)) ? (int) $ctx : 2;
return $this->getService()->sourceChunks((int) $sourceId, $aroundInt, $ctxInt);
}
public function postActionAsk(Request $request, Response $response): array
@@ -0,0 +1,128 @@
<?php
namespace Espo\Modules\KnowledgeBase\EntryPoints;
use Espo\Core\Api\Request;
use Espo\Core\Api\Response;
use Espo\Core\EntryPoint\EntryPoint;
use Espo\Core\Exceptions\BadRequest;
use Espo\Core\Exceptions\Error;
use Espo\Core\Exceptions\Forbidden;
use Espo\Core\InjectableFactory;
use Espo\Entities\User;
use Espo\Modules\KnowledgeBase\Services\KnowledgeBaseService;
/**
* Streams Server-Sent Events from shira-hermes /kb/ask/stream to the
* browser's EventSource. The browser opens this with a GET (EventSource
* has no POST), passes the question via ?message=, and we forward it
* to shira-hermes with the firm-wide API key — the credential never
* leaves the EspoCRM server.
*
* Response is text/event-stream with no buffering. We bypass EspoCRM's
* Response wrapper entirely (echo + flush + exit) because the framework
* is wired for buffered JSON responses, not infinite streams.
*
* Access: `?entryPoint=KnowledgeBaseAskStream&message=<urlencoded>`
*/
class KnowledgeBaseAskStream implements EntryPoint
{
public function __construct(
private InjectableFactory $injectableFactory,
private User $user,
) {}
public function run(Request $request, Response $response): void
{
if ($this->user->isPortal()) {
throw new Forbidden('Portal users have no KB access.');
}
$message = $request->getQueryParam('message');
if (!is_string($message) || trim($message) === '') {
throw new BadRequest('message (string) is required.');
}
if (strlen($message) > 2000) {
throw new BadRequest('message is too long.');
}
$service = $this->injectableFactory->create(KnowledgeBaseService::class);
try {
$upstreamUrl = $service->getStreamingUrl('/kb/ask/stream');
$apiKey = $service->getStreamingApiKey();
} catch (\Throwable $e) {
throw new Error('SmartAssistant integration is not configured: ' . $e->getMessage());
}
// Disable PHP/Apache output buffering and compression so each SSE
// event flushes to the wire as soon as it arrives from upstream.
// Order matters: clean any open buffers BEFORE setting headers.
while (ob_get_level() > 0) {
@ob_end_clean();
}
@ini_set('zlib.output_compression', 'Off');
@ini_set('output_buffering', 'Off');
@ini_set('implicit_flush', '1');
if (function_exists('apache_setenv')) {
@apache_setenv('no-gzip', '1');
}
ignore_user_abort(false);
@set_time_limit(300);
header('Content-Type: text/event-stream; charset=utf-8');
header('Cache-Control: no-cache, no-transform');
header('X-Accel-Buffering: no');
header('Connection: keep-alive');
// Defense in depth: same locked-down CSP we use for the PDF entry.
header("Content-Security-Policy: default-src 'self'");
$payload = json_encode(['message' => $message], JSON_UNESCAPED_UNICODE);
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => $upstreamUrl,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $payload,
CURLOPT_HTTPHEADER => [
'Content-Type: application/json',
'Accept: text/event-stream',
'X-Api-Key: ' . $apiKey,
],
CURLOPT_RETURNTRANSFER => false,
CURLOPT_BUFFERSIZE => 1024,
CURLOPT_TIMEOUT => 300,
CURLOPT_CONNECTTIMEOUT => 10,
CURLOPT_WRITEFUNCTION => function ($ch, $data) {
if (connection_aborted()) {
// Browser closed the EventSource — bail so cURL stops.
return -1;
}
echo $data;
@ob_flush();
@flush();
return strlen($data);
},
]);
$ok = curl_exec($ch);
$errno = curl_errno($ch);
$errmsg = curl_error($ch);
curl_close($ch);
if (!$ok && $errno !== 23 /* aborted by callback */) {
// Best effort: if upstream errored before any data was
// flushed, surface a final SSE error event so the client
// doesn't silently spin.
echo "data: " . json_encode(
['type' => 'error', 'message' => 'upstream: ' . $errmsg],
JSON_UNESCAPED_UNICODE,
) . "\n\n";
@ob_flush();
@flush();
}
// We've fully written the response. Bypass EspoCRM's Response
// emit() to avoid double-headers / extra buffering.
exit;
}
}
@@ -1,5 +1,8 @@
{
"KnowledgeBasePdf": {
"className": "Espo\\Modules\\KnowledgeBase\\EntryPoints\\KnowledgeBasePdf"
},
"KnowledgeBaseAskStream": {
"className": "Espo\\Modules\\KnowledgeBase\\EntryPoints\\KnowledgeBaseAskStream"
}
}
@@ -48,9 +48,13 @@ class KnowledgeBaseService
return $this->get('/kb/sources');
}
public function sourceChunks(int $sourceId): array
public function sourceChunks(int $sourceId, ?int $around = null, int $ctx = 2): array
{
return $this->get('/kb/source/' . $sourceId . '/chunks');
$path = '/kb/source/' . $sourceId . '/chunks';
if ($around !== null) {
$path .= '?around=' . $around . '&ctx=' . $ctx;
}
return $this->get($path);
}
public function ask(string $message, ?string $conversationId, array $history): array
@@ -136,6 +140,25 @@ class KnowledgeBaseService
return $data->apiKey ?? null;
}
/**
* Public accessors so the streaming EntryPoint can build a cURL
* connection to shira-hermes without going through the JSON request()
* helper (which buffers the whole body).
*/
public function getStreamingUrl(string $path): string
{
return rtrim($this->getBaseUrl(), '/') . $path;
}
public function getStreamingApiKey(): string
{
$key = $this->getApiKey();
if (!$key) {
throw new Error('SmartAssistant API key is not configured.');
}
return $key;
}
private function get(string $path): array
{
return $this->request('GET', $path, null);
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "KnowledgeBase",
"module": "KnowledgeBase",
"version": "0.1.9",
"version": "0.2.0",
"acceptableVersions": [
">=8.0.0"
],