feat(kb/ask): live SSE streaming of Shira's progress (v0.2.0)

User wanted to see what Shira is doing while she thinks — like Claude
does — instead of staring at an incrementing seconds counter. Now the
ask UI shows one line per agent step as it happens:

  3s  🧠  קוראת את השאלה
  4s  🔍  מחפשת בבסיס הידע: "תקנה 36"
  9s  ✓   נמצאו 5 קטעים רלוונטיים
 10s  🔍  מחפשת בבסיס הידע: "תקנה 36 ביטוח לאומי"
 14s  ✓   נמצאו 5 קטעים רלוונטיים
 15s  🧠  ממשיכה לחקור
 22s  ✍️  מנסחת תשובה
 → תשובה

Client wiring:
- runAsk now opens an EventSource at ?entryPoint=KnowledgeBaseAskStream
  instead of POSTing JSON. Each SSE event of type thinking/tool_start/
  tool_done/tool_error/writing appends a row to a stacked progress log.
  The final {type:answer,text,sources} event triggers renderAskAnswer
  with the existing split view. {type:error} surfaces via showError.
- Module-level state from v0.1.9 unchanged in spirit but now stores the
  EventSource + accumulated events. afterRender on view re-mount replays
  the entire event list and re-binds onmessage/onerror — the SSE
  connection itself doesn't drop because it lives at module scope.
- Elapsed-time pill stays for cadence but is decoration; the real
  signal is the per-event log.

EspoCRM proxy:
- New EntryPoint KnowledgeBaseAskStream — required because EventSource
  is GET-only and ?message= must come via the URL. PHP buffering is
  disabled (zlib + ob + apache_setenv no-gzip), Apache headers are set
  for SSE (no-cache, X-Accel-Buffering: no), and we cURL into
  shira-hermes /kb/ask/stream with CURLOPT_WRITEFUNCTION echoing each
  byte and flushing immediately. The handler ends with exit; to bypass
  EspoCRM's Response wrapper which would otherwise emit headers/body
  on top of our raw stream.
- KnowledgeBaseService gained two public accessors (getStreamingUrl,
  getStreamingApiKey) so the EntryPoint can build the cURL without
  going through request() (which buffers the whole body).
- entryPoints.json registers the new class.

Server (shira-hermes 76fd77f):
- AgentRunner.run accepts an on_event sync callback fired before each
  LLM call (thinking) and around each tool call (tool_start, tool_done,
  tool_error). Hebrew labels in agent_runner._tool_label / _tool_done_label
  ship over the wire ready-to-display — keeps the client free of
  translation concerns.
- POST /kb/ask/stream wraps the existing agent runner in an asyncio
  task feeding an asyncio.Queue, returns StreamingResponse with proper
  SSE headers (X-Accel-Buffering, Cache-Control). 15s comment-line
  heartbeats keep the connection open through proxy idle-timeouts.
- Original POST /kb/ask still works — additive change.

Refs Task Master #8

.taskmaster/tasks/tasks.json

@@ -37,6 +37,18 @@
         "dependencies": [],
         "createdAt": "2026-04-25T10:00:00Z"
       },
+      {
+        "id": 8,
+        "title": "feat(kb/ask): SSE streaming with live agent progress",
+        "description": "User wanted Shira's progress to be visible while she thinks — like Claude does — instead of just an incrementing seconds counter. Implementation: AgentRunner.run accepts an optional sync on_event callback that emits {type, label, ...} dicts at each agent step. New POST /kb/ask/stream returns text/event-stream, with the runner running in an asyncio task whose events feed an asyncio.Queue that the response generator drains as SSE. Client opens EventSource against a new EspoCRM EntryPoint KnowledgeBaseAskStream that proxies the SSE bytes through cURL with output buffering disabled (echo + flush + exit, bypassing the framework Response). The progress UI now stacks one line per event (thinking, tool_start with the query Shira chose, tool_done with the chunk count, writing) instead of just a seconds counter. v0.1.9 _activeAsk persistence pattern preserved — events accumulate at module scope so a view switch + return replays the live log.",
+        "status": "done",
+        "priority": "high",
+        "details": "shira-hermes 76fd77f. EspoCRM EntryPoint registered in metadata/app/entryPoints.json. Hebrew labels live in agent_runner._tool_label / _tool_done_label so the wire format already carries display-ready strings (no client-side mapping). 15s heartbeats keep the connection open through proxy idle-timeouts.",
+        "testStrategy": "curl -N POST /kb/ask/stream {message:'מהי תקנה 36'} should produce data: lines with thinking → tool_start (with query) → tool_done (with chunk count) → ... → writing → answer. Browser EventSource should drive the same flow into the UI.",
+        "subtasks": [],
+        "dependencies": [],
+        "createdAt": "2026-04-25T13:00:00Z"
+      },
       {
         "id": 7,
         "title": "fix: search preview duplicates left card for text sources",

files/client/custom/modules/knowledge-base/src/views/kb/index.js

@@ -115,7 +115,9 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
                     const $input = this.$el.find('input[data-name="query"]');
                     if ($input.length) $input.val(_activeAsk.question);
                     this.setLoading(true);
-                    this.startAskProgress(_activeAsk.startedAt);
+                    // Pass the whole ask so startAskProgress can replay the
+                    // accumulated event log into the freshly-rendered DOM.
+                    this.startAskProgress(_activeAsk);
                     this._attachAskHandlers(_activeAsk);
                 } else if (_lastAsk) {
                     const $input = this.$el.find('input[data-name="query"]');
@@ -205,93 +207,173 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
             _clear(sessionStorage, SS_ASK);
 
             this.setLoading(true);
-            // Ask goes through the full agent loop (search + rerank + LLM),
+            // Streaming via Server-Sent Events. The browser opens an
-            // which routinely takes 30-90s. The default jQuery Ajax timeout
+            // EventSource against KnowledgeBaseAskStream which proxies
-            // is too short; override it here so the browser doesn't give up
+            // /kb/ask/stream on shira-hermes; one SSE event per agent step
-            // before shira-hermes responds.
+            // (thinking, tool_start, tool_done) plus a final answer event.
-            const promise = Espo.Ajax.postRequest('KnowledgeBase/action/ask', {
+            const url = '?entryPoint=KnowledgeBaseAskStream&message='
-                message: message,
+                + encodeURIComponent(message);
-            }, {timeout: 240000});
+            const es = new EventSource(url);
-            _activeAsk = {question: message, promise: promise, startedAt: Date.now()};
+            _activeAsk = {
-            this.startAskProgress(_activeAsk.startedAt);
+                question: message,
+                es: es,
+                events: [],
+                startedAt: Date.now(),
+            };
+            this.startAskProgress(_activeAsk);
             this._attachAskHandlers(_activeAsk);
         },
 
-        // Wire .then/.catch on a (possibly module-level) ask promise to
+        // Wire SSE handlers on a (possibly module-level) ask. Called from
-        // whichever view is currently mounted. Called from runAsk for the
+        // runAsk for the initial bind and from afterRender when the user
-        // initial bind and from afterRender when the user navigates back to
+        // navigates back to the KB view while an earlier stream is still
-        // the KB view while an earlier request is still in flight.
+        // open. The EventSource itself lives at module scope (in _activeAsk)
+        // so handlers can be re-bound without dropping the connection.
         _attachAskHandlers: function (ask) {
             const self = this;
-            ask.promise.then(res => {
+            ask.es.onmessage = function (msgEvent) {
-                const entry = {
+                let data;
-                    question: ask.question,
+                try { data = JSON.parse(msgEvent.data); }
-                    text: res.text || '',
+                catch (e) {
-                    sources: res.sources || [],
+                    console.warn('KB: malformed SSE payload', e);
-                    completedAt: Date.now(),
+                    return;
-                };
+                }
-                _lastAsk = entry;
+                if (data.type === 'answer') {
-                _saveJson(sessionStorage, SS_ASK, entry);
+                    self._finishAsk(ask, {
-                if (_activeAsk === ask) _activeAsk = null;
+                        question: ask.question,
-                if (self.$el && self.$el.length) {
+                        text: data.text || '',
-                    self.stopAskProgress();
+                        sources: data.sources || [],
-                    self.setLoading(false);
+                        completedAt: Date.now(),
-                    if (self.mode === 'ask') {
+                    });
-                        self.renderAskAnswer(entry.text, entry.sources);
+                } else if (data.type === 'error') {
+                    self._failAsk(ask, data.message || 'שגיאה בשרת');
+                } else {
+                    // Status event — record it and append to the live log.
+                    ask.events.push(Object.assign({_clientTs: Date.now()}, data));
+                    if (self.$el && self.$el.length && self.mode === 'ask') {
+                        self._appendAskEvent(ask, ask.events[ask.events.length - 1]);
                     }
                 }
-            }).catch(err => {
+            };
-                if (_activeAsk === ask) _activeAsk = null;
+            ask.es.onerror = function () {
-                if (self.$el && self.$el.length) {
+                // Don't tear down on transient blips while the answer is
-                    self.stopAskProgress();
+                // already arriving; only treat it as a hard failure if no
-                    self.setLoading(false);
+                // answer has been delivered.
-                    self.showError(err);
+                if (ask._completed) return;
-                }
+                self._failAsk(ask, 'הזרם נסגר לפני שהגיעה תשובה');
-            });
+            };
         },
 
-        startAskProgress: function (startedAt) {
+        _finishAsk: function (ask, entry) {
-            // Show a live elapsed-time indicator so users know the request
+            ask._completed = true;
-            // is still running rather than stuck. Typical answers arrive in
+            try { ask.es.close(); } catch (e) { /* ignore */ }
-            // 30-90s; we warn after 2 minutes. `startedAt` is passed in so
+            _lastAsk = entry;
-            // a request that began before a view re-mount keeps showing the
+            _saveJson(sessionStorage, SS_ASK, entry);
-            // accumulated elapsed time, not 0.
+            if (_activeAsk === ask) _activeAsk = null;
+            if (this.$el && this.$el.length) {
+                this.stopAskProgress();
+                this.setLoading(false);
+                if (this.mode === 'ask') {
+                    this.renderAskAnswer(entry.text, entry.sources);
+                }
+            }
+        },
+
+        _failAsk: function (ask, message) {
+            try { ask.es.close(); } catch (e) { /* ignore */ }
+            if (_activeAsk === ask) _activeAsk = null;
+            if (this.$el && this.$el.length) {
+                this.stopAskProgress();
+                this.setLoading(false);
+                this.showError(new Error(message));
+            }
+        },
+
+        startAskProgress: function (askOrStartedAt) {
+            // Render a live progress panel: a "thinking" header with elapsed
+            // time, plus a stacked log of the events Shira's emitted so far
+            // (one line per LLM step / tool call). `askOrStartedAt` is
+            // either the active-ask object (from runAsk and afterRender) or
+            // a bare timestamp (legacy call sites). Re-mounting on view
+            // switch replays the accumulated event list from ask.events.
             if (this.mode !== 'ask') return;
             this.stopAskProgress();
-            startedAt = startedAt || Date.now();
+            let ask = null;
+            let startedAt;
+            if (askOrStartedAt && typeof askOrStartedAt === 'object') {
+                ask = askOrStartedAt;
+                startedAt = ask.startedAt || Date.now();
+            } else {
+                startedAt = askOrStartedAt || Date.now();
+            }
+
             const $results = this.$el.find('.kb-results');
             $results.html(
                 '<div class="kb-ask-progress panel panel-info">' +
                 '  <div class="panel-body" style="direction:rtl;text-align:right;">' +
-                '    <i class="fas fa-circle-notch fa-spin" style="margin-left:8px;"></i>' +
+                '    <div style="display:flex;align-items:center;gap:8px;margin-bottom:10px;">' +
-                '    <strong>שירה חושבת…</strong>' +
+                '      <i class="fas fa-circle-notch fa-spin"></i>' +
-                '    <div class="small text-muted" style="margin-top:6px;">' +
+                '      <strong>שירה חושבת</strong>' +
-                '      שירה מחפשת בבסיס הידע, מסכמת ומציגה ציטוטים מדויקים.' +
+                '      <span class="text-muted small kb-elapsed-pill" ' +
-                '      זה יכול לקחת בין 30 ל-90 שניות.' +
+                          'style="margin-right:auto;font-variant-numeric:tabular-nums;">0 שניות</span>' +
-                '    </div>' +
-                '    <div class="small" style="margin-top:10px;">' +
-                '      <span>זמן שעבר: </span>' +
-                '      <span class="kb-elapsed">0</span>' +
-                '      <span> שניות</span>' +
                 '    </div>' +
+                '    <div class="kb-events" style="' +
+                       'border-top:1px solid #d8e0e8;padding-top:8px;' +
+                       'max-height:60vh;overflow-y:auto;font-size:0.95em;"></div>' +
                 '  </div>' +
                 '</div>'
             );
+
+            // Replay any events already collected (matters when the user
+            // navigates away mid-think and comes back to a still-streaming
+            // ask — the stream kept going at module scope, but the DOM was
+            // freshly re-rendered).
+            if (ask && Array.isArray(ask.events)) {
+                ask.events.forEach(ev => this._appendAskEvent(ask, ev));
+            }
+
             const tick = () => {
                 const sec = Math.round((Date.now() - startedAt) / 1000);
-                const $elapsed = this.$el.find('.kb-elapsed');
+                this.$el.find('.kb-elapsed-pill').text(sec + ' שניות');
-                if ($elapsed.length) $elapsed.text(sec);
-                if (sec === 120) {
-                    this.$el.find('.kb-ask-progress .panel-body').append(
-                        '<div class="alert alert-warning" style="margin-top:10px;">' +
-                        'לוקח יותר זמן מהרגיל. עדיין ממתינים…</div>'
-                    );
-                }
             };
             tick();
             this._askTimer = setInterval(tick, 1000);
         },
 
+        _appendAskEvent: function (ask, ev) {
+            const $events = this.$el.find('.kb-events');
+            if (!$events.length) return;
+            const startedAt = (ask && ask.startedAt) || Date.now();
+            const ts = ev._clientTs || Date.now();
+            const sec = Math.max(0, Math.round((ts - startedAt) / 1000));
+            const icon = {
+                thinking: '🧠',
+                tool_start: '🔍',
+                tool_done: '✓',
+                tool_error: '⚠',
+                writing: '✍️',
+            }[ev.type] || '·';
+            const colour = {
+                thinking: '#475569',
+                tool_start: '#0f5298',
+                tool_done: '#1e6f40',
+                tool_error: '#a02622',
+                writing: '#5a3a91',
+            }[ev.type] || '#475569';
+            const label = this.escape(ev.label || ev.type);
+            const row = $(
+                '<div style="padding:3px 0;color:' + colour + ';display:flex;gap:8px;align-items:flex-start;">' +
+                '  <span style="display:inline-block;min-width:40px;color:#94a3b8;font-variant-numeric:tabular-nums;">' +
+                       sec + 'ש</span>' +
+                '  <span style="margin:0 2px;">' + icon + '</span>' +
+                '  <span style="flex:1 1 auto;word-break:break-word;">' + label + '</span>' +
+                '</div>'
+            );
+            $events.append(row);
+            // Always keep the latest line visible.
+            $events.scrollTop($events[0].scrollHeight);
+        },
+
         stopAskProgress: function () {
             if (this._askTimer) {
                 clearInterval(this._askTimer);

files/custom/Espo/Modules/KnowledgeBase/EntryPoints/KnowledgeBaseAskStream.php

@@ -0,0 +1,128 @@
+<?php
+
+namespace Espo\Modules\KnowledgeBase\EntryPoints;
+
+use Espo\Core\Api\Request;
+use Espo\Core\Api\Response;
+use Espo\Core\EntryPoint\EntryPoint;
+use Espo\Core\Exceptions\BadRequest;
+use Espo\Core\Exceptions\Error;
+use Espo\Core\Exceptions\Forbidden;
+use Espo\Core\InjectableFactory;
+use Espo\Entities\User;
+use Espo\Modules\KnowledgeBase\Services\KnowledgeBaseService;
+
+/**
+ * Streams Server-Sent Events from shira-hermes /kb/ask/stream to the
+ * browser's EventSource. The browser opens this with a GET (EventSource
+ * has no POST), passes the question via ?message=, and we forward it
+ * to shira-hermes with the firm-wide API key — the credential never
+ * leaves the EspoCRM server.
+ *
+ * Response is text/event-stream with no buffering. We bypass EspoCRM's
+ * Response wrapper entirely (echo + flush + exit) because the framework
+ * is wired for buffered JSON responses, not infinite streams.
+ *
+ * Access: `?entryPoint=KnowledgeBaseAskStream&message=<urlencoded>`
+ */
+class KnowledgeBaseAskStream implements EntryPoint
+{
+    public function __construct(
+        private InjectableFactory $injectableFactory,
+        private User $user,
+    ) {}
+
+    public function run(Request $request, Response $response): void
+    {
+        if ($this->user->isPortal()) {
+            throw new Forbidden('Portal users have no KB access.');
+        }
+
+        $message = $request->getQueryParam('message');
+        if (!is_string($message) || trim($message) === '') {
+            throw new BadRequest('message (string) is required.');
+        }
+        if (strlen($message) > 2000) {
+            throw new BadRequest('message is too long.');
+        }
+
+        $service = $this->injectableFactory->create(KnowledgeBaseService::class);
+        try {
+            $upstreamUrl = $service->getStreamingUrl('/kb/ask/stream');
+            $apiKey = $service->getStreamingApiKey();
+        } catch (\Throwable $e) {
+            throw new Error('SmartAssistant integration is not configured: ' . $e->getMessage());
+        }
+
+        // Disable PHP/Apache output buffering and compression so each SSE
+        // event flushes to the wire as soon as it arrives from upstream.
+        // Order matters: clean any open buffers BEFORE setting headers.
+        while (ob_get_level() > 0) {
+            @ob_end_clean();
+        }
+        @ini_set('zlib.output_compression', 'Off');
+        @ini_set('output_buffering', 'Off');
+        @ini_set('implicit_flush', '1');
+        if (function_exists('apache_setenv')) {
+            @apache_setenv('no-gzip', '1');
+        }
+
+        ignore_user_abort(false);
+        @set_time_limit(300);
+
+        header('Content-Type: text/event-stream; charset=utf-8');
+        header('Cache-Control: no-cache, no-transform');
+        header('X-Accel-Buffering: no');
+        header('Connection: keep-alive');
+        // Defense in depth: same locked-down CSP we use for the PDF entry.
+        header("Content-Security-Policy: default-src 'self'");
+
+        $payload = json_encode(['message' => $message], JSON_UNESCAPED_UNICODE);
+
+        $ch = curl_init();
+        curl_setopt_array($ch, [
+            CURLOPT_URL => $upstreamUrl,
+            CURLOPT_POST => true,
+            CURLOPT_POSTFIELDS => $payload,
+            CURLOPT_HTTPHEADER => [
+                'Content-Type: application/json',
+                'Accept: text/event-stream',
+                'X-Api-Key: ' . $apiKey,
+            ],
+            CURLOPT_RETURNTRANSFER => false,
+            CURLOPT_BUFFERSIZE => 1024,
+            CURLOPT_TIMEOUT => 300,
+            CURLOPT_CONNECTTIMEOUT => 10,
+            CURLOPT_WRITEFUNCTION => function ($ch, $data) {
+                if (connection_aborted()) {
+                    // Browser closed the EventSource — bail so cURL stops.
+                    return -1;
+                }
+                echo $data;
+                @ob_flush();
+                @flush();
+                return strlen($data);
+            },
+        ]);
+        $ok = curl_exec($ch);
+        $errno = curl_errno($ch);
+        $errmsg = curl_error($ch);
+        curl_close($ch);
+
+        if (!$ok && $errno !== 23 /* aborted by callback */) {
+            // Best effort: if upstream errored before any data was
+            // flushed, surface a final SSE error event so the client
+            // doesn't silently spin.
+            echo "data: " . json_encode(
+                ['type' => 'error', 'message' => 'upstream: ' . $errmsg],
+                JSON_UNESCAPED_UNICODE,
+            ) . "\n\n";
+            @ob_flush();
+            @flush();
+        }
+
+        // We've fully written the response. Bypass EspoCRM's Response
+        // emit() to avoid double-headers / extra buffering.
+        exit;
+    }
+}

files/custom/Espo/Modules/KnowledgeBase/Resources/metadata/app/entryPoints.json

@@ -1,5 +1,8 @@
 {
     "KnowledgeBasePdf": {
         "className": "Espo\\Modules\\KnowledgeBase\\EntryPoints\\KnowledgeBasePdf"
+    },
+    "KnowledgeBaseAskStream": {
+        "className": "Espo\\Modules\\KnowledgeBase\\EntryPoints\\KnowledgeBaseAskStream"
     }
 }

files/custom/Espo/Modules/KnowledgeBase/Services/KnowledgeBaseService.php

@@ -140,6 +140,25 @@ class KnowledgeBaseService
         return $data->apiKey ?? null;
     }
 
+    /**
+     * Public accessors so the streaming EntryPoint can build a cURL
+     * connection to shira-hermes without going through the JSON request()
+     * helper (which buffers the whole body).
+     */
+    public function getStreamingUrl(string $path): string
+    {
+        return rtrim($this->getBaseUrl(), '/') . $path;
+    }
+
+    public function getStreamingApiKey(): string
+    {
+        $key = $this->getApiKey();
+        if (!$key) {
+            throw new Error('SmartAssistant API key is not configured.');
+        }
+        return $key;
+    }
+
     private function get(string $path): array
     {
         return $this->request('GET', $path, null);

manifest.json

@@ -1,7 +1,7 @@
 {
     "name": "KnowledgeBase",
     "module": "KnowledgeBase",
-    "version": "0.1.11",
+    "version": "0.2.0",
     "acceptableVersions": [
         ">=8.0.0"
     ],