- mark task #21 (commit-race + collapsible panels fix) as done — followup to v0.8.1 release commit 95c48c7
- add SECURITY_AUDIT_2026-04-25.md from the deep audit of v0.8.0 (1 critical, 4 high, 5 medium, 4 low findings — kept for reference; remediation tracked separately)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two unrelated UX issues in the Knowledge Base manage tab:
1. Double-commit 400 error. The polling tick at index.js:1217 was
blindly overwriting a locally-set 'committing' status with whatever
the server reported. If the GET /batch poll raced ahead of the POST
commitJob reaching the DB, the card flipped back to 'awaiting_review',
the "אשר הכל" button re-enabled, and a second click triggered a duplicate
commit which the backend rejected with HTTP 400 ("job N is in status
'done', expected 'awaiting_review'"). Fix: never downgrade
'committing' → 'awaiting_review' from the polling response.
2. Admin tab scrolling. The "מקורות בנושא", "תוויות תת-נושא" and
"משימות אחרונות" panels were always expanded on tab entry, pushing
the actual upload form far down the page. Wrapped each in a
collapsible body with a chevron toggle, default collapsed. Data
lazy-loads on first expand.
Refs Task Master #21
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v0.8.0 ships the four-feature bundle from the v0.8.0 plan plus
mid-flight fixes discovered when batch-testing 5 circulars.
New capabilities:
* Multi-file batch upload. AI classifier (Sonnet via ai-gateway)
reads the first 3 pages of each PDF and proposes kind / title /
identifier / labels / dates / summary. User reviews + edits per
card before committing. Two-phase ingest: classify →
awaiting_review → embed.
* 'tool' kind for academic assessment instruments (GMFCS, MACS).
* Labels (kb_label / kb_source_label) for sub-topic navigation;
classifier proposes shared label slugs so the graph builds
itself.
* 'תוויות' admin sub-section for merge/delete of unused labels.
* NEW 'ממתינים לאישור' panel: lists batches still in
awaiting_review with status counters so a hard-refreshed user
can resume their review — closes the RAM-only _activeBatch gap.
Mid-flight fixes folded in:
* classifier timeout 45s→90s, concurrency 5→2. ai-gateway
serializes through a single Claude OAuth session; with conc=5,
jobs 4-5 of a 5-file batch consistently timed out.
* labels schema array-of-string → comma-separated string. Claude
via ai-gateway returned [{}, {}, {}] for items:{type:"string"}.
_normalize accepts both shapes defensively.
Backfilled ai_classified_at + description + 1-3 labels for the 10
sources ingested in earlier sessions so the filter UX is uniform.
Backend (shira-hermes) in commits 0d678da (Phase 6) + bb23b8a
(this session's fixes).
Refs Task Master #20
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a topics-management panel above the sources table. Admins can
create new legal domains (דיני עבודה, דין פלילי, נדל"ן) without
running SQL — slug + name + description + system prompt addendum
all live in one inline form. Per-row actions:
✏ Edit — opens inline form with all fields except slug
(slug is locked post-create — the S3 layout
depends on it)
👁 Toggle active — soft-disable: hides the topic from the user-facing
<select> while keeping data intact. Eye icon
flips between "השבת" and "הפעל"
🗑 Delete — hard-delete; only allowed when source_count=0.
Otherwise the UI directs the user to soft-disable
instead
Any topic mutation (create / rename / toggle / delete) invalidates
the cached _topics list so the user-facing dropdown re-fetches
immediately — admins don't have to refresh the page to see their
own changes.
Service.request() now distinguishes 4xx from 5xx upstream errors:
4xx is rethrown as BadRequest (so the user sees "slug already exists"
instead of a generic 500), 5xx stays as Error.
Backend: depends on shira-hermes commit 49503ca (admin topic
endpoints). No new migration — kb_topic schema from migration 001
is sufficient.
Description field in manifest was rewritten to Hebrew to match the
in-CRM scopeNames label ("מאגר ידע").
Refs Task Master #15
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds full CRUD for KB sources from the EspoCRM UI: a sortable table at
the top of the 'ניהול' tab with kind / title-identifier-updated / chunk
count / actions columns, a kind filter (חוק / תקנות / חוזרים / פסיקה),
and three per-row actions:
✏ Edit — inline form with title, identifier, dates, source URL
↻ Reingest — re-parses the original file in place, replacing chunks
while keeping the source row + hand-edited metadata
🗑 Delete — confirm dialog with chunk count; double confirm above
100 chunks; also deletes the MinIO object
Browser polls the standard /KnowledgeBase/action/job endpoint for
re-ingest progress (same banner as upload). On terminal status
(done|failed) the sources table auto-refreshes so chunk_count and
last_ingest reflect the new state. Topic switch invalidates the
per-topic admin-sources cache so a stale list doesn't bleed across
topics.
Backend: depends on shira-hermes commit 17f93b5
(GET/PUT/DELETE /admin/kb/sources + POST /reingest).
Refs Task Master #14
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Court rulings get their own kind so users can filter search results
to "פסיקה" specifically and upload case-law PDFs from the ניהול tab.
PHP validators in Controller + Service accept the new kind; the search
dropdown and upload kind dropdown gain a "פסיקה" option; kindHe in
index.js maps caselaw → "פסיקה" so jobs list, search results, and
browse view all label it consistently.
Backend: depends on shira-hermes commit 16eeeff (kind validators +
chunker fallback) and migration 003_caselaw_kind.sql (already applied
on dev — alters CHECK constraints on kb_source.kind and
kb_ingest_job.kind to include 'caselaw').
Refs Task Master #18
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a "ניהול" tab with multipart file upload, async job tracking, and
a recent-jobs list with live status polling. Browser POSTs to
/KnowledgeBase/action/upload (PHP proxy), which forwards as multipart
to shira-hermes /admin/kb/upload. shira-hermes returns a job_id
immediately and processes parse/chunk/embed in a background task; the
browser polls every 2s until status hits done|failed.
New EspoCRM endpoints:
POST /KnowledgeBase/action/upload (multipart, $_FILES['file'])
GET /KnowledgeBase/action/jobs (list, filtered by topicId/status)
GET /KnowledgeBase/action/job?id (single-job detail)
The X-User-Name header is forwarded so kb_ingest_job records who
uploaded each file. Cross-topic guard in switchTopic stops polling
when the user changes topics mid-upload (the job continues server-side).
Depends on shira-hermes commit 0cb89fb (admin upload endpoints +
migration 002).
Refs Task Master #13
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generalize the Knowledge Base from insurance-only to multi-domain.
Introduces kb_topic table (shira-hermes), topic-aware endpoints
(/kb/topics, /kb/search, /kb/ask, /kb/ask/stream all accept topic_id),
a topic picker <select> in the EspoCRM UI (localStorage-persisted),
and renames the LLM tool search_insurance_kb → search_legal_kb.
All existing sources migrate to topic_id=1 (ביטוח לאומי); the UI
defaults to that topic so no visible regression for current users.
Refs Task Master #12
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive Coolify-prod runbook covering: pgvector PG provisioning,
MinIO bucket, shira-hermes dockerimage app, Infisical /prod secrets,
data migration choice (pg_dump+S3 mirror vs fresh ingest), KB extension
install on prod EspoCRM, SmartAssistant integration config,
optional n8n auto-scan workflow clone, DNS, and end-to-end smoke test
sequence. Currently the entire stack is dev-only — this task makes
KB live on the production CRM at crm.prod.marcus-law.co.il.
Captures the open-but-not-blocking items from the v0.2.1 sessions plus
the comprehensive plan for generalizing the KB beyond ביטוח לאומי.
#9-11: carry-over polish noted at the v0.2.1 close
- #9 SSE rejoin (in-flight stream survives F5 / tab close)
- #10 n8n release-asset auto-attach has been failing for every release
from v0.1.8 through v0.2.1; manual curl uploads as workaround
- #11 ask mode source picker hides text-source citations (Wikisource law)
because /kb/ask filters sources_used to PDFs only
#12-16: multi-topic KB plan (next session)
- #12 Phase 1 — DB schema with kb_topic + kb_source.topic_id, /kb/topics
endpoint, topic-aware system prompts, topic dropdown in UI;
backwards-compat by seeding topic_id=1 ביטוח לאומי and migrating
the 6 existing sources to it
- #13 Phase 2 — file upload from the management panel, async ingestion
via kb_ingest_job table, live job-status polling
- #14 Phase 3 — sources management UI: edit metadata, delete (with
cascade messaging), re-ingest, jobs/failures view
- #15 Phase 4 — topic CRUD UI for admins (add דיני עבודה, דין פלילי,
etc., edit prompts, soft-disable)
- #16 Phase 5 (optional) — per-topic ACL by EspoCRM role with view vs
manage permissions
Phases 1-3 are the must-ship subset for what the user asked. Phase 4 is
admin convenience. Phase 5 is scoped only if firms ask for it.
User wanted to see what Shira is doing while she thinks — like Claude
does — instead of staring at an incrementing seconds counter. Now the
ask UI shows one line per agent step as it happens:
3s 🧠 קוראת את השאלה
4s 🔍 מחפשת בבסיס הידע: "תקנה 36"
9s ✓ נמצאו 5 קטעים רלוונטיים
10s 🔍 מחפשת בבסיס הידע: "תקנה 36 ביטוח לאומי"
14s ✓ נמצאו 5 קטעים רלוונטיים
15s 🧠 ממשיכה לחקור
22s ✍️ מנסחת תשובה
→ תשובה
Client wiring:
- runAsk now opens an EventSource at ?entryPoint=KnowledgeBaseAskStream
instead of POSTing JSON. Each SSE event of type thinking/tool_start/
tool_done/tool_error/writing appends a row to a stacked progress log.
The final {type:answer,text,sources} event triggers renderAskAnswer
with the existing split view. {type:error} surfaces via showError.
- Module-level state from v0.1.9 unchanged in spirit but now stores the
EventSource + accumulated events. afterRender on view re-mount replays
the entire event list and re-binds onmessage/onerror — the SSE
connection itself doesn't drop because it lives at module scope.
- Elapsed-time pill stays for cadence but is decoration; the real
signal is the per-event log.
EspoCRM proxy:
- New EntryPoint KnowledgeBaseAskStream — required because EventSource
is GET-only and ?message= must come via the URL. PHP buffering is
disabled (zlib + ob + apache_setenv no-gzip), Apache headers are set
for SSE (no-cache, X-Accel-Buffering: no), and we cURL into
shira-hermes /kb/ask/stream with CURLOPT_WRITEFUNCTION echoing each
byte and flushing immediately. The handler ends with exit; to bypass
EspoCRM's Response wrapper which would otherwise emit headers/body
on top of our raw stream.
- KnowledgeBaseService gained two public accessors (getStreamingUrl,
getStreamingApiKey) so the EntryPoint can build the cURL without
going through request() (which buffers the whole body).
- entryPoints.json registers the new class.
Server (shira-hermes 76fd77f):
- AgentRunner.run accepts an on_event sync callback fired before each
LLM call (thinking) and around each tool call (tool_start, tool_done,
tool_error). Hebrew labels in agent_runner._tool_label / _tool_done_label
ship over the wire ready-to-display — keeps the client free of
translation concerns.
- POST /kb/ask/stream wraps the existing agent runner in an asyncio
task feeding an asyncio.Queue, returns StreamingResponse with proper
SSE headers (X-Accel-Buffering, Cache-Control). 15s comment-line
heartbeats keep the connection open through proxy idle-timeouts.
- Original POST /kb/ask still works — additive change.
Refs Task Master #8
User reported the search card "looks broken" when the selected hit is the
law (Wikisource source 5) — the right preview pane just duplicates the
chunk text already visible in the left list card. For PDF sources the
right pane shows the PDF page, but for text-only sources it had nothing
useful to add and fell back to a copy.
Fix:
- /kb/search hits gained chunk_index (shira-hermes b127caf), and
/kb/source/{id}/chunks gained an `around=<chunk_index>&ctx=<k>` window.
- Espo proxy plumbing: getActionChunks reads `around`/`ctx` query
params and forwards them; KnowledgeBaseService::sourceChunks accepts
optional $around + $ctx and appends them to the upstream URL.
- showSearchPreview for non-PDF hits now fetches the matched chunk plus
2 sections before and 2 after, renders them as stacked panels: the
matched section gets a yellow #fff3cd highlight, neighbors render as
muted context blocks. The panel header carries a prominent
"פתח ב-Wikisource" button so users can jump to the public source page.
After render the matched section auto-scrolls into view.
- Race protection via _previewToken: a click on a different hit while a
fetch is in flight cancels the stale render so the user always sees
the section that matches the currently-selected card.
PDF path unchanged.
Refs Task Master #7
Two requests on top of v0.1.9:
1. Drag-to-resize splitter
The previous fixed Bootstrap col-md-5/col-md-7 split locked the
user into 41.66% answer / 58.33% PDF. Replaced with a flex-row
layout containing a 6px drag handle. mousedown spawns a fixed
transparent overlay so the PDF.js iframe doesn't swallow mouse
events while dragging; mouseup persists the chosen ratio (clamped
15-85%) to localStorage as kb-split-pct, so the next page-load and
the next session both come back to the same layout. Both the
ask split view and the search split view share _buildSplitShell +
_wireSplitter — single source of truth for the geometry.
2. Search survives a view switch
Search had the same view-bound promise problem ask had before
v0.1.9. Hoisted the in-flight request to a module-level
_activeSearch and the most recent {query, kind, hits, selectedIdx}
to _lastSearch + sessionStorage. afterRender restores the input,
the kind filter, the hit list (with the same hit highlighted), and
the right-pane preview. Click handler updates selectedIdx so a
round-trip lands on the user's last-clicked hit, not always on the
top-ranked one.
The persistence helpers grew a tiny refactor: _loadJson/_saveJson/_clear
replace the ask-only SS_KEY shim and now back both ask + search keys.
Refs Task Master #5, #6
Two recurring frustrations from the v0.1.8 demo:
1. Searching "מהי תקנה 37" only returned the תקנה 37 חוזר and the law —
ספר הליקויים, the canonical medical reference, was crowded out by the
document whose title literally contained the query terms. Fixed in
shira-hermes 1441a41 by adding an LLM-driven query-expansion path to
/kb/search (default expand=true). Same query now also returns 2 hits
from src 29 (ספר הליקויים).
2. Asking Shira a question and clicking another EspoCRM view dropped the
in-flight request — coming back rendered a blank kb-results pane, even
though the request had completed in the background. Fixed here:
{question, promise, startedAt} are now stored at module scope (closure
shared across remounts of this view). afterRender re-binds handlers
for any active ask, resumes the progress panel with the original start
time so the elapsed counter doesn't reset, and replays the most recent
completed answer from sessionStorage so the user sees their result
even after a hard reload. State is intentionally session-scoped — no
answers persist across browser sessions.
Refs Task Master #3, #4
Search and ask modes now render results as a two-column layout: a left
column with ranked hit cards (search) or Shira's answer + source picker
(ask), and a right column with an iframe showing the source PDF scrolled
to page_number. Clicking a hit (search) or source pill (ask) swaps the
iframe's src, so users can verify a quote against the original PDF
without leaving the KB tab.
- search: renderSearchResults lays out results as panel cards on the
left (with kind + title + section + "עמ׳ N" label); the top hit is
pre-selected and its PDF loads on the right. Clicking any card
re-highlights it and swaps the preview. Law-kind hits (Wikisource
text) gracefully fall back to a chunk-text panel with a Wikisource
link so the right pane never 404s on a text source.
- ask: renderAskAnswer dedups /kb/ask's sources[] by source_id,
collects all cited pages per source, and renders a picker row plus
per-source page-jump buttons. First source's first page loads on
initial render; buttons swap the iframe without re-running the query.
Depends on shira-hermes commits e534709 + 1ca1cc0 (chunker page_number
propagation + null-byte strip) — without them, every PDF collapses to
page 1 in the DB and the jump links are cosmetic.
Refs Task Master #1
chaim