feat: Phase 3 — sources management table in 'ניהול' tab
Adds full CRUD for KB sources from the EspoCRM UI: a sortable table at
the top of the 'ניהול' tab with kind / title-identifier-updated / chunk
count / actions columns, a kind filter (חוק / תקנות / חוזרים / פסיקה),
and three per-row actions:
✏ Edit — inline form with title, identifier, dates, source URL
↻ Reingest — re-parses the original file in place, replacing chunks
while keeping the source row + hand-edited metadata
🗑 Delete — confirm dialog with chunk count; double confirm above
100 chunks; also deletes the MinIO object
Browser polls the standard /KnowledgeBase/action/job endpoint for
re-ingest progress (same banner as upload). On terminal status
(done|failed) the sources table auto-refreshes so chunk_count and
last_ingest reflect the new state. Topic switch invalidates the
per-topic admin-sources cache so a stale list doesn't bleed across
topics.
Backend: depends on shira-hermes commit 17f93b5
(GET/PUT/DELETE /admin/kb/sources + POST /reingest).
Refs Task Master #14
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -165,7 +165,7 @@
|
||||
"id": "14",
|
||||
"title": "Phase 3 — Management panel UI (sources table, edit metadata, delete, re-ingest)",
|
||||
"description": "Today the only way to manage existing sources is via SQL on the shared PG (DELETE FROM kb_source WHERE id=…) plus an mc command on MinIO. End users need a panel where they can: see all sources for the selected topic, with chunk counts and ingestion dates; edit source metadata (title, identifier, dates, source_url); delete a source (with confirm and clear cascade messaging); re-ingest a source (re-fetch its file from MinIO processed/, drop chunks, re-run parse+chunk+embed); see the most recent ingest jobs (success + failure both). The panel sits in the same KB extension under a new tab 'ניהול'.",
|
||||
"status": "pending",
|
||||
"status": "in-progress",
|
||||
"priority": "high",
|
||||
"details": "shira-hermes endpoints:\n- GET /admin/kb/sources?topic_id=&kind=&limit=: list with id, kind, title, identifier, chunk_count, ingestion_date, last_ingest_status (joining kb_ingest_job).\n- PUT /admin/kb/sources/{id}: update editable fields (title, identifier, source_url, published_at, effective_at).\n- DELETE /admin/kb/sources/{id}: hard delete (cascades chunks via FK, drops processed/ files in S3 too).\n- POST /admin/kb/sources/{id}/reingest: requires source_id; finds the original file at original_path s3:// URI; creates a new kb_ingest_job row with status=queued and the existing source_id; background task replaces chunks+embeddings (does NOT change source_id, so all v0.1.11 chunk_index references remain valid IF the chunker output is stable; otherwise old _lastSearch sessionStorage will point at chunk_indexes that no longer exist — flag for testing). Should optionally just delete + re-create the source — simpler.\n\nEspoCRM client:\n- 'ניהול' tab. Top: header showing topic name + a count badge ('X מסמכים בנושא'). Below: 3 collapsed sections — uploaded jobs (collapsed if all done), sources table, recent failures.\n- Sources table: id, kind label, title, identifier, chunk_count, ingestion_date. Rightmost column: action buttons (✏ edit, 🗑 delete, ↻ re-ingest, 👁 view in browse mode).\n- Edit: opens a modal with the editable fields. PUT on save.\n- Delete: confirm modal that shows chunk_count + 'this will also remove the PDF from the search results'. Two-step confirmation if source has >100 chunks.\n- Re-ingest: confirms; submits POST /reingest; surfaces in the jobs section.\n- Use jQuery + Espo.Ui.dialog patterns consistent with the rest of the extension.\n\nACL gate: this tab visible only to users with role 'KB Admin' (define in EspoCRM if not present) or fall back to !isPortal for now.",
|
||||
"testStrategy": "1) Open 'ניהול' tab. See the 6 existing sources for ביטוח לאומי. 2) Edit one source's title — refresh → new title sticks. 3) Re-ingest ספר הליקויים. Job appears in the jobs section processing → done. Source still searchable end-to-end after. 4) Delete one source. Confirm modal appears with chunk count. After delete: source gone from /kb/sources, /kb/search no longer surfaces it. 5) Failed re-ingest (e.g. delete the file in MinIO first) — surfaces in failures section with error_message.",
|
||||
@@ -173,7 +173,8 @@
|
||||
"dependencies": [
|
||||
"13"
|
||||
],
|
||||
"createdAt": "2026-04-25T13:30:00Z"
|
||||
"createdAt": "2026-04-25T13:30:00Z",
|
||||
"updatedAt": "2026-04-25T17:03:34.486Z"
|
||||
},
|
||||
{
|
||||
"id": "15",
|
||||
@@ -222,11 +223,11 @@
|
||||
"description": "Add 'caselaw' alongside law/regulation/circular across the whole stack. Touches: DB CHECK constraints (migration 003), shira-hermes Python (Literal types, _KINDS, kind validation, chunker section detection), EspoCRM Controller/Service validators, template dropdowns, JS kindHe mapping.",
|
||||
"details": "DB: migration 003_caselaw_kind.sql alters CHECK on kb_source.kind and kb_ingest_job.kind to include 'caselaw'. shira-hermes: update Literal types in ingest.py, admin_kb.py, kb_public.py SearchRequest; add 'caselaw' to s3.py _KINDS; chunker.py — fall back to generic section detector (regex for 'פסק דין', 'תיק' / case number, paragraph numbers). EspoCRM: update validators in Controller postActionUpload + Service search/uploadFile; add 'caselaw' option to kind <select> in index.tpl (search box + upload form); update kindHe in index.js.",
|
||||
"testStrategy": "",
|
||||
"status": "in-progress",
|
||||
"status": "done",
|
||||
"dependencies": [],
|
||||
"priority": "medium",
|
||||
"subtasks": [],
|
||||
"updatedAt": "2026-04-25T16:49:30.816Z"
|
||||
"updatedAt": "2026-04-25T17:02:14.241Z"
|
||||
},
|
||||
{
|
||||
"id": "19",
|
||||
@@ -243,9 +244,9 @@
|
||||
],
|
||||
"metadata": {
|
||||
"version": "1.0.0",
|
||||
"lastModified": "2026-04-25T16:49:30.816Z",
|
||||
"taskCount": 18,
|
||||
"completedCount": 10,
|
||||
"lastModified": "2026-04-25T17:03:34.486Z",
|
||||
"taskCount": 19,
|
||||
"completedCount": 11,
|
||||
"tags": [
|
||||
"master"
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user