feat(KB): v0.8.0 — bulk upload, AI classifier, labels, tool kind

v0.8.0 ships the four-feature bundle from the v0.8.0 plan plus
mid-flight fixes discovered when batch-testing 5 circulars.

New capabilities:
  * Multi-file batch upload. AI classifier (Sonnet via ai-gateway)
    reads the first 3 pages of each PDF and proposes kind / title /
    identifier / labels / dates / summary. User reviews + edits per
    card before committing. Two-phase ingest: classify →
    awaiting_review → embed.
  * 'tool' kind for academic assessment instruments (GMFCS, MACS).
  * Labels (kb_label / kb_source_label) for sub-topic navigation;
    classifier proposes shared label slugs so the graph builds
    itself.
  * 'תוויות' admin sub-section for merge/delete of unused labels.
  * NEW 'ממתינים לאישור' panel: lists batches still in
    awaiting_review with status counters so a hard-refreshed user
    can resume their review — closes the RAM-only _activeBatch gap.

Mid-flight fixes folded in:
  * classifier timeout 45s→90s, concurrency 5→2. ai-gateway
    serializes through a single Claude OAuth session; with conc=5,
    jobs 4-5 of a 5-file batch consistently timed out.
  * labels schema array-of-string → comma-separated string. Claude
    via ai-gateway returned [{}, {}, {}] for items:{type:"string"}.
    _normalize accepts both shapes defensively.

Backfilled ai_classified_at + description + 1-3 labels for the 10
sources ingested in earlier sessions so the filter UX is uniform.

Backend (shira-hermes) in commits 0d678da (Phase 6) + bb23b8a
(this session's fixes).

Refs Task Master #20

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-25 20:18:34 +00:00
parent 30a9c397e8
commit 8a4f16a50f
7 changed files with 1020 additions and 66 deletions
+21 -8
View File
@@ -180,7 +180,7 @@
"id": "15",
"title": "Phase 4 — Topic CRUD UI (admins can add/edit/disable topics from the panel)",
"description": "Once Phase 1 ships there will be a single seeded topic ('ביטוח לאומי'). For Klear and other firms to actually use the multi-topic capability they need a UI to create new topics — דיני עבודה, דין פלילי, נדל\"ן — without anyone running SQL. Add a topics-management section in the 'ניהול' tab: list active+inactive topics, add new (slug + name + system_prompt_addendum), edit (rename, change prompt addendum), soft-delete (sets is_active=false). Soft-delete keeps the data but hides the topic from the user-facing dropdown.",
"status": "in-progress",
"status": "done",
"priority": "normal",
"details": "shira-hermes endpoints:\n- GET /admin/kb/topics: list including inactive ones (with source counts).\n- POST /admin/kb/topics: create. Validates slug (lowercase, hyphens only, unique). Returns created row.\n- PUT /admin/kb/topics/{id}: update name, description, system_prompt_addendum, is_active.\n- DELETE /admin/kb/topics/{id}: 409 Conflict if topic has sources; otherwise hard-delete. Soft-delete (is_active=false) is the normal path.\n\nEspoCRM client:\n- New section in 'ניהול' tab: 'נושאים' table — slug, name, source count, is_active toggle, edit button.\n- Add new: modal with slug, name, description, prompt-addendum textarea (pre-populated with a generic template like 'You are answering legal questions about <topic>. Cite sections explicitly when possible. Never answer from generic legal training if the KB has a matching section').\n- Edit: same modal pre-filled. Saving updates the topic; if name changed, the topic dropdown refreshes.\n- The 'ניהול' tab itself only shows the topics section if user is admin. Regular users see only sources management for the topics they have access to.\n\nDefault topic on first install of multi-topic version: still id=1 'ביטוח לאומי'. New installs get the same seed.",
"testStrategy": "1) Create topic 'דיני עבודה' (slug 'employment-law'). 2) Verify it appears in GET /kb/topics within seconds. 3) The user-facing topic dropdown lists it. 4) Upload a PDF under it (Phase 2 flow), verify search/ask scoped to it works. 5) Edit prompt addendum — /kb/ask answers in context of employment law. 6) Soft-delete the test topic — disappears from user dropdown but kept in admin list.",
@@ -189,7 +189,7 @@
"12"
],
"createdAt": "2026-04-25T13:30:00Z",
"updatedAt": "2026-04-25T17:30:11.758Z"
"updatedAt": "2026-04-25T17:47:57.615Z"
},
{
"id": "17",
@@ -207,16 +207,17 @@
"id": "16",
"title": "Phase 5 (optional) — Per-topic ACL by EspoCRM role",
"description": "Not every user in a multi-topic firm should see every topic. Example: only the criminal-law department needs the criminal KB; the family-law team shouldn't. Wire EspoCRM's existing role/team model to the KB topics: each topic can be restricted to specific roles, with two permission levels — 'view' (use search/ask) and 'manage' (upload/edit/delete). Without role mapping, all non-portal users see all topics (current behavior). With mapping, the topic dropdown filters to topics the user has at least 'view' on, and the management UI gates 'manage' actions to users with the manage permission for that topic.",
"status": "pending",
"status": "deferred",
"priority": "low",
"details": "DB: new table kb_topic_acl: topic_id FK, role_name TEXT, permission TEXT CHECK in ('view', 'manage'), PRIMARY KEY (topic_id, role_name, permission).\n\nshira-hermes endpoints:\n- /kb/topics: filter by user's roles. The user's role list comes from the EspoCRM proxy (sent in a header or JWT claim — needs design).\n- /admin/kb/* actions: enforce 'manage' permission per source's topic_id.\n\nEspoCRM client:\n- Topic dropdown: only shows topics the user has 'view' on.\n- Management tab: only visible if user has 'manage' on at least one topic.\n- Edit/delete buttons in the sources table: visible only when user has 'manage' on that source's topic.\n\nThis is the most-likely-to-skip phase if the user's firm is small and everyone needs everything. Worth scoping out depth before committing.",
"details": "DB: new table kb_topic_acl: topic_id FK, role_name TEXT, permission TEXT CHECK in ('view', 'manage'), PRIMARY KEY (topic_id, role_name, permission).\n\nshira-hermes endpoints:\n- /kb/topics: filter by user's roles. The user's role list comes from the EspoCRM proxy (sent in a header or JWT claim — needs design).\n- /admin/kb/* actions: enforce 'manage' permission per source's topic_id.\n\nEspoCRM client:\n- Topic dropdown: only shows topics the user has 'view' on.\n- Management tab: only visible if user has 'manage' on at least one topic.\n- Edit/delete buttons in the sources table: visible only when user has 'manage' on that source's topic.\n\nThis is the most-likely-to-skip phase if the user's firm is small and everyone needs everything. Worth scoping out depth before committing.\n\n---\nDEFERRED 2026-04-25: small firm, all users need access to all topics. Re-open if the firm grows past ~3 lawyers AND topics start to genuinely need siloing (e.g. confidential criminal cases the partners shouldn't browse). Until then, the only access control is `User::isPortal()` already enforced in `Controller::checkAccess()`.",
"testStrategy": "Set up two test users: lawyer_a with role 'criminal-law', lawyer_b with role 'employment-law'. Topics: 'דין פלילי' restricted to 'criminal-law' role only, 'דיני עבודה' restricted to 'employment-law'. lawyer_a sees only the criminal topic; lawyer_b only employment. Cross-user attempts return 403.",
"subtasks": [],
"dependencies": [
"12",
"15"
],
"createdAt": "2026-04-25T13:30:00Z"
"createdAt": "2026-04-25T13:30:00Z",
"updatedAt": "2026-04-25T17:51:07.124Z"
},
{
"id": "18",
@@ -241,13 +242,25 @@
"subtasks": [],
"dependencies": [],
"createdAt": "2026-04-25T16:54:54.478470Z"
},
{
"id": "20",
"title": "v0.8.0 — kind 'tool' + labels + bulk upload with AI classification",
"description": "Phase 6. Adds 'tool' kind for academic assessment instruments (GMFCS, MACS, etc), label-based sub-topic grouping (kb_label many-to-many), multi-file drag-drop upload, and AI-extracted metadata via Claude/ai-gateway. Two-phase ingest: classify → await_review → embed. Replaces single-file upload with batch-review screen. Also creates 'תוויות' admin tab. Subsumes task #19 (labels). Plan: ~/.claude/plans/hidden-tickling-ullman.md",
"details": "See /home/chaim/.claude/plans/hidden-tickling-ullman.md for the full architecture, migration scripts, API surface, classifier prompt, UX, and file-level breakdown. ~7 days estimated.",
"testStrategy": "",
"status": "in-progress",
"dependencies": [],
"priority": "medium",
"subtasks": [],
"updatedAt": "2026-04-25T18:21:44.731Z"
}
],
"metadata": {
"version": "1.0.0",
"lastModified": "2026-04-25T17:30:11.759Z",
"taskCount": 19,
"completedCount": 12,
"lastModified": "2026-04-25T18:21:44.731Z",
"taskCount": 20,
"completedCount": 13,
"tags": [
"master"
]