Ports the pdfplumber-based parser from the (currently inactive) n8n
workflow legal-aid-report-ingest.json so EspoCRM's GreenInvoiceBilling
extension can do manual upload + dry-run preview of monthly Legal Aid
payment reports without depending on n8n being live.
Auth: X-Api-Key (matches admin_kb pattern).
Response shape: same JSON the existing LegalAidPaymentReportService
ingest pipeline expects, so the EspoCRM controller can hand it
straight back to ::ingest($payload, $pdf, dryRun=true).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
/opt/data on shira-hermes is now a Docker named volume on both dev and prod
(added 2026-04-27 so learned skills/profiles/cases survive redeploys). Named
volumes copy image content only on FIRST creation — after that the volume is
opaque to image updates. So a new entry added to config/skills/ in the repo
would never reach an instance that already has a populated volume.
Fix: at FastAPI startup, walk /app/config/skills/ (baked into the image,
outside the volume mount) and copy any directory whose name does not yet
exist under /opt/data/skills/. Skills already present — whether learned by
the model, edited by a user, or seeded earlier — are never touched.
- skills.py: BAKED_SKILLS_DIR derived from __file__ so it stays correct
regardless of WORKDIR; seed_skills_from_image() returns (added, skipped)
- main.py: call after ensure_data_dirs() and log the result
- Verified with a unit fixture: 2 new baked skills copied, 1 pre-existing
skill with custom content left untouched
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- pyproject.toml -> 0.3.0 (significant changes since 0.2.0:
full Phase 1-7 KB stack, classifier+labels, batch-upload UI,
pending-review panel, bucket rename to legal-kb)
- api/__init__.py reads version from pyproject at import time so
/api/health and OpenAPI both reflect reality without a second
hardcoded copy
- api/services/kb/voyage.py: _post_with_retry wraps embed/rerank
POSTs with exponential backoff on 429+5xx (max 5 retries, base
2s, capped 30s, +25% jitter, honors Retry-After header). Dev and
prod share a single VOYAGE_API_KEY today, so a simultaneous bulk
re-ingest on both used to fail half the calls; now the burst is
absorbed in seconds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three endpoints under /kb/*, same X-Api-Key auth as the admin and
SmartAssistant routes:
- POST /kb/search — hybrid search + rerank, returns ranked chunks as JSON.
No LLM in the path, so it's fast enough for a live UI.
- GET /kb/sources — list of active sources (law / regulation / circular)
with basic metadata, for the browse tab.
- GET /kb/source/{id}/chunks — all chunks of one source, ordered, so
the UI can show a document inline with its hierarchical headings.
- POST /kb/ask — full agent loop (same runner as SmartAssistant but with
allowed_toolsets=legal), for the ask-shira mode of the UI.
Dates are serialized to ISO strings so the frontend doesn't have to deal
with Python date objects.
Refs Task Master #2
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a searchable KB for חוק הביטוח הלאומי, תקנות הביטוח הלאומי, and חוזרי
הביטוח הלאומי. Hybrid search (pgvector cosine + tsvector/trgm) fused with
Reciprocal Rank Fusion, exposed to Shira as the `search_insurance_kb` tool.
- api/services/kb/: asyncpg pool, Voyage voyage-multilingual-2 client,
per-kind chunker (statute-by-section / circular header-aware), ingest
pipeline with supersession (old source → superseded_by), hybrid search.
- api/routes/admin_kb.py: POST /admin/kb/ingest (multipart), GET /admin/kb/stats.
- mcp_server/tools/legal_kb_tools.py: `search_insurance_kb` registered under
the `legal` toolset.
- pyproject.toml: +asyncpg, +python-docx, +boto3, +python-multipart.
Infra (external): pgvector/pgvector:pg16 on the shared PG, insurance_kb DB
with hnsw + gin indexes, MinIO bucket insurance-kb, KB_DATABASE_URL in
Infisical /espocrm, VOYAGE_API_KEY/VOYAGE_MODEL/KB_DATABASE_URL on the
shira-hermes Coolify app.
Refs Task Master #2
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chaim