Ports the pdfplumber-based parser from the (currently inactive) n8n
workflow legal-aid-report-ingest.json so EspoCRM's GreenInvoiceBilling
extension can do manual upload + dry-run preview of monthly Legal Aid
payment reports without depending on n8n being live.
Auth: X-Api-Key (matches admin_kb pattern).
Response shape: same JSON the existing LegalAidPaymentReportService
ingest pipeline expects, so the EspoCRM controller can hand it
straight back to ::ingest($payload, $pdf, dryRun=true).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- pyproject.toml -> 0.3.0 (significant changes since 0.2.0:
full Phase 1-7 KB stack, classifier+labels, batch-upload UI,
pending-review panel, bucket rename to legal-kb)
- api/__init__.py reads version from pyproject at import time so
/api/health and OpenAPI both reflect reality without a second
hardcoded copy
- api/services/kb/voyage.py: _post_with_retry wraps embed/rerank
POSTs with exponential backoff on 429+5xx (max 5 retries, base
2s, capped 30s, +25% jitter, honors Retry-After header). Dev and
prod share a single VOYAGE_API_KEY today, so a simultaneous bulk
re-ingest on both used to fail half the calls; now the burst is
absorbed in seconds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends scripts/fetch_national_insurance_law.py with a --regulations mode
that pulls all canonical (non-redirect) תקנות הביטוח הלאומי pages from the
MediaWiki API and writes <slug>.txt + <slug>.txt.meta.json pairs ready for
the existing scan-inbox cron.
Used to seed the live insurance KB with 92 regulations (~1,500 chunks,
~22MB Postgres) on dev. Wikisource's redirect graph dedups spelling
variants (קיצבה/קצבה, etc.) for free.
Bumps version to 0.2.0.
Refs Task Master #3
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a searchable KB for חוק הביטוח הלאומי, תקנות הביטוח הלאומי, and חוזרי
הביטוח הלאומי. Hybrid search (pgvector cosine + tsvector/trgm) fused with
Reciprocal Rank Fusion, exposed to Shira as the `search_insurance_kb` tool.
- api/services/kb/: asyncpg pool, Voyage voyage-multilingual-2 client,
per-kind chunker (statute-by-section / circular header-aware), ingest
pipeline with supersession (old source → superseded_by), hybrid search.
- api/routes/admin_kb.py: POST /admin/kb/ingest (multipart), GET /admin/kb/stats.
- mcp_server/tools/legal_kb_tools.py: `search_insurance_kb` registered under
the `legal` toolset.
- pyproject.toml: +asyncpg, +python-docx, +boto3, +python-multipart.
Infra (external): pgvector/pgvector:pg16 on the shared PG, insurance_kb DB
with hnsw + gin indexes, MinIO bucket insurance-kb, KB_DATABASE_URL in
Infisical /espocrm, VOYAGE_API_KEY/VOYAGE_MODEL/KB_DATABASE_URL on the
shira-hermes Coolify app.
Refs Task Master #2
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the CRM's readDocument returns success:false (scanned PDFs,
image-only DOCX, corrupted files), shira-hermes now:
1. Fetches the raw file bytes via new getDocumentBytes endpoint
2. Sends them to Claude Vision through ai-gateway:
- Images (PNG/JPG/TIFF/etc): direct vision call
- PDFs: render each page at 180 DPI via PyMuPDF, vision call per page
- DOCX: extract images from word/media/, vision call per image
New tool: read_document_ocr (explicit OCR request)
Existing tool read_document auto-falls-back to OCR on extraction failure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hatchling couldn't find the package directory (expected shira_hermes/).
Added [tool.hatch.build.targets.wheel] packages to include api/ and
mcp_server/ directories.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security:
- Path traversal: validate user_id, case_id, skill_name against safe regex
- Input sanitization in memory.py and skills.py file path construction
Correctness:
- ISR timezone: replace hardcoded UTC+3 with ZoneInfo("Asia/Jerusalem") for DST
- save_rule: use EspoCrmApiError.status_code instead of fragile string matching
- Background tasks: store references to prevent GC before completion
- Lock race: use dict.setdefault() for atomic lock creation
- Version: health.py now matches main.py (0.2.0)
- Skill names: normalized to lowercase for dedup consistency
Build:
- Dockerfile: install from pyproject.toml instead of hardcoded pip list
- Remove unused mcp>=1.0.0 dependency from pyproject.toml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chaim