Commit Graph

4 Commits

Author SHA1 Message Date
chaim 1051ed3332 feat(legal-aid): async parse jobs to survive multi-minute vision latency
Claude Vision via ai-gateway's claude-code OAuth provider runs ~140s per
rendered page (a 3-page report measured at 417s), which blows past any safe
synchronous HTTP timeout — the EspoCRM controller's 240s curl gave up with
"0 bytes received" while ai-gateway kept working.

parse-pdf-report now accepts ?mode=async: it returns {jobId, status:"pending"}
immediately and runs the parse in the background, persisting the result/error
to a file-based job store under /opt/data (survives multi-worker + restarts).
Poll GET /parse-pdf-report/jobs/{jobId}. Default sync mode is unchanged so
n8n and other server-side callers keep working.

Refs prod ai-gateway log id cee4c17c (claudeMs=417513).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 09:56:21 +00:00
chaim 2628a37c56 fix(legal-aid): render PDF pages as JPEG q85 instead of PNG
Bundling all report pages as 180-DPI PNG base64 in one ai-gateway
chat-completions request blew past its body limit and returned 413.
A rendered table page is mostly white, so q85 JPEG is several times
smaller while keeping the table text crisp enough for Claude Vision.
Pairs with the ai-gateway limit bump to 25mb.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 08:26:45 +00:00
chaim 098a86e3fd fix(legal-aid): use Claude Vision for PDF parsing instead of pdfplumber
The real Legal Aid monthly reports have multi-line cells (sub-types like
"הודעה בהתאם להחלטה" wrap to two visual lines), RTL Hebrew column ordering
that flips depending on whether a row contains long Hebrew text, and
amounts that wrap across rows ("1478.5\n4" → 1478.54). pdfplumber's
extract_tables() returned 0 rows on a real report — the v1 parser
silently produced an empty preview.

Switching to the Claude-Vision path (mirrors api/services/ocr.py): render
each PDF page via PyMuPDF, send to ai-gateway with a JSON-output prompt,
parse the structured response. Claude reads RTL Hebrew + multi-line cells
natively and produces the same JSON shape that
LegalAidPaymentReportService.ingest() expects.

`?debug=true` added to return the raw model response for diagnostics
when a future PDF format breaks extraction.

pdfplumber stays in pyproject.toml for now in case we want a non-AI
fallback later, but the runtime path no longer touches it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 16:53:53 +00:00
chaim 3fa0775ffe feat(legal-aid): add /api/legal-aid/parse-pdf-report endpoint
Ports the pdfplumber-based parser from the (currently inactive) n8n
workflow legal-aid-report-ingest.json so EspoCRM's GreenInvoiceBilling
extension can do manual upload + dry-run preview of monthly Legal Aid
payment reports without depending on n8n being live.

Auth: X-Api-Key (matches admin_kb pattern).
Response shape: same JSON the existing LegalAidPaymentReportService
ingest pipeline expects, so the EspoCRM controller can hand it
straight back to ::ingest($payload, $pdf, dryRun=true).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 10:56:25 +00:00