Two bugs surfaced during Phase 1 validation on real circulars:
1. The chunker produced one giant chunk for large statutes (ספר הליקויים,
ספר המבחנים) that the section regex could not split. This exceeded
Voyage's 120K-token batch limit and the ingest failed. Add a hard
2000-token ceiling per chunk with a character-based fallback split that
preserves heading_path and section_ref.
2. ingest_source matched "existing source" by kind + COALESCE(identifier,
''), so any two sources with NULL identifier compared equal. The second
ingest would then supersede the first unrelated document. Match by
identifier only when one is provided; otherwise fall back to
(kind + title) and require the candidate to still be active.
Refs Task Master #2
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chaim
fd4430791b