chaim
e534709706
Commit 426b0d6 fixed page propagation through the Chunk reconstruction in
chunk(), but two downstream paths still collapsed every sub-chunk to the
parent's starting page:
1. _split_oversized operated on markerless content (since _as_dicts inside
chunk_statute/chunk_circular stripped markers before returning), so when
a single giant chunk was sliced into pieces — the common path for
ספר הליקויים / ספר המבחנים, which have no statute-style section markers —
every piece inherited c.page_number = parent's start page. 209/209 chunks
landed on page 1.
2. chunk_circular packed paragraphs into sub-chunks all tagged with
seg_page (the segment's opening page), with no attempt to track where
each sub-chunk actually begins within the original text. חוזר תקנה 37
(8 chunks) and חוזר נכות 1941 (4 chunks) — neither had header matches —
landed entirely on page 1.
Fixes:
- chunk_statute and chunk_circular now return list[Chunk] with markers
still embedded in content; _as_dicts is called once at the end of
chunk() so markers are stripped AFTER splitting.
- _split_oversized builds a local page map from markers in each parent
chunk's content, with a virtual (0, parent.page_number) entry so pieces
before the first real marker still inherit the correct page.
- chunk_circular tracks each paragraph's absolute offset in the original
text and computes each emitted sub-chunk's page_number from its first
paragraph's offset against the full-text page_map.
Verified on synthetic 3-page input: regulation path now emits pages 1,1,2,3
across 4 pieces (was 1,1,1,1); circular path emits 1,1,2,2,3,3 (was
1,1,1,1,1,1).
Refs Task Master #2