Nitzan Pomerantz
75ee79bbfd
Add percentage-based backup outlier filtering for heterogeneous data
Root cause: IQR filtering fails with wide distributions (mixed room counts, property types).
Example: 9,459 NIS/sqm deal (68% below average) passed IQR with k=1.0 due to wide IQR.
Solution: Dual filtering approach - IQR + percentage backup
Changes:
- Add ANALYSIS_USE_PERCENTAGE_BACKUP config (default: true)
- Add ANALYSIS_PERCENTAGE_THRESHOLD config (default: 0.5 = 50%)
- Apply percentage backup after IQR when method=iqr
- Update outlier report with percentage backup parameters
- Update CLAUDE.md docs with dual filtering explanation
Behavior:
1. Hard bounds filter (1K-100K/sqm, 100K min total)
2. IQR filter (k=1.0) - catches mild outliers
3. Percentage filter (50% from median) - catches extreme outliers
4. Deal removed if flagged by ANY method
Example: 12K/sqm deal (60% below 30K median) now caught by percentage backup
even when IQR bounds are permissive due to heterogeneous data.
Tests: All 326 tests pass, manual verification confirms extreme outliers removed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 14:44:14 +02:00
..
2025-11-29 14:44:14 +02:00
2025-10-30 22:24:40 +02:00
2025-11-29 14:44:14 +02:00
2025-11-28 22:40:29 +02:00
2025-10-30 22:24:40 +02:00