The test_market_activity_trend_stable test was failing intermittently
because it used get_recent_date() which approximates months as 30 days,
causing inconsistent month boundary calculations depending on when the
test runs.
Root cause:
- Using months_ago * 30 creates shifting month boundaries
- Calendar dates don't align consistently with 30-day periods
- Deals could fall into unexpected months based on test execution date
- Trend calculation compares first half vs second half of months
- Inconsistent month grouping led to "decreasing" instead of "stable"
Solution:
- Use explicit, deterministic dates (15th of each month)
- Calculate year-month combinations going back from current month
- Ensures exactly 1 deal per month for 12 consecutive months
- All deals guaranteed to be within time_period_months=12 window
- Month boundaries are now consistent regardless of test execution date
Before: Test result varied based on current date (flaky)
After: Test result is always "stable" (deterministic)
This ensures the test validates the trend detection logic without
temporal flakiness.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement configurable outlier detection and robust statistical measures to
improve analysis accuracy for real estate data. Addresses issues with data
entry errors, partial deals, and other anomalies that skew statistics.
Key Features:
- IQR-based outlier detection (moderate filtering by default, k=1.5)
- Hard bounds filtering for obvious errors (price_per_sqm, deal_amount)
- Robust volatility using IQR instead of std_dev for investment analysis
- Transparent reporting with both filtered and unfiltered statistics
Implementation:
- Add outlier_detection.py module with IQR/percent/hard bounds methods
- Add OutlierReport model and enhance DealStatistics with filtered fields
- Update calculate_deal_statistics() to support optional outlier filtering
- Update analyze_investment_potential() to use robust volatility
- Add 9 new configuration parameters for customization
- Add comprehensive test suite (24 tests) for outlier detection
- Update CLAUDE.md with usage documentation
Configuration (all via env vars):
- ANALYSIS_OUTLIER_METHOD=iqr (default, or percent/none)
- ANALYSIS_IQR_MULTIPLIER=1.5 (moderate, 3.0=conservative)
- ANALYSIS_PRICE_PER_SQM_MIN/MAX=1000/100000 (bounds in NIS/sqm)
- ANALYSIS_MIN_DEAL_AMOUNT=100000 (catches partial deals)
- ANALYSIS_USE_ROBUST_VOLATILITY=true (IQR-based CV)
- ANALYSIS_USE_ROBUST_TRENDS=true (filter before regression)
Testing:
- All existing tests pass (326 passed)
- 24 new comprehensive outlier detection tests
- Real-world scenario tests (partial deals, data errors)
Backward Compatible:
- Default behavior improves accuracy without breaking changes
- All new fields in models are optional
- Config parameters have sensible defaults
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adjusted sample_deals fixture to ensure 5 deals across 5 distinct months.
Previous dates collapsed into 3 months due to naive 30-day calculations.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Nitzan P