mcp-servers/nadlan-mcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Change percentage threshold from 50% to 40% for tighter outlier filtering

Browse Source 
Rationale: 50% threshold too permissive for Israeli real estate market.
Within same room count/area, legitimate price variance typically ±30-35%, not 50%.

Example: 13,402 NIS/sqm deal (47% below 25,120 median) was passing with 50% threshold.
This is almost certainly data error or partial deal, not legitimate market variation.

Changes:
- ANALYSIS_PERCENTAGE_THRESHOLD default: 0.5 → 0.4 (40%)
- Update CLAUDE.md docs to reflect 40% threshold

Impact:
- New lower bound: median × 0.6 (was median × 0.5)
- New upper bound: median × 1.4 (was median × 1.5)
- Tighter filtering while preserving legitimate high/low-end deals
- Better aligned with actual Israeli real estate market variance

Tests: All 326 tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Nitzan Pomerantz
2025-11-29 15:11:47 +02:00
parent 75ee79bbfd
commit 107e4079d9
2 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+3 -3
View File
@@ -244,7 +244,7 @@ The MCP now includes configurable outlier detection and robust statistical measu
**Key Features:** **Key Features:**
- **IQR-based outlier detection**: Uses Interquartile Range (robust to skewed data) - **IQR-based outlier detection**: Uses Interquartile Range (robust to skewed data)
- **Percentage backup filtering**: Catches extreme outliers (>50% from median) in heterogeneous data - **Percentage backup filtering**: Catches extreme outliers (>40% from median) in heterogeneous data
- **Hard bounds filtering**: Removes obvious errors (price_per_sqm < 1K or > 100K NIS/sqm, deals < 100K NIS) - **Hard bounds filtering**: Removes obvious errors (price_per_sqm < 1K or > 100K NIS/sqm, deals < 100K NIS)
- **Robust volatility**: Investment analysis uses IQR instead of std_dev for stability ratings - **Robust volatility**: Investment analysis uses IQR instead of std_dev for stability ratings
- **Transparent reporting**: Returns both filtered and unfiltered statistics with outlier reports - **Transparent reporting**: Returns both filtered and unfiltered statistics with outlier reports
@@ -258,7 +258,7 @@ ANALYSIS_MIN_DEALS_FOR_OUTLIER_DETECTION=10 # Minimum deals needed (default: 10
# Percentage-based Backup Filtering (catches extreme outliers in heterogeneous data) # Percentage-based Backup Filtering (catches extreme outliers in heterogeneous data)
ANALYSIS_USE_PERCENTAGE_BACKUP=true # Enable percentage backup (default: true) ANALYSIS_USE_PERCENTAGE_BACKUP=true # Enable percentage backup (default: true)
ANALYSIS_PERCENTAGE_THRESHOLD=0.5 # Remove deals >50% from median (default: 0.5) ANALYSIS_PERCENTAGE_THRESHOLD=0.4 # Remove deals >40% from median (default: 0.4)
# Hard Bounds (catches obvious data errors) # Hard Bounds (catches obvious data errors)
ANALYSIS_PRICE_PER_SQM_MIN=1000 # 1K NIS/sqm minimum (default: 1000) ANALYSIS_PRICE_PER_SQM_MIN=1000 # 1K NIS/sqm minimum (default: 1000)
@@ -301,7 +301,7 @@ original_mean = stats.price_per_sqm_statistics["mean"]
``` ```
**Design Principles:** **Design Principles:**
- **Dual filtering approach**: IQR (k=1.0) + percentage backup (50%) catch both mild and extreme outliers - **Dual filtering approach**: IQR (k=1.0) + percentage backup (40%) catch both mild and extreme outliers
- **Handles heterogeneous data**: Percentage backup catches outliers when IQR becomes too permissive (mixed room counts, property types) - **Handles heterogeneous data**: Percentage backup catches outliers when IQR becomes too permissive (mixed room counts, property types)
- **Transparent**: Both filtered and unfiltered statistics returned - **Transparent**: Both filtered and unfiltered statistics returned
- **Conservative with real data**: Hard bounds + IQR + percentage preserve legitimate high-end properties - **Conservative with real data**: Hard bounds + IQR + percentage preserve legitimate high-end properties
nadlan_mcp/config.py
+1 -1
View File
@@ -73,7 +73,7 @@ class GovmapConfig:
== "true" == "true"
) )
analysis_percentage_threshold: float = field( analysis_percentage_threshold: float = field(
default_factory=lambda: float(os.getenv("ANALYSIS_PERCENTAGE_THRESHOLD", "0.5")) default_factory=lambda: float(os.getenv("ANALYSIS_PERCENTAGE_THRESHOLD", "0.4"))
) )
# Hard Bounds for Price per Sqm (catches obvious data errors) # Hard Bounds for Price per Sqm (catches obvious data errors)