Change default IQR multiplier to 1.0 for more aggressive outlier filtering

- Change ANALYSIS_IQR_MULTIPLIER default from 1.5 to 1.0 in config.py
- Add iqr_multiplier parameter to all filtering & statistics functions
- Allow runtime override via MCP tools (get_valuation_comparables, get_deal_statistics)
- Update CLAUDE.md docs with new default & override examples

Rationale: k=1.0 catches more suspicious deals (e.g. 43% below median) while
still preserving legitimate edge cases via hard bounds. Users can override
per-call for more conservative filtering (k=1.5) if needed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Nitzan Pomerantz
2025-11-28 22:40:29 +02:00
parent 1bc94e39ba
commit 8e591423da
6 changed files with 43 additions and 17 deletions
+8 -4
View File
@@ -252,7 +252,7 @@ The MCP now includes configurable outlier detection and robust statistical measu
```bash
# Outlier Detection Strategy
ANALYSIS_OUTLIER_METHOD=iqr # Options: iqr, percent, none (default: iqr)
ANALYSIS_IQR_MULTIPLIER=1.5 # 1.5=moderate, 3.0=conservative (default: 1.5)
ANALYSIS_IQR_MULTIPLIER=1.0 # 1.0=aggressive, 1.5=moderate, 3.0=conservative (default: 1.0)
ANALYSIS_MIN_DEALS_FOR_OUTLIER_DETECTION=10 # Minimum deals needed (default: 10)
# Hard Bounds (catches obvious data errors)
@@ -273,10 +273,14 @@ ANALYSIS_INCLUDE_UNFILTERED_STATS=true # Report both filtered/unfiltered (defau
from nadlan_mcp.govmap.statistics import calculate_deal_statistics
from nadlan_mcp.config import GovmapConfig
# With outlier filtering (default behavior)
# With outlier filtering (default behavior, k=1.0)
config = GovmapConfig() # Uses env vars or defaults
stats = calculate_deal_statistics(deals, config)
# Override IQR multiplier for more/less aggressive filtering
stats = calculate_deal_statistics(deals, config, iqr_multiplier=1.5) # More conservative
stats = calculate_deal_statistics(deals, config, iqr_multiplier=0.5) # More aggressive
# Access filtered statistics (main results after outlier removal)
filtered_mean = stats.filtered_price_per_sqm_statistics["mean"]
filtered_median = stats.filtered_price_per_sqm_statistics["median"]
@@ -292,10 +296,10 @@ original_mean = stats.price_per_sqm_statistics["mean"]
```
**Design Principles:**
- **Enabled by default**: Moderate filtering (k=1.5) to catch common errors
- **Enabled by default**: Aggressive filtering (k=1.0) to catch common errors & suspicious deals
- **Transparent**: Both filtered and unfiltered statistics returned
- **Conservative with real data**: Hard bounds + IQR preserve legitimate high-end properties
- **Configurable**: Adjust sensitivity via environment variables
- **Configurable**: Adjust sensitivity via environment variables or function parameters
- **MCP provides data, LLM provides intelligence**: Outlier detection improves data quality; LLM interprets results
### Retry Logic