Commit Graph

27 Commits

Author SHA1 Message Date
Nitzan Pomerantz 0ac9d136cd Implementation of phase 4.1 2025-10-26 10:58:46 +02:00
Nitzan Pomerantz 48bfb00628 CR fix 2025-10-25 14:01:53 +03:00
Nitzan Pomerantz c87750f57f Fix median calculation
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-10-25 13:57:25 +03:00
Nitzan Pomerantz 4d9febf527 Fix autocomplete_address bug and add test coverage report
Bug Fix:
- Fixed autocomplete_address tool to use correct API response fields
- Changed from non-existent fields (addressLabel, settlementNameHeb, coordinates)
  to actual API fields (text, id, type, score, shape)
- Added coordinate parsing from WKT POINT format
- Impact: HIGH - tool was returning empty data for all fields

Test Coverage Report:
- Added comprehensive test coverage analysis
- Documented 34 passing tests
- Identified e2e test results for MCP tools
- Listed missing test cases and recommendations
- Overall assessment: Good coverage, one bug found and fixed

E2E Test Results:
-  find_recent_deals_for_address - WORKING
-  analyze_market_trends - WORKING
-  get_valuation_comparables - WORKING
-  autocomplete_address - FIXED (needs MCP server restart to take effect)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 13:45:52 +03:00
Nitzan Pomerantz ab19d86b63 Phase 3: Remove old monolithic govmap.py file (step 3/3)
All functionality has been extracted to the govmap package:
- validators.py - Input validation functions
- utils.py - Utility functions
- filters.py - Deal filtering logic
- statistics.py - Statistical calculations
- market_analysis.py - Market analysis functions
- client.py - GovmapClient class with API methods
- __init__.py - Package exports

Backward compatibility maintained through package __init__.py exports.
Existing imports like `from nadlan_mcp.govmap import GovmapClient` now
use the govmap package instead of the old monolithic file.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 13:21:17 +03:00
Nitzan Pomerantz 7757694077 Phase 3: Complete govmap package extraction (step 2/3)
- Created market_analysis.py with market analysis functions:
  - parse_deal_dates() - Date parsing and filtering helper
  - calculate_market_activity_score() - Activity metrics
  - analyze_investment_potential() - Investment analysis
  - get_market_liquidity() - Liquidity metrics

- Created client.py with GovmapClient class:
  - Core API methods (autocomplete, get deals, etc.)
  - Validation methods delegating to validators module
  - Utility methods delegating to utils module
  - Filtering, statistics, and analysis methods delegating to respective modules

- Updated govmap/__init__.py to export GovmapClient

All modules maintain backward compatibility through delegation pattern.
Next: Delete old govmap.py and update imports.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 13:18:19 +03:00
Nitzan Pomerantz b565c7fb08 Phase 3: Create govmap package structure (step 1/3)
Created modular package structure:
- validators.py: Input validation functions (3 functions, ~100 lines)
- utils.py: Helper utilities (3 functions, ~140 lines)
- filters.py: Deal filtering logic (1 main function, ~140 lines)
- statistics.py: Statistical calculations (2 functions, ~130 lines)

All functions extracted from monolithic govmap.py as pure functions.
Next: Extract market analysis and create client.py with API methods.

Part of Phase 3 refactoring - no functionality changes yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 13:06:48 +03:00
Nitzan Pomerantz fde0afa7a1 Apply suggestion from @gemini-code-assist[bot]
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-10-25 00:43:26 +03:00
Nitzan Pomerantz b62815ec08 Fix: Strip bloat fields to resolve MCP token limit issues
Problem:
- get_valuation_comparables failed with 61,565 tokens (exceeded 25K MCP limit)
- Large MULTIPOLYGON shape data consumed ~40-50% of response tokens
- Not useful for LLM analysis, only bloating responses

Solution:
1. Added strip_bloat_fields() helper to remove:
   - shape: Large coordinate data
   - sourceorder: Internal ordering field
   - source_polygon_id: Internal reference field

2. Applied to 5 MCP tools returning deal data:
   - get_deals_by_radius
   - get_street_deals
   - find_recent_deals_for_address
   - get_neighborhood_deals
   - get_valuation_comparables

3. Reduced get_valuation_comparables default max_comparables: 200 → 50

Results:
- Token usage reduced by ~87% (61K → ~7-8K tokens)
- All tools now work within MCP token limits
- Cleaner, more efficient responses
- No loss of useful data for LLM analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 00:40:31 +03:00
Nitzan Pomerantz ee0b9466d4 Fix: Critical filtering bugs - missing data handling
Fixed two critical bugs in filter_deals_by_criteria():

Bug #1: CRASH on None property type
- Issue: Code called .lower() on None when propertyTypeDescription was null
- Impact: Would crash MCP tool in production with incomplete data
- Fix: Check if deal_type is None/empty before calling .lower()

Bug #2: Missing data passes through filters
- Issue: Deals with None/missing values passed filters when they shouldn't
- Example: Filtering by area=60-70 would include deals with assetArea=None
- Impact: get_valuation_comparables returned inflated results with bad data
- Fix: Explicitly check for None when filter is active and exclude those deals

Changes:
- Property type filter: Check for None/empty before normalization
- Area filter: Exclude deals with missing area when min/max_area specified
- Room filter: Exclude deals with missing rooms when min/max_rooms specified
- Price filter: Exclude deals with missing price when min/max_price specified
- Invalid data: Changed from 'pass' to 'continue' to exclude bad data

Added 6 comprehensive unit tests:
- test_filter_excludes_missing_property_type
- test_filter_excludes_missing_area
- test_filter_excludes_missing_rooms
- test_filter_excludes_missing_price
- test_filter_excludes_invalid_numeric_data
- test_filter_allows_missing_data_when_no_filter

This fixes the E2E issue where get_valuation_comparables was returning
incomplete results. Now filters properly exclude deals with missing data.

Test results: 34/34 passing (6 new tests added)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 00:08:05 +03:00
Nitzan Pomerantz fe0a96ab83 Fix: Address E2E bugs from real-world MCP testing
Fixes three critical bugs discovered during live testing with Claude:

Bug #1: Same building detection always returned 0
- Root cause: deal.get('address') returned empty string
- Fix: Construct address from streetNameHeb + houseNum fields
- Added test: test_same_building_detection_with_api_fields

Bug #2: Property type filter too strict
- Root cause: Exact match failed for variants like 'דירת גג'
- Fix: Use flexible substring matching with normalization

Bug #3: Deal deduplication used non-existent field
- Root cause: Deduplication key referenced deal.get('address')
- Fix: Removed non-existent field from deduplication key

Also added _calculate_distance() helper for future radius filtering.

Test results: 26/28 passing (2 pre-existing failures unrelated)
References: .cursor/plans/MCP-E2E-TEST-SUMMARY.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 23:47:07 +03:00
Nitzan Pomerantz ac6c780419 Improve fastmcp_server error handling and code quality
Refactor repetitive try-except blocks using _safe_calculate_metric helper
function to reduce code duplication and improve maintainability.

Changes:
- Add _safe_calculate_metric helper to centralize error handling
- Remove misleading default values (0/"unknown") in summary fields
- Use None defaults instead to clearly indicate unavailable data
- Add final newline to file per convention

This prevents LLMs from misinterpreting 0 as "zero investment potential"
when it actually means "data unavailable".

Addresses PR #2 review comments on lines 733, 749, and 759.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 19:26:52 +03:00
Nitzan Pomerantz e5fb218ac0 Refactor date parsing and implement time period filtering
Extract duplicated date parsing logic into a reusable _parse_deal_dates
helper method. This centralizes error handling, validation, and date
grouping logic across multiple market analysis functions.

Key improvements:
- DRY: Eliminates code duplication across three functions
- Implements time_period_months filtering (fixes unused parameter bug)
- Returns both monthly and quarterly breakdowns in one pass
- Consistent error handling and logging
- Better maintainability

This change fixes the unused time_period_months parameters in both
calculate_market_activity_score and get_market_liquidity functions.

Addresses PR #2 review comments on lines 1013, 1058, and 1257.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 19:23:06 +03:00
Nitzan Pomerantz 11512045dd Extract magic numbers into named constants
Define module-level constants for market activity, volatility, and liquidity
thresholds. This improves code readability and makes threshold values easier
to maintain and adjust in the future.

Constants added:
- ACTIVITY_VERY_HIGH_THRESHOLD, ACTIVITY_HIGH_THRESHOLD, etc.
- VOLATILITY_VERY_VOLATILE_THRESHOLD, VOLATILITY_VOLATILE_THRESHOLD, etc.
- LIQUIDITY_VERY_HIGH_THRESHOLD, LIQUIDITY_HIGH_THRESHOLD, etc.

Addresses PR #2 review comments on lines 1084 and 1224.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 19:22:03 +03:00
Nitzan Pomerantz 2968711307 Complete Phase 2: Market Analysis, Filtering & Documentation
This commit implements all Phase 2 functionality with architectural
improvements over the original plan.

## Phase 2.1: Property Valuation Data 
- filter_deals_by_criteria() with comprehensive filtering
- calculate_deal_statistics() for statistical aggregations
- _extract_floor_number() for Hebrew floor parsing
- _calculate_std_dev() helper function
- MCP tools: get_valuation_comparables, get_deal_statistics

## Phase 2.2: Market Activity & Investment Analysis 
- calculate_market_activity_score() - deal frequency & velocity
  * Activity score (0-100), trend analysis, monthly distribution
  * Classifies markets: very_high, high, moderate, low, very_low
- analyze_investment_potential() - price trends & stability
  * Price appreciation rate via linear regression
  * Volatility score using coefficient of variation
  * Investment score combining appreciation & stability
- get_market_liquidity() - turnover & liquidity metrics
  * Quarterly/monthly breakdowns, velocity scoring
  * Trend direction, most active periods
- MCP tool: get_market_activity_metrics (unified tool)

## Phase 2.3: Enhanced Deal Filtering 
- Property type, room count, price, area, floor filtering
- All integrated into existing tools
- Hebrew floor number parsing support

## Testing 
- Added 15 comprehensive unit tests (all passing)
- Coverage: market activity, investment analysis, liquidity, filtering
- Edge cases: empty data, invalid dates, insufficient data

## Documentation 
- Created CLAUDE.md (~250 lines) - AI agent guidance
  * Development commands, architecture overview
  * Product vision from USECASES.md
  * Available tools with status indicators
- Updated TASKS.md - Phase 2 marked 100% complete

## Architectural Improvements
- 1 unified MCP tool instead of 6 separate tools (simpler API)
- 1 flexible filtering function instead of 3 (more composable)
- All logic in govmap.py (no new files, better cohesion)
- ~955 lines added with comprehensive documentation

## Design Principles Followed
 MCP provides data, LLM provides intelligence
 No predictions - only statistical calculations
 Comprehensive error handling & input validation
 Well-documented with detailed docstrings

Phase 2 Progress: 100% complete (60% overall project completion)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 18:51:50 +03:00
Nitzan Pomerantz d9e42bea17 Some more small fixes 2025-10-23 00:11:41 +03:00
Nitzan Pomerantz 49bfc7b940 CR Fixes and update markdownlint 2025-10-22 23:59:07 +03:00
Nitzan Pomerantz 85f52a8108 Main updates of Phase 1 2025-10-19 00:58:46 +03:00
Nitzan Pomerantz 9ee5372b1c Another iteration 2025-07-14 14:26:53 +03:00
Nitzan Pomerantz 650048445f Optimization because of long running times 2025-07-14 13:40:36 +03:00
Nitzan Pomerantz 80dfaa28f0 Improving parameters 2025-07-14 13:12:44 +03:00
Nitzan Pomerantz 4f3b74ceaf Refactor fastmcp 2025-07-13 01:05:16 +03:00
Nitzan Pomerantz dd6d78b48b Refactor govmap 2025-07-13 01:03:15 +03:00
Nitzan Pomerantz a1fa75a878 mcp clean up 2025-07-13 00:56:47 +03:00
Nitzan Pomerantz d8c18cb2f7 mcp second iteration 2025-07-13 00:50:51 +03:00
Nitzan Pomerantz 76bdc323df MCP server first iteration 2025-07-12 22:11:12 +03:00
Nitzan Pomerantz f422aeaa79 First iteration 2025-07-12 17:45:54 +03:00