mcp-servers/nadlan-mcp

Fork 0

Files

T

Nitzan Pomerantz c71f950eb7 Added documentation

2025-10-19 00:58:11 +03:00

15 KiB

Raw Blame History

Nadlan-MCP Architecture

Overview

Nadlan-MCP is a Model Context Protocol (MCP) server that provides Israeli real estate data to AI agents (LLMs). The architecture follows a clean separation of concerns with three main layers:

MCP Tools Layer - Exposes functions to LLM clients
Business Logic Layer - Data analysis and processing
API Client Layer - Communicates with Govmap API

Design Principles

1. MCP Provides Data, LLM Provides Intelligence

The server's role is to retrieve, structure, and optionally summarize data. Complex analysis, decision-making, and predictions are left to the LLM client.

Example:

✅ MCP: Provides comparable property deals with filters
✅ LLM: Analyzes comparables and estimates property value
❌ MCP: Calculates ML-based property valuation

2. Structured by Default

All tools return detailed, structured data by default. Optional summarized_response parameter provides condensed summaries when needed.

3. Reliability First

Automatic retry with exponential backoff
Rate limiting to respect API limits
Comprehensive input validation
Detailed error messages

4. Configuration Over Hard-coding

All settings (timeouts, retries, rate limits) are configurable via environment variables or code.

System Architecture

┌─────────────────────────────────────────────────────────────┐
│                       LLM Client (AI Agent)                  │
│                    (Claude, GPT, etc.)                       │
└────────────────────────────┬────────────────────────────────┘
                             │ MCP Protocol
                             ↓
┌─────────────────────────────────────────────────────────────┐
│                    MCP Tools Layer                          │
│  ┌───────────────┐  ┌─────────────────┐  ┌──────────────┐ │
│  │autocomplete   │  │find_recent_deals│  │analyze_market│ │
│  │_address       │  │_for_address     │  │_trends       │ │
│  └───────┬───────┘  └────────┬────────┘  └──────┬───────┘ │
│          │                   │                    │         │
│  ┌───────┴───────────────────┴────────────────────┴──────┐ │
│  │            fastmcp_server.py (Tool Definitions)       │ │
│  └───────────────────────────┬───────────────────────────┘ │
└────────────────────────────┬─┴────────────────────────────┘
                             │
                             ↓
┌─────────────────────────────────────────────────────────────┐
│                 Business Logic Layer                         │
│  ┌────────────────────┐  ┌──────────────────────────────┐  │
│  │ Market Analysis    │  │ Deal Processing & Filtering  │  │
│  │ (analyze_market    │  │ (_is_same_building, etc.)    │  │
│  │  _trends logic)    │  │                              │  │
│  └────────────────────┘  └──────────────────────────────┘  │
│  ┌────────────────────────────────────────────────────────┐ │
│  │            govmap.py (GovmapClient)                    │ │
│  │  • Validation  • Retry Logic  • Rate Limiting         │ │
│  └─────────────────────────┬──────────────────────────────┘ │
└───────────────────────────┬┴──────────────────────────────┘
                            │ HTTP/JSON
                            ↓
┌─────────────────────────────────────────────────────────────┐
│                    Govmap API Layer                         │
│        (Israeli Government Real Estate API)                 │
│  ┌───────────────┐  ┌────────────┐  ┌──────────────────┐  │
│  │ Autocomplete  │  │ Deals by   │  │ Street/          │  │
│  │ Endpoint      │  │ Radius     │  │ Neighborhood     │  │
│  │               │  │ Endpoint   │  │ Deals Endpoints  │  │
│  └───────────────┘  └────────────┘  └──────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Component Details

Configuration Layer (`config.py`)

Purpose: Centralized configuration management

Key Components:

GovmapConfig - Dataclass with all configuration settings
Environment variable support for all settings
Validation on initialization

Configuration Options:

API settings (base URL, user agent)
Timeout settings (connect, read)
Retry settings (max retries, backoff)
Rate limiting (requests per second)
Default search parameters

Usage:

from nadlan_mcp.config import get_config, set_config, GovmapConfig

# Use global config
config = get_config()

# Override config
custom_config = GovmapConfig(max_retries=5, requests_per_second=10.0)
set_config(custom_config)

API Client Layer (`govmap.py`)

Purpose: Communicate with Govmap API with reliability features

Key Components:

GovmapClient Class

Responsibilities:

Make HTTP requests to Govmap API
Implement retry logic with exponential backoff
Enforce rate limiting
Validate inputs
Parse and validate responses

Key Methods:

autocomplete_address() - Search for addresses
get_gush_helka() - Get block/parcel data
get_deals_by_radius() - Get nearby deals
get_street_deals() - Get street-level deals
get_neighborhood_deals() - Get neighborhood deals
find_recent_deals_for_address() - Main comprehensive search

Reliability Features:

Retry Logic - Exponential backoff on failures
Rate Limiting - Tracks request times, enforces delays
Input Validation - Validates all inputs before API calls
Timeouts - Configurable connect and read timeouts

MCP Tools Layer (`fastmcp_server.py`)

Purpose: Expose functionality to LLM clients via MCP protocol

Key Components:

FastMCP server instance
Tool definitions (decorated functions)
JSON response formatting
Error handling for user-friendly messages

Tool Design Pattern:

@mcp.tool()
def tool_name(param: type, summarized_response: bool = False) -> str:
    """
    Tool description for LLM.
    
    Args:
        param: Parameter description
        summarized_response: 
            - False (default): Full structured data for LLM processing
            - True: Condensed summary with key insights
    
    Returns:
        JSON string with data or summary
    """
    # 1. Call business logic / API client
    # 2. Format response
    # 3. Return JSON string

Data Flow

Example: Find Recent Deals for Address

1. LLM Request
   └─> "Find deals for Sokolov 38 Holon in last 2 years"

2. MCP Tool: find_recent_deals_for_address()
   ├─> Validate inputs
   └─> Call GovmapClient

3. GovmapClient Workflow:
   ├─> autocomplete_address("סוקולוב 38 חולון")
   │   ├─> Rate limit check
   │   ├─> HTTP POST (with retry)
   │   └─> Parse coordinates
   │
   ├─> get_deals_by_radius(coordinates, 30m)
   │   ├─> Rate limit check
   │   ├─> HTTP GET (with retry)
   │   └─> Extract polygon_ids
   │
   ├─> For each polygon_id:
   │   ├─> get_street_deals(polygon_id)
   │   └─> get_neighborhood_deals(polygon_id)
   │
   ├─> Filter & Prioritize:
   │   ├─> Same building deals (priority 0)
   │   ├─> Street deals (priority 1)
   │   └─> Neighborhood deals (priority 2)
   │
   └─> Sort by priority then date

4. Format Response
   └─> JSON with deals + metadata

5. Return to LLM
   └─> LLM analyzes and responds to user

Error Handling Strategy

Validation Errors (ValueError)

Invalid address format
Negative/zero parameters
Out-of-range values
Action: Immediate failure with clear message

Network Errors (requests.RequestException)

Connection failures
Timeouts
HTTP errors
Action: Retry with exponential backoff (up to configured max_retries)

API Response Errors

Invalid JSON
Unexpected format
Missing required fields
Action: Raise ValueError with specific details

Rate Limiting

Track last request time
Sleep if necessary before request
Action: Transparent to caller, automatic

Performance Considerations

Current Implementation

Optimizations:

Connection pooling (requests.Session)
Rate limiting prevents API throttling
Early validation reduces unnecessary API calls
Polygon deduplication in find_recent_deals_for_address()

Limitations:

Synchronous (blocking) API calls
No caching (MVP decision)
Sequential polygon queries

Future Optimizations (Not in MVP)

Phase 1: In-Memory Caching

Cache autocomplete results (1 hour TTL)
Cache deal results (30 minute TTL)
Reduce API load by 60-80%

Phase 2: Async/Parallel

Convert to async/await
Parallel polygon queries
3-5x faster for multi-polygon searches

Phase 3: Production Caching

Redis for distributed caching
Cache warming strategies
Cross-instance consistency

Security & Rate Limiting

Rate Limiting

Current: 5 requests/second (configurable)

Rationale:

Respects Govmap API (no published limits, being conservative)
Prevents accidental DoS
Balances responsiveness with safety

Implementation:

Tracks last request timestamp
Sleeps if interval too short
Per-instance (not distributed)

Input Validation

All user inputs are validated before use:

Address strings: length, type, non-empty
Coordinates: numeric, reasonable bounds
Integers: positive, max values
Deal types: enum validation (1 or 2)

Prevents:

Injection attacks
API errors from malformed requests
Expensive/dangerous operations

API Key Management

Current: No API keys required (public API)

Future: If API keys added:

Store in environment variables only
Never log or expose in responses
Rotate regularly

Testing Strategy

Unit Tests

Mock API responses
Test validation logic
Test error handling
Test retry logic

Integration Tests

Real API calls (marked @pytest.mark.integration)
VCR.py for recording/replaying
Test complete workflows

Validation Tests

Invalid inputs
Edge cases
Boundary conditions

Configuration Options

Environment Variables

# API Settings
GOVMAP_BASE_URL=https://www.govmap.gov.il/api/
GOVMAP_USER_AGENT=NadlanMCP/1.0.0

# Timeouts (seconds)
GOVMAP_CONNECT_TIMEOUT=10
GOVMAP_READ_TIMEOUT=30

# Retry Settings
GOVMAP_MAX_RETRIES=3
GOVMAP_RETRY_MIN_WAIT=1
GOVMAP_RETRY_MAX_WAIT=10

# Rate Limiting
GOVMAP_REQUESTS_PER_SECOND=5.0

# Defaults
GOVMAP_DEFAULT_RADIUS=50
GOVMAP_DEFAULT_YEARS_BACK=2
GOVMAP_DEFAULT_DEAL_LIMIT=100

Tuning Guidelines

High Reliability (Production):

MAX_RETRIES=5
RETRY_MAX_WAIT=30
REQUESTS_PER_SECOND=3.0

High Performance (Development):

MAX_RETRIES=2
RETRY_MAX_WAIT=5
REQUESTS_PER_SECOND=10.0

Conservative (Shared API):

MAX_RETRIES=3
RETRY_MAX_WAIT=10
REQUESTS_PER_SECOND=2.0

Future Architecture Evolution

Phase 2: Data Models (Pydantic)

Add structured models for type safety:

class Deal(BaseModel):
    deal_id: str
    address: str
    deal_amount: float
    asset_area: float
    price_per_sqm: float
    deal_date: datetime
    property_type: str
    # ...

Phase 3: Separation of Concerns

nadlan_mcp/
├── api_client.py       # Pure API communication
├── analyzers/
│   ├── market.py       # Market analysis logic
│   ├── valuation.py    # Valuation helpers
│   └── filtering.py    # Deal filtering
├── models.py           # Pydantic models
└── fastmcp_server.py   # Thin MCP tool definitions

Phase 4: Database Layer (Optional)

For historical tracking and faster queries:

├── database/
│   ├── models.py       # SQLAlchemy models
│   ├── crud.py         # CRUD operations
│   └── migrations/     # Alembic migrations

Deployment

Development

python run_fastmcp_server.py

Production (MCP Client)

{
  "servers": {
    "nadlan-mcp": {
      "command": "python",
      "args": ["/path/to/nadlan-mcp/run_fastmcp_server.py"],
      "env": {
        "GOVMAP_REQUESTS_PER_SECOND": "3.0",
        "GOVMAP_MAX_RETRIES": "5"
      }
    }
  }
}

API Limitations

Govmap API Constraints

Known Limitations:

No published rate limits (being conservative with 5 req/sec)
No authentication required (public API)
Data freshness: Updated periodically (not real-time)
Coverage: Official Israeli government records only

Best Practices:

Use appropriate time ranges (2-5 years recommended)
Limit radius searches (under 1000m for performance)
Cache results when possible (future feature)
Respect rate limiting

MCP Protocol Constraints

Token Limits:

Large deal lists can exceed LLM context windows
Use summarized_response=True for large datasets
Default limits (50-100 deals) are token-optimized

Monitoring & Debugging

Logging Levels

INFO: Normal operations

Requests being made
Successful responses

WARNING: Recoverable issues

Retrying after failures
Rate limiting delays
Validation warnings

ERROR: Failures

Exhausted retries
Invalid responses
Configuration errors

Debug Mode

Enable debug logging:

import logging
logging.basicConfig(level=logging.DEBUG)

15 KiB Raw Blame History

Nadlan-MCP Architecture

Overview

Design Principles

1. MCP Provides Data, LLM Provides Intelligence

2. Structured by Default

3. Reliability First

4. Configuration Over Hard-coding

System Architecture

Component Details

Configuration Layer (config.py)

API Client Layer (govmap.py)

GovmapClient Class

MCP Tools Layer (fastmcp_server.py)

Data Flow

Example: Find Recent Deals for Address

Error Handling Strategy

Validation Errors (ValueError)

Network Errors (requests.RequestException)

API Response Errors

Rate Limiting

Performance Considerations

Current Implementation

Future Optimizations (Not in MVP)

Security & Rate Limiting

Rate Limiting

Input Validation

API Key Management

Testing Strategy

Unit Tests

Integration Tests

Validation Tests

Configuration Options

Environment Variables

Tuning Guidelines

Future Architecture Evolution

Phase 2: Data Models (Pydantic)

Phase 3: Separation of Concerns

Phase 4: Database Layer (Optional)

Deployment

Development

Production (MCP Client)

API Limitations

Govmap API Constraints

MCP Protocol Constraints

Monitoring & Debugging

Logging Levels

Debug Mode

References

15 KiB

Raw Blame History

Configuration Layer (`config.py`)

API Client Layer (`govmap.py`)

MCP Tools Layer (`fastmcp_server.py`)