# Nadlan-MCP Architecture

## Overview

Nadlan-MCP is a Model Context Protocol (MCP) server that provides Israeli real estate data to AI agents (LLMs). The architecture follows a clean separation of concerns with three main layers:

1. **MCP Tools Layer** - Exposes functions to LLM clients
2. **Business Logic Layer** - Data analysis and processing
3. **API Client Layer** - Communicates with Govmap API

## Design Principles

### 1. MCP Provides Data, LLM Provides Intelligence

The server's role is to retrieve, structure, and optionally summarize data. Complex analysis, decision-making, and predictions are left to the LLM client.

**Example:**
- ✅ MCP: Provides comparable property deals with filters
- ✅ LLM: Analyzes comparables and estimates property value
- ❌ MCP: Calculates ML-based property valuation

### 2. Structured by Default

All tools return detailed, structured data by default. Optional `summarized_response` parameter provides condensed summaries when needed.

### 3. Reliability First

- Automatic retry with exponential backoff
- Rate limiting to respect API limits
- Comprehensive input validation
- Detailed error messages

### 4. Configuration Over Hard-coding

All settings (timeouts, retries, rate limits) are configurable via environment variables or code.

## System Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                       LLM Client (AI Agent)                  │
│                    (Claude, GPT, etc.)                       │
└────────────────────────────┬────────────────────────────────┘
                             │ MCP Protocol
                             ↓
┌─────────────────────────────────────────────────────────────┐
│                    MCP Tools Layer                          │
│  ┌───────────────┐  ┌─────────────────┐  ┌──────────────┐ │
│  │autocomplete   │  │find_recent_deals│  │analyze_market│ │
│  │_address       │  │_for_address     │  │_trends       │ │
│  └───────┬───────┘  └────────┬────────┘  └──────┬───────┘ │
│          │                   │                    │         │
│  ┌───────┴───────────────────┴────────────────────┴──────┐ │
│  │            fastmcp_server.py (Tool Definitions)       │ │
│  └───────────────────────────┬───────────────────────────┘ │
└────────────────────────────┬─┴────────────────────────────┘
                             │
                             ↓
┌─────────────────────────────────────────────────────────────┐
│                 Business Logic Layer                         │
│  ┌────────────────────┐  ┌──────────────────────────────┐  │
│  │ Market Analysis    │  │ Deal Processing & Filtering  │  │
│  │ (analyze_market    │  │ (_is_same_building, etc.)    │  │
│  │  _trends logic)    │  │                              │  │
│  └────────────────────┘  └──────────────────────────────┘  │
│  ┌────────────────────────────────────────────────────────┐ │
│  │            govmap.py (GovmapClient)                    │ │
│  │  • Validation  • Retry Logic  • Rate Limiting         │ │
│  └─────────────────────────┬──────────────────────────────┘ │
└───────────────────────────┬┴──────────────────────────────┘
                            │ HTTP/JSON
                            ↓
┌─────────────────────────────────────────────────────────────┐
│                    Govmap API Layer                         │
│        (Israeli Government Real Estate API)                 │
│  ┌───────────────┐  ┌────────────┐  ┌──────────────────┐  │
│  │ Autocomplete  │  │ Deals by   │  │ Street/          │  │
│  │ Endpoint      │  │ Radius     │  │ Neighborhood     │  │
│  │               │  │ Endpoint   │  │ Deals Endpoints  │  │
│  └───────────────┘  └────────────┘  └──────────────────┘  │
└─────────────────────────────────────────────────────────────┘
```

## Component Details

### Configuration Layer (`config.py`)

**Purpose:** Centralized configuration management

**Key Components:**
- `GovmapConfig` - Dataclass with all configuration settings
- Environment variable support for all settings
- Validation on initialization

**Configuration Options:**
- API settings (base URL, user agent)
- Timeout settings (connect, read)
- Retry settings (max retries, backoff)
- Rate limiting (requests per second)
- Default search parameters

**Usage:**
```python
from nadlan_mcp.config import get_config, set_config, GovmapConfig

# Use global config
config = get_config()

# Override config
custom_config = GovmapConfig(max_retries=5, requests_per_second=10.0)
set_config(custom_config)
```

### API Client Layer (`govmap.py`)

**Purpose:** Communicate with Govmap API with reliability features

**Key Components:**

#### GovmapClient Class

**Responsibilities:**
- Make HTTP requests to Govmap API
- Implement retry logic with exponential backoff
- Enforce rate limiting
- Validate inputs
- Parse and validate responses

**Key Methods:**
- `autocomplete_address()` - Search for addresses
- `get_gush_helka()` - Get block/parcel data
- `get_deals_by_radius()` - Get nearby deals
- `get_street_deals()` - Get street-level deals
- `get_neighborhood_deals()` - Get neighborhood deals
- `find_recent_deals_for_address()` - Main comprehensive search

**Reliability Features:**
1. **Retry Logic** - Exponential backoff on failures
2. **Rate Limiting** - Tracks request times, enforces delays
3. **Input Validation** - Validates all inputs before API calls
4. **Timeouts** - Configurable connect and read timeouts

### MCP Tools Layer (`fastmcp_server.py`)

**Purpose:** Expose functionality to LLM clients via MCP protocol

**Key Components:**
- FastMCP server instance
- Tool definitions (decorated functions)
- JSON response formatting
- Error handling for user-friendly messages

**Tool Design Pattern:**
```python
@mcp.tool()
def tool_name(param: type, summarized_response: bool = False) -> str:
    """
    Tool description for LLM.
    
    Args:
        param: Parameter description
        summarized_response: 
            - False (default): Full structured data for LLM processing
            - True: Condensed summary with key insights
    
    Returns:
        JSON string with data or summary
    """
    # 1. Call business logic / API client
    # 2. Format response
    # 3. Return JSON string
```

## Data Flow

### Example: Find Recent Deals for Address

```
1. LLM Request
   └─> "Find deals for Sokolov 38 Holon in last 2 years"

2. MCP Tool: find_recent_deals_for_address()
   ├─> Validate inputs
   └─> Call GovmapClient

3. GovmapClient Workflow:
   ├─> autocomplete_address("סוקולוב 38 חולון")
   │   ├─> Rate limit check
   │   ├─> HTTP POST (with retry)
   │   └─> Parse coordinates
   │
   ├─> get_deals_by_radius(coordinates, 30m)
   │   ├─> Rate limit check
   │   ├─> HTTP GET (with retry)
   │   └─> Extract polygon_ids
   │
   ├─> For each polygon_id:
   │   ├─> get_street_deals(polygon_id)
   │   └─> get_neighborhood_deals(polygon_id)
   │
   ├─> Filter & Prioritize:
   │   ├─> Same building deals (priority 0)
   │   ├─> Street deals (priority 1)
   │   └─> Neighborhood deals (priority 2)
   │
   └─> Sort by priority then date

4. Format Response
   └─> JSON with deals + metadata

5. Return to LLM
   └─> LLM analyzes and responds to user
```

## Error Handling Strategy

### Validation Errors (ValueError)
- Invalid address format
- Negative/zero parameters
- Out-of-range values
- **Action:** Immediate failure with clear message

### Network Errors (requests.RequestException)
- Connection failures
- Timeouts
- HTTP errors
- **Action:** Retry with exponential backoff (up to configured max_retries)

### API Response Errors
- Invalid JSON
- Unexpected format
- Missing required fields
- **Action:** Raise ValueError with specific details

### Rate Limiting
- Track last request time
- Sleep if necessary before request
- **Action:** Transparent to caller, automatic

## Performance Considerations

### Current Implementation

**Optimizations:**
- Connection pooling (requests.Session)
- Rate limiting prevents API throttling
- Early validation reduces unnecessary API calls
- Polygon deduplication in find_recent_deals_for_address()

**Limitations:**
- Synchronous (blocking) API calls
- No caching (MVP decision)
- Sequential polygon queries

### Future Optimizations (Not in MVP)

**Phase 1: In-Memory Caching**
- Cache autocomplete results (1 hour TTL)
- Cache deal results (30 minute TTL)
- Reduce API load by 60-80%

**Phase 2: Async/Parallel**
- Convert to async/await
- Parallel polygon queries
- 3-5x faster for multi-polygon searches

**Phase 3: Production Caching**
- Redis for distributed caching
- Cache warming strategies
- Cross-instance consistency

## Security & Rate Limiting

### Rate Limiting

**Current:** 5 requests/second (configurable)

**Rationale:**
- Respects Govmap API (no published limits, being conservative)
- Prevents accidental DoS
- Balances responsiveness with safety

**Implementation:**
- Tracks last request timestamp
- Sleeps if interval too short
- Per-instance (not distributed)

### Input Validation

All user inputs are validated before use:
- Address strings: length, type, non-empty
- Coordinates: numeric, reasonable bounds
- Integers: positive, max values
- Deal types: enum validation (1 or 2)

**Prevents:**
- Injection attacks
- API errors from malformed requests
- Expensive/dangerous operations

### API Key Management

**Current:** No API keys required (public API)

**Future:** If API keys added:
- Store in environment variables only
- Never log or expose in responses
- Rotate regularly

## Testing Strategy

### Unit Tests
- Mock API responses
- Test validation logic
- Test error handling
- Test retry logic

### Integration Tests
- Real API calls (marked `@pytest.mark.integration`)
- VCR.py for recording/replaying
- Test complete workflows

### Validation Tests
- Invalid inputs
- Edge cases
- Boundary conditions

## Configuration Options

### Environment Variables

```bash
# API Settings
GOVMAP_BASE_URL=https://www.govmap.gov.il/api/
GOVMAP_USER_AGENT=NadlanMCP/1.0.0

# Timeouts (seconds)
GOVMAP_CONNECT_TIMEOUT=10
GOVMAP_READ_TIMEOUT=30

# Retry Settings
GOVMAP_MAX_RETRIES=3
GOVMAP_RETRY_MIN_WAIT=1
GOVMAP_RETRY_MAX_WAIT=10

# Rate Limiting
GOVMAP_REQUESTS_PER_SECOND=5.0

# Defaults
GOVMAP_DEFAULT_RADIUS=50
GOVMAP_DEFAULT_YEARS_BACK=2
GOVMAP_DEFAULT_DEAL_LIMIT=100
```

### Tuning Guidelines

**High Reliability (Production):**
- `MAX_RETRIES=5`
- `RETRY_MAX_WAIT=30`
- `REQUESTS_PER_SECOND=3.0`

**High Performance (Development):**
- `MAX_RETRIES=2`
- `RETRY_MAX_WAIT=5`
- `REQUESTS_PER_SECOND=10.0`

**Conservative (Shared API):**
- `MAX_RETRIES=3`
- `RETRY_MAX_WAIT=10`
- `REQUESTS_PER_SECOND=2.0`

## Future Architecture Evolution

### Phase 2: Data Models (Pydantic)

Add structured models for type safety:
```python
class Deal(BaseModel):
    deal_id: str
    address: str
    deal_amount: float
    asset_area: float
    price_per_sqm: float
    deal_date: datetime
    property_type: str
    # ...
```

### Phase 3: Separation of Concerns

```
nadlan_mcp/
├── api_client.py       # Pure API communication
├── analyzers/
│   ├── market.py       # Market analysis logic
│   ├── valuation.py    # Valuation helpers
│   └── filtering.py    # Deal filtering
├── models.py           # Pydantic models
└── fastmcp_server.py   # Thin MCP tool definitions
```

### Phase 4: Database Layer (Optional)

For historical tracking and faster queries:
```
├── database/
│   ├── models.py       # SQLAlchemy models
│   ├── crud.py         # CRUD operations
│   └── migrations/     # Alembic migrations
```

## Deployment

### Development
```bash
python run_fastmcp_server.py
```

### Production (MCP Client)
```json
{
  "servers": {
    "nadlan-mcp": {
      "command": "python",
      "args": ["/path/to/nadlan-mcp/run_fastmcp_server.py"],
      "env": {
        "GOVMAP_REQUESTS_PER_SECOND": "3.0",
        "GOVMAP_MAX_RETRIES": "5"
      }
    }
  }
}
```

## API Limitations

### Govmap API Constraints

**Known Limitations:**
- No published rate limits (being conservative with 5 req/sec)
- No authentication required (public API)
- Data freshness: Updated periodically (not real-time)
- Coverage: Official Israeli government records only

**Best Practices:**
- Use appropriate time ranges (2-5 years recommended)
- Limit radius searches (under 1000m for performance)
- Cache results when possible (future feature)
- Respect rate limiting

### MCP Protocol Constraints

**Token Limits:**
- Large deal lists can exceed LLM context windows
- Use `summarized_response=True` for large datasets
- Default limits (50-100 deals) are token-optimized

## Monitoring & Debugging

### Logging Levels

**INFO:** Normal operations
- Requests being made
- Successful responses

**WARNING:** Recoverable issues
- Retrying after failures
- Rate limiting delays
- Validation warnings

**ERROR:** Failures
- Exhausted retries
- Invalid responses
- Configuration errors

### Debug Mode

Enable debug logging:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
```

## References

- [Govmap API](https://www.govmap.gov.il/)
- [MCP Protocol](https://modelcontextprotocol.io/)
- [FastMCP Documentation](https://github.com/jlowin/fastmcp)
- [Israeli Real Estate Data](https://data.gov.il/)