mcp-servers/nadlan-mcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added documentation

Browse Source
This commit is contained in:
Nitzan Pomerantz
2025-10-19 00:58:11 +03:00
parent ffd7c84aa7
commit c71f950eb7
1 changed files with 490 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ARCHITECTURE.md
+490
View File
@@ -0,0 +1,490 @@
# Nadlan-MCP Architecture
## Overview
Nadlan-MCP is a Model Context Protocol (MCP) server that provides Israeli real estate data to AI agents (LLMs). The architecture follows a clean separation of concerns with three main layers:
1. **MCP Tools Layer** - Exposes functions to LLM clients
2. **Business Logic Layer** - Data analysis and processing
3. **API Client Layer** - Communicates with Govmap API
## Design Principles
### 1. MCP Provides Data, LLM Provides Intelligence
The server's role is to retrieve, structure, and optionally summarize data. Complex analysis, decision-making, and predictions are left to the LLM client.
**Example:**
- ✅ MCP: Provides comparable property deals with filters
- ✅ LLM: Analyzes comparables and estimates property value
- ❌ MCP: Calculates ML-based property valuation
### 2. Structured by Default
All tools return detailed, structured data by default. Optional `summarized_response` parameter provides condensed summaries when needed.
### 3. Reliability First
- Automatic retry with exponential backoff
- Rate limiting to respect API limits
- Comprehensive input validation
- Detailed error messages
### 4. Configuration Over Hard-coding
All settings (timeouts, retries, rate limits) are configurable via environment variables or code.
## System Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ LLM Client (AI Agent) │
│ (Claude, GPT, etc.) │
└────────────────────────────┬────────────────────────────────┘
│ MCP Protocol
↓
┌─────────────────────────────────────────────────────────────┐
│ MCP Tools Layer │
│ ┌───────────────┐ ┌─────────────────┐ ┌──────────────┐ │
│ │autocomplete │ │find_recent_deals│ │analyze_market│ │
│ │_address │ │_for_address │ │_trends │ │
│ └───────┬───────┘ └────────┬────────┘ └──────┬───────┘ │
│ │ │ │ │
│ ┌───────┴───────────────────┴────────────────────┴──────┐ │
│ │ fastmcp_server.py (Tool Definitions) │ │
│ └───────────────────────────┬───────────────────────────┘ │
└────────────────────────────┬─┴────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────┐
│ Business Logic Layer │
│ ┌────────────────────┐ ┌──────────────────────────────┐ │
│ │ Market Analysis │ │ Deal Processing & Filtering │ │
│ │ (analyze_market │ │ (_is_same_building, etc.) │ │
│ │ _trends logic) │ │ │ │
│ └────────────────────┘ └──────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ govmap.py (GovmapClient) │ │
│ │ • Validation • Retry Logic • Rate Limiting │ │
│ └─────────────────────────┬──────────────────────────────┘ │
└───────────────────────────┬┴──────────────────────────────┘
│ HTTP/JSON
↓
┌─────────────────────────────────────────────────────────────┐
│ Govmap API Layer │
│ (Israeli Government Real Estate API) │
│ ┌───────────────┐ ┌────────────┐ ┌──────────────────┐ │
│ │ Autocomplete │ │ Deals by │ │ Street/ │ │
│ │ Endpoint │ │ Radius │ │ Neighborhood │ │
│ │ │ │ Endpoint │ │ Deals Endpoints │ │
│ └───────────────┘ └────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
## Component Details
### Configuration Layer (`config.py`)
**Purpose:** Centralized configuration management
**Key Components:**
- `GovmapConfig` - Dataclass with all configuration settings
- Environment variable support for all settings
- Validation on initialization
**Configuration Options:**
- API settings (base URL, user agent)
- Timeout settings (connect, read)
- Retry settings (max retries, backoff)
- Rate limiting (requests per second)
- Default search parameters
**Usage:**
```python
from nadlan_mcp.config import get_config, set_config, GovmapConfig
# Use global config
config = get_config()
# Override config
custom_config = GovmapConfig(max_retries=5, requests_per_second=10.0)
set_config(custom_config)
```
### API Client Layer (`govmap.py`)
**Purpose:** Communicate with Govmap API with reliability features
**Key Components:**
#### GovmapClient Class
**Responsibilities:**
- Make HTTP requests to Govmap API
- Implement retry logic with exponential backoff
- Enforce rate limiting
- Validate inputs
- Parse and validate responses
**Key Methods:**
- `autocomplete_address()` - Search for addresses
- `get_gush_helka()` - Get block/parcel data
- `get_deals_by_radius()` - Get nearby deals
- `get_street_deals()` - Get street-level deals
- `get_neighborhood_deals()` - Get neighborhood deals
- `find_recent_deals_for_address()` - Main comprehensive search
**Reliability Features:**
1. **Retry Logic** - Exponential backoff on failures
2. **Rate Limiting** - Tracks request times, enforces delays
3. **Input Validation** - Validates all inputs before API calls
4. **Timeouts** - Configurable connect and read timeouts
### MCP Tools Layer (`fastmcp_server.py`)
**Purpose:** Expose functionality to LLM clients via MCP protocol
**Key Components:**
- FastMCP server instance
- Tool definitions (decorated functions)
- JSON response formatting
- Error handling for user-friendly messages
**Tool Design Pattern:**
```python
@mcp.tool()
def tool_name(param: type, summarized_response: bool = False) -> str:
"""
Tool description for LLM.
Args:
param: Parameter description
summarized_response:
- False (default): Full structured data for LLM processing
- True: Condensed summary with key insights
Returns:
JSON string with data or summary
"""
# 1. Call business logic / API client
# 2. Format response
# 3. Return JSON string
```
## Data Flow
### Example: Find Recent Deals for Address
```
1. LLM Request
└─> "Find deals for Sokolov 38 Holon in last 2 years"
2. MCP Tool: find_recent_deals_for_address()
├─> Validate inputs
└─> Call GovmapClient
3. GovmapClient Workflow:
├─> autocomplete_address("סוקולוב 38 חולון")
│ ├─> Rate limit check
│ ├─> HTTP POST (with retry)
│ └─> Parse coordinates
│
├─> get_deals_by_radius(coordinates, 30m)
│ ├─> Rate limit check
│ ├─> HTTP GET (with retry)
│ └─> Extract polygon_ids
│
├─> For each polygon_id:
│ ├─> get_street_deals(polygon_id)
│ └─> get_neighborhood_deals(polygon_id)
│
├─> Filter & Prioritize:
│ ├─> Same building deals (priority 0)
│ ├─> Street deals (priority 1)
│ └─> Neighborhood deals (priority 2)
│
└─> Sort by priority then date
4. Format Response
└─> JSON with deals + metadata
5. Return to LLM
└─> LLM analyzes and responds to user
```
## Error Handling Strategy
### Validation Errors (ValueError)
- Invalid address format
- Negative/zero parameters
- Out-of-range values
- **Action:** Immediate failure with clear message
### Network Errors (requests.RequestException)
- Connection failures
- Timeouts
- HTTP errors
- **Action:** Retry with exponential backoff (up to configured max_retries)
### API Response Errors
- Invalid JSON
- Unexpected format
- Missing required fields
- **Action:** Raise ValueError with specific details
### Rate Limiting
- Track last request time
- Sleep if necessary before request
- **Action:** Transparent to caller, automatic
## Performance Considerations
### Current Implementation
**Optimizations:**
- Connection pooling (requests.Session)
- Rate limiting prevents API throttling
- Early validation reduces unnecessary API calls
- Polygon deduplication in find_recent_deals_for_address()
**Limitations:**
- Synchronous (blocking) API calls
- No caching (MVP decision)
- Sequential polygon queries
### Future Optimizations (Not in MVP)
**Phase 1: In-Memory Caching**
- Cache autocomplete results (1 hour TTL)
- Cache deal results (30 minute TTL)
- Reduce API load by 60-80%
**Phase 2: Async/Parallel**
- Convert to async/await
- Parallel polygon queries
- 3-5x faster for multi-polygon searches
**Phase 3: Production Caching**
- Redis for distributed caching
- Cache warming strategies
- Cross-instance consistency
## Security & Rate Limiting
### Rate Limiting
**Current:** 5 requests/second (configurable)
**Rationale:**
- Respects Govmap API (no published limits, being conservative)
- Prevents accidental DoS
- Balances responsiveness with safety
**Implementation:**
- Tracks last request timestamp
- Sleeps if interval too short
- Per-instance (not distributed)
### Input Validation
All user inputs are validated before use:
- Address strings: length, type, non-empty
- Coordinates: numeric, reasonable bounds
- Integers: positive, max values
- Deal types: enum validation (1 or 2)
**Prevents:**
- Injection attacks
- API errors from malformed requests
- Expensive/dangerous operations
### API Key Management
**Current:** No API keys required (public API)
**Future:** If API keys added:
- Store in environment variables only
- Never log or expose in responses
- Rotate regularly
## Testing Strategy
### Unit Tests
- Mock API responses
- Test validation logic
- Test error handling
- Test retry logic
### Integration Tests
- Real API calls (marked `@pytest.mark.integration`)
- VCR.py for recording/replaying
- Test complete workflows
### Validation Tests
- Invalid inputs
- Edge cases
- Boundary conditions
## Configuration Options
### Environment Variables
```bash
# API Settings
GOVMAP_BASE_URL=https://www.govmap.gov.il/api/
GOVMAP_USER_AGENT=NadlanMCP/1.0.0
# Timeouts (seconds)
GOVMAP_CONNECT_TIMEOUT=10
GOVMAP_READ_TIMEOUT=30
# Retry Settings
GOVMAP_MAX_RETRIES=3
GOVMAP_RETRY_MIN_WAIT=1
GOVMAP_RETRY_MAX_WAIT=10
# Rate Limiting
GOVMAP_REQUESTS_PER_SECOND=5.0
# Defaults
GOVMAP_DEFAULT_RADIUS=50
GOVMAP_DEFAULT_YEARS_BACK=2
GOVMAP_DEFAULT_DEAL_LIMIT=100
```
### Tuning Guidelines
**High Reliability (Production):**
- `MAX_RETRIES=5`
- `RETRY_MAX_WAIT=30`
- `REQUESTS_PER_SECOND=3.0`
**High Performance (Development):**
- `MAX_RETRIES=2`
- `RETRY_MAX_WAIT=5`
- `REQUESTS_PER_SECOND=10.0`
**Conservative (Shared API):**
- `MAX_RETRIES=3`
- `RETRY_MAX_WAIT=10`
- `REQUESTS_PER_SECOND=2.0`
## Future Architecture Evolution
### Phase 2: Data Models (Pydantic)
Add structured models for type safety:
```python
class Deal(BaseModel):
deal_id: str
address: str
deal_amount: float
asset_area: float
price_per_sqm: float
deal_date: datetime
property_type: str
# ...
```
### Phase 3: Separation of Concerns
```
nadlan_mcp/
├── api_client.py # Pure API communication
├── analyzers/
│ ├── market.py # Market analysis logic
│ ├── valuation.py # Valuation helpers
│ └── filtering.py # Deal filtering
├── models.py # Pydantic models
└── fastmcp_server.py # Thin MCP tool definitions
```
### Phase 4: Database Layer (Optional)
For historical tracking and faster queries:
```
├── database/
│ ├── models.py # SQLAlchemy models
│ ├── crud.py # CRUD operations
│ └── migrations/ # Alembic migrations
```
## Deployment
### Development
```bash
python run_fastmcp_server.py
```
### Production (MCP Client)
```json
{
"servers": {
"nadlan-mcp": {
"command": "python",
"args": ["/path/to/nadlan-mcp/run_fastmcp_server.py"],
"env": {
"GOVMAP_REQUESTS_PER_SECOND": "3.0",
"GOVMAP_MAX_RETRIES": "5"
}
}
}
}
```
## API Limitations
### Govmap API Constraints
**Known Limitations:**
- No published rate limits (being conservative with 5 req/sec)
- No authentication required (public API)
- Data freshness: Updated periodically (not real-time)
- Coverage: Official Israeli government records only
**Best Practices:**
- Use appropriate time ranges (2-5 years recommended)
- Limit radius searches (under 1000m for performance)
- Cache results when possible (future feature)
- Respect rate limiting
### MCP Protocol Constraints
**Token Limits:**
- Large deal lists can exceed LLM context windows
- Use `summarized_response=True` for large datasets
- Default limits (50-100 deals) are token-optimized
## Monitoring & Debugging
### Logging Levels
**INFO:** Normal operations
- Requests being made
- Successful responses
**WARNING:** Recoverable issues
- Retrying after failures
- Rate limiting delays
- Validation warnings
**ERROR:** Failures
- Exhausted retries
- Invalid responses
- Configuration errors
### Debug Mode
Enable debug logging:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
```
## References
- [Govmap API](https://www.govmap.gov.il/)
- [MCP Protocol](https://modelcontextprotocol.io/)
- [FastMCP Documentation](https://github.com/jlowin/fastmcp)
- [Israeli Real Estate Data](https://data.gov.il/)