domverse/ciaovolo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add flight comparator web app with full scan pipeline

Browse Source 
Full-stack flight price scanner built on fast-flights v3 (SOCS cookie bypass):

Backend (FastAPI + SQLite):
- REST API with rate limiting, Pydantic v2 validation, paginated responses
- Scan pipeline: resolves airports, queries every day in the window, saves
  individual flights + aggregate route stats to SQLite
- Background async scan processor with real-time progress tracking
- Airport search endpoint backed by OpenFlights dataset
- Daily scan window (all dates, not monthly samples)

Frontend (React 19 + TypeScript + Tailwind CSS v4):
- Dashboard with live scan status and recent scans
- Create scan form: country mode or specific airports (searchable dropdown)
- Scan detail page with expandable route rows showing individual flights
  (date, airline, departure, arrival, price) loaded on demand
- AirportSearch component with debounced live search and multi-select

Database:
- scans → routes → flights schema with FK cascade and auto-update triggers
- Migrations for schema evolution (relaxed country constraint)

Tests:
- 74 tests: unit + integration, isolated per-test SQLite DB
- Confirmed flight fixtures in tests/confirmed_flights.json (50 real flights,
  BDS→FMM Ryanair + BDS→DUS Eurowings, scraped Feb 2026)
- Integration tests parametrized from confirmed routes

Docker:
- Multi-stage builds, Compose orchestration, Nginx reverse proxy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
domverse
2026-02-26 17:11:51 +01:00
parent aea7590874
commit 6421f83ca7
67 changed files with 37173 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
flight-comparator/.coveragerc Normal file
View File

@@ -0,0 +1,69 @@
[run]
# Coverage configuration for Flight Radar Web App
# Source directories
source = .
# Omit these files from coverage
omit =
*/tests/*
*/test_*.py
*/__pycache__/*
*/venv/*
*/env/*
*/.venv/*
*/site-packages/*
*/dist-packages/*
*/airports.py
*/cache.py
*/cache_admin.py
*/date_resolver.py
*/formatter.py
*/main.py
*/progress.py
*/searcher_*.py
setup.py
# Include only api_server.py and database files
include =
api_server.py
database/*.py
[report]
# Reporting options
# Precision for coverage percentage
precision = 2
# Show missing lines
show_missing = True
# Skip empty files
skip_empty = True
# Skip covered files
skip_covered = False
# Exclude lines from coverage
exclude_lines =
# Standard pragma
pragma: no cover
# Don't complain about missing debug code
def __repr__
# Don't complain if tests don't hit defensive assertion code
raise AssertionError
raise NotImplementedError
# Don't complain if non-runnable code isn't run
if __name__ == .__main__.:
if TYPE_CHECKING:
# Don't complain about abstract methods
@abstractmethod
[html]
# HTML report options
directory = htmlcov
title = Flight Radar Web App - Test Coverage Report

82
flight-comparator/.dockerignore Normal file
View File

@@ -0,0 +1,82 @@
# Git
.git
.gitignore
.github
# Python
__pycache__
*.py[cod]
*$py.class
*.so
.Python
env/
venv/
.venv/
ENV/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# Node
node_modules/
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.npm
.eslintcache
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
.DS_Store
# Testing
.coverage
htmlcov/
.pytest_cache/
.tox/
# Documentation
*.md
docs/
README.md
PRD.MD
CLAUDE.md
*.log
# Frontend build (built in Docker)
frontend/dist/
frontend/node_modules/
# Development files
.env
*.local
# Session files
SESSION_*.md
STEP_*.md
PHASE_*.md
DEBUG_*.md
IMPLEMENTATION_*.md
MIGRATION_*.md
DECISIONS.md
CACHING.md
DAILY_SCAN_FEATURE.md
FAST_FLIGHTS_TEST_REPORT.md
WEB_APP_PRD.md
RESUME_PROMPT.md

67
flight-comparator/.env.example Normal file
View File

@@ -0,0 +1,67 @@
# Flight Radar Web App - Environment Configuration
# Copy this file to .env and customize for your environment
# ============================================================================
# Backend Configuration
# ============================================================================
# Server Settings
PORT=8000
HOST=0.0.0.0
# Database
DATABASE_PATH=/app/data/cache.db
# CORS Origins (comma-separated)
# Development: http://localhost:5173,http://localhost:3000
# Production: https://yourdomain.com
ALLOWED_ORIGINS=http://localhost,http://localhost:80
# ============================================================================
# Frontend Configuration
# ============================================================================
# API Base URL (used during build)
VITE_API_BASE_URL=http://localhost:8000
# ============================================================================
# Docker Configuration
# ============================================================================
# Backend Port (external)
BACKEND_PORT=8000
# Frontend Port (external)
FRONTEND_PORT=80
# ============================================================================
# Optional: Production Settings
# ============================================================================
# Logging Level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
LOG_LEVEL=INFO
# Rate Limiting (requests per minute)
RATE_LIMIT_SCANS=10
RATE_LIMIT_LOGS=30
RATE_LIMIT_AIRPORTS=100
RATE_LIMIT_DEFAULT=60
# Cache Settings
CACHE_THRESHOLD_HOURS=24
# ============================================================================
# Notes
# ============================================================================
#
# Development:
# - Use default settings
# - CORS allows localhost origins
# - Verbose logging enabled
#
# Production:
# - Set proper ALLOWED_ORIGINS
# - Use HTTPS if possible
# - Adjust rate limits as needed
# - Consider using environment-specific .env files
#

59
flight-comparator/.gitignore vendored Normal file
View File

@@ -0,0 +1,59 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# Virtual environments
venv/
ENV/
env/
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Testing
.pytest_cache/
.coverage
htmlcov/
# Output / generated data
*.csv
*.log
# JSON — keep fixture and airport data, ignore everything else
*.json
!data/airports_by_country.json
!tests/confirmed_flights.json
# Database files
*.db
# Node
frontend/node_modules/
frontend/dist/

857
flight-comparator/CLAUDE.md Normal file
View File

@@ -0,0 +1,857 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This repository contains **two applications**:
1. **Flight Airport Comparator CLI** - Python CLI tool for flight comparisons
2. **Flight Radar Web App** - Full-stack web application with REST API, React frontend, and Docker deployment
### CLI Tool
A Python CLI tool that compares direct flights from multiple airports in a country to a single destination, using Google Flights data via the fast-flights library.
**Core question it answers:** "I want to fly to [DESTINATION]. Which airport in [COUNTRY] should I depart from — and when in the next 6 months does the best route open up?"
### Web Application
A production-ready web application providing:
- REST API (FastAPI) with rate limiting, validation, and error handling
- React + TypeScript frontend with real-time updates
- SQLite database with automatic schema migrations
- Docker deployment with health checks
- 43 passing tests with 75% code coverage
## Web Application Architecture
### Tech Stack
**Backend:**
- FastAPI 0.104+ with Pydantic v2 for validation
- SQLite database with foreign keys enabled
- Uvicorn ASGI server
- Python 3.11+
**Frontend:**
- React 19 with TypeScript (strict mode)
- Vite 7 for build tooling
- Tailwind CSS v4 with @tailwindcss/postcss
- React Router v7 for client-side routing
- Axios for API requests
**Infrastructure:**
- Docker multi-stage builds
- Docker Compose orchestration
- Nginx reverse proxy for production
- Volume persistence for database
### Web App File Structure
```
flight-comparator/
├── api_server.py # FastAPI app (1,300+ lines)
├── database/
│ ├── __init__.py # Connection utilities
│ ├── init_db.py # Schema initialization
│ └── schema.sql # Database schema (scans, routes tables)
├── frontend/
│ ├── src/
│ │ ├── api.ts # Type-safe API client (308 lines)
│ │ ├── components/ # React components
│ │ │ ├── Layout.tsx
│ │ │ ├── AirportSearch.tsx
│ │ │ ├── ErrorBoundary.tsx
│ │ │ ├── Toast.tsx
│ │ │ └── LoadingSpinner.tsx
│ │ └── pages/ # Page components
│ │ ├── Dashboard.tsx
│ │ ├── Scans.tsx
│ │ ├── ScanDetails.tsx
│ │ ├── Airports.tsx
│ │ └── Logs.tsx
│ ├── package.json
│ └── vite.config.ts # Vite config with API proxy
├── tests/
│ ├── conftest.py # Pytest fixtures
│ ├── test_api_endpoints.py # 26 unit tests
│ └── test_integration.py # 15 integration tests
├── Dockerfile.backend # Python backend container
├── Dockerfile.frontend # Node + Nginx container
├── docker-compose.yml # Service orchestration
└── nginx.conf # Nginx configuration
Total: ~3,300 lines of production code
```
### Database Schema
**Table: scans**
- Tracks scan requests with status (pending → running → completed/failed)
- Foreign keys enabled with CASCADE deletes
- CHECK constraints for IATA codes (3 chars) and ISO country codes (2 chars)
- Auto-updated timestamps via triggers
- Indexes on `(origin, country)`, `status`, and `created_at`
**Table: routes**
- Stores discovered routes per scan (foreign key to scans.id)
- Flight statistics: min/max/avg price, flight count, airlines array (JSON)
- Composite index on `(scan_id, min_price)` for sorted queries
**Views:**
- `scan_statistics` - Aggregated stats per scan
- `recent_scans` - Last 100 scans with route counts
### API Architecture (api_server.py)
**Key Classes:**
1. **LogBuffer + BufferedLogHandler** (lines 48-100)
- Thread-safe circular buffer for application logs
- Custom logging handler that stores logs in memory
- Supports filtering by level and search
2. **RateLimiter** (lines 102-150)
- Sliding window rate limiting per endpoint per IP
- Independent tracking for each endpoint
- X-Forwarded-For support for proxy setups
- Rate limit headers on all responses
3. **Pydantic Models** (lines 152-300)
- Input validation with auto-normalization (lowercase → uppercase)
- Custom validators for IATA codes (3 chars), ISO codes (2 chars), dates
- Generic PaginatedResponse[T] model for consistent pagination
- Detailed validation error messages
**API Endpoints:**
| Method | Path | Purpose | Rate Limit |
|--------|------|---------|------------|
| GET | `/health` | Health check | No limit |
| GET | `/api/v1/airports` | Search airports | 100/min |
| POST | `/api/v1/scans` | Create scan | 10/min |
| GET | `/api/v1/scans` | List scans | 30/min |
| GET | `/api/v1/scans/{id}` | Get scan details | 30/min |
| GET | `/api/v1/scans/{id}/routes` | Get routes | 30/min |
| GET | `/api/v1/logs` | View logs | 30/min |
**Middleware Stack:**
1. Request ID middleware (UUID per request)
2. CORS middleware (configurable origins via `ALLOWED_ORIGINS` env var)
3. Rate limiting middleware (per-endpoint per-IP)
4. Custom exception handlers (validation, HTTP, general)
**Startup Logic:**
- Downloads airport data from OpenFlights
- Initializes database schema
- Detects and fixes stuck scans (status=running with no update > 1 hour)
- Enables SQLite foreign keys globally
### Frontend Architecture
**Routing:**
- `/` - Dashboard with stats cards and recent scans
- `/scans` - Create new scan form
- `/scans/:id` - View scan details and routes table
- `/airports` - Search airport database
- `/logs` - Application log viewer
**State Management:**
- Local component state with React hooks (useState, useEffect)
- No global state library (Redux, Context) - API is source of truth
- Optimistic UI updates with error rollback
**API Client Pattern (src/api.ts):**
```typescript
// Type-safe interfaces for all API responses
export interface Scan { id: number; origin: string; ... }
export interface Route { id: number; destination: string; ... }
// Organized by resource
export const scanApi = {
list: (page, limit, status?) => api.get<PaginatedResponse<Scan>>(...),
create: (data) => api.post<CreateScanResponse>(...),
get: (id) => api.get<Scan>(...),
routes: (id, page, limit) => api.get<PaginatedResponse<Route>>(...)
};
```
**Error Handling:**
- ErrorBoundary component catches React errors
- Toast notifications for user feedback (4 types: success, error, info, warning)
- LoadingSpinner for async operations
- Graceful fallbacks for missing data
**TypeScript Strict Mode:**
- `verbatimModuleSyntax` enabled
- Type-only imports required: `import type { Scan } from '../api'`
- Explicit `ReturnType<typeof setTimeout>` for timer refs
- No implicit any
## CLI Tool Architecture
### Key Technical Components
1. **Google Flights Scraping with SOCS Cookie Bypass**
- Uses `fast-flights v3.0rc1` (must install from GitHub, not PyPI)
- Custom `SOCSCookieIntegration` class in `searcher_v3.py` (lines 32-79) bypasses Google's EU consent page
- SOCS cookie value from: https://github.com/AWeirdDev/flights/issues/46
- Uses `primp` library for browser impersonation (Chrome 145, macOS)
2. **Async + Threading Hybrid Pattern**
- Main async layer: `search_multiple_routes()` uses asyncio with semaphore for concurrency
- Sync bridge: `asyncio.to_thread()` wraps the synchronous `get_flights()` calls
- Random delays (0.5-1.5s) between requests to avoid rate limiting
- Default concurrency: 5 workers (configurable with `--workers`)
3. **SQLite Caching System** (`cache.py`)
- Two-table schema: `flight_searches` (queries) + `flight_results` (flight data)
- Cache key: SHA256 hash of `origin|destination|date|seat_class|adults`
- Default threshold: 24 hours (configurable with `--cache-threshold`)
- Automatic cache hit detection with progress indicator
- Admin tool: `cache_admin.py` for stats/cleanup
4. **Seasonal Scanning & New Connection Detection**
- `resolve_dates()`: Generates one date per month (default: 15th) across window
- `detect_new_connections()`: Compares route sets month-over-month
- Tags routes as ✨ NEW the first month they appear after being absent
### Critical Error Handling Pattern
**IMPORTANT:** The parser in `searcher_v3.py` (lines 218-302) uses defensive None-checking throughout:
```python
# Always validate before accessing list elements
if not isinstance(flight_segments, list):
continue
if len(flight_segments) == 0:
continue
segment = flight_segments[0]
# Validate segment is not None
if segment is None:
continue
```
**Why:** Google Flights returns different JSON structures depending on availability. Some "no results" responses contain `None` elements or unexpected structures. See `DEBUG_SESSION_2026-02-22_RESOLVED.md` for full analysis.
**Known Issue:** The fast-flights library itself has a bug at `parser.py:55` where it tries to access `payload[3][0]` when `payload[3]` is None. This affects ~11% of edge cases (routes with no flights on specific dates). Our error handling gracefully catches this and returns empty results instead of crashing. Success rate: 89%.
### Module Responsibilities
- **`main.py`**: CLI entrypoint (Click), argument parsing, orchestration
- **`searcher_v3.py`**: Flight queries with SOCS cookie integration, caching, concurrency
- **`date_resolver.py`**: Date logic, seasonal window generation, new connection detection
- **`airports.py`**: Airport data management (OpenFlights dataset), country resolution
- **`formatter.py`**: Output formatting (Rich tables, JSON, CSV)
- **`cache.py`**: SQLite caching layer with timestamp-based invalidation
- **`progress.py`**: Real-time progress display using Rich Live tables
## Common Development Commands
### Web Application
**Backend Development:**
```bash
# Start API server (development mode with auto-reload)
python api_server.py
# Access: http://localhost:8000
# API docs: http://localhost:8000/docs
# Initialize/reset database
python database/init_db.py
# Run backend tests only
pytest tests/test_api_endpoints.py -v
# Run integration tests
pytest tests/test_integration.py -v
# Run all tests with coverage
pytest tests/ -v --cov=api_server --cov=database --cov-report=html
```
**Frontend Development:**
```bash
cd frontend
# Install dependencies (first time)
npm install
# Start dev server with hot reload
npm run dev
# Access: http://localhost:5173
# Note: Vite proxy forwards /api/* to http://localhost:8000
# Type checking
npm run build # Runs tsc -b first
# Lint
npm run lint
# Production build
npm run build
# Output: frontend/dist/
# Preview production build
npm run preview
```
**Docker Deployment:**
```bash
# Quick start (build + start both services)
docker-compose up -d
# View logs
docker-compose logs -f
# Rebuild after code changes
docker-compose up --build
# Stop services
docker-compose down
# Access application
# Frontend: http://localhost
# Backend API: http://localhost:8000
# Database backup
docker cp flight-radar-backend:/app/cache.db ./backup.db
# Database restore
docker cp ./backup.db flight-radar-backend:/app/cache.db
docker-compose restart backend
```
**Testing Web App:**
```bash
# Run all 43 tests
pytest tests/ -v
# Run specific test file
pytest tests/test_api_endpoints.py::test_health_endpoint -v
# Run tests with markers
pytest tests/ -v -m "unit"
pytest tests/ -v -m "integration"
# Coverage report
pytest tests/ --cov-report=term --cov-report=html
# Open: htmlcov/index.html
```
### CLI Tool
**Running the Tool**
```bash
# Single date query
python main.py --to BDS --country DE --date 2026-04-15
# Seasonal scan (6 months, queries 15th of each month)
python main.py --to BDS --country DE
# Daily scan (every day for 3 months) - NEW in 2026-02-22
python main.py --from BDS --to DUS --daily-scan --window 3
# Daily scan with custom date range - NEW in 2026-02-22
python main.py --from BDS --to-country DE --daily-scan --start-date 2026-04-01 --end-date 2026-04-30
# Dry run (preview without API calls)
python main.py --to BDS --country DE --dry-run
# With specific airports and custom workers
python main.py --to BDS --from DUS,MUC,FMM --date 2026-04-15 --workers 1
# Force fresh queries (ignore cache)
python main.py --to BDS --country DE --no-cache
```
### Testing
```bash
# Run full test suite
pytest tests/ -v
# Run integration tests (make real API calls — slow)
pytest tests/ -v -m integration
# Module-specific smoke tests
pytest tests/test_date_resolver.py tests/test_airports.py tests/test_searcher.py tests/test_formatter.py -v
```
### Cache Management
```bash
# View cache statistics
python cache_admin.py stats
# Clean old entries (30+ days)
python cache_admin.py clean --days 30
# Clear entire cache
python cache_admin.py clear-all
```
### Installation & Dependencies
```bash
# CRITICAL: Must install fast-flights v3 from GitHub (not PyPI)
pip install --upgrade git+https://github.com/AWeirdDev/flights.git
# Install other dependencies
pip install -r requirements.txt
# Build airport database (runs automatically on first use)
python airports.py
```
## Code Patterns & Conventions
### Web Application Patterns
**CRITICAL: Foreign Keys Must Be Enabled**
SQLite disables foreign keys by default. **Always** execute `PRAGMA foreign_keys = ON` after creating a connection:
```python
# Correct pattern (database/__init__.py)
conn = sqlite3.connect(db_path)
conn.execute("PRAGMA foreign_keys = ON")
# In tests (tests/conftest.py)
@pytest.fixture
def clean_database(test_db_path):
conn = get_connection()
conn.execute("PRAGMA foreign_keys = ON") # ← REQUIRED
# ... rest of fixture
```
**Why:** Without this, CASCADE deletes don't work, foreign key constraints aren't enforced, and data integrity is compromised.
**Rate Limiting: Per-Endpoint Per-IP**
The RateLimiter class tracks limits independently for each endpoint:
```python
# api_server.py lines 102-150
class RateLimiter:
def __init__(self):
self.requests = defaultdict(lambda: defaultdict(deque))
# Structure: {endpoint: {ip: deque([timestamps])}}
```
**Why:** Prevents a single IP from exhausting the scan quota (10/min) by making log requests (30/min). Each endpoint has independent limits.
**Validation: Auto-Normalization**
Pydantic validators auto-normalize inputs:
```python
class CreateScanRequest(BaseModel):
origin: str
country: str
@validator('origin', 'country', pre=True)
def uppercase_codes(cls, v):
return v.strip().upper() if v else v
```
**Result:** Frontend can send lowercase codes, backend normalizes them. Consistent database format.
**API Responses: Consistent Format**
All endpoints return:
- Success: `{ data: T, metadata?: {...} }`
- Error: `{ detail: string | object, request_id: string }`
- Paginated: `{ items: T[], pagination: { page, limit, total, pages } }`
**Database: JSON Arrays for Airlines**
The `routes.airlines` column stores JSON arrays:
```python
# Saving (api_server.py line ~1311)
json.dumps(route['airlines'])
# Loading (api_server.py line ~1100)
json.loads(row['airlines']) if row['airlines'] else []
```
**Why:** SQLite doesn't have array types. JSON serialization maintains type safety.
**Frontend: Type-Only Imports**
With `verbatimModuleSyntax` enabled:
```typescript
// ❌ Wrong - runtime import of type
import { Scan } from '../api'
// ✅ Correct - type-only import
import type { Scan } from '../api'
```
**Error if wrong:** `'Scan' is a type and must be imported using a type-only import`
**Frontend: Timer Refs**
```typescript
// ❌ Wrong - no NodeJS in browser
const timer = useRef<NodeJS.Timeout>()
// ✅ Correct - ReturnType utility
const timer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined)
```
**Frontend: Debounced Search**
Pattern used in AirportSearch.tsx:
```typescript
const debounceTimer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
const handleInputChange = (e) => {
if (debounceTimer.current) {
clearTimeout(debounceTimer.current);
}
debounceTimer.current = setTimeout(() => {
// API call here
}, 300);
};
// Cleanup on unmount
useEffect(() => {
return () => {
if (debounceTimer.current) {
clearTimeout(debounceTimer.current);
}
};
}, []);
```
### CLI Tool Error Handling Philosophy
**Graceful degradation over crashes:**
- Always wrap parsing in try/except with detailed logging
- Return empty lists `[]` instead of raising exceptions
- Log errors with full traceback but continue processing other routes
- Progress callback reports errors but search continues
Example from `searcher_v3.py`:
```python
except Exception as parse_error:
import traceback
print(f"\n=== PARSING ERROR ===")
print(f"Query: {origin}→{destination} on {date}")
traceback.print_exc()
# Return empty list instead of crashing
return []
```
### Defensive Programming for API Responses
When working with flight data from fast-flights:
1. **Always** check `isinstance()` before assuming type
2. **Always** validate list is not empty before accessing `[0]`
3. **Always** check element is not `None` after accessing
4. **Always** use `getattr(obj, 'attr', default)` for optional fields
5. **Always** handle both `[H]` and `[H, M]` time formats
### Async/Await Patterns
- Use `asyncio.to_thread()` to bridge sync libraries (fast-flights) with async code
- Use `asyncio.Semaphore()` to limit concurrent requests
- Use `asyncio.gather()` to execute all tasks in parallel
- Add random delays (`asyncio.sleep(random.uniform(0.5, 1.5))`) to avoid rate limiting
### Cache-First Strategy
1. Check cache first with `get_cached_results()`
2. On cache miss, query API and save with `save_results()`
3. Report cache hits via progress callback
4. Respect `use_cache` flag and `cache_threshold_hours` parameter
## Important Constants
From `date_resolver.py`:
```python
SEARCH_WINDOW_MONTHS = 6 # Default seasonal scan window
SAMPLE_DAY_OF_MONTH = 15 # Which day to query each month (seasonal mode only)
```
From `cache.py`:
```python
DEFAULT_CACHE_THRESHOLD_HOURS = 24
```
## Debugging Tips
### When Flight Searches Fail
1. Look for patterns in error logs:
- `'NoneType' object is not subscriptable` → Missing None validation in `searcher_v3.py`
- `fast-flights/parser.py line 55` → Library bug, can't fix without patching (~11% of edge cases)
2. Verify SOCS cookie is still valid (see `docs/MIGRATION_V3.md` for refresh instructions)
3. Run with `--workers 1` to rule out concurrency as the cause
### Performance Issues
- Reduce `--window` for faster seasonal scans
- Increase `--workers` (but watch rate limiting)
- Use `--from` with specific airports instead of `--country`
- Check cache hit rate with `cache_admin.py stats`
### Concurrency Issues
- Start with `--workers 1` to isolate non-concurrency bugs
- Gradually increase workers while monitoring error rates
- Note: Error rates can differ between sequential and concurrent execution, suggesting rate limiting or response variation
## Testing Philosophy
- **Smoke tests** in `tests/` verify each module works independently
- **Integration tests** (`-m integration`) make real API calls — use confirmed routes from `tests/confirmed_flights.json`
- Always test with `--workers 1` first when debugging to isolate concurrency issues
## Known Limitations
1. **fast-flights library dependency:** Subject to Google's anti-bot measures and API changes
2. **Rate limiting:** Large scans (100+ airports) may hit rate limits despite delays
3. **EU consent flow:** Relies on SOCS cookie workaround which may break if Google changes their system
4. **Parser bug in fast-flights:** ~11% failure rate on edge cases (gracefully handled — returns empty result)
5. **Prices are snapshots:** Not final booking prices, subject to availability changes
6. **Prices are snapshots:** Not final booking prices, subject to availability changes
## Documentation
- **`README.md`**: Main entry point and usage guide
- **`docs/DEPLOYMENT.md`**: Comprehensive deployment guide (Docker + manual)
- **`docs/DOCKER_README.md`**: Docker quick-start guide
- **`docs/DECISIONS.md`**: Architecture and design decisions
- **`docs/MIGRATION_V3.md`**: fast-flights v2→v3 migration and SOCS cookie refresh
- **`docs/CACHING.md`**: SQLite caching layer reference
- **`database/schema.sql`**: Database schema with full comments
- **`tests/confirmed_flights.json`**: Ground-truth flight data for integration tests
## Environment Variables
### Web Application
**Backend (api_server.py):**
```bash
# Server configuration
PORT=8000 # API server port
HOST=0.0.0.0 # Bind address (0.0.0.0 for Docker)
# Database
DATABASE_PATH=/app/data/cache.db # SQLite database path
# CORS
ALLOWED_ORIGINS=http://localhost,http://localhost:80 # Comma-separated
# Logging
LOG_LEVEL=INFO # DEBUG, INFO, WARNING, ERROR, CRITICAL
# Rate limiting (requests per minute per IP)
RATE_LIMIT_SCANS=10
RATE_LIMIT_LOGS=30
RATE_LIMIT_AIRPORTS=100
```
**Frontend (vite.config.ts):**
```bash
# Build-time only
VITE_API_BASE_URL=/api/v1 # API base URL (usually use proxy instead)
```
**Docker (.env file):**
```bash
# Service ports
BACKEND_PORT=8000
FRONTEND_PORT=80
# All backend variables above also apply
```
**Configuration Files:**
- `.env.example` - Template with all variables documented (72 lines)
- `frontend/vite.config.ts` - API proxy for development
- `nginx.conf` - API proxy for production
### CLI Tool Environment
No environment variables required for CLI tool. All configuration via command-line flags.
## When Making Changes
### Web Application Changes
**Before Modifying API Endpoints (api_server.py):**
1. **Always read existing code first** to understand request/response patterns
2. **Update Pydantic models** if adding new fields
3. **Add validation** with descriptive error messages
4. **Update frontend API client** (frontend/src/api.ts) with new types
5. **Add tests** in tests/test_api_endpoints.py
6. **Update rate limiting** if adding new endpoint
7. **Document in API docs** (FastAPI auto-generates from docstrings)
**Before Modifying Database Schema (database/schema.sql):**
1. **CRITICAL:** Test migration path from existing data
2. **Add migration logic** to database/init_db.py
3. **Update CHECK constraints** if changing validation rules
4. **Add/update indexes** for new query patterns
5. **Test foreign key cascades** work correctly
6. **Update tests** in tests/test_integration.py
7. **Backup production data** before applying
Example migration pattern:
```python
# database/init_db.py
def migrate_to_v2(conn):
"""Add new column with default value."""
try:
conn.execute("ALTER TABLE scans ADD COLUMN new_field TEXT DEFAULT 'default'")
conn.commit()
except sqlite3.OperationalError:
# Column already exists, skip
pass
```
**Before Modifying Frontend Components:**
1. **Check TypeScript strict mode** requirements (type-only imports)
2. **Update API client types** (src/api.ts) if API changed
3. **Test responsive design** on mobile/tablet/desktop
4. **Verify error handling** with network failures
5. **Check accessibility** (keyboard navigation, screen readers)
6. **Update tests** if adding testable logic
7. **Verify production build** with `npm run build`
**Before Modifying Docker Configuration:**
1. **Validate docker-compose.yml** with `docker-compose config`
2. **Test build** with `docker-compose build --no-cache`
3. **Verify health checks** work correctly
4. **Test volume persistence** (database survives restarts)
5. **Check environment variables** are properly passed
6. **Update documentation** (`docs/DEPLOYMENT.md`, `docs/DOCKER_README.md`)
7. **Test full deployment** from scratch
**Rate Limiting Changes:**
When modifying rate limits:
1. Update constants in api_server.py
2. Update .env.example with new defaults
3. Consider impact on user experience (too strict = frustrated users)
4. Test with concurrent requests
5. Document in API response headers
**Common Pitfalls:**
1. **Forgetting foreign keys:** Add `PRAGMA foreign_keys = ON` to every connection
2. **Type-only imports:** Use `import type` for interfaces in TypeScript
3. **JSON arrays:** Remember to `json.loads()` when reading airlines from database
4. **Rate limiting:** New endpoints need rate limit decorator
5. **CORS:** Add new origins to ALLOWED_ORIGINS env var
6. **Cache invalidation:** Frontend may cache old data, handle with ETags or timestamps
### CLI Tool Changes
**Before Modifying Parser (`searcher_v3.py`)
### Before Modifying Parser (`searcher_v3.py`)
1. Maintain the layered validation pattern: type check → empty check → None check (see lines 218-302)
2. Run `pytest tests/test_scan_pipeline.py -m integration` to verify known routes still return flights
3. Add comprehensive error logging with tracebacks for debugging
### Before Modifying Caching (`cache.py`)
1. Understand the two-table schema: searches + results
2. Remember that cache keys include ALL query parameters (origin, destination, date, seat_class, adults)
3. Test cache invalidation logic with different threshold values
4. Verify foreign key cascade deletes work correctly
### Before Modifying Async Logic (`searcher_v3.py`, `main.py`)
1. Respect the sync/async boundary: fast-flights is synchronous, use `asyncio.to_thread()`
2. Always use semaphores to limit concurrency (prevent rate limiting)
3. Test with different `--workers` values (1, 3, 5, 10) to verify behavior
4. Add random delays between requests to avoid anti-bot detection
### Before Adding New CLI Arguments (`main.py`)
1. Update Click options with proper help text and defaults
2. Update `README.md` usage examples
3. Update `PRD.MD` if changing core functionality
4. Consider cache implications (new parameter = new cache key dimension)
---
## Project Status
### Web Application: ✅ PRODUCTION READY
**Completed:** All 30 steps across 4 phases (100% complete)
**Phase 1: Backend Foundation** - ✅ 10/10 steps
- Database schema with triggers and views
- FastAPI REST API with validation
- Error handling and rate limiting
- Startup cleanup for stuck scans
- Log viewer endpoint
**Phase 2: Testing Infrastructure** - ✅ 5/5 steps
- pytest configuration
- 43 passing tests (26 unit + 15 integration)
- 75% code coverage
- Database isolation in tests
- Test fixtures and factories
**Phase 3: Frontend Development** - ✅ 10/10 steps
- React + TypeScript app with Vite
- Tailwind CSS v4 styling
- 5 pages + 5 components
- Type-safe API client
- Error boundary and toast notifications
- Production build: 293 KB (93 KB gzipped)
**Phase 4: Docker Deployment** - ✅ 5/5 steps
- Multi-stage Docker builds
- Docker Compose orchestration
- Nginx reverse proxy
- Volume persistence
- Health checks and auto-restart
**Quick Start:**
```bash
docker-compose up -d
open http://localhost
```
### CLI Tool: ✅ FUNCTIONAL
- Successfully queries Google Flights via fast-flights v3 with SOCS cookie
- 89% success rate on real flight queries
- Caching system reduces API calls
- Seasonal scanning and new route detection
- Rich terminal output
**Known limitations:** fast-flights library parser bug affects ~11% of edge cases (documented in DEBUG_SESSION_2026-02-22_RESOLVED.md)
---
**Total Project:**
- ~3,300+ lines of production code
- ~2,500+ lines of documentation
- 43/43 tests passing
- Zero TODO/FIXME comments
- Docker validated
- Ready for deployment

47
flight-comparator/Dockerfile.backend Normal file
View File

@@ -0,0 +1,47 @@
# Backend Dockerfile for Flight Radar API
FROM python:3.11-slim
# Set working directory
WORKDIR /app
# Set environment variables
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1
# Install system dependencies
RUN apt-get update && apt-get install -y \
gcc \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements file
COPY requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY api_server.py .
COPY airports.py .
COPY cache.py .
COPY database/ ./database/
# Create necessary directories
RUN mkdir -p data
# Download airport data on build
RUN python -c "from airports import download_and_build_airport_data; download_and_build_airport_data()"
# Initialize database
RUN python database/init_db.py
# Expose port
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD python -c "import requests; requests.get('http://localhost:8000/health').raise_for_status()"
# Run the application
CMD ["python", "api_server.py"]

36
flight-comparator/Dockerfile.frontend Normal file
View File

@@ -0,0 +1,36 @@
# Frontend Dockerfile for Flight Radar UI
# Stage 1: Build React application
FROM node:20-alpine AS builder
WORKDIR /app
# Copy package files
COPY frontend/package*.json ./
# Install dependencies
RUN npm ci
# Copy source code
COPY frontend/ .
# Build production app
RUN npm run build
# Stage 2: Serve with nginx
FROM nginx:alpine
# Copy built assets from builder stage
COPY --from=builder /app/dist /usr/share/nginx/html
# Copy nginx configuration
COPY nginx.conf /etc/nginx/conf.d/default.conf
# Expose port
EXPOSE 80
# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --quiet --tries=1 --spider http://localhost/ || exit 1
# Start nginx
CMD ["nginx", "-g", "daemon off;"]

295
flight-comparator/README.md Normal file
View File

@@ -0,0 +1,295 @@
# Flight Airport Comparator CLI ✈️
A Python CLI tool that helps you find the best departure airport for your destination by comparing direct flights from all airports in a country.
**✅ NOW WITH WORKING FLIGHT DATA!** Uses fast-flights v3.0rc1 with SOCS cookie integration to successfully bypass Google's consent page.
## What It Does
Answers the question: **"I want to fly to [DESTINATION]. Which airport in [COUNTRY] should I depart from — and when in the next 6 months does the best route open up?"**
### Key Features
- 🌍 **Multi-Airport Comparison**: Automatically scans all airports in a country
- 📅 **Seasonal Scanning**: Discover new routes and price trends across 6 months
- ⚡ **Direct Flights Only**: Filters out connections automatically
- 🆕 **New Route Detection**: Highlights routes that appear in later months
- 🎨 **Beautiful Tables**: Rich terminal output with color and formatting
- 🚀 **Fast & Concurrent**: Parallel API requests for quick results
- ✅ **SOCS Cookie Integration**: Bypasses Google consent page for real flight data!
- 💾 **Smart Caching**: SQLite cache reduces API calls and prevents rate limiting
## Installation
```bash
# Clone or download this repository
cd flight-comparator
# Install fast-flights v3.0rc1 (REQUIRED for working flight data)
pip install --upgrade git+https://github.com/AWeirdDev/flights.git
# Install other dependencies
pip install -r requirements.txt
# Build airport database (runs automatically on first use)
python airports.py
```
### Requirements
- Python 3.10+
- **fast-flights v3.0rc1** (install from GitHub, not PyPI)
- Dependencies: click, rich, python-dateutil, primp
## Quick Test
Verify it works with real flight data:
```bash
python test_v3_with_cookies.py
```
Expected output:
```
✅ SUCCESS! Found 1 flight option(s):
1. Ryanair
Price: €89
BER → BRI
06:10 - 08:20 (130 min)
```
## Usage
### Basic Examples
**Single date query:**
```bash
python main.py --to JFK --country DE --date 2026-06-15
```
**Seasonal scan (6 months):**
```bash
python main.py --to JFK --country DE
```
**Custom airport list:**
```bash
python main.py --to JFK --from FRA,MUC,BER --date 2026-06-15
```
**Dry run (preview without API calls):**
```bash
python main.py --to JFK --country DE --dry-run
```
### All Options
```
Options:
--to TEXT Destination airport IATA code (e.g., JFK) [required]
--country TEXT Origin country ISO code (e.g., DE, US)
--date TEXT Departure date YYYY-MM-DD. Omit for seasonal scan.
--window INTEGER Months to scan in seasonal mode (default: 6)
--seat [economy|premium|business|first]
Cabin class (default: economy)
--adults INTEGER Number of passengers (default: 1)
--sort [price|duration] Sort order (default: price)
--from TEXT Comma-separated IATA codes (overrides --country)
--top INTEGER Max results per airport (default: 3)
--output [table|json|csv]
Output format (default: table)
--workers INTEGER Concurrency level (default: 5)
--dry-run List airports and dates without API calls
--help Show this message and exit.
```
### Advanced Examples
**Business class, sorted by duration:**
```bash
python main.py --to SIN --country DE --date 2026-07-20 --seat business --sort duration
```
**Seasonal scan with 12-month window:**
```bash
python main.py --to LAX --country GB --window 12
```
**Output as JSON:**
```bash
python main.py --to CDG --country NL --date 2026-05-10 --output json
```
**Force fresh queries (disable cache):**
```bash
python main.py --to JFK --country DE --no-cache
```
**Custom cache threshold (48 hours):**
```bash
python main.py --to JFK --country DE --cache-threshold 48
```
## How It Works
1. **Airport Resolution**: Loads airports for your country from the OpenFlights dataset
2. **Date Resolution**: Single date or generates monthly dates (15th of each month)
3. **Flight Search**: Queries Google Flights via fast-flights for each airport × date
4. **Filtering**: Keeps only direct flights (0 stops)
5. **Analysis**: Detects new connections in seasonal mode
6. **Formatting**: Presents results in beautiful tables, JSON, or CSV
## Seasonal Scan Mode
When you omit `--date`, the tool automatically:
- Queries one date per month (default: 15th) across the next 6 months
- Detects routes that appear in later months but not earlier ones
- Tags new connections with ✨ NEW indicator
- Helps you discover seasonal schedule changes
This is especially useful for:
- Finding when summer routes start
- Discovering new airline schedules
- Comparing price trends over time
## Country Codes
Common country codes:
- 🇩🇪 DE (Germany)
- 🇺🇸 US (United States)
- 🇬🇧 GB (United Kingdom)
- 🇫🇷 FR (France)
- 🇪🇸 ES (Spain)
- 🇮🇹 IT (Italy)
- 🇳🇱 NL (Netherlands)
- 🇦🇺 AU (Australia)
- 🇯🇵 JP (Japan)
[Full list of supported countries available in data/airports_by_country.json]
## Architecture
```
flight-comparator/
├── main.py # CLI entrypoint (Click)
├── date_resolver.py # Date logic & new connection detection
├── airports.py # Airport data management
├── searcher.py # Flight search with concurrency
├── formatter.py # Output formatting (Rich tables, JSON, CSV)
├── data/
│ └── airports_by_country.json # Generated airport database
├── tests/ # Smoke tests for each module
└── requirements.txt
```
## Caching System
The tool uses SQLite to cache flight search results, reducing API calls and preventing rate limiting.
### How It Works
- **Automatic caching**: All search results are saved to `data/flight_cache.db`
- **Cache hits**: If a query was made recently, results are retrieved instantly from cache
- **Default threshold**: 24 hours (configurable with `--cache-threshold`)
- **Cache indicator**: Shows `💾 Cache hit:` when using cached data
### Cache Management
**View cache statistics:**
```bash
python cache_admin.py stats
```
**Clean old entries (30+ days):**
```bash
python cache_admin.py clean --days 30
```
**Clear entire cache:**
```bash
python cache_admin.py clear-all
```
### CLI Options
- `--cache-threshold N`: Set cache validity in hours (default: 24)
- `--no-cache`: Force fresh API queries, ignore cache
### Benefits
- ⚡ **Instant results** for repeated queries (0.0s vs 2-3s per query)
- 🛡️ **Rate limit protection**: Avoid hitting Google's API limits
- 💰 **Reduced API load**: Fewer requests = lower risk of being blocked
- 📊 **Historical data**: Cache preserves price history
## Configuration
Key constants in `date_resolver.py`:
```python
SEARCH_WINDOW_MONTHS = 6 # Default seasonal scan window
SAMPLE_DAY_OF_MONTH = 15 # Which day to query each month
```
You can override the window at runtime with `--window N`.
## Limitations
- ⚠️ Relies on fast-flights scraping Google Flights (subject to rate limits and anti-bot measures)
- ⚠️ EU users may encounter consent flow issues (use fallback mode, which is default)
- ⚠️ Prices are as shown on Google Flights, not final booking prices
- ⚠️ Seasonal scan queries only the 15th of each month as a sample
- ⚠️ Large scans (many airports × months) can take 2-3 minutes
## Performance
Single date scan:
- ~20 airports: < 30s (with --workers 5)
Seasonal scan (6 months):
- ~20 airports: 2-3 minutes
- Total requests: 120 (20 × 6)
## Testing
Run smoke tests for each module:
```bash
cd tests
python test_date_resolver.py
python test_airports.py
python test_searcher.py
python test_formatter.py
```
## Troubleshooting
**"fast-flights not installed"**
```bash
pip install fast-flights
```
**"Country code 'XX' not found"**
- Check the country code is correct (2-letter ISO code)
- Verify it exists in `data/airports_by_country.json`
**Slow performance**
- Reduce `--window` for seasonal scans
- Increase `--workers` (but watch out for rate limiting)
- Use `--from` with specific airports instead of entire country
**No results found**
- Try a different date (some routes are seasonal)
- Check the destination airport code is correct
- Verify there actually are direct flights on that route
## License
This tool is for personal use and research. Respect Google Flights' terms of service and rate limits.
## Credits
- Uses [fast-flights](https://github.com/shmuelzon/fast-flights) for Google Flights scraping
- Airport data from [OpenFlights](https://openflights.org/)
- Built with [Click](https://click.palletsprojects.com/) and [Rich](https://rich.readthedocs.io/)

235
flight-comparator/airports.py Normal file
View File

@@ -0,0 +1,235 @@
"""
Airport data resolution by country.
Handles loading and filtering airport data from OpenFlights dataset.
"""
import json
import csv
from pathlib import Path
from typing import Optional
import urllib.request
# Try to import pycountry, fall back to manual mapping if not available
try:
import pycountry
HAS_PYCOUNTRY = True
except ImportError:
HAS_PYCOUNTRY = False
AIRPORTS_JSON_PATH = Path(__file__).parent / "data" / "airports_by_country.json"
OPENFLIGHTS_URL = "https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat"
# Manual mapping for common countries (fallback if pycountry not available)
COUNTRY_NAME_TO_ISO = {
"Germany": "DE",
"United States": "US",
"United Kingdom": "GB",
"France": "FR",
"Spain": "ES",
"Italy": "IT",
"Netherlands": "NL",
"Belgium": "BE",
"Austria": "AT",
"Switzerland": "CH",
"Poland": "PL",
"Czech Republic": "CZ",
"Denmark": "DK",
"Sweden": "SE",
"Norway": "NO",
"Finland": "FI",
"Ireland": "IE",
"Portugal": "PT",
"Greece": "GR",
"Turkey": "TR",
"Japan": "JP",
"China": "CN",
"South Korea": "KR",
"India": "IN",
"Australia": "AU",
"New Zealand": "NZ",
"Canada": "CA",
"Mexico": "MX",
"Brazil": "BR",
"Argentina": "AR",
"Chile": "CL",
"Colombia": "CO",
"Peru": "PE",
"South Africa": "ZA",
"Egypt": "EG",
"United Arab Emirates": "AE",
"Thailand": "TH",
"Singapore": "SG",
"Malaysia": "MY",
"Indonesia": "ID",
"Philippines": "PH",
"Vietnam": "VN",
}
def country_name_to_iso_code(country_name: str) -> Optional[str]:
"""
Convert country name to ISO 2-letter code.
Args:
country_name: Full country name
Returns:
ISO 2-letter code or None if not found
"""
if HAS_PYCOUNTRY:
try:
country = pycountry.countries.search_fuzzy(country_name)[0]
return country.alpha_2
except (LookupError, AttributeError):
pass
# Fallback to manual mapping
return COUNTRY_NAME_TO_ISO.get(country_name)
def download_and_build_airport_data(force_rebuild: bool = False) -> None:
"""
Download OpenFlights dataset and build airports_by_country.json.
Filters to airports with valid IATA codes only.
Groups by ISO 2-letter country code.
Args:
force_rebuild: If True, rebuild even if file exists
"""
if AIRPORTS_JSON_PATH.exists() and not force_rebuild:
return
print(f"Downloading OpenFlights airport data from {OPENFLIGHTS_URL}...")
# Download the data
response = urllib.request.urlopen(OPENFLIGHTS_URL)
data = response.read().decode('utf-8')
# Parse CSV
# Format: AirportID,Name,City,Country,IATA,ICAO,Lat,Lon,Alt,Timezone,DST,Tz,Type,Source
airports_by_country = {}
for line in data.strip().split('\n'):
# Use csv reader to handle quoted fields properly
row = next(csv.reader([line]))
if len(row) < 5:
continue
airport_id = row[0]
name = row[1]
city = row[2]
country_name = row[3]
iata = row[4]
icao = row[5] if len(row) > 5 else ""
# Skip if no valid IATA code
if not iata or iata == "\\N" or len(iata) != 3:
continue
# Skip if country name is missing
if not country_name or country_name == "\\N":
continue
# Convert country name to ISO code
country_code = country_name_to_iso_code(country_name)
if not country_code:
# Skip if we can't map the country
continue
# Build airport entry
airport = {
"iata": iata,
"name": name,
"city": city,
"icao": icao if icao != "\\N" else ""
}
# Group by country ISO code
if country_code not in airports_by_country:
airports_by_country[country_code] = []
airports_by_country[country_code].append(airport)
# Ensure data directory exists
AIRPORTS_JSON_PATH.parent.mkdir(parents=True, exist_ok=True)
# Write to JSON file
with open(AIRPORTS_JSON_PATH, 'w', encoding='utf-8') as f:
json.dump(airports_by_country, f, indent=2, ensure_ascii=False)
total_airports = sum(len(v) for v in airports_by_country.values())
print(f"✓ Built airport data: {len(airports_by_country)} countries, {total_airports} airports")
def get_airports_for_country(country_code: str) -> list[dict]:
"""
Get list of airports for a given country code.
Args:
country_code: ISO 2-letter country code (e.g., "DE", "US")
Returns:
List of airport dicts with keys: iata, name, city, icao
Raises:
FileNotFoundError: If airports data file doesn't exist
ValueError: If country code not found
"""
# Ensure data file exists
if not AIRPORTS_JSON_PATH.exists():
download_and_build_airport_data()
# Load from JSON
with open(AIRPORTS_JSON_PATH, 'r', encoding='utf-8') as f:
airports_by_country = json.load(f)
country_code = country_code.upper()
if country_code not in airports_by_country:
available = sorted(airports_by_country.keys())[:10]
raise ValueError(
f"Country code '{country_code}' not found. "
f"Available codes (sample): {', '.join(available)}..."
)
return airports_by_country[country_code]
def resolve_airport_list(country: Optional[str], from_airports: Optional[str]) -> list[dict]:
"""
Resolve the final list of origin airports to scan.
Args:
country: ISO 2-letter country code (if --from not provided)
from_airports: Comma-separated IATA codes (overrides country)
Returns:
List of airport dicts with keys: iata, name, city
Raises:
ValueError: If neither country nor from_airports provided, or if invalid
"""
if from_airports:
# Parse custom airport list
iata_codes = [code.strip().upper() for code in from_airports.split(',')]
# Create minimal airport dicts
return [{"iata": code, "name": code, "city": ""} for code in iata_codes]
if country:
return get_airports_for_country(country)
raise ValueError("Either --country or --from must be provided")
if __name__ == "__main__":
# Build the dataset if run directly
download_and_build_airport_data(force_rebuild=True)
print("\nSample data for Germany (DE):")
de_airports = get_airports_for_country("DE")
for airport in de_airports[:5]:
print(f" {airport['iata']} - {airport['name']} ({airport['city']})")
print(f" ... and {len(de_airports) - 5} more")

1645
flight-comparator/api_server.py Normal file
View File

File diff suppressed because it is too large Load Diff

311
flight-comparator/cache.py Normal file
View File

@@ -0,0 +1,311 @@
"""
SQLite caching layer for flight search results.
Stores search results with timestamps to avoid unnecessary API calls
and reduce rate limiting issues.
"""
import sqlite3
import hashlib
import json
from datetime import datetime, timedelta
from pathlib import Path
from typing import Optional
# Cache database location
CACHE_DB_PATH = Path(__file__).parent / "data" / "flight_cache.db"
# Default cache threshold in hours
DEFAULT_CACHE_THRESHOLD_HOURS = 24
def init_database():
"""Initialize SQLite database with required tables."""
CACHE_DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(CACHE_DB_PATH)
cursor = conn.cursor()
# Table for search queries
cursor.execute("""
CREATE TABLE IF NOT EXISTS flight_searches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
query_hash TEXT NOT NULL UNIQUE,
origin TEXT NOT NULL,
destination TEXT NOT NULL,
search_date TEXT NOT NULL,
seat_class TEXT NOT NULL,
adults INTEGER NOT NULL,
query_timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
)
""")
# Table for flight results
cursor.execute("""
CREATE TABLE IF NOT EXISTS flight_results (
id INTEGER PRIMARY KEY AUTOINCREMENT,
search_id INTEGER NOT NULL,
airline TEXT,
departure_time TEXT,
arrival_time TEXT,
duration_minutes INTEGER,
price REAL,
currency TEXT,
plane_type TEXT,
FOREIGN KEY (search_id) REFERENCES flight_searches(id) ON DELETE CASCADE
)
""")
# Indexes for performance
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_query_hash
ON flight_searches(query_hash)
""")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_query_timestamp
ON flight_searches(query_timestamp)
""")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_search_id
ON flight_results(search_id)
""")
conn.commit()
conn.close()
def get_cache_key(origin: str, destination: str, date: str, seat_class: str, adults: int) -> str:
"""
Generate a unique cache key for a flight search query.
Args:
origin: Origin airport IATA code
destination: Destination airport IATA code
date: Search date (YYYY-MM-DD)
seat_class: Cabin class
adults: Number of passengers
Returns:
SHA256 hash of the query parameters
"""
query_string = f"{origin}|{destination}|{date}|{seat_class}|{adults}"
return hashlib.sha256(query_string.encode()).hexdigest()
def get_cached_results(
origin: str,
destination: str,
date: str,
seat_class: str,
adults: int,
threshold_hours: int = DEFAULT_CACHE_THRESHOLD_HOURS,
) -> Optional[list[dict]]:
"""
Retrieve cached flight results if they exist and are recent enough.
Args:
origin: Origin airport IATA code
destination: Destination airport IATA code
date: Search date (YYYY-MM-DD)
seat_class: Cabin class
adults: Number of passengers
threshold_hours: Maximum age of cached results in hours
Returns:
List of flight dicts if cache hit, None if cache miss or expired
"""
init_database()
cache_key = get_cache_key(origin, destination, date, seat_class, adults)
threshold_time = datetime.now() - timedelta(hours=threshold_hours)
conn = sqlite3.connect(CACHE_DB_PATH)
cursor = conn.cursor()
# Find recent search
cursor.execute("""
SELECT id, query_timestamp
FROM flight_searches
WHERE query_hash = ?
AND query_timestamp > ?
ORDER BY query_timestamp DESC
LIMIT 1
""", (cache_key, threshold_time.isoformat()))
search_row = cursor.fetchone()
if not search_row:
conn.close()
return None
search_id, timestamp = search_row
# Retrieve flight results
cursor.execute("""
SELECT airline, departure_time, arrival_time, duration_minutes,
price, currency, plane_type
FROM flight_results
WHERE search_id = ?
""", (search_id,))
flight_rows = cursor.fetchall()
conn.close()
# Convert to flight dicts
flights = []
for row in flight_rows:
flights.append({
"origin": origin,
"destination": destination,
"airline": row[0],
"departure_time": row[1],
"arrival_time": row[2],
"duration_minutes": row[3],
"price": row[4],
"currency": row[5],
"plane_type": row[6],
"stops": 0, # Only direct flights are cached
})
return flights
def save_results(
origin: str,
destination: str,
date: str,
seat_class: str,
adults: int,
flights: list[dict],
) -> None:
"""
Save flight search results to cache database.
Args:
origin: Origin airport IATA code
destination: Destination airport IATA code
date: Search date (YYYY-MM-DD)
seat_class: Cabin class
adults: Number of passengers
flights: List of flight dicts to cache
"""
init_database()
cache_key = get_cache_key(origin, destination, date, seat_class, adults)
conn = sqlite3.connect(CACHE_DB_PATH)
cursor = conn.cursor()
try:
# Delete old search with same cache key (replace with fresh data)
cursor.execute("""
DELETE FROM flight_searches
WHERE query_hash = ?
""", (cache_key,))
# Insert search query
cursor.execute("""
INSERT INTO flight_searches
(query_hash, origin, destination, search_date, seat_class, adults)
VALUES (?, ?, ?, ?, ?, ?)
""", (cache_key, origin, destination, date, seat_class, adults))
search_id = cursor.lastrowid
# Insert flight results
for flight in flights:
cursor.execute("""
INSERT INTO flight_results
(search_id, airline, departure_time, arrival_time, duration_minutes,
price, currency, plane_type)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""", (
search_id,
flight.get("airline"),
flight.get("departure_time"),
flight.get("arrival_time"),
flight.get("duration_minutes"),
flight.get("price"),
flight.get("currency"),
flight.get("plane_type"),
))
conn.commit()
except Exception as e:
conn.rollback()
print(f"⚠️ Cache save failed: {e}")
finally:
conn.close()
def clear_old_cache(days: int = 30) -> int:
"""
Delete cached results older than specified number of days.
Args:
days: Maximum age of cached results to keep
Returns:
Number of deleted search records
"""
init_database()
threshold_time = datetime.now() - timedelta(days=days)
conn = sqlite3.connect(CACHE_DB_PATH)
cursor = conn.cursor()
cursor.execute("""
DELETE FROM flight_searches
WHERE query_timestamp < ?
""", (threshold_time.isoformat(),))
deleted_count = cursor.rowcount
conn.commit()
conn.close()
return deleted_count
def get_cache_stats() -> dict:
"""
Get statistics about cached data.
Returns:
Dict with cache statistics
"""
init_database()
conn = sqlite3.connect(CACHE_DB_PATH)
cursor = conn.cursor()
# Count total searches
cursor.execute("SELECT COUNT(*) FROM flight_searches")
total_searches = cursor.fetchone()[0]
# Count total flight results
cursor.execute("SELECT COUNT(*) FROM flight_results")
total_results = cursor.fetchone()[0]
# Get oldest and newest entries
cursor.execute("""
SELECT MIN(query_timestamp), MAX(query_timestamp)
FROM flight_searches
""")
oldest, newest = cursor.fetchone()
# Get database file size
db_size_bytes = CACHE_DB_PATH.stat().st_size if CACHE_DB_PATH.exists() else 0
conn.close()
return {
"total_searches": total_searches,
"total_results": total_results,
"oldest_entry": oldest,
"newest_entry": newest,
"db_size_mb": db_size_bytes / (1024 * 1024),
}

104
flight-comparator/cache_admin.py Executable file
View File

@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""
Cache administration utility for flight search results.
Provides commands to view cache statistics and clean up old entries.
"""
import click
from cache import get_cache_stats, clear_old_cache, init_database, CACHE_DB_PATH
@click.group()
def cli():
"""Flight search cache administration."""
pass
@cli.command()
def stats():
"""Display cache statistics."""
init_database()
stats = get_cache_stats()
click.echo()
click.echo("Flight Search Cache Statistics")
click.echo("=" * 50)
click.echo(f"Database location: {CACHE_DB_PATH}")
click.echo(f"Total searches cached: {stats['total_searches']}")
click.echo(f"Total flight results: {stats['total_results']}")
click.echo(f"Database size: {stats['db_size_mb']:.2f} MB")
if stats['oldest_entry']:
click.echo(f"Oldest entry: {stats['oldest_entry']}")
if stats['newest_entry']:
click.echo(f"Newest entry: {stats['newest_entry']}")
click.echo("=" * 50)
click.echo()
@cli.command()
@click.option('--days', default=30, type=int, help='Delete entries older than N days')
@click.option('--confirm', is_flag=True, help='Skip confirmation prompt')
def clean(days: int, confirm: bool):
"""Clean up old cache entries."""
init_database()
stats = get_cache_stats()
click.echo()
click.echo(f"Current cache: {stats['total_searches']} searches, {stats['db_size_mb']:.2f} MB")
click.echo(f"Will delete entries older than {days} days.")
click.echo()
if not confirm:
if not click.confirm('Proceed with cleanup?'):
click.echo("Cancelled.")
return
deleted = clear_old_cache(days)
# Get new stats
new_stats = get_cache_stats()
click.echo(f"✓ Deleted {deleted} old search(es)")
click.echo(f"New cache size: {new_stats['total_searches']} searches, {new_stats['db_size_mb']:.2f} MB")
click.echo()
@cli.command()
@click.option('--confirm', is_flag=True, help='Skip confirmation prompt')
def clear_all(confirm: bool):
"""Delete all cached data."""
init_database()
stats = get_cache_stats()
click.echo()
click.echo(f"⚠️ WARNING: This will delete ALL {stats['total_searches']} cached searches!")
click.echo()
if not confirm:
if not click.confirm('Are you sure?'):
click.echo("Cancelled.")
return
deleted = clear_old_cache(days=0) # Delete everything
click.echo(f"✓ Deleted all {deleted} cached search(es)")
click.echo()
@cli.command()
def init():
"""Initialize cache database (create tables if not exist)."""
click.echo("Initializing cache database...")
init_database()
click.echo(f"✓ Database initialized at: {CACHE_DB_PATH}")
click.echo()
if __name__ == '__main__':
cli()

24896
flight-comparator/data/airports_by_country.json Normal file
View File

File diff suppressed because it is too large Load Diff

9
flight-comparator/database/__init__.py Normal file
View File

@@ -0,0 +1,9 @@
"""
Database package for Flight Radar Web App.
This package handles database initialization, migrations, and connections.
"""
from .init_db import initialize_database, get_connection
__all__ = ['initialize_database', 'get_connection']

296
flight-comparator/database/init_db.py Normal file
View File

@@ -0,0 +1,296 @@
#!/usr/bin/env python3
"""
Database Initialization Script for Flight Radar Web App
This script initializes or migrates the SQLite database for the web app.
It extends the existing cache.db with new tables while preserving existing data.
Usage:
# As script
python database/init_db.py
# As module
from database import initialize_database
initialize_database()
"""
import os
import sqlite3
import sys
from pathlib import Path
def get_db_path():
"""Get path to cache.db in the flight-comparator directory."""
# Assuming this script is in flight-comparator/database/
script_dir = Path(__file__).parent
project_dir = script_dir.parent
db_path = project_dir / "cache.db"
return db_path
def get_schema_path():
"""Get path to schema.sql"""
return Path(__file__).parent / "schema.sql"
def load_schema():
"""Load schema.sql file content."""
schema_path = get_schema_path()
if not schema_path.exists():
raise FileNotFoundError(f"Schema file not found: {schema_path}")
with open(schema_path, 'r', encoding='utf-8') as f:
return f.read()
def check_existing_tables(conn):
"""Check which tables already exist in the database."""
cursor = conn.execute("""
SELECT name FROM sqlite_master
WHERE type='table'
ORDER BY name
""")
return [row[0] for row in cursor.fetchall()]
def get_schema_version(conn):
"""Get current schema version, or 0 if schema_version table doesn't exist."""
existing_tables = check_existing_tables(conn)
if 'schema_version' not in existing_tables:
return 0
cursor = conn.execute("SELECT MAX(version) FROM schema_version")
result = cursor.fetchone()[0]
return result if result is not None else 0
def _migrate_relax_country_constraint(conn, verbose=True):
"""
Migration: Relax the country column CHECK constraint from = 2 to >= 2.
The country column stores either a 2-letter ISO country code (e.g., 'DE')
or a comma-separated list of destination IATA codes (e.g., 'MUC,FRA,BER').
The original CHECK(length(country) = 2) only allowed country codes.
"""
cursor = conn.execute(
"SELECT sql FROM sqlite_master WHERE type='table' AND name='scans'"
)
row = cursor.fetchone()
if not row or 'length(country) = 2' not in row[0]:
return # Table doesn't exist yet or already migrated
if verbose:
print(" 🔄 Migrating scans table: relaxing country CHECK constraint...")
# SQLite doesn't support ALTER TABLE MODIFY COLUMN, so we must recreate.
# Steps:
# 1. Disable FK checks (routes has FK to scans)
# 2. Drop triggers that reference scans from routes table (they'll be
# recreated by executescript(schema_sql) below)
# 3. Create scans_new with relaxed constraint
# 4. Copy data, drop scans, rename scans_new -> scans
# 5. Re-enable FK checks
conn.execute("PRAGMA foreign_keys = OFF")
# Drop triggers on routes that reference scans in their bodies.
# They are recreated by the subsequent executescript(schema_sql) call.
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_insert")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_update")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_delete")
conn.execute("""
CREATE TABLE scans_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
origin TEXT NOT NULL CHECK(length(origin) = 3),
country TEXT NOT NULL CHECK(length(country) >= 2),
start_date TEXT NOT NULL,
end_date TEXT NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
status TEXT NOT NULL DEFAULT 'pending'
CHECK(status IN ('pending', 'running', 'completed', 'failed')),
total_routes INTEGER NOT NULL DEFAULT 0 CHECK(total_routes >= 0),
routes_scanned INTEGER NOT NULL DEFAULT 0 CHECK(routes_scanned >= 0),
total_flights INTEGER NOT NULL DEFAULT 0 CHECK(total_flights >= 0),
error_message TEXT,
seat_class TEXT DEFAULT 'economy',
adults INTEGER DEFAULT 1 CHECK(adults > 0 AND adults <= 9),
CHECK(end_date >= start_date),
CHECK(routes_scanned <= total_routes OR total_routes = 0)
)
""")
conn.execute("INSERT INTO scans_new SELECT * FROM scans")
conn.execute("DROP TABLE scans")
conn.execute("ALTER TABLE scans_new RENAME TO scans")
conn.execute("PRAGMA foreign_keys = ON")
conn.commit()
if verbose:
print(" ✅ Migration complete: country column now accepts >= 2 chars")
def initialize_database(db_path=None, verbose=True):
"""
Initialize or migrate the database.
Args:
db_path: Path to database file (default: cache.db or DATABASE_PATH env var)
verbose: Print status messages
Returns:
sqlite3.Connection: Database connection with foreign keys enabled
"""
if db_path is None:
# Check for DATABASE_PATH environment variable first (used by tests)
env_db_path = os.environ.get('DATABASE_PATH')
if env_db_path:
db_path = Path(env_db_path)
else:
db_path = get_db_path()
db_exists = db_path.exists()
if verbose:
if db_exists:
print(f"📊 Extending existing database: {db_path}")
else:
print(f"📊 Creating new database: {db_path}")
# Connect to database
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row # Access columns by name
# Get existing state
existing_tables = check_existing_tables(conn)
current_version = get_schema_version(conn)
if verbose:
if existing_tables:
print(f" Existing tables: {', '.join(existing_tables)}")
print(f" Current schema version: {current_version}")
else:
print(" No existing tables found")
# Apply migrations before running schema
_migrate_relax_country_constraint(conn, verbose)
# Load and execute schema
schema_sql = load_schema()
if verbose:
print(" Executing schema...")
try:
# Execute schema (uses CREATE TABLE IF NOT EXISTS, so safe)
conn.executescript(schema_sql)
conn.commit()
if verbose:
print(" ✅ Schema executed successfully")
except sqlite3.Error as e:
conn.rollback()
if verbose:
print(f" ❌ Schema execution failed: {e}")
raise
# Verify foreign keys are enabled
cursor = conn.execute("PRAGMA foreign_keys")
fk_enabled = cursor.fetchone()[0]
if fk_enabled != 1:
if verbose:
print(" ⚠️ Warning: Foreign keys not enabled!")
print(" Enabling foreign keys for this connection...")
conn.execute("PRAGMA foreign_keys = ON")
# Check what was created
new_tables = check_existing_tables(conn)
new_version = get_schema_version(conn)
if verbose:
print(f"\n✅ Database initialized successfully!")
print(f" Total tables: {len(new_tables)}")
print(f" Schema version: {new_version}")
print(f" Foreign keys: {'✅ Enabled' if fk_enabled else '❌ Disabled'}")
# Count indexes, triggers, views
cursor = conn.execute("""
SELECT type, COUNT(*) as count
FROM sqlite_master
WHERE type IN ('index', 'trigger', 'view')
GROUP BY type
""")
for row in cursor:
print(f" {row[0].capitalize()}s: {row[1]}")
# Check for web app tables specifically
web_tables = [t for t in new_tables if t in ('scans', 'routes', 'schema_version')]
if web_tables:
print(f"\n📦 Web app tables: {', '.join(web_tables)}")
# Check for existing cache tables
cache_tables = [t for t in new_tables if 'flight' in t.lower()]
if cache_tables:
print(f"💾 Existing cache tables: {', '.join(cache_tables)}")
return conn
def get_connection(db_path=None):
"""
Get a database connection with foreign keys enabled.
Args:
db_path: Path to database file (default: cache.db or DATABASE_PATH env var)
Returns:
sqlite3.Connection: Database connection
"""
if db_path is None:
# Check for DATABASE_PATH environment variable first (used by tests)
env_db_path = os.environ.get('DATABASE_PATH')
if env_db_path:
db_path = Path(env_db_path)
else:
db_path = get_db_path()
if not db_path.exists():
raise FileNotFoundError(
f"Database not found: {db_path}\n"
"Run 'python database/init_db.py' to create it."
)
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
return conn
def main():
"""CLI entry point."""
print("=" * 60)
print("Flight Radar Web App - Database Initialization")
print("=" * 60)
print()
try:
conn = initialize_database(verbose=True)
conn.close()
print()
print("=" * 60)
print("✅ Done! Database is ready.")
print("=" * 60)
return 0
except Exception as e:
print()
print("=" * 60)
print(f"❌ Error: {e}")
print("=" * 60)
return 1
if __name__ == "__main__":
sys.exit(main())

261
flight-comparator/database/schema.sql Normal file
View File

@@ -0,0 +1,261 @@
-- Flight Radar Web App - Database Schema
-- Version: 2.0
-- Date: 2026-02-23
-- Database: SQLite 3
--
-- This schema extends the existing cache.db with new tables for the web app.
-- Existing tables (flight_searches, flight_results) are preserved.
-- ============================================================================
-- CRITICAL: Enable Foreign Keys (SQLite default is OFF!)
-- ============================================================================
PRAGMA foreign_keys = ON;
-- ============================================================================
-- Table: scans
-- Purpose: Track flight scan requests and their status
-- ============================================================================
CREATE TABLE IF NOT EXISTS scans (
-- Primary key with auto-increment
id INTEGER PRIMARY KEY AUTOINCREMENT,
-- Search parameters (validated by CHECK constraints)
origin TEXT NOT NULL CHECK(length(origin) = 3),
country TEXT NOT NULL CHECK(length(country) >= 2),
start_date TEXT NOT NULL, -- ISO 8601: YYYY-MM-DD
end_date TEXT NOT NULL,
-- Timestamps (auto-managed)
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- Scan status (enforced enum via CHECK)
status TEXT NOT NULL DEFAULT 'pending'
CHECK(status IN ('pending', 'running', 'completed', 'failed')),
-- Progress tracking
total_routes INTEGER NOT NULL DEFAULT 0 CHECK(total_routes >= 0),
routes_scanned INTEGER NOT NULL DEFAULT 0 CHECK(routes_scanned >= 0),
total_flights INTEGER NOT NULL DEFAULT 0 CHECK(total_flights >= 0),
-- Error information (NULL if no error)
error_message TEXT,
-- Additional search parameters
seat_class TEXT DEFAULT 'economy',
adults INTEGER DEFAULT 1 CHECK(adults > 0 AND adults <= 9),
-- Constraints across columns
CHECK(end_date >= start_date),
CHECK(routes_scanned <= total_routes OR total_routes = 0)
);
-- Performance indexes for scans table
CREATE INDEX IF NOT EXISTS idx_scans_origin_country
ON scans(origin, country);
CREATE INDEX IF NOT EXISTS idx_scans_status
ON scans(status)
WHERE status IN ('pending', 'running'); -- Partial index for active scans
CREATE INDEX IF NOT EXISTS idx_scans_created_at
ON scans(created_at DESC); -- For recent scans query
-- ============================================================================
-- Table: routes
-- Purpose: Store discovered routes with flight statistics
-- ============================================================================
CREATE TABLE IF NOT EXISTS routes (
-- Primary key
id INTEGER PRIMARY KEY AUTOINCREMENT,
-- Foreign key to scans (cascade delete)
scan_id INTEGER NOT NULL,
-- Destination airport
destination TEXT NOT NULL CHECK(length(destination) = 3),
destination_name TEXT NOT NULL,
destination_city TEXT,
-- Flight statistics
flight_count INTEGER NOT NULL DEFAULT 0 CHECK(flight_count >= 0),
airlines TEXT NOT NULL, -- JSON array: ["Ryanair", "Lufthansa"]
-- Price statistics (NULL if no flights)
min_price REAL CHECK(min_price >= 0),
max_price REAL CHECK(max_price >= 0),
avg_price REAL CHECK(avg_price >= 0),
-- Timestamp
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- Foreign key constraint with cascade delete
FOREIGN KEY (scan_id)
REFERENCES scans(id)
ON DELETE CASCADE,
-- Price consistency constraints
CHECK(max_price >= min_price OR max_price IS NULL),
CHECK(avg_price >= min_price OR avg_price IS NULL),
CHECK(avg_price <= max_price OR avg_price IS NULL)
);
-- Performance indexes for routes table
CREATE INDEX IF NOT EXISTS idx_routes_scan_id
ON routes(scan_id);
CREATE INDEX IF NOT EXISTS idx_routes_destination
ON routes(destination);
CREATE INDEX IF NOT EXISTS idx_routes_min_price
ON routes(min_price)
WHERE min_price IS NOT NULL; -- Partial index for routes with prices
-- ============================================================================
-- Triggers: Auto-update timestamps and aggregates
-- ============================================================================
-- Trigger: Update scans.updated_at on any update
CREATE TRIGGER IF NOT EXISTS update_scans_timestamp
AFTER UPDATE ON scans
FOR EACH ROW
BEGIN
UPDATE scans
SET updated_at = CURRENT_TIMESTAMP
WHERE id = NEW.id;
END;
-- Trigger: Update total_flights count when routes are inserted
CREATE TRIGGER IF NOT EXISTS update_scan_flight_count_insert
AFTER INSERT ON routes
FOR EACH ROW
BEGIN
UPDATE scans
SET total_flights = (
SELECT COALESCE(SUM(flight_count), 0)
FROM routes
WHERE scan_id = NEW.scan_id
)
WHERE id = NEW.scan_id;
END;
-- Trigger: Update total_flights count when routes are updated
CREATE TRIGGER IF NOT EXISTS update_scan_flight_count_update
AFTER UPDATE OF flight_count ON routes
FOR EACH ROW
BEGIN
UPDATE scans
SET total_flights = (
SELECT COALESCE(SUM(flight_count), 0)
FROM routes
WHERE scan_id = NEW.scan_id
)
WHERE id = NEW.scan_id;
END;
-- Trigger: Update total_flights count when routes are deleted
CREATE TRIGGER IF NOT EXISTS update_scan_flight_count_delete
AFTER DELETE ON routes
FOR EACH ROW
BEGIN
UPDATE scans
SET total_flights = (
SELECT COALESCE(SUM(flight_count), 0)
FROM routes
WHERE scan_id = OLD.scan_id
)
WHERE id = OLD.scan_id;
END;
-- ============================================================================
-- Table: flights
-- Purpose: Store individual flights discovered per scan
-- ============================================================================
CREATE TABLE IF NOT EXISTS flights (
id INTEGER PRIMARY KEY AUTOINCREMENT,
-- Foreign key to scans (cascade delete)
scan_id INTEGER NOT NULL,
-- Route
destination TEXT NOT NULL CHECK(length(destination) = 3),
date TEXT NOT NULL, -- ISO 8601: YYYY-MM-DD
-- Flight details
airline TEXT,
departure_time TEXT, -- HH:MM
arrival_time TEXT, -- HH:MM
price REAL CHECK(price >= 0),
stops INTEGER NOT NULL DEFAULT 0,
-- Timestamp
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (scan_id)
REFERENCES scans(id)
ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_flights_scan_id
ON flights(scan_id);
CREATE INDEX IF NOT EXISTS idx_flights_scan_dest
ON flights(scan_id, destination);
CREATE INDEX IF NOT EXISTS idx_flights_price
ON flights(scan_id, price ASC)
WHERE price IS NOT NULL;
-- ============================================================================
-- Views: Useful queries
-- ============================================================================
-- View: Recent scans with route counts
CREATE VIEW IF NOT EXISTS recent_scans AS
SELECT
s.id,
s.origin,
s.country,
s.status,
s.created_at,
s.total_routes,
s.total_flights,
COUNT(r.id) as routes_found,
MIN(r.min_price) as cheapest_flight,
s.error_message
FROM scans s
LEFT JOIN routes r ON r.scan_id = s.id
GROUP BY s.id
ORDER BY s.created_at DESC
LIMIT 10;
-- View: Active scans (pending or running)
CREATE VIEW IF NOT EXISTS active_scans AS
SELECT *
FROM scans
WHERE status IN ('pending', 'running')
ORDER BY created_at ASC;
-- ============================================================================
-- Initial Data: None (tables start empty)
-- ============================================================================
-- Schema version tracking (for future migrations)
CREATE TABLE IF NOT EXISTS schema_version (
version INTEGER PRIMARY KEY,
applied_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
description TEXT
);
INSERT OR IGNORE INTO schema_version (version, description)
VALUES (1, 'Initial web app schema with scans and routes tables');
-- ============================================================================
-- Verification Queries (for testing)
-- ============================================================================
-- Uncomment to verify schema creation:
-- SELECT name, type FROM sqlite_master WHERE type IN ('table', 'index', 'trigger', 'view') ORDER BY type, name;
-- PRAGMA foreign_keys;
-- PRAGMA table_info(scans);
-- PRAGMA table_info(routes);

132
flight-comparator/date_resolver.py Normal file
View File

@@ -0,0 +1,132 @@
"""
Date resolution and seasonal scan logic for flight comparator.
"""
from datetime import date, timedelta
from dateutil.relativedelta import relativedelta
from typing import Optional
# Primary configuration constants
SEARCH_WINDOW_MONTHS = 6 # Default seasonal scan window
SAMPLE_DAY_OF_MONTH = 15 # Representative mid-month date for seasonal queries
def resolve_dates(date_arg: Optional[str], window: int) -> list[str]:
"""
Resolve query dates based on CLI input.
Returns a single date if --date is provided; otherwise generates one date
per month across the window for seasonal scanning.
Args:
date_arg: Optional date string in YYYY-MM-DD format
window: Number of months to scan (only used if date_arg is None)
Returns:
List of date strings in YYYY-MM-DD format
"""
if date_arg:
return [date_arg]
today = date.today()
dates = []
for i in range(1, window + 1):
# Generate dates starting from next month
target_date = today + relativedelta(months=i)
# Set to the sample day of month (default: 15th)
try:
target_date = target_date.replace(day=SAMPLE_DAY_OF_MONTH)
except ValueError:
# Handle months with fewer days (e.g., February with day=31)
# Use the last day of the month instead
target_date = target_date.replace(day=1) + relativedelta(months=1) - relativedelta(days=1)
dates.append(target_date.strftime('%Y-%m-%d'))
return dates
def resolve_dates_daily(
start_date: Optional[str],
end_date: Optional[str],
window: int,
) -> list[str]:
"""
Generate a list of ALL dates in a range, day-by-day (Monday-Sunday).
This is used for comprehensive daily scans to catch flights that only operate
on specific days of the week (e.g., Saturday-only routes).
Args:
start_date: Optional start date in YYYY-MM-DD format
end_date: Optional end date in YYYY-MM-DD format
window: Number of months to scan if dates not specified
Returns:
List of date strings in YYYY-MM-DD format, one per day
Examples:
# Scan next 3 months daily
resolve_dates_daily(None, None, 3)
# Scan specific range
resolve_dates_daily("2026-04-01", "2026-06-30", 3)
"""
today = date.today()
# Determine start date
if start_date:
start = date.fromisoformat(start_date)
else:
# Start from tomorrow (or first day of next month)
start = today + timedelta(days=1)
# Determine end date
if end_date:
end = date.fromisoformat(end_date)
else:
# End after window months
end = today + relativedelta(months=window)
# Generate all dates in range
dates = []
current = start
while current <= end:
dates.append(current.strftime('%Y-%m-%d'))
current += timedelta(days=1)
return dates
def detect_new_connections(monthly_results: dict[str, list]) -> dict[str, str]:
"""
Detect routes that appear for the first time in later months.
Compares route sets month-over-month and tags routes that weren't present
in any previous month.
Args:
monthly_results: Dict mapping month strings to lists of flight objects
Each flight must have 'origin' and 'destination' attributes
Returns:
Dict mapping route keys (e.g., "FRA->JFK") to the first month they appeared
"""
seen = set()
new_connections = {}
# Process months in chronological order
for month in sorted(monthly_results.keys()):
flights = monthly_results[month]
current = {f"{f['origin']}->{f['destination']}" for f in flights}
# Find routes in current month that weren't in any previous month
for route in current - seen:
if seen: # Only tag as NEW if it's not the very first month
new_connections[route] = month
# Add all current routes to seen set
seen |= current
return new_connections

281
flight-comparator/discover_routes.py Normal file
View File

@@ -0,0 +1,281 @@
#!/usr/bin/env python3
"""
Route Discovery Tool
Phase 1: Quickly discover which routes have direct flights
- Scans one sample date per month across the window
- Identifies which destination airports have ANY flights
- Saves results to discovered_routes.json
Phase 2: Targeted daily scans (use main.py)
- Run detailed daily scans only on discovered routes
- Much faster than scanning all airports
Example workflow:
# Phase 1: Discover routes (fast)
python discover_routes.py --from BDS --to-country DE --window 3
# Phase 2: Daily scan each discovered route (targeted)
python main.py --from BDS --to DUS --daily-scan --window 3
python main.py --from BDS --to FMM --daily-scan --window 3
...
"""
import asyncio
import json
import sys
from datetime import date
from dateutil.relativedelta import relativedelta
from typing import Optional
try:
import click
except ImportError:
print("Error: click library not installed. Install with: pip install click")
sys.exit(1)
from airports import resolve_airport_list, download_and_build_airport_data
try:
from searcher_v3 import search_multiple_routes
except ImportError:
print("Error: searcher_v3 not found")
sys.exit(1)
def generate_discovery_dates(window_months: int) -> list[str]:
"""
Generate sample dates for route discovery.
Uses one date per month (15th) to quickly check which routes exist.
Args:
window_months: Number of months to check
Returns:
List of date strings (YYYY-MM-DD)
"""
today = date.today()
dates = []
for i in range(1, window_months + 1):
target_date = today + relativedelta(months=i)
try:
target_date = target_date.replace(day=15)
except ValueError:
# Handle months with fewer days
target_date = target_date.replace(day=1) + relativedelta(months=1) - relativedelta(days=1)
dates.append(target_date.strftime('%Y-%m-%d'))
return dates
@click.command()
@click.option('--from', 'origin', required=True, help='Origin airport IATA code (e.g., BDS)')
@click.option('--to-country', 'country', required=True, help='Destination country ISO code (e.g., DE)')
@click.option('--window', default=3, type=int, help='Months to scan (default: 3)')
@click.option('--output', default='discovered_routes.json', help='Output file (default: discovered_routes.json)')
@click.option('--workers', default=5, type=int, help='Concurrency level (default: 5)')
def discover(origin: str, country: str, window: int, output: str, workers: int):
"""
Discover which routes have direct flights.
Quickly scans sample dates to find which destination airports have ANY flights.
Much faster than daily scanning all airports.
Example:
python discover_routes.py --from BDS --to-country DE --window 3
"""
print()
print("=" * 70)
print("ROUTE DISCOVERY SCAN")
print("=" * 70)
print(f"Origin: {origin}")
print(f"Destinations: All airports in {country}")
print(f"Strategy: Sample one date per month for {window} months")
print()
# Ensure airport data exists
try:
download_and_build_airport_data()
except Exception as e:
click.echo(f"Error building airport data: {e}", err=True)
sys.exit(1)
# Get all destination airports
try:
destination_airports = resolve_airport_list(country, None)
except ValueError as e:
click.echo(f"Error: {e}", err=True)
sys.exit(1)
print(f"Found {len(destination_airports)} airports in {country}")
# Generate sample dates (one per month)
sample_dates = generate_discovery_dates(window)
print(f"Sample dates: {', '.join(sample_dates)}")
print()
# Build routes to scan
routes = []
for airport in destination_airports:
for sample_date in sample_dates:
routes.append((origin, airport['iata'], sample_date))
total_routes = len(routes)
print(f"Scanning {total_routes} routes ({len(destination_airports)} airports × {len(sample_dates)} dates)...")
print()
# Execute discovery scan
try:
results = asyncio.run(
search_multiple_routes(
routes,
seat_class="economy",
adults=1,
max_workers=workers,
cache_threshold_hours=24,
use_cache=True,
progress_callback=None, # Suppress detailed progress
)
)
except Exception as e:
click.echo(f"Error during scan: {e}", err=True)
sys.exit(1)
# Analyze results to find which destinations have flights
destinations_with_flights = set()
destination_details = {}
for (orig, dest, query_date), flights in results.items():
if flights: # Has at least one flight
destinations_with_flights.add(dest)
if dest not in destination_details:
destination_details[dest] = {
"iata": dest,
"flights_found": 0,
"airlines": set(),
"sample_dates": [],
"price_range": {"min": None, "max": None},
}
destination_details[dest]["flights_found"] += len(flights)
destination_details[dest]["sample_dates"].append(query_date)
for flight in flights:
destination_details[dest]["airlines"].add(flight.get("airline", "Unknown"))
price = flight.get("price")
if price:
if destination_details[dest]["price_range"]["min"] is None:
destination_details[dest]["price_range"]["min"] = price
destination_details[dest]["price_range"]["max"] = price
else:
destination_details[dest]["price_range"]["min"] = min(
destination_details[dest]["price_range"]["min"], price
)
destination_details[dest]["price_range"]["max"] = max(
destination_details[dest]["price_range"]["max"], price
)
# Convert sets to lists for JSON serialization
for dest in destination_details:
destination_details[dest]["airlines"] = sorted(list(destination_details[dest]["airlines"]))
# Get airport names
airport_map = {ap['iata']: ap for ap in destination_airports}
# Prepare output
discovered_routes = {
"scan_date": date.today().strftime('%Y-%m-%d'),
"origin": origin,
"country": country,
"window_months": window,
"total_airports_scanned": len(destination_airports),
"destinations_with_flights": len(destinations_with_flights),
"sample_dates": sample_dates,
"routes": []
}
for dest in sorted(destinations_with_flights):
details = destination_details[dest]
airport_info = airport_map.get(dest, {})
route_info = {
"destination": dest,
"destination_name": airport_info.get('name', 'Unknown'),
"destination_city": airport_info.get('city', ''),
"flights_found": details["flights_found"],
"airlines": details["airlines"],
"dates_with_flights": sorted(details["sample_dates"]),
"price_range": details["price_range"],
}
discovered_routes["routes"].append(route_info)
# Save to file
with open(output, 'w') as f:
json.dump(discovered_routes, f, indent=2)
# Display results
print()
print("=" * 70)
print("DISCOVERY RESULTS")
print("=" * 70)
print(f"Total airports scanned: {len(destination_airports)}")
print(f"Destinations with flights: {len(destinations_with_flights)}")
print(f"Success rate: {len(destinations_with_flights) / len(destination_airports) * 100:.1f}%")
print()
if destinations_with_flights:
print("Routes with direct flights:")
print()
print(f"{'IATA':<6} {'City':<25} {'Airlines':<30} {'Flights':<8} {'Price Range'}")
print("-" * 90)
for route in discovered_routes["routes"]:
airlines_str = ", ".join(route["airlines"][:3]) # Show up to 3 airlines
if len(route["airlines"]) > 3:
airlines_str += f" +{len(route['airlines']) - 3}"
price_min = route["price_range"]["min"]
price_max = route["price_range"]["max"]
if price_min and price_max:
price_range = f"€{price_min}-€{price_max}"
else:
price_range = "—"
print(f"{route['destination']:<6} {route['destination_city'][:24]:<25} "
f"{airlines_str[:29]:<30} {route['flights_found']:<8} {price_range}")
print()
print(f"✅ Saved to: {output}")
print()
print("=" * 70)
print("NEXT STEP: Targeted Daily Scans")
print("=" * 70)
print("Run detailed daily scans on discovered routes:")
print()
for route in discovered_routes["routes"][:5]: # Show first 5 examples
dest = route['destination']
print(f"python main.py --from {origin} --to {dest} --daily-scan --window {window}")
if len(discovered_routes["routes"]) > 5:
print(f"... and {len(discovered_routes['routes']) - 5} more routes")
print()
print("Or use the automated batch script:")
print(f"python scan_discovered_routes.py {output}")
else:
print("⚠️ No routes with direct flights found")
print()
print("This could mean:")
print(" - No direct flights exist for these routes")
print(" - API errors prevented detection")
print(" - Try expanding the date range with --window")
print()
if __name__ == '__main__':
discover()

54
flight-comparator/docker-compose.yml Normal file
View File

@@ -0,0 +1,54 @@
services:
# Backend API Server
backend:
build:
context: .
dockerfile: Dockerfile.backend
container_name: flight-radar-backend
restart: unless-stopped
ports:
- "8000:8000"
environment:
- PORT=8000
- DATABASE_PATH=/app/data/cache.db
- ALLOWED_ORIGINS=http://localhost,http://localhost:80,http://frontend
volumes:
- backend-data:/app/data
- ./cache.db:/app/cache.db:rw
networks:
- flight-radar-network
healthcheck:
test: ["CMD", "python", "-c", "import requests; requests.get('http://localhost:8000/health').raise_for_status()"]
interval: 30s
timeout: 3s
retries: 3
start_period: 10s
# Frontend UI
frontend:
build:
context: .
dockerfile: Dockerfile.frontend
container_name: flight-radar-frontend
restart: unless-stopped
ports:
- "80:80"
depends_on:
backend:
condition: service_healthy
networks:
- flight-radar-network
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/"]
interval: 30s
timeout: 3s
retries: 3
start_period: 5s
networks:
flight-radar-network:
driver: bridge
volumes:
backend-data:
driver: local

316
flight-comparator/docs/CACHING.md Normal file
View File

@@ -0,0 +1,316 @@
# Flight Search Caching System
## Overview
The Flight Airport Comparator now includes a **SQLite-based caching system** to reduce API calls, prevent rate limiting, and provide instant results for repeated queries.
## How It Works
### Automatic Caching
- Every flight search is automatically saved to `data/flight_cache.db`
- Includes: origin, destination, date, seat class, adults, timestamp
- Stores all flight results: airline, price, times, duration, etc.
### Cache Lookup
Before making an API call, the tool:
1. Generates a unique cache key (SHA256 hash of query parameters)
2. Checks if results exist in database
3. Verifies results are within threshold (default: 24 hours)
4. Returns cached data if valid, otherwise queries API
### Cache Indicators
```
💾 Cache hit: BER->BRI on 2026-03-23 (1 flights) # Instant result (0.0s)
```
No indicator = Cache miss, fresh API query made (~2-3s per route)
## Usage
### CLI Options
**Use default cache (24 hours):**
```bash
python main.py --to JFK --country DE
```
**Custom cache threshold (48 hours):**
```bash
python main.py --to JFK --country DE --cache-threshold 48
```
**Disable cache (force fresh queries):**
```bash
python main.py --to JFK --country DE --no-cache
```
### Cache Management
**View statistics:**
```bash
python cache_admin.py stats
# Output:
# Flight Search Cache Statistics
# ==================================================
# Database location: /Users/.../flight_cache.db
# Total searches cached: 42
# Total flight results: 156
# Database size: 0.15 MB
# Oldest entry: 2026-02-20 10:30:00
# Newest entry: 2026-02-21 18:55:50
```
**Clean old entries:**
```bash
# Delete entries older than 30 days
python cache_admin.py clean --days 30
# Delete entries older than 7 days
python cache_admin.py clean --days 7 --confirm
```
**Clear entire cache:**
```bash
python cache_admin.py clear-all
# ⚠️ WARNING: Requires confirmation
```
## Database Schema
### flight_searches table
```sql
CREATE TABLE flight_searches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
query_hash TEXT NOT NULL UNIQUE, -- SHA256 of query params
origin TEXT NOT NULL,
destination TEXT NOT NULL,
search_date TEXT NOT NULL, -- YYYY-MM-DD
seat_class TEXT NOT NULL,
adults INTEGER NOT NULL,
query_timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
);
```
### flight_results table
```sql
CREATE TABLE flight_results (
id INTEGER PRIMARY KEY AUTOINCREMENT,
search_id INTEGER NOT NULL, -- FK to flight_searches
airline TEXT,
departure_time TEXT,
arrival_time TEXT,
duration_minutes INTEGER,
price REAL,
currency TEXT,
plane_type TEXT,
FOREIGN KEY (search_id) REFERENCES flight_searches(id) ON DELETE CASCADE
);
```
### Indexes
- `idx_query_hash` on `flight_searches(query_hash)` - Fast cache lookup
- `idx_query_timestamp` on `flight_searches(query_timestamp)` - Fast expiry checks
- `idx_search_id` on `flight_results(search_id)` - Fast result retrieval
## Benefits
### ⚡ Speed
- **Cache hit**: 0.0s (instant)
- **Cache miss**: ~2-3s (API call + save to cache)
- Example: 95 airports × 3 dates = 285 queries
- First run: ~226s (fresh API calls)
- Second run: ~0.1s (all cache hits!)
### 🛡️ Rate Limit Protection
- Prevents identical repeated queries
- Especially useful for:
- Testing and development
- Re-running seasonal scans
- Comparing different output formats
- Experimenting with sort orders
### 💰 Reduced API Load
- Fewer requests to Google Flights
- Lower risk of being rate-limited or blocked
- Respectful of Google's infrastructure
### 📊 Historical Data
- Cache preserves price snapshots over time
- Can compare prices from different query times
- Useful for tracking price trends
## Performance Example
**First Query (Cache Miss):**
```bash
$ python main.py --to BDS --country DE --window 3
# Searching 285 routes (95 airports × 3 dates)...
# Done in 226.2s
```
**Second Query (Cache Hit):**
```bash
$ python main.py --to BDS --country DE --window 3
# 💾 Cache hit: FMM->BDS on 2026-04-15 (1 flights)
# Done in 0.0s
```
**Savings:** 226.2s → 0.0s (100% cache hit rate)
## Cache Key Generation
Cache keys are SHA256 hashes of query parameters:
```python
# Example query
origin = "BER"
destination = "BRI"
date = "2026-03-23"
seat_class = "economy"
adults = 1
# Cache key
query_string = "BER|BRI|2026-03-23|economy|1"
cache_key = sha256(query_string) = "a7f3c8d2..."
```
Different parameters = different cache key:
- `BER->BRI, 2026-03-23, economy, 1` ≠ `BER->BRI, 2026-03-24, economy, 1`
- `BER->BRI, 2026-03-23, economy, 1` ≠ `BER->BRI, 2026-03-23, business, 1`
## Maintenance
### Recommended Cache Cleaning Schedule
**For regular users:**
```bash
# Clean monthly (keep last 30 days)
python cache_admin.py clean --days 30 --confirm
```
**For developers/testers:**
```bash
# Clean weekly (keep last 7 days)
python cache_admin.py clean --days 7 --confirm
```
**For one-time users:**
```bash
# Clear all after use
python cache_admin.py clear-all --confirm
```
### Database Growth
**Typical sizes:**
- 1 search = ~1 KB
- 100 searches = ~100 KB
- 1000 searches = ~1 MB
- 10,000 searches = ~10 MB
Most users will stay under 1 MB even with heavy use.
## Testing
**Test cache functionality:**
```bash
python test_cache.py
# Output:
# ======================================================================
# TESTING CACHE OPERATIONS
# ======================================================================
#
# 1. Clearing old cache...
# ✓ Cache cleared
# 2. Testing cache miss (first query)...
# ✓ Cache miss (as expected)
# 3. Saving flight results to cache...
# ✓ Results saved
# 4. Testing cache hit (second query)...
# ✓ Cache hit: Found 1 flight(s)
# ...
# ✅ ALL CACHE TESTS PASSED!
```
## Architecture
### Integration Points
1. **searcher_v3.py**:
- `search_direct_flights()` checks cache before API call
- Saves results after successful query
2. **main.py**:
- `--cache-threshold` CLI option
- `--no-cache` flag
- Passes cache settings to searcher
3. **cache.py**:
- `get_cached_results()`: Check for valid cached data
- `save_results()`: Store flight results
- `clear_old_cache()`: Maintenance operations
- `get_cache_stats()`: Database statistics
4. **cache_admin.py**:
- CLI for cache management
- Human-readable statistics
- Safe deletion with confirmations
## Implementation Details
### Thread Safety
SQLite handles concurrent reads automatically. Writes are serialized by SQLite's locking mechanism.
### Error Handling
- Database errors are caught and logged
- Failed cache operations fall through to API queries
- No crash on corrupted database (graceful degradation)
### Data Persistence
- Cache survives program restarts
- Located in `data/flight_cache.db`
- Can be backed up, copied, or shared
## Future Enhancements
Potential improvements:
- [ ] Cache invalidation based on flight departure time
- [ ] Compression for large result sets
- [ ] Export cache to CSV for analysis
- [ ] Cache warming (pre-populate common routes)
- [ ] Distributed cache (Redis/Memcached)
- [ ] Cache analytics (hit rate, popular routes)
## Troubleshooting
**Cache not working:**
```bash
# Check if cache module is available
python -c "import cache; print('✓ Cache available')"
# Initialize database manually
python cache_admin.py init
```
**Database locked:**
```bash
# Close all running instances
# Or delete and reinitialize
rm data/flight_cache.db
python cache_admin.py init
```
**Disk space issues:**
```bash
# Check database size
python cache_admin.py stats
# Clean aggressively
python cache_admin.py clean --days 1 --confirm
```
## Credits
Caching implementation by Claude Code, integrated with fast-flights v3.0rc1 SOCS cookie bypass.

209
flight-comparator/docs/DECISIONS.md Normal file
View File

@@ -0,0 +1,209 @@
# Implementation Decisions & Notes
This document tracks decisions made during implementation and deviations from the PRD.
## Date: 2026-02-21
### Country Code Mapping
**Decision**: Used manual country name to ISO code mapping instead of downloading separate OpenFlights countries.dat
**Rationale**:
- OpenFlights airports.dat contains full country names, not ISO codes
- Added optional pycountry library support for broader coverage
- Fallback to manual mapping for 40+ common countries
- Simpler and more reliable than fuzzy matching country names
**Impact**:
- Works for most common travel countries (DE, US, GB, FR, ES, IT, etc.)
- Less common countries may not be available unless pycountry is installed
- Can be easily extended by adding to COUNTRY_NAME_TO_ISO dict
### fast-flights Integration
**Decision**: Implemented defensive handling for fast-flights library structure
**Rationale**:
- fast-flights documentation is limited on exact flight object structure
- Implemented multiple fallback methods to detect direct flights:
1. Check `stops` attribute
2. Check if only one flight segment
3. Verify departure/arrival airports match query
- Added retry logic with exponential backoff
**Impact**:
- More resilient to library API changes
- May filter differently than expected if library structure differs
- Graceful degradation: returns empty results on error rather than crashing
### Price Level Indicator
**Decision**: Simplified market indicator to always show "Typical" in initial implementation
**Rationale**:
- PRD mentions "Low ✅ / Typical / High" indicators
- Proper implementation would require:
- Calculating price distribution across all results
- Defining percentile thresholds
- Maintaining historical price data
- Out of scope for v1, can be added later
**Impact**:
- Current implementation just shows "Typical" for all flights
- Still provides full price information for manual comparison
- Future enhancement: calculate percentiles and add Low/High markers
### Airport Filtering
**Decision**: No filtering by airport size (large_airport / medium_airport)
**Rationale**:
- OpenFlights airports.dat does not include a "type" field in the public CSV
- Would need additional dataset or API to classify airports
- PRD mentioned filtering to large/medium airports, but not critical for functionality
- Users can manually filter with --from flag if needed
**Impact**:
- May include some smaller regional airports that don't have international flights
- Results in more comprehensive coverage
- ~95 airports for Germany vs ~10-15 major ones
### Error Handling Philosophy
**Decision**: Fail-soft approach throughout - partial results preferred over full crash
**Rationale**:
- PRD explicitly states: "Partial results preferred over full crash in all cases"
- Scraping can be unreliable (rate limits, network issues, anti-bot measures)
- Better to show 15/20 airports than fail completely
**Implementation**:
- Each airport/date query wrapped in try/except
- Warnings logged but execution continues
- Empty results returned on failure
- Summary shows how many airports succeeded
### Dry Run Mode
**Decision**: Enhanced dry-run output beyond PRD specification
**Addition**:
- Shows estimated API call count
- Displays estimated time based on worker count
- Lists sample of airports that will be scanned
- Shows all dates that will be queried
**Rationale**:
- Helps users understand the scope before running expensive queries
- Useful for estimating how long a scan will take
- Can catch configuration errors early
### Module Organization
**Decision**: Followed PRD build order exactly: date_resolver → airports → searcher → formatter → main
**Result**:
- Clean separation of concerns
- Each module is independently testable
- Natural dependency flow with no circular imports
### Testing Approach
**Decision**: Basic smoke tests rather than comprehensive unit tests
**Rationale**:
- PRD asked for "quick smoke test before moving to the next"
- Full integration tests require live API access to fast-flights
- Focused on testing pure functions (date resolution, duration parsing, formatting)
- API integration can only be validated with real network calls
**Coverage**:
- ✅ date_resolver: date generation and new connection detection logic
- ✅ airports: country resolution and custom airport lists
- ✅ searcher: duration parsing (API mocked/skipped)
- ✅ formatter: duration formatting
- ❌ Full end-to-end API integration (requires live Google Flights access)
### Dependencies
**Decision**: All dependencies are optional with graceful fallbacks
**Implementation**:
- fast-flights: Required for actual flight search, but code handles missing import
- rich: Falls back to plain text output if not available
- pycountry: Optional enhancement for country mapping
- click, python-dateutil: Core requirements
**Rationale**:
- Better developer experience
- Can run tests and --dry-run without all dependencies
- Clear error messages when missing required deps for actual searches
## Future Enhancements Noted
These were considered but deferred to keep v1 scope focused:
1. **Price level calculation**: Requires statistical analysis of result set
2. **Airport size filtering**: Needs additional data source
3. **Return trip support**: PRD lists as v2 feature
4. **Historical price tracking**: PRD lists as v2 feature
5. **Better fast-flights integration**: Depends on library documentation/stability
## Known Issues
1. **fast-flights structure unknown**: Implemented defensive checks, may need adjustment based on real API responses
2. **Limited country coverage without pycountry**: Only 40+ manually mapped countries
3. **No caching**: Each run hits the API fresh (could add in future)
4. **Rate limiting**: Basic 0.5-1.5s random delay, may need tuning based on actual API behavior
## Testing Notes
All modules tested with smoke tests:
- ✅ date_resolver: PASSED
- ✅ airports: PASSED
- ✅ searcher: PASSED (logic only, no API calls)
- ✅ formatter: PASSED
End-to-end testing requires:
1. Installing fast-flights
2. Running actual queries against Google Flights
3. May encounter rate limiting or anti-bot measures
## fast-flights Integration Test Results (2026-02-21)
**Status**: Implementation verified, but live scraping encounters anti-bot measures
**What was tested**:
- ✅ Corrected API integration (FlightData + get_flights parameters)
- ✅ Tool correctly calls fast-flights with proper arguments
- ✅ Error handling works as designed (graceful degradation)
- ❌ Google Flights scraping blocked by language selection/consent pages
**API Corrections Made**:
1. `FlightData()` does not accept `trip` parameter (moved to `get_flights()`)
2. `flight_data` must be a list: `[flight]` not `flight`
3. `seat` uses strings ('economy', 'premium-economy', 'business', 'first') not codes
4. `max_stops=0` parameter in FlightData for direct flights
**Observed Errors**:
- HTTP 401 with 'fallback' mode (requires Playwright cloud service subscription)
- Language selection page returned with 'common' mode (anti-bot detection)
- This is **expected behavior** as noted in PRD: "subject to rate limiting, anti-bot measures"
**Recommendation**:
The tool implementation is correct and complete. The fast-flights library itself has limitations with Google Flights scraping due to:
1. Anti-bot measures (CAPTCHA, consent flows, language selection redirects)
2. Potential need for Playwright cloud service subscription
3. Regional restrictions (EU consent flows mentioned in PRD)
Users should be aware that:
- The tool's **logic and architecture are sound**
- All **non-API components work perfectly**
- **Live flight data** may be unavailable due to Google Flights anti-scraping measures
- This is a **limitation of web scraping in general**, not our implementation
Alternative approaches for future versions:
1. Use official flight API services (Amadeus, Skyscanner, etc.)
2. Implement local browser automation with Selenium/Playwright
3. Add CAPTCHA solving service integration
4. Use cached/sample data for demonstrations

480
flight-comparator/docs/DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,480 @@
# Flight Radar Web App - Deployment Guide
**Complete Docker deployment instructions for production and development environments.**
---
## Table of Contents
- [Quick Start](#quick-start)
- [Prerequisites](#prerequisites)
- [Docker Deployment](#docker-deployment)
- [Manual Deployment](#manual-deployment)
- [Environment Configuration](#environment-configuration)
- [Troubleshooting](#troubleshooting)
- [Monitoring](#monitoring)
---
## Quick Start
### Using Docker Compose (Recommended)
```bash
# 1. Clone the repository
git clone <repository-url>
cd flight-comparator
# 2. Build and start services
docker-compose up -d
# 3. Access the application
# Frontend: http://localhost
# Backend API: http://localhost:8000
# API Docs: http://localhost:8000/docs
```
That's it! The application is now running.
---
## Prerequisites
### For Docker Deployment
- Docker Engine 20.10+
- Docker Compose 2.0+
- 2GB RAM minimum
- 5GB disk space
### For Manual Deployment
- Python 3.11+
- Node.js 20+
- npm or yarn
- 4GB RAM recommended
---
## Docker Deployment
### Production Deployment
#### 1. Configure Environment
```bash
# Copy environment template
cp .env.example .env
# Edit configuration
nano .env
```
**Production Environment Variables:**
```bash
# Backend
PORT=8000
ALLOWED_ORIGINS=https://yourdomain.com
# Logging
LOG_LEVEL=INFO
# Rate Limits (adjust based on traffic)
RATE_LIMIT_SCANS=10
RATE_LIMIT_AIRPORTS=100
```
#### 2. Build Images
```bash
# Build both frontend and backend
docker-compose build
# Or build individually
docker build -f Dockerfile.backend -t flight-radar-backend .
docker build -f Dockerfile.frontend -t flight-radar-frontend .
```
#### 3. Start Services
```bash
# Start in detached mode
docker-compose up -d
# View logs
docker-compose logs -f
# Check status
docker-compose ps
```
#### 4. Verify Deployment
```bash
# Check backend health
curl http://localhost:8000/health
# Check frontend
curl http://localhost/
# Check API endpoints
curl http://localhost:8000/api/v1/scans
```
### Development Deployment
```bash
# Start with logs attached
docker-compose up
# Rebuild after code changes
docker-compose up --build
# Stop services
docker-compose down
```
---
## Manual Deployment
### Backend Deployment
```bash
# 1. Install dependencies
pip install -r requirements.txt
# 2. Initialize database
python database/init_db.py
# 3. Download airport data
python -c "from airports import download_and_build_airport_data; download_and_build_airport_data()"
# 4. Start server
python api_server.py
```
Backend runs on: http://localhost:8000
### Frontend Deployment
```bash
# 1. Navigate to frontend directory
cd frontend
# 2. Install dependencies
npm install
# 3. Build for production
npm run build
# 4. Serve with nginx or static server
# Option 1: Preview with Vite
npm run preview
# Option 2: Use a static server
npx serve -s dist -l 80
```
Frontend runs on: http://localhost
---
## Environment Configuration
### Backend Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8000` | Backend server port |
| `HOST` | `0.0.0.0` | Server bind address |
| `DATABASE_PATH` | `cache.db` | SQLite database path |
| `ALLOWED_ORIGINS` | `localhost` | CORS allowed origins |
| `LOG_LEVEL` | `INFO` | Logging level |
| `RATE_LIMIT_SCANS` | `10` | Scans per minute per IP |
| `RATE_LIMIT_LOGS` | `30` | Log requests per minute |
| `RATE_LIMIT_AIRPORTS` | `100` | Airport searches per minute |
### Frontend Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `VITE_API_BASE_URL` | `/api/v1` | API base URL (build time) |
**Note:** Frontend uses Vite proxy in development, and nginx proxy in production.
---
## Docker Commands Reference
### Managing Services
```bash
# Start services
docker-compose up -d
# Stop services
docker-compose down
# Restart services
docker-compose restart
# View logs
docker-compose logs -f [service-name]
# Execute command in container
docker-compose exec backend bash
docker-compose exec frontend sh
```
### Image Management
```bash
# List images
docker images | grep flight-radar
# Remove images
docker rmi flight-radar-backend flight-radar-frontend
# Prune unused images
docker image prune -a
```
### Volume Management
```bash
# List volumes
docker volume ls
# Inspect backend data volume
docker volume inspect flight-comparator_backend-data
# Backup database
docker cp flight-radar-backend:/app/cache.db ./backup.db
# Restore database
docker cp ./backup.db flight-radar-backend:/app/cache.db
```
### Health Checks
```bash
# Check container health
docker ps
# Backend health check
docker-compose exec backend python -c "import requests; print(requests.get('http://localhost:8000/health').json())"
# Frontend health check
docker-compose exec frontend wget -qO- http://localhost/
```
---
## Troubleshooting
### Backend Issues
**Problem:** Backend fails to start
```bash
# Check logs
docker-compose logs backend
# Common issues:
# - Database not initialized: Rebuild image
# - Port already in use: Change BACKEND_PORT in .env
# - Missing dependencies: Check requirements.txt
```
**Problem:** API returns 500 errors
```bash
# Check application logs
docker-compose logs backend | grep ERROR
# Check database
docker-compose exec backend ls -la cache.db
# Restart service
docker-compose restart backend
```
### Frontend Issues
**Problem:** Frontend shows blank page
```bash
# Check nginx logs
docker-compose logs frontend
# Verify build
docker-compose exec frontend ls -la /usr/share/nginx/html
# Check nginx config
docker-compose exec frontend cat /etc/nginx/conf.d/default.conf
```
**Problem:** API calls fail from frontend
```bash
# Check nginx proxy configuration
docker-compose exec frontend cat /etc/nginx/conf.d/default.conf | grep proxy_pass
# Verify backend is accessible from frontend container
docker-compose exec frontend wget -qO- http://backend:8000/health
# Check CORS configuration
curl -H "Origin: http://localhost" -v http://localhost:8000/health
```
### Database Issues
**Problem:** Database locked error
```bash
# Stop all services
docker-compose down
# Remove database volume
docker volume rm flight-comparator_backend-data
# Restart services (database will be recreated)
docker-compose up -d
```
**Problem:** Database corruption
```bash
# Backup current database
docker cp flight-radar-backend:/app/cache.db ./corrupted.db
# Stop services
docker-compose down
# Remove volume
docker volume rm flight-comparator_backend-data
# Start services (fresh database)
docker-compose up -d
```
---
## Monitoring
### Application Logs
```bash
# View all logs
docker-compose logs -f
# Backend logs only
docker-compose logs -f backend
# Frontend logs only
docker-compose logs -f frontend
# Last 100 lines
docker-compose logs --tail=100
# Logs since specific time
docker-compose logs --since 2024-01-01T00:00:00
```
### Resource Usage
```bash
# Container stats
docker stats flight-radar-backend flight-radar-frontend
# Disk usage
docker system df
# Detailed container info
docker inspect flight-radar-backend
```
### Health Monitoring
```bash
# Health check status
docker ps --filter "name=flight-radar"
# Backend API health
curl http://localhost:8000/health
# Check recent scans
curl http://localhost:8000/api/v1/scans?limit=5
# Check logs endpoint
curl "http://localhost:8000/api/v1/logs?limit=10"
```
---
## Production Best Practices
### Security
1. **Use HTTPS:** Deploy behind a reverse proxy (nginx, Caddy, Traefik)
2. **Environment Variables:** Never commit `.env` files
3. **Update CORS:** Set proper `ALLOWED_ORIGINS`
4. **Rate Limiting:** Adjust limits based on traffic
5. **Secrets Management:** Use Docker secrets or external secret managers
### Performance
1. **Resource Limits:** Set memory/CPU limits in docker-compose.yml
2. **Volumes:** Use named volumes for persistent data
3. **Caching:** Enable nginx caching for static assets
4. **CDN:** Consider CDN for frontend assets
5. **Database:** Regular backups and optimization
### Reliability
1. **Health Checks:** Monitor `/health` endpoint
2. **Restart Policy:** Use `restart: unless-stopped`
3. **Logging:** Centralized logging (ELK, Loki, CloudWatch)
4. **Backups:** Automated database backups
5. **Updates:** Regular dependency updates
---
## Scaling
### Horizontal Scaling
```yaml
# docker-compose.yml
services:
backend:
deploy:
replicas: 3
# Add load balancer (nginx, HAProxy)
load-balancer:
image: nginx
# Configure upstream servers
```
### Vertical Scaling
```yaml
services:
backend:
deploy:
resources:
limits:
cpus: '2'
memory: 2G
reservations:
cpus: '1'
memory: 1G
```
---
## Support
For issues and questions:
- Check logs: `docker-compose logs`
- Review documentation: `/docs` endpoints
- Check health: `/health` endpoint
---
**Last Updated:** 2026-02-23
**Version:** 2.0

353
flight-comparator/docs/DOCKER_README.md Normal file
View File

@@ -0,0 +1,353 @@
# Flight Radar Web App - Docker Quick Start
**Get the entire application running in under 2 minutes!** 🚀
---
## One-Command Deployment
```bash
docker-compose up -d
```
**Access the application:**
- **Frontend:** http://localhost
- **Backend API:** http://localhost:8000
- **API Docs:** http://localhost:8000/docs
---
## What Gets Deployed?
### Backend (Python FastAPI)
- RESTful API server
- SQLite database with schema
- Airport data (auto-downloaded)
- Rate limiting & logging
- Health checks
### Frontend (React + Nginx)
- Production-optimized React build
- Nginx web server
- API proxy configuration
- Static asset caching
- Health checks
### Networking
- Internal bridge network
- Backend accessible at `backend:8000`
- Frontend proxies API requests
---
## Architecture
```
┌─────────────────────────────────────────────────┐
│ Docker Host │
│ │
│ ┌──────────────────┐ ┌─────────────────┐ │
│ │ Frontend │ │ Backend │ │
│ │ (nginx:80) │◄────►│ (Python:8000) │ │
│ │ │ │ │ │
│ │ - React App │ │ - FastAPI │ │
│ │ - Static Files │ │ - SQLite DB │ │
│ │ - API Proxy │ │ - Rate Limit │ │
│ └──────────────────┘ └─────────────────┘ │
│ │ │ │
│ │ │ │
│ Port 80 Port 8000 │
└─────────┼─────────────────────────┼─────────────┘
│ │
└─────────────────────────┘
Host Machine Access
```
---
## Quick Commands
### Starting & Stopping
```bash
# Start (detached)
docker-compose up -d
# Start (with logs)
docker-compose up
# Stop
docker-compose down
# Restart
docker-compose restart
```
### Monitoring
```bash
# View logs
docker-compose logs -f
# Check status
docker-compose ps
# Resource usage
docker stats flight-radar-backend flight-radar-frontend
```
### Database
```bash
# Backup
docker cp flight-radar-backend:/app/cache.db ./backup.db
# Restore
docker cp ./backup.db flight-radar-backend:/app/cache.db
# Access database
docker-compose exec backend sqlite3 cache.db
```
### Rebuilding
```bash
# Rebuild after code changes
docker-compose up --build
# Force rebuild
docker-compose build --no-cache
```
---
## Ports
| Service | Internal | External | Purpose |
|---------|----------|----------|---------|
| Frontend | 80 | 80 | Web UI |
| Backend | 8000 | 8000 | API Server |
**Change ports in `.env`:**
```bash
FRONTEND_PORT=8080
BACKEND_PORT=8001
```
---
## Volumes
### Backend Data
- **Volume:** `backend-data`
- **Mount:** `/app/data`
- **Contents:** Database, cache files
- **Persistence:** Survives container restarts
### Database File
- **Mount:** `./cache.db:/app/cache.db`
- **Type:** Bind mount (optional)
- **Purpose:** Easy backup access
---
## Environment Variables
Create `.env` from template:
```bash
cp .env.example .env
```
**Key Variables:**
```bash
# Backend
PORT=8000
ALLOWED_ORIGINS=http://localhost
# Rate Limits
RATE_LIMIT_SCANS=10
RATE_LIMIT_AIRPORTS=100
```
---
## Health Checks
Both services have automatic health checks:
```bash
# Backend
curl http://localhost:8000/health
# Frontend
curl http://localhost/
# Docker health status
docker ps
```
**Health indicators:**
- `healthy` - Service operational
- `starting` - Initialization in progress
- `unhealthy` - Service down
---
## Troubleshooting
### Container won't start
```bash
# Check logs
docker-compose logs [service-name]
# Common issues:
# - Port already in use: Change port in .env
# - Build failed: Run docker-compose build --no-cache
# - Permission denied: Check file permissions
```
### API not accessible from frontend
```bash
# Check nginx proxy config
docker-compose exec frontend cat /etc/nginx/conf.d/default.conf
# Test backend from frontend container
docker-compose exec frontend wget -qO- http://backend:8000/health
```
### Database issues
```bash
# Reset database
docker-compose down
docker volume rm flight-comparator_backend-data
docker-compose up -d
```
---
## Development Workflow
### Code Changes
**Backend changes:**
```bash
# Edit Python files
# Rebuild and restart
docker-compose up --build backend
```
**Frontend changes:**
```bash
# Edit React files
# Rebuild and restart
docker-compose up --build frontend
```
### Hot Reload (Development)
For development with hot reload, run services manually:
**Backend:**
```bash
python api_server.py
```
**Frontend:**
```bash
cd frontend
npm run dev
```
---
## Production Deployment
### Security Checklist
- [ ] Set `ALLOWED_ORIGINS` to production domain
- [ ] Use HTTPS (reverse proxy with SSL)
- [ ] Update rate limits for expected traffic
- [ ] Configure logging level to `INFO` or `WARNING`
- [ ] Set up automated backups
- [ ] Enable monitoring
- [ ] Review nginx security headers
### Performance Optimization
```yaml
# docker-compose.yml
services:
backend:
deploy:
resources:
limits:
cpus: '1'
memory: 1G
```
---
## Useful Docker Commands
```bash
# Remove everything (reset)
docker-compose down -v
# View logs since 1 hour ago
docker-compose logs --since 1h
# Execute command in backend
docker-compose exec backend python --version
# Shell access
docker-compose exec backend bash
docker-compose exec frontend sh
# Copy files from container
docker cp flight-radar-backend:/app/cache.db ./
# Network inspection
docker network inspect flight-comparator_flight-radar-network
```
---
## Files Created
- `Dockerfile.backend` - Backend container image
- `Dockerfile.frontend` - Frontend container image
- `docker-compose.yml` - Service orchestration
- `nginx.conf` - Nginx web server config
- `.env.example` - Environment template
- `.dockerignore` - Build optimization
---
## Resource Requirements
**Minimum:**
- CPU: 1 core
- RAM: 2GB
- Disk: 5GB
**Recommended:**
- CPU: 2 cores
- RAM: 4GB
- Disk: 10GB
---
## Next Steps
1. ✅ Start application: `docker-compose up -d`
2. ✅ Open browser: http://localhost
3. ✅ Create a scan
4. ✅ View results
5. ✅ Explore logs: http://localhost/logs
---
**Need help?** See [DEPLOYMENT.md](DEPLOYMENT.md) for detailed documentation.

234
flight-comparator/docs/MIGRATION_V3.md Normal file
View File

@@ -0,0 +1,234 @@
# Migration Guide: fast-flights v3.0rc1 with SOCS Cookie
## What Changed
The Flight Airport Comparator now uses **fast-flights v3.0rc1** with **SOCS cookie integration** to successfully bypass Google's consent page and retrieve real flight data.
## Quick Start
### 1. Install fast-flights v3.0rc1
```bash
pip install --upgrade git+https://github.com/AWeirdDev/flights.git
```
### 2. Verify Installation
```bash
python -c "import fast_flights; print('✓ v3.0rc1 installed')"
```
### 3. Test It Works
```bash
cd flight-comparator
python test_v3_with_cookies.py
```
You should see:
```
✅ SUCCESS! Found 1 flight option(s):
1. Ryanair
Price: €89
BER → BRI
...
```
## What's New
### ✅ SOCS Cookie Integration
The breakthrough solution! A custom `Integration` class injects Google's SOCS (consent) cookie into every request:
```python
class SOCSCookieIntegration(Integration):
SOCS_COOKIE = 'CAESHwgBEhJnd3NfMjAyNTAyMjctMF9SQzIaBXpoLUNOIAEaBgiAy6O-Bg'
def fetch_html(self, q: Query | str, /) -> str:
client = primp.Client(...)
response = client.get(
"https://www.google.com/travel/flights",
params=params,
cookies={'SOCS': self.SOCS_COOKIE}, # ← Magic happens here
)
return response.text
```
This tells Google the user has accepted cookies, bypassing the consent page entirely.
### ✅ v3 API Changes
**Old (v2.2):**
```python
from fast_flights import FlightData, get_flights
flight = FlightData(
date="2026-03-23",
from_airport="BER",
to_airport="BRI"
)
result = get_flights(
flight,
passengers=Passengers(adults=1),
seat=1,
fetch_mode='fallback'
)
```
**New (v3.0rc1):**
```python
from fast_flights import FlightQuery, create_query, get_flights
flights = [FlightQuery(
date="2026-03-23",
from_airport="BER",
to_airport="BRI",
max_stops=0
)]
query = create_query(
flights=flights,
seat="economy", # String, not number
trip="one-way",
passengers=Passengers(adults=1) # Keyword argument
)
result = get_flights(query, integration=cookie_integration)
```
### ✅ Automatic Fallback
The tool automatically uses `searcher_v3.py` if v3.0rc1 is installed, otherwise falls back to the legacy searcher:
```python
try:
from searcher_v3 import search_multiple_routes
print("✓ Using fast-flights v3.0rc1 with SOCS cookie integration")
except ImportError:
from searcher import search_multiple_routes
print("⚠️ Using legacy searcher (v2.2)")
```
## File Structure
```
flight-comparator/
├── searcher_v3.py # NEW: v3 searcher with SOCS cookie
├── searcher.py # OLD: v2 searcher (kept for fallback)
├── main.py # UPDATED: Auto-detects v3 or v2
├── test_v3_with_cookies.py # NEW: v3 cookie integration test
├── tests/
│ └── test_comprehensive_v3.py # NEW: Full test suite
├── MIGRATION_V3.md # This file
└── FAST_FLIGHTS_TEST_REPORT.md # Research findings
```
## Troubleshooting
### "fast-flights not found"
```bash
pip install --upgrade git+https://github.com/AWeirdDev/flights.git
```
### "Cannot import FlightQuery"
You have v2.2 installed. Uninstall and reinstall v3:
```bash
pip uninstall fast-flights
pip install git+https://github.com/AWeirdDev/flights.git
```
### "Still getting consent page"
The SOCS cookie may have expired (13-month lifetime). Get a fresh one:
1. Open Google Flights in your browser
2. Accept cookies
3. Check browser dev tools → Application → Cookies → `SOCS`
4. Copy the value
5. Update `SOCS_COOKIE` in `searcher_v3.py`
### "Protobuf version conflict"
v3.0rc1 requires protobuf >= 5.27.0, which may conflict with other packages:
```bash
pip install --upgrade protobuf
# OR
pip install protobuf==5.27.0 --force-reinstall
```
If conflicts persist, use a virtual environment:
```bash
python -m venv venv
source venv/bin/activate # or `venv\Scripts\activate` on Windows
pip install -r requirements.txt
pip install git+https://github.com/AWeirdDev/flights.git
```
## Testing
### Run Full Test Suite
```bash
cd tests
python test_comprehensive_v3.py
```
This tests:
- ✅ SOCS cookie integration
- ✅ Single route queries
- ✅ Multiple routes batch processing
- ✅ Different dates
- ✅ No direct flights handling
- ✅ Invalid airport codes
- ✅ Concurrent requests (10 routes)
- ✅ Price validation
### Quick Smoke Test
```bash
python test_v3_with_cookies.py
```
### Test Your Tool End-to-End
```bash
python main.py --to BDS --from BER,FRA,MUC --date 2026-06-15
```
## Performance
With v3.0rc1 + SOCS cookie:
| Metric | Performance |
|--------|-------------|
| Single query | ~3-5s |
| 10 concurrent routes | ~20-30s |
| Success rate | ~80-90% (some routes have no direct flights) |
| Consent page bypass | ✅ 100% |
## What's Next
1. **Monitor SOCS cookie validity** - May need refresh after 13 months
2. **Consider caching** - Save results to avoid repeated API calls
3. **Add retry logic** - For transient network errors
4. **Rate limiting awareness** - Google may still throttle excessive requests
## Credits
- Solution based on [GitHub Issue #46](https://github.com/AWeirdDev/flights/issues/46)
- SOCS cookie research from [Cookie Library](https://cookielibrary.org/cookie_consent/socs/)
- fast-flights by [@AWeirdDev](https://github.com/AWeirdDev/flights)
## Support
If you encounter issues:
1. Check [FAST_FLIGHTS_TEST_REPORT.md](./FAST_FLIGHTS_TEST_REPORT.md) for detailed findings
2. Review [GitHub Issues](https://github.com/AWeirdDev/flights/issues)
3. Ensure you're on v3.0rc1: `python -c "import fast_flights; print(dir(fast_flights))"`

345
flight-comparator/formatter.py Normal file
View File

@@ -0,0 +1,345 @@
"""
Output formatting for flight comparison results.
Supports table, JSON, and CSV formats.
"""
import json
import csv
import sys
from typing import Optional
from datetime import datetime
try:
from rich.console import Console
from rich.table import Table
from rich import box
HAS_RICH = True
except ImportError:
HAS_RICH = False
def format_duration(minutes: int) -> str:
"""
Format duration in minutes to human-readable format.
Args:
minutes: Duration in minutes
Returns:
Formatted string like "9h 30m"
"""
if minutes == 0:
return "—"
hours = minutes // 60
mins = minutes % 60
if mins == 0:
return f"{hours}h"
return f"{hours}h {mins}m"
def format_table_single_date(
results: dict[str, list[dict]],
destination: str,
country: str,
date: str,
seat_class: str,
sort_by: str,
total_airports: int,
elapsed_time: float,
) -> None:
"""
Format and print results for single-date mode as a table.
Args:
results: Dict mapping airport IATA codes to lists of flights
destination: Destination airport code
country: Origin country code
date: Query date
seat_class: Cabin class
sort_by: Sort criterion (price or duration)
total_airports: Total number of airports scanned
elapsed_time: Total execution time in seconds
"""
if not HAS_RICH:
print("Rich library not installed, using plain text output")
_format_plain_single_date(results, destination, country, date, seat_class, sort_by, total_airports, elapsed_time)
return
console = Console()
# Print header
console.print()
console.print(
f"Flight Comparator: {country} → {destination} | {date} | "
f"{seat_class.title()} | Sorted by: {sort_by.title()}",
style="bold cyan"
)
console.print()
# Create table
table = Table(box=box.DOUBLE_EDGE, show_header=True, header_style="bold magenta")
table.add_column("#", justify="right", style="dim")
table.add_column("From", style="cyan")
table.add_column("Airline", style="green")
table.add_column("Depart", justify="center")
table.add_column("Arrive", justify="center")
table.add_column("Duration", justify="center")
table.add_column("Price", justify="right", style="yellow")
table.add_column("Market", justify="center")
# Flatten and sort results
all_flights = []
for airport_code, flights in results.items():
for flight in flights:
flight['airport_code'] = airport_code
all_flights.append(flight)
# Sort by price or duration
if sort_by == "duration":
all_flights.sort(key=lambda f: f.get('duration_minutes', 999999))
else: # price
all_flights.sort(key=lambda f: f.get('price', 999999))
# Add rows
airports_with_flights = set()
for idx, flight in enumerate(all_flights, 1):
airport_code = flight['airport_code']
airports_with_flights.add(airport_code)
from_text = f"{airport_code} {flight.get('city', '')}"
airline = flight.get('airline', 'Unknown')
depart = flight.get('departure_time', '—')
arrive = flight.get('arrival_time', '—')
duration = format_duration(flight.get('duration_minutes', 0))
price = f"{flight.get('currency', '€')}{flight.get('price', 0):.0f}"
# Simple market indicator (Low/Typical/High)
# In a real implementation, this would compare against price distribution
market = "Typical"
table.add_row(
str(idx),
from_text,
airline,
depart,
arrive,
duration,
price,
market
)
# Add rows for airports with no direct flights
no_direct_count = 0
for airport_code in results.keys():
if airport_code not in airports_with_flights:
from_text = f"{airport_code}"
table.add_row(
"—",
from_text,
"—",
"—",
"—",
"no direct flights found",
"—",
"—",
style="dim"
)
no_direct_count += 1
console.print(table)
console.print()
# Summary
console.print(
f"Scanned {total_airports} airports • "
f"{len(airports_with_flights)} with direct flights • "
f"Done in {elapsed_time:.1f}s",
style="dim"
)
console.print()
def format_table_seasonal(
results_by_month: dict[str, dict[str, list[dict]]],
new_connections: dict[str, str],
destination: str,
country: str,
seat_class: str,
total_airports: int,
elapsed_time: float,
) -> None:
"""
Format and print results for seasonal scan mode.
Args:
results_by_month: Dict mapping month strings to airport->flights dicts
new_connections: Dict mapping route keys to first appearance month
destination: Destination airport code
country: Origin country code
seat_class: Cabin class
total_airports: Total airports scanned
elapsed_time: Execution time in seconds
"""
if not HAS_RICH:
print("Rich library not installed, using plain text output")
_format_plain_seasonal(results_by_month, new_connections, destination, country, seat_class, total_airports, elapsed_time)
return
console = Console()
# Print header
console.print()
console.print(
f"Flight Comparator: {country} → {destination} | "
f"Seasonal scan: {len(results_by_month)} months | {seat_class.title()}",
style="bold cyan"
)
console.print()
# Process each month
for month in sorted(results_by_month.keys()):
month_results = results_by_month[month]
# Create month header
console.print(f"[bold yellow]{month.upper()}[/bold yellow]")
console.print()
# Flatten flights for this month
all_flights = []
for airport_code, flights in month_results.items():
for flight in flights:
flight['airport_code'] = airport_code
all_flights.append(flight)
# Sort by price
all_flights.sort(key=lambda f: f.get('price', 999999))
# Show top results
for idx, flight in enumerate(all_flights[:5], 1): # Top 5 per month
airport_code = flight['airport_code']
from_text = f"{airport_code} {flight.get('city', '')}"
airline = flight.get('airline', 'Unknown')
price = f"{flight.get('currency', '€')}{flight.get('price', 0):.0f}"
# Check if this is a new connection
route_key = f"{airport_code}->{destination}"
is_new = route_key in new_connections and new_connections[route_key] == month
market = "✨ NEW" if is_new else "Typical"
console.print(
f"{idx} │ {from_text:20} │ {airline:15} │ {price:8} │ {market}"
)
console.print()
console.print()
# Summary
if new_connections:
console.print("[bold]New connections detected:[/bold]")
for route, first_month in sorted(new_connections.items()):
console.print(f" • {route} (from {first_month})", style="green")
console.print()
console.print(
f"Scanned {len(results_by_month)} months × {total_airports} airports • "
f"Done in {elapsed_time:.1f}s",
style="dim"
)
console.print()
def _format_plain_single_date(results, destination, country, date, seat_class, sort_by, total_airports, elapsed_time):
"""Plain text fallback for single-date mode."""
print()
print(f"Flight Comparator: {country} → {destination} | {date} | {seat_class.title()} | Sorted by: {sort_by.title()}")
print("=" * 80)
print()
all_flights = []
for airport_code, flights in results.items():
for flight in flights:
flight['airport_code'] = airport_code
all_flights.append(flight)
if sort_by == "duration":
all_flights.sort(key=lambda f: f.get('duration_minutes', 999999))
else:
all_flights.sort(key=lambda f: f.get('price', 999999))
for idx, flight in enumerate(all_flights, 1):
print(f"{idx}. {flight['airport_code']} - {flight.get('airline', 'Unknown')} - "
f"{flight.get('currency', '€')}{flight.get('price', 0):.0f} - "
f"{format_duration(flight.get('duration_minutes', 0))}")
print()
print(f"Scanned {total_airports} airports • Done in {elapsed_time:.1f}s")
print()
def _format_plain_seasonal(results_by_month, new_connections, destination, country, seat_class, total_airports, elapsed_time):
"""Plain text fallback for seasonal mode."""
print()
print(f"Flight Comparator: {country} → {destination} | Seasonal scan: {len(results_by_month)} months | {seat_class.title()}")
print("=" * 80)
for month in sorted(results_by_month.keys()):
print(f"\n{month.upper()}")
print("-" * 40)
month_results = results_by_month[month]
all_flights = []
for airport_code, flights in month_results.items():
for flight in flights:
flight['airport_code'] = airport_code
all_flights.append(flight)
all_flights.sort(key=lambda f: f.get('price', 999999))
for idx, flight in enumerate(all_flights[:5], 1):
route_key = f"{flight['airport_code']}->{destination}"
is_new = route_key in new_connections and new_connections[route_key] == month
new_tag = " ✨ NEW" if is_new else ""
print(f"{idx}. {flight['airport_code']} - {flight.get('airline', 'Unknown')} - "
f"{flight.get('currency', '€')}{flight.get('price', 0):.0f}{new_tag}")
print()
if new_connections:
print("New connections detected:")
for route, first_month in sorted(new_connections.items()):
print(f" • {route} (from {first_month})")
print()
print(f"Scanned {len(results_by_month)} months × {total_airports} airports • Done in {elapsed_time:.1f}s")
print()
def format_json(results, **kwargs) -> None:
"""Format results as JSON."""
print(json.dumps(results, indent=2))
def format_csv(results: dict[str, list[dict]], **kwargs) -> None:
"""Format results as CSV."""
writer = csv.writer(sys.stdout)
writer.writerow(['Origin', 'Destination', 'Airline', 'Departure', 'Arrival', 'Duration_Min', 'Price', 'Currency'])
for airport_code, flights in results.items():
for flight in flights:
writer.writerow([
airport_code,
flight.get('destination', ''),
flight.get('airline', ''),
flight.get('departure_time', ''),
flight.get('arrival_time', ''),
flight.get('duration_minutes', 0),
flight.get('price', 0),
flight.get('currency', ''),
])

24
flight-comparator/frontend/.gitignore vendored Normal file
View File

@@ -0,0 +1,24 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?

23
flight-comparator/frontend/eslint.config.js Normal file
View File

@@ -0,0 +1,23 @@
import js from '@eslint/js'
import globals from 'globals'
import reactHooks from 'eslint-plugin-react-hooks'
import reactRefresh from 'eslint-plugin-react-refresh'
import tseslint from 'typescript-eslint'
import { defineConfig, globalIgnores } from 'eslint/config'
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
js.configs.recommended,
tseslint.configs.recommended,
reactHooks.configs.flat.recommended,
reactRefresh.configs.vite,
],
languageOptions: {
ecmaVersion: 2020,
globals: globals.browser,
},
},
])

13
flight-comparator/frontend/index.html Normal file
View File

@@ -0,0 +1,13 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>frontend</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>

6
flight-comparator/frontend/postcss.config.js Normal file
View File

@@ -0,0 +1,6 @@
export default {
plugins: {
'@tailwindcss/postcss': {},
autoprefixer: {},
},
}

1
flight-comparator/frontend/public/vite.svg Normal file
View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

42
flight-comparator/frontend/src/App.css Normal file
View File

@@ -0,0 +1,42 @@
#root {
max-width: 1280px;
margin: 0 auto;
padding: 2rem;
text-align: center;
}
.logo {
height: 6em;
padding: 1.5em;
will-change: filter;
transition: filter 300ms;
}
.logo:hover {
filter: drop-shadow(0 0 2em #646cffaa);
}
.logo.react:hover {
filter: drop-shadow(0 0 2em #61dafbaa);
}
@keyframes logo-spin {
from {
transform: rotate(0deg);
}
to {
transform: rotate(360deg);
}
}
@media (prefers-reduced-motion: no-preference) {
a:nth-of-type(2) .logo {
animation: logo-spin infinite 20s linear;
}
}
.card {
padding: 2em;
}
.read-the-docs {
color: #888;
}

28
flight-comparator/frontend/src/App.tsx Normal file
View File

@@ -0,0 +1,28 @@
import { BrowserRouter, Routes, Route } from 'react-router-dom';
import Layout from './components/Layout';
import Dashboard from './pages/Dashboard';
import Scans from './pages/Scans';
import ScanDetails from './pages/ScanDetails';
import Airports from './pages/Airports';
import Logs from './pages/Logs';
import ErrorBoundary from './components/ErrorBoundary';
function App() {
return (
<ErrorBoundary>
<BrowserRouter>
<Routes>
<Route path="/" element={<Layout />}>
<Route index element={<Dashboard />} />
<Route path="scans" element={<Scans />} />
<Route path="scans/:id" element={<ScanDetails />} />
<Route path="airports" element={<Airports />} />
<Route path="logs" element={<Logs />} />
</Route>
</Routes>
</BrowserRouter>
</ErrorBoundary>
);
}
export default App;

145
flight-comparator/frontend/src/api.ts Normal file
View File

@@ -0,0 +1,145 @@
import axios from 'axios';
const api = axios.create({
baseURL: '/api/v1',
headers: {
'Content-Type': 'application/json',
},
});
// Types
export interface Scan {
id: number;
origin: string;
country: string;
start_date: string;
end_date: string;
status: 'pending' | 'running' | 'completed' | 'failed';
created_at: string;
updated_at: string;
total_routes: number;
routes_scanned: number;
total_flights: number;
error_message?: string;
seat_class: string;
adults: number;
}
export interface Route {
id: number;
scan_id: number;
destination: string;
destination_name: string;
destination_city?: string;
flight_count: number;
airlines: string[];
min_price?: number;
max_price?: number;
avg_price?: number;
created_at: string;
}
export interface Flight {
id: number;
scan_id: number;
destination: string;
date: string;
airline?: string;
departure_time?: string;
arrival_time?: string;
price?: number;
stops: number;
}
export interface Airport {
iata: string;
name: string;
city: string;
country: string;
}
export interface LogEntry {
timestamp: string;
level: string;
message: string;
module?: string;
function?: string;
line?: number;
}
export interface PaginatedResponse<T> {
data: T[];
pagination: {
page: number;
limit: number;
total: number;
pages: number;
has_next: boolean;
has_prev: boolean;
};
}
export interface CreateScanRequest {
origin: string;
country?: string; // Optional: provide either country or destinations
destinations?: string[]; // Optional: provide either country or destinations
start_date?: string;
end_date?: string;
window_months?: number;
seat_class?: 'economy' | 'premium' | 'business' | 'first';
adults?: number;
}
export interface CreateScanResponse {
status: string;
id: number;
scan: Scan;
}
// API functions
export const scanApi = {
list: (page = 1, limit = 20, status?: string) => {
const params: any = { page, limit };
if (status) params.status = status;
return api.get<PaginatedResponse<Scan>>('/scans', { params });
},
get: (id: number) => {
return api.get<Scan>(`/scans/${id}`);
},
create: (data: CreateScanRequest) => {
return api.post<CreateScanResponse>('/scans', data);
},
getRoutes: (id: number, page = 1, limit = 20) => {
return api.get<PaginatedResponse<Route>>(`/scans/${id}/routes`, {
params: { page, limit }
});
},
getFlights: (id: number, destination?: string, page = 1, limit = 50) => {
const params: Record<string, unknown> = { page, limit };
if (destination) params.destination = destination;
return api.get<PaginatedResponse<Flight>>(`/scans/${id}/flights`, { params });
},
};
export const airportApi = {
search: (query: string, page = 1, limit = 20) => {
return api.get<PaginatedResponse<Airport>>('/airports', {
params: { q: query, page, limit }
});
},
};
export const logApi = {
list: (page = 1, limit = 50, level?: string, search?: string) => {
const params: any = { page, limit };
if (level) params.level = level;
if (search) params.search = search;
return api.get<PaginatedResponse<LogEntry>>('/logs', { params });
},
};
export default api;

1
flight-comparator/frontend/src/assets/react.svg Normal file
View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.0 KiB

132
flight-comparator/frontend/src/components/AirportSearch.tsx Normal file
View File

@@ -0,0 +1,132 @@
import { useState, useEffect, useRef } from 'react';
import { airportApi } from '../api';
import type { Airport } from '../api';
interface AirportSearchProps {
value: string;
onChange: (value: string) => void;
placeholder?: string;
clearAfterSelect?: boolean;
required?: boolean;
}
export default function AirportSearch({ value, onChange, placeholder, clearAfterSelect, required = true }: AirportSearchProps) {
const [query, setQuery] = useState(value);
const [airports, setAirports] = useState<Airport[]>([]);
const [loading, setLoading] = useState(false);
const [showDropdown, setShowDropdown] = useState(false);
const debounceTimer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
const containerRef = useRef<HTMLDivElement>(null);
useEffect(() => {
setQuery(value);
}, [value]);
useEffect(() => {
// Close dropdown when clicking outside
const handleClickOutside = (event: MouseEvent) => {
if (containerRef.current && !containerRef.current.contains(event.target as Node)) {
setShowDropdown(false);
}
};
document.addEventListener('mousedown', handleClickOutside);
return () => document.removeEventListener('mousedown', handleClickOutside);
}, []);
const searchAirports = async (searchQuery: string) => {
if (searchQuery.length < 2) {
setAirports([]);
setShowDropdown(false);
return;
}
try {
setLoading(true);
const response = await airportApi.search(searchQuery, 1, 10);
setAirports(response.data.data);
setShowDropdown(true);
} catch (error) {
console.error('Failed to search airports:', error);
setAirports([]);
} finally {
setLoading(false);
}
};
const handleInputChange = (e: React.ChangeEvent<HTMLInputElement>) => {
const newQuery = e.target.value.toUpperCase();
setQuery(newQuery);
onChange(newQuery);
// Debounce search
if (debounceTimer.current) {
clearTimeout(debounceTimer.current);
}
debounceTimer.current = setTimeout(() => {
searchAirports(newQuery);
}, 300);
};
const handleSelectAirport = (airport: Airport) => {
onChange(airport.iata);
if (clearAfterSelect) {
setQuery('');
setAirports([]);
} else {
setQuery(airport.iata);
}
setShowDropdown(false);
};
return (
<div ref={containerRef} className="relative">
<input
type="text"
value={query}
onChange={handleInputChange}
onFocus={() => {
if (airports.length > 0) {
setShowDropdown(true);
}
}}
maxLength={3}
required={required}
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
placeholder={placeholder || 'Search airports...'}
/>
{/* Dropdown */}
{showDropdown && airports.length > 0 && (
<div className="absolute z-10 w-full mt-1 bg-white border border-gray-300 rounded-md shadow-lg max-h-60 overflow-y-auto">
{loading && (
<div className="px-4 py-2 text-sm text-gray-500">
Searching...
</div>
)}
{!loading && airports.map((airport) => (
<div
key={airport.iata}
onClick={() => handleSelectAirport(airport)}
className="px-4 py-2 hover:bg-gray-100 cursor-pointer"
>
<div className="flex items-center justify-between">
<div>
<span className="font-medium text-gray-900">{airport.iata}</span>
<span className="ml-2 text-sm text-gray-600">
{airport.name}
</span>
</div>
<span className="text-sm text-gray-500">
{airport.city}, {airport.country}
</span>
</div>
</div>
))}
</div>
)}
</div>
);
}

73
flight-comparator/frontend/src/components/ErrorBoundary.tsx Normal file
View File

@@ -0,0 +1,73 @@
import React from 'react';
interface Props {
children: React.ReactNode;
}
interface State {
hasError: boolean;
error: Error | null;
}
export default class ErrorBoundary extends React.Component<Props, State> {
constructor(props: Props) {
super(props);
this.state = { hasError: false, error: null };
}
static getDerivedStateFromError(error: Error): State {
return { hasError: true, error };
}
componentDidCatch(error: Error, errorInfo: React.ErrorInfo) {
console.error('Error caught by boundary:', error, errorInfo);
}
render() {
if (this.state.hasError) {
return (
<div className="min-h-screen flex items-center justify-center bg-gray-50">
<div className="max-w-md w-full bg-white shadow-lg rounded-lg p-6">
<div className="flex items-center justify-center w-12 h-12 mx-auto bg-red-100 rounded-full">
<svg
className="w-6 h-6 text-red-600"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M6 18L18 6M6 6l12 12"
/>
</svg>
</div>
<h2 className="mt-4 text-xl font-bold text-center text-gray-900">
Something went wrong
</h2>
<p className="mt-2 text-sm text-center text-gray-600">
{this.state.error?.message || 'An unexpected error occurred'}
</p>
<div className="mt-6 flex justify-center space-x-3">
<button
onClick={() => window.location.href = '/'}
className="px-4 py-2 bg-blue-500 hover:bg-blue-600 text-white rounded-md text-sm font-medium"
>
Go to Dashboard
</button>
<button
onClick={() => window.location.reload()}
className="px-4 py-2 border border-gray-300 hover:bg-gray-50 text-gray-700 rounded-md text-sm font-medium"
>
Reload Page
</button>
</div>
</div>
</div>
);
}
return this.props.children;
}
}

72
flight-comparator/frontend/src/components/Layout.tsx Normal file
View File

@@ -0,0 +1,72 @@
import { Link, Outlet, useLocation } from 'react-router-dom';
export default function Layout() {
const location = useLocation();
const isActive = (path: string) => {
return location.pathname === path;
};
return (
<div className="min-h-screen bg-gray-50">
{/* Header */}
<header className="bg-white shadow-sm border-b border-gray-200">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-4">
<div className="flex flex-col sm:flex-row justify-between items-center gap-4">
<h1 className="text-2xl font-bold text-blue-600 flex items-center">
<span className="text-3xl mr-2">✈️</span>
Flight Radar
</h1>
<nav className="flex flex-wrap justify-center gap-2">
<Link
to="/"
className={`px-3 py-2 rounded-md text-sm font-medium ${
isActive('/')
? 'bg-blue-500 text-white'
: 'text-gray-700 hover:bg-gray-100'
}`}
>
Dashboard
</Link>
<Link
to="/scans"
className={`px-3 py-2 rounded-md text-sm font-medium ${
isActive('/scans')
? 'bg-blue-500 text-white'
: 'text-gray-700 hover:bg-gray-100'
}`}
>
Scans
</Link>
<Link
to="/airports"
className={`px-3 py-2 rounded-md text-sm font-medium ${
isActive('/airports')
? 'bg-blue-500 text-white'
: 'text-gray-700 hover:bg-gray-100'
}`}
>
Airports
</Link>
<Link
to="/logs"
className={`px-3 py-2 rounded-md text-sm font-medium ${
isActive('/logs')
? 'bg-blue-500 text-white'
: 'text-gray-700 hover:bg-gray-100'
}`}
>
Logs
</Link>
</nav>
</div>
</div>
</header>
{/* Main Content */}
<main className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
<Outlet />
</main>
</div>
);
}

15
flight-comparator/frontend/src/components/LoadingSpinner.tsx Normal file
View File

@@ -0,0 +1,15 @@
export default function LoadingSpinner({ size = 'md' }: { size?: 'sm' | 'md' | 'lg' }) {
const sizeClasses = {
sm: 'w-4 h-4',
md: 'w-8 h-8',
lg: 'w-12 h-12',
};
return (
<div className="flex justify-center items-center">
<div
className={`${sizeClasses[size]} border-4 border-blue-200 border-t-blue-600 rounded-full animate-spin`}
/>
</div>
);
}

97
flight-comparator/frontend/src/components/Toast.tsx Normal file
View File

@@ -0,0 +1,97 @@
import { useEffect } from 'react';
export type ToastType = 'success' | 'error' | 'info' | 'warning';
interface ToastProps {
message: string;
type: ToastType;
onClose: () => void;
duration?: number;
}
export default function Toast({ message, type, onClose, duration = 5000 }: ToastProps) {
useEffect(() => {
const timer = setTimeout(onClose, duration);
return () => clearTimeout(timer);
}, [duration, onClose]);
const getColors = () => {
switch (type) {
case 'success':
return 'bg-green-50 border-green-200 text-green-800';
case 'error':
return 'bg-red-50 border-red-200 text-red-800';
case 'warning':
return 'bg-yellow-50 border-yellow-200 text-yellow-800';
case 'info':
return 'bg-blue-50 border-blue-200 text-blue-800';
}
};
const getIcon = () => {
switch (type) {
case 'success':
return (
<svg className="w-5 h-5 text-green-600" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z"
clipRule="evenodd"
/>
</svg>
);
case 'error':
return (
<svg className="w-5 h-5 text-red-600" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z"
clipRule="evenodd"
/>
</svg>
);
case 'warning':
return (
<svg className="w-5 h-5 text-yellow-600" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M8.257 3.099c.765-1.36 2.722-1.36 3.486 0l5.58 9.92c.75 1.334-.213 2.98-1.742 2.98H4.42c-1.53 0-2.493-1.646-1.743-2.98l5.58-9.92zM11 13a1 1 0 11-2 0 1 1 0 012 0zm-1-8a1 1 0 00-1 1v3a1 1 0 002 0V6a1 1 0 00-1-1z"
clipRule="evenodd"
/>
</svg>
);
case 'info':
return (
<svg className="w-5 h-5 text-blue-600" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M18 10a8 8 0 11-16 0 8 8 0 0116 0zm-7-4a1 1 0 11-2 0 1 1 0 012 0zM9 9a1 1 0 000 2v3a1 1 0 001 1h1a1 1 0 100-2v-3a1 1 0 00-1-1H9z"
clipRule="evenodd"
/>
</svg>
);
}
};
return (
<div
className={`fixed bottom-4 right-4 flex items-center p-4 border rounded-lg shadow-lg ${getColors()} animate-slide-up`}
style={{ minWidth: '300px', maxWidth: '500px' }}
>
<div className="flex-shrink-0">{getIcon()}</div>
<p className="ml-3 text-sm font-medium flex-1">{message}</p>
<button
onClick={onClose}
className="ml-4 flex-shrink-0 text-gray-400 hover:text-gray-600"
>
<svg className="w-4 h-4" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M4.293 4.293a1 1 0 011.414 0L10 8.586l4.293-4.293a1 1 0 111.414 1.414L11.414 10l4.293 4.293a1 1 0 01-1.414 1.414L10 11.414l-4.293 4.293a1 1 0 01-1.414-1.414L8.586 10 4.293 5.707a1 1 0 010-1.414z"
clipRule="evenodd"
/>
</svg>
</button>
</div>
);
}

18
flight-comparator/frontend/src/index.css Normal file
View File

@@ -0,0 +1,18 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
@keyframes slide-up {
from {
transform: translateY(100%);
opacity: 0;
}
to {
transform: translateY(0);
opacity: 1;
}
}
.animate-slide-up {
animation: slide-up 0.3s ease-out;
}

10
flight-comparator/frontend/src/main.tsx Normal file
View File

@@ -0,0 +1,10 @@
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import './index.css'
import App from './App.tsx'
createRoot(document.getElementById('root')!).render(
<StrictMode>
<App />
</StrictMode>,
)

144
flight-comparator/frontend/src/pages/Airports.tsx Normal file
View File

@@ -0,0 +1,144 @@
import { useState } from 'react';
import { airportApi } from '../api';
import type { Airport } from '../api';
export default function Airports() {
const [query, setQuery] = useState('');
const [airports, setAirports] = useState<Airport[]>([]);
const [loading, setLoading] = useState(false);
const [page, setPage] = useState(1);
const [totalPages, setTotalPages] = useState(0);
const [total, setTotal] = useState(0);
const handleSearch = async (searchQuery: string, searchPage = 1) => {
if (searchQuery.length < 2) {
setAirports([]);
return;
}
try {
setLoading(true);
const response = await airportApi.search(searchQuery, searchPage, 20);
setAirports(response.data.data);
setTotalPages(response.data.pagination.pages);
setTotal(response.data.pagination.total);
setPage(searchPage);
} catch (error) {
console.error('Failed to search airports:', error);
} finally {
setLoading(false);
}
};
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault();
handleSearch(query, 1);
};
return (
<div>
<h2 className="text-2xl font-bold text-gray-900 mb-6">Airport Search</h2>
{/* Search Form */}
<div className="bg-white rounded-lg shadow p-6 mb-6">
<form onSubmit={handleSubmit}>
<div className="flex space-x-4">
<input
type="text"
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder="Search by IATA code, city, or airport name..."
className="flex-1 px-4 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
<button
type="submit"
disabled={loading || query.length < 2}
className="px-6 py-2 bg-blue-500 hover:bg-blue-600 text-white rounded-md font-medium disabled:opacity-50 disabled:cursor-not-allowed"
>
{loading ? 'Searching...' : 'Search'}
</button>
</div>
<p className="mt-2 text-sm text-gray-500">
Enter at least 2 characters to search
</p>
</form>
</div>
{/* Results */}
{airports.length > 0 && (
<div className="bg-white rounded-lg shadow overflow-hidden">
<div className="px-6 py-4 border-b border-gray-200 flex justify-between items-center">
<h3 className="text-lg font-semibold text-gray-900">
Search Results
</h3>
<span className="text-sm text-gray-500">
{total} airport{total !== 1 ? 's' : ''} found
</span>
</div>
<div className="divide-y divide-gray-200">
{airports.map((airport) => (
<div key={airport.iata} className="px-6 py-4 hover:bg-gray-50">
<div className="flex items-center justify-between">
<div className="flex-1">
<div className="flex items-center space-x-3">
<span className="font-bold text-lg text-blue-600">
{airport.iata}
</span>
<span className="font-medium text-gray-900">
{airport.name}
</span>
</div>
<div className="mt-1 text-sm text-gray-500">
{airport.city}, {airport.country}
</div>
</div>
<button
onClick={() => {
navigator.clipboard.writeText(airport.iata);
}}
className="px-3 py-1 text-sm text-blue-600 hover:bg-blue-50 rounded"
>
Copy Code
</button>
</div>
</div>
))}
</div>
{/* Pagination */}
{totalPages > 1 && (
<div className="px-6 py-4 border-t border-gray-200 flex justify-between items-center">
<div className="text-sm text-gray-500">
Page {page} of {totalPages}
</div>
<div className="flex space-x-2">
<button
onClick={() => handleSearch(query, page - 1)}
disabled={page === 1 || loading}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Previous
</button>
<button
onClick={() => handleSearch(query, page + 1)}
disabled={page === totalPages || loading}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Next
</button>
</div>
</div>
)}
</div>
)}
{/* Empty State */}
{!loading && airports.length === 0 && query.length >= 2 && (
<div className="bg-white rounded-lg shadow p-12 text-center">
<p className="text-gray-500">No airports found for "{query}"</p>
</div>
)}
</div>
);
}

157
flight-comparator/frontend/src/pages/Dashboard.tsx Normal file
View File

@@ -0,0 +1,157 @@
import { useEffect, useState } from 'react';
import { Link } from 'react-router-dom';
import { scanApi } from '../api';
import type { Scan } from '../api';
export default function Dashboard() {
const [scans, setScans] = useState<Scan[]>([]);
const [loading, setLoading] = useState(true);
const [stats, setStats] = useState({
total: 0,
pending: 0,
running: 0,
completed: 0,
failed: 0,
});
useEffect(() => {
loadScans();
}, []);
const loadScans = async () => {
try {
setLoading(true);
const response = await scanApi.list(1, 10);
const scanList = response.data.data;
setScans(scanList);
// Calculate stats
setStats({
total: response.data.pagination.total,
pending: scanList.filter(s => s.status === 'pending').length,
running: scanList.filter(s => s.status === 'running').length,
completed: scanList.filter(s => s.status === 'completed').length,
failed: scanList.filter(s => s.status === 'failed').length,
});
} catch (error) {
console.error('Failed to load scans:', error);
} finally {
setLoading(false);
}
};
const getStatusColor = (status: string) => {
switch (status) {
case 'completed':
return 'bg-green-100 text-green-800';
case 'running':
return 'bg-blue-100 text-blue-800';
case 'pending':
return 'bg-yellow-100 text-yellow-800';
case 'failed':
return 'bg-red-100 text-red-800';
default:
return 'bg-gray-100 text-gray-800';
}
};
const formatDate = (dateString: string) => {
return new Date(dateString).toLocaleString();
};
if (loading) {
return (
<div className="flex justify-center items-center h-64">
<div className="text-gray-500">Loading...</div>
</div>
);
}
return (
<div>
<div className="flex justify-between items-center mb-6">
<h2 className="text-2xl font-bold text-gray-900">Dashboard</h2>
<Link
to="/scans"
className="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded-md text-sm font-medium"
>
+ New Scan
</Link>
</div>
{/* Stats Cards */}
<div className="grid grid-cols-1 md:grid-cols-5 gap-4 mb-8">
<div className="bg-white p-6 rounded-lg shadow">
<div className="text-sm font-medium text-gray-500">Total Scans</div>
<div className="text-3xl font-bold text-gray-900 mt-2">{stats.total}</div>
</div>
<div className="bg-white p-6 rounded-lg shadow">
<div className="text-sm font-medium text-gray-500">Pending</div>
<div className="text-3xl font-bold text-yellow-600 mt-2">{stats.pending}</div>
</div>
<div className="bg-white p-6 rounded-lg shadow">
<div className="text-sm font-medium text-gray-500">Running</div>
<div className="text-3xl font-bold text-blue-600 mt-2">{stats.running}</div>
</div>
<div className="bg-white p-6 rounded-lg shadow">
<div className="text-sm font-medium text-gray-500">Completed</div>
<div className="text-3xl font-bold text-green-600 mt-2">{stats.completed}</div>
</div>
<div className="bg-white p-6 rounded-lg shadow">
<div className="text-sm font-medium text-gray-500">Failed</div>
<div className="text-3xl font-bold text-red-600 mt-2">{stats.failed}</div>
</div>
</div>
{/* Recent Scans */}
<div className="bg-white rounded-lg shadow">
<div className="px-6 py-4 border-b border-gray-200">
<h3 className="text-lg font-semibold text-gray-900">Recent Scans</h3>
</div>
<div className="divide-y divide-gray-200">
{scans.length === 0 ? (
<div className="px-6 py-12 text-center text-gray-500">
No scans yet. Create your first scan to get started!
</div>
) : (
scans.map((scan) => (
<Link
key={scan.id}
to={`/scans/${scan.id}`}
className="block px-6 py-4 hover:bg-gray-50 cursor-pointer"
>
<div className="flex items-center justify-between">
<div className="flex-1">
<div className="flex items-center space-x-3">
<span className="font-medium text-gray-900">
{scan.origin} → {scan.country}
</span>
<span
className={`px-2 py-1 text-xs font-medium rounded-full ${getStatusColor(
scan.status
)}`}
>
{scan.status}
</span>
</div>
<div className="mt-1 text-sm text-gray-500">
{scan.start_date} to {scan.end_date} • {scan.adults} adult(s) • {scan.seat_class}
</div>
{scan.total_routes > 0 && (
<div className="mt-1 text-sm text-gray-500">
{scan.total_routes} routes • {scan.total_flights} flights found
</div>
)}
</div>
<div className="text-sm text-gray-500">
{formatDate(scan.created_at)}
</div>
</div>
</Link>
))
)}
</div>
</div>
</div>
);
}

194
flight-comparator/frontend/src/pages/Logs.tsx Normal file
View File

@@ -0,0 +1,194 @@
import { useEffect, useState } from 'react';
import { logApi } from '../api';
import type { LogEntry } from '../api';
export default function Logs() {
const [logs, setLogs] = useState<LogEntry[]>([]);
const [loading, setLoading] = useState(true);
const [page, setPage] = useState(1);
const [totalPages, setTotalPages] = useState(1);
const [level, setLevel] = useState<string>('');
const [search, setSearch] = useState('');
const [searchQuery, setSearchQuery] = useState('');
useEffect(() => {
loadLogs();
}, [page, level, searchQuery]);
const loadLogs = async () => {
try {
setLoading(true);
const response = await logApi.list(page, 50, level || undefined, searchQuery || undefined);
setLogs(response.data.data);
setTotalPages(response.data.pagination.pages);
} catch (error) {
console.error('Failed to load logs:', error);
} finally {
setLoading(false);
}
};
const handleSearch = (e: React.FormEvent) => {
e.preventDefault();
setSearchQuery(search);
setPage(1);
};
const getLevelColor = (logLevel: string) => {
switch (logLevel) {
case 'DEBUG': return 'bg-gray-100 text-gray-700';
case 'INFO': return 'bg-blue-100 text-blue-700';
case 'WARNING': return 'bg-yellow-100 text-yellow-700';
case 'ERROR': return 'bg-red-100 text-red-700';
case 'CRITICAL': return 'bg-red-200 text-red-900';
default: return 'bg-gray-100 text-gray-700';
}
};
const formatTimestamp = (timestamp: string) => {
return new Date(timestamp).toLocaleString();
};
return (
<div>
<h2 className="text-2xl font-bold text-gray-900 mb-6">Logs</h2>
{/* Filters */}
<div className="bg-white rounded-lg shadow p-6 mb-6">
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
{/* Level Filter */}
<div>
<label htmlFor="level" className="block text-sm font-medium text-gray-700 mb-2">
Log Level
</label>
<select
id="level"
value={level}
onChange={(e) => {
setLevel(e.target.value);
setPage(1);
}}
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
>
<option value="">All Levels</option>
<option value="DEBUG">DEBUG</option>
<option value="INFO">INFO</option>
<option value="WARNING">WARNING</option>
<option value="ERROR">ERROR</option>
<option value="CRITICAL">CRITICAL</option>
</select>
</div>
{/* Search */}
<div>
<label htmlFor="search" className="block text-sm font-medium text-gray-700 mb-2">
Search Messages
</label>
<form onSubmit={handleSearch} className="flex space-x-2">
<input
type="text"
id="search"
value={search}
onChange={(e) => setSearch(e.target.value)}
placeholder="Search log messages..."
className="flex-1 px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
<button
type="submit"
className="px-4 py-2 bg-blue-500 hover:bg-blue-600 text-white rounded-md text-sm font-medium"
>
Search
</button>
</form>
</div>
</div>
{/* Clear Filters */}
{(level || searchQuery) && (
<div className="mt-4">
<button
onClick={() => {
setLevel('');
setSearch('');
setSearchQuery('');
setPage(1);
}}
className="text-sm text-blue-600 hover:text-blue-700"
>
Clear Filters
</button>
</div>
)}
</div>
{/* Logs List */}
{loading ? (
<div className="flex justify-center items-center h-64">
<div className="text-gray-500">Loading logs...</div>
</div>
) : (
<div className="bg-white rounded-lg shadow overflow-hidden">
<div className="px-6 py-4 border-b border-gray-200">
<h3 className="text-lg font-semibold text-gray-900">Log Entries</h3>
</div>
{logs.length === 0 ? (
<div className="px-6 py-12 text-center text-gray-500">
No logs found
</div>
) : (
<>
<div className="divide-y divide-gray-200">
{logs.map((log, index) => (
<div key={index} className="px-6 py-4">
<div className="flex items-start space-x-3">
<span className={`px-2 py-1 text-xs font-medium rounded ${getLevelColor(log.level)}`}>
{log.level}
</span>
<div className="flex-1 min-w-0">
<p className="text-sm text-gray-900 break-words">
{log.message}
</p>
<div className="mt-1 flex items-center space-x-4 text-xs text-gray-500">
<span>{formatTimestamp(log.timestamp)}</span>
{log.module && <span>Module: {log.module}</span>}
{log.function && <span>Function: {log.function}</span>}
{log.line && <span>Line: {log.line}</span>}
</div>
</div>
</div>
</div>
))}
</div>
{/* Pagination */}
{totalPages > 1 && (
<div className="px-6 py-4 border-t border-gray-200 flex justify-between items-center">
<div className="text-sm text-gray-500">
Page {page} of {totalPages}
</div>
<div className="flex space-x-2">
<button
onClick={() => setPage(page - 1)}
disabled={page === 1}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Previous
</button>
<button
onClick={() => setPage(page + 1)}
disabled={page === totalPages}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Next
</button>
</div>
</div>
)}
</>
)}
</div>
)}
</div>
);
}

384
flight-comparator/frontend/src/pages/ScanDetails.tsx Normal file
View File

@@ -0,0 +1,384 @@
import { Fragment, useEffect, useState } from 'react';
import { useParams, useNavigate } from 'react-router-dom';
import { scanApi } from '../api';
import type { Scan, Route, Flight } from '../api';
export default function ScanDetails() {
const { id } = useParams<{ id: string }>();
const navigate = useNavigate();
const [scan, setScan] = useState<Scan | null>(null);
const [routes, setRoutes] = useState<Route[]>([]);
const [loading, setLoading] = useState(true);
const [page, setPage] = useState(1);
const [totalPages, setTotalPages] = useState(1);
const [sortField, setSortField] = useState<'min_price' | 'destination' | 'flight_count'>('min_price');
const [sortDirection, setSortDirection] = useState<'asc' | 'desc'>('asc');
const [expandedRoute, setExpandedRoute] = useState<string | null>(null);
const [flightsByDest, setFlightsByDest] = useState<Record<string, Flight[]>>({});
const [loadingFlights, setLoadingFlights] = useState<string | null>(null);
useEffect(() => {
if (id) {
loadScanDetails();
}
}, [id, page]);
// Auto-refresh while scan is running
useEffect(() => {
if (!scan || (scan.status !== 'pending' && scan.status !== 'running')) {
return;
}
const interval = setInterval(() => {
loadScanDetails();
}, 3000); // Poll every 3 seconds
return () => clearInterval(interval);
}, [scan?.status, id]);
useEffect(() => {
// Sort routes when sort field or direction changes
const sorted = [...routes].sort((a, b) => {
let aVal: any = a[sortField];
let bVal: any = b[sortField];
if (sortField === 'min_price') {
aVal = aVal ?? Infinity;
bVal = bVal ?? Infinity;
}
if (aVal < bVal) return sortDirection === 'asc' ? -1 : 1;
if (aVal > bVal) return sortDirection === 'asc' ? 1 : -1;
return 0;
});
setRoutes(sorted);
}, [sortField, sortDirection]);
const loadScanDetails = async () => {
try {
setLoading(true);
const [scanResponse, routesResponse] = await Promise.all([
scanApi.get(Number(id)),
scanApi.getRoutes(Number(id), page, 20),
]);
setScan(scanResponse.data);
setRoutes(routesResponse.data.data);
setTotalPages(routesResponse.data.pagination.pages);
} catch (error) {
console.error('Failed to load scan details:', error);
} finally {
setLoading(false);
}
};
const handleSort = (field: typeof sortField) => {
if (sortField === field) {
setSortDirection(sortDirection === 'asc' ? 'desc' : 'asc');
} else {
setSortField(field);
setSortDirection('asc');
}
};
const toggleFlights = async (destination: string) => {
if (expandedRoute === destination) {
setExpandedRoute(null);
return;
}
setExpandedRoute(destination);
if (flightsByDest[destination]) return; // already loaded
setLoadingFlights(destination);
try {
const resp = await scanApi.getFlights(Number(id), destination, 1, 200);
setFlightsByDest((prev) => ({ ...prev, [destination]: resp.data.data }));
} catch {
setFlightsByDest((prev) => ({ ...prev, [destination]: [] }));
} finally {
setLoadingFlights(null);
}
};
const getStatusColor = (status: string) => {
switch (status) {
case 'completed': return 'bg-green-100 text-green-800';
case 'running': return 'bg-blue-100 text-blue-800';
case 'pending': return 'bg-yellow-100 text-yellow-800';
case 'failed': return 'bg-red-100 text-red-800';
default: return 'bg-gray-100 text-gray-800';
}
};
const formatPrice = (price?: number) => {
return price ? `€${price.toFixed(2)}` : 'N/A';
};
if (loading && !scan) {
return (
<div className="flex justify-center items-center h-64">
<div className="text-gray-500">Loading...</div>
</div>
);
}
if (!scan) {
return (
<div className="text-center py-12">
<p className="text-gray-500">Scan not found</p>
</div>
);
}
return (
<div>
{/* Header */}
<div className="mb-6">
<button
onClick={() => navigate('/')}
className="text-blue-500 hover:text-blue-700 mb-4"
>
← Back to Dashboard
</button>
<div className="flex justify-between items-start">
<div>
<h2 className="text-2xl font-bold text-gray-900">
{scan.origin} → {scan.country}
</h2>
<p className="text-gray-600 mt-1">
{scan.start_date} to {scan.end_date} • {scan.adults} adult(s) • {scan.seat_class}
</p>
</div>
<span className={`px-3 py-1 text-sm font-medium rounded-full ${getStatusColor(scan.status)}`}>
{scan.status}
</span>
</div>
</div>
{/* Progress Bar (for running scans) */}
{(scan.status === 'pending' || scan.status === 'running') && (
<div className="bg-white p-4 rounded-lg shadow mb-6">
<div className="flex justify-between items-center mb-2">
<span className="text-sm font-medium text-gray-700">
{scan.status === 'pending' ? 'Initializing...' : 'Scanning in progress...'}
</span>
<span className="text-sm text-gray-600">
{scan.routes_scanned} / {scan.total_routes > 0 ? scan.total_routes : '?'} routes
</span>
</div>
<div className="w-full bg-gray-200 rounded-full h-2.5">
<div
className="bg-blue-600 h-2.5 rounded-full transition-all duration-300"
style={{
width: scan.total_routes > 0
? `${Math.min((scan.routes_scanned / scan.total_routes) * 100, 100)}%`
: '0%'
}}
></div>
</div>
<p className="text-xs text-gray-500 mt-2">
Auto-refreshing every 3 seconds...
</p>
</div>
)}
{/* Stats */}
<div className="grid grid-cols-1 md:grid-cols-3 gap-4 mb-6">
<div className="bg-white p-4 rounded-lg shadow">
<div className="text-sm text-gray-500">Total Routes</div>
<div className="text-2xl font-bold text-gray-900 mt-1">{scan.total_routes}</div>
</div>
<div className="bg-white p-4 rounded-lg shadow">
<div className="text-sm text-gray-500">Routes Scanned</div>
<div className="text-2xl font-bold text-gray-900 mt-1">{scan.routes_scanned}</div>
</div>
<div className="bg-white p-4 rounded-lg shadow">
<div className="text-sm text-gray-500">Total Flights</div>
<div className="text-2xl font-bold text-gray-900 mt-1">{scan.total_flights}</div>
</div>
</div>
{/* Routes Table */}
<div className="bg-white rounded-lg shadow overflow-hidden">
<div className="px-6 py-4 border-b border-gray-200">
<h3 className="text-lg font-semibold text-gray-900">Routes Found</h3>
</div>
{routes.length === 0 ? (
<div className="px-6 py-12 text-center">
{scan.status === 'completed' ? (
<div>
<p className="text-gray-500 text-lg">No routes found</p>
<p className="text-gray-400 text-sm mt-2">No flights available for the selected route and dates.</p>
</div>
) : scan.status === 'failed' ? (
<div>
<p className="text-red-500 text-lg">Scan failed</p>
{scan.error_message && (
<p className="text-gray-500 text-sm mt-2">{scan.error_message}</p>
)}
</div>
) : (
<div>
<div className="inline-block animate-spin rounded-full h-8 w-8 border-b-2 border-blue-500 mb-4"></div>
<p className="text-gray-500 text-lg">Scanning in progress...</p>
<p className="text-gray-400 text-sm mt-2">
Routes will appear here as they are discovered.
</p>
</div>
)}
</div>
) : (
<>
<div className="overflow-x-auto">
<table className="w-full">
<thead className="bg-gray-50">
<tr>
<th
className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider cursor-pointer hover:bg-gray-100"
onClick={() => handleSort('destination')}
>
Destination {sortField === 'destination' && (sortDirection === 'asc' ? '↑' : '↓')}
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
City
</th>
<th
className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider cursor-pointer hover:bg-gray-100"
onClick={() => handleSort('flight_count')}
>
Flights {sortField === 'flight_count' && (sortDirection === 'asc' ? '↑' : '↓')}
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Airlines
</th>
<th
className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider cursor-pointer hover:bg-gray-100"
onClick={() => handleSort('min_price')}
>
Min Price {sortField === 'min_price' && (sortDirection === 'asc' ? '↑' : '↓')}
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Avg Price
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Max Price
</th>
</tr>
</thead>
<tbody className="bg-white divide-y divide-gray-200">
{routes.map((route) => (
<Fragment key={route.id}>
<tr
key={route.id}
className="hover:bg-gray-50 cursor-pointer"
onClick={() => toggleFlights(route.destination)}
>
<td className="px-6 py-4 whitespace-nowrap">
<div className="flex items-center gap-2">
<span className="text-gray-400 text-xs">
{expandedRoute === route.destination ? '▼' : '▶'}
</span>
<div>
<div className="font-medium text-gray-900">{route.destination}</div>
<div className="text-sm text-gray-500">{route.destination_name}</div>
</div>
</div>
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{route.destination_city || 'N/A'}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-900">
{route.flight_count}
</td>
<td className="px-6 py-4 text-sm text-gray-500">
<div className="max-w-xs truncate">
{route.airlines.join(', ')}
</div>
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm font-medium text-green-600">
{formatPrice(route.min_price)}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{formatPrice(route.avg_price)}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{formatPrice(route.max_price)}
</td>
</tr>
{expandedRoute === route.destination && (
<tr key={`${route.id}-flights`}>
<td colSpan={7} className="px-0 py-0 bg-gray-50">
{loadingFlights === route.destination ? (
<div className="px-8 py-4 text-sm text-gray-500">Loading flights...</div>
) : (
<table className="w-full text-sm">
<thead>
<tr className="bg-gray-100 text-xs text-gray-500 uppercase">
<th className="px-8 py-2 text-left">Date</th>
<th className="px-4 py-2 text-left">Airline</th>
<th className="px-4 py-2 text-left">Departure</th>
<th className="px-4 py-2 text-left">Arrival</th>
<th className="px-4 py-2 text-left font-semibold text-green-700">Price</th>
</tr>
</thead>
<tbody>
{(flightsByDest[route.destination] || []).map((f) => (
<tr key={f.id} className="border-t border-gray-200 hover:bg-white">
<td className="px-8 py-2 text-gray-700">{f.date}</td>
<td className="px-4 py-2 text-gray-600">{f.airline || '—'}</td>
<td className="px-4 py-2 text-gray-600">{f.departure_time || '—'}</td>
<td className="px-4 py-2 text-gray-600">{f.arrival_time || '—'}</td>
<td className="px-4 py-2 font-medium text-green-600">
{f.price != null ? `€${f.price.toFixed(2)}` : '—'}
</td>
</tr>
))}
{(flightsByDest[route.destination] || []).length === 0 && (
<tr>
<td colSpan={5} className="px-8 py-3 text-gray-400 text-center">
No flight details available
</td>
</tr>
)}
</tbody>
</table>
)}
</td>
</tr>
)}
</Fragment>
))}
</tbody>
</table>
</div>
{/* Pagination */}
{totalPages > 1 && (
<div className="px-6 py-4 border-t border-gray-200 flex justify-between items-center">
<div className="text-sm text-gray-500">
Page {page} of {totalPages}
</div>
<div className="flex space-x-2">
<button
onClick={() => setPage(page - 1)}
disabled={page === 1}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Previous
</button>
<button
onClick={() => setPage(page + 1)}
disabled={page === totalPages}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Next
</button>
</div>
</div>
)}
</>
)}
</div>
</div>
);
}

299
flight-comparator/frontend/src/pages/Scans.tsx Normal file
View File

@@ -0,0 +1,299 @@
import { useState } from 'react';
import { scanApi } from '../api';
import type { CreateScanRequest } from '../api';
import AirportSearch from '../components/AirportSearch';
export default function Scans() {
const [destinationMode, setDestinationMode] = useState<'country' | 'airports'>('country');
const [formData, setFormData] = useState<CreateScanRequest>({
origin: '',
country: '',
window_months: 3,
seat_class: 'economy',
adults: 1,
});
const [selectedAirports, setSelectedAirports] = useState<string[]>([]);
const [loading, setLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const [success, setSuccess] = useState<string | null>(null);
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError(null);
setSuccess(null);
setLoading(true);
try {
// Validate airports mode has at least one airport selected
if (destinationMode === 'airports' && selectedAirports.length === 0) {
setError('Please add at least one destination airport');
setLoading(false);
return;
}
// Build request based on destination mode
const requestData: any = {
origin: formData.origin,
window_months: formData.window_months,
seat_class: formData.seat_class,
adults: formData.adults,
};
if (destinationMode === 'country') {
requestData.country = formData.country;
} else {
requestData.destinations = selectedAirports;
}
const response = await scanApi.create(requestData);
setSuccess(`Scan created successfully! ID: ${response.data.id}`);
// Reset form
setFormData({
origin: '',
country: '',
window_months: 3,
seat_class: 'economy',
adults: 1,
});
setSelectedAirports([]);
// Redirect to dashboard after 2 seconds
setTimeout(() => {
window.location.href = '/';
}, 2000);
} catch (err: any) {
const errorMessage = err.response?.data?.message || 'Failed to create scan';
setError(errorMessage);
} finally {
setLoading(false);
}
};
const handleChange = (
e: React.ChangeEvent<HTMLInputElement | HTMLSelectElement>
) => {
const { name, value } = e.target;
setFormData((prev) => ({
...prev,
[name]: name === 'adults' || name === 'window_months' ? parseInt(value) : value,
}));
};
return (
<div>
<h2 className="text-2xl font-bold text-gray-900 mb-6">Create New Scan</h2>
<div className="bg-white rounded-lg shadow p-6 max-w-2xl">
<form onSubmit={handleSubmit} className="space-y-6">
{/* Origin Airport */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">
Origin Airport (IATA Code)
</label>
<AirportSearch
value={formData.origin}
onChange={(value) => setFormData((prev) => ({ ...prev, origin: value }))}
placeholder="e.g., BDS, MUC, FRA"
/>
<p className="mt-1 text-sm text-gray-500">
Enter 3-letter IATA code (e.g., BDS for Brindisi)
</p>
</div>
{/* Destination Mode Toggle */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-3">
Destination Mode
</label>
<div className="flex space-x-2 mb-4">
<button
type="button"
onClick={() => setDestinationMode('country')}
className={`flex-1 px-4 py-2 text-sm font-medium rounded-md ${
destinationMode === 'country'
? 'bg-blue-500 text-white'
: 'bg-gray-100 text-gray-700 hover:bg-gray-200'
}`}
>
Search by Country
</button>
<button
type="button"
onClick={() => setDestinationMode('airports')}
className={`flex-1 px-4 py-2 text-sm font-medium rounded-md ${
destinationMode === 'airports'
? 'bg-blue-500 text-white'
: 'bg-gray-100 text-gray-700 hover:bg-gray-200'
}`}
>
Search by Airports
</button>
</div>
{/* Country Mode */}
{destinationMode === 'country' ? (
<div>
<label htmlFor="country" className="block text-sm font-medium text-gray-700 mb-2">
Destination Country (2-letter code)
</label>
<input
type="text"
id="country"
name="country"
value={formData.country}
onChange={handleChange}
maxLength={2}
required
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
placeholder="e.g., DE, IT, ES"
/>
<p className="mt-1 text-sm text-gray-500">
ISO 2-letter country code (e.g., DE for Germany)
</p>
</div>
) : (
/* Airports Mode */
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">
Destination Airports
</label>
<div className="space-y-2">
<AirportSearch
value=""
onChange={(code) => {
if (code && code.length === 3 && !selectedAirports.includes(code)) {
setSelectedAirports([...selectedAirports, code]);
}
}}
clearAfterSelect
required={false}
placeholder="Search and add airports..."
/>
{/* Selected airports list */}
{selectedAirports.length > 0 && (
<div className="flex flex-wrap gap-2 mt-2">
{selectedAirports.map((code) => (
<div
key={code}
className="inline-flex items-center px-3 py-1 bg-blue-100 text-blue-800 rounded-full text-sm"
>
<span className="font-medium">{code}</span>
<button
type="button"
onClick={() => setSelectedAirports(selectedAirports.filter((c) => c !== code))}
className="ml-2 text-blue-600 hover:text-blue-800"
>
×
</button>
</div>
))}
</div>
)}
<p className="text-sm text-gray-500">
{selectedAirports.length === 0
? 'Search and add destination airports (up to 50)'
: `${selectedAirports.length} airport(s) selected`}
</p>
</div>
</div>
)}
</div>
{/* Search Window */}
<div>
<label htmlFor="window_months" className="block text-sm font-medium text-gray-700 mb-2">
Search Window (months)
</label>
<input
type="number"
id="window_months"
name="window_months"
value={formData.window_months}
onChange={handleChange}
min={1}
max={12}
required
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
<p className="mt-1 text-sm text-gray-500">
Number of months to search (1-12)
</p>
</div>
{/* Seat Class */}
<div>
<label htmlFor="seat_class" className="block text-sm font-medium text-gray-700 mb-2">
Seat Class
</label>
<select
id="seat_class"
name="seat_class"
value={formData.seat_class}
onChange={handleChange}
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
>
<option value="economy">Economy</option>
<option value="premium">Premium Economy</option>
<option value="business">Business</option>
<option value="first">First Class</option>
</select>
</div>
{/* Number of Adults */}
<div>
<label htmlFor="adults" className="block text-sm font-medium text-gray-700 mb-2">
Number of Adults
</label>
<input
type="number"
id="adults"
name="adults"
value={formData.adults}
onChange={handleChange}
min={1}
max={9}
required
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
<p className="mt-1 text-sm text-gray-500">
Number of adult passengers (1-9)
</p>
</div>
{/* Error Message */}
{error && (
<div className="bg-red-50 border border-red-200 text-red-700 px-4 py-3 rounded">
{error}
</div>
)}
{/* Success Message */}
{success && (
<div className="bg-green-50 border border-green-200 text-green-700 px-4 py-3 rounded">
{success}
</div>
)}
{/* Submit Button */}
<div className="flex justify-end space-x-3">
<button
type="button"
onClick={() => window.location.href = '/'}
className="px-4 py-2 border border-gray-300 rounded-md text-sm font-medium text-gray-700 hover:bg-gray-50"
>
Cancel
</button>
<button
type="submit"
disabled={loading}
className="px-4 py-2 bg-blue-500 hover:bg-blue-600 text-white rounded-md text-sm font-medium disabled:opacity-50 disabled:cursor-not-allowed"
>
{loading ? 'Creating...' : 'Create Scan'}
</button>
</div>
</form>
</div>
</div>
);
}

11
flight-comparator/frontend/tailwind.config.js Normal file
View File

@@ -0,0 +1,11 @@
/** @type {import('tailwindcss').Config} */
export default {
content: [
"./index.html",
"./src/**/*.{js,ts,jsx,tsx}",
],
theme: {
extend: {},
},
plugins: [],
}

20
flight-comparator/frontend/vite.config.ts Normal file
View File

@@ -0,0 +1,20 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
// https://vite.dev/config/
export default defineConfig({
plugins: [react()],
server: {
port: 5173,
proxy: {
'/api': {
target: 'http://localhost:8000',
changeOrigin: true,
},
'/health': {
target: 'http://localhost:8000',
changeOrigin: true,
}
}
}
})

411
flight-comparator/main.py Executable file
View File

@@ -0,0 +1,411 @@
#!/usr/bin/env python3
"""
Flight Airport Comparator CLI
Compares direct flight options from multiple airports in a country to a single destination.
Supports both single-date queries and seasonal scanning across multiple months.
"""
import asyncio
import time
import sys
from typing import Optional
try:
import click
except ImportError:
print("Error: click library not installed. Install with: pip install click")
sys.exit(1)
from date_resolver import resolve_dates, resolve_dates_daily, detect_new_connections, SEARCH_WINDOW_MONTHS
from airports import resolve_airport_list, download_and_build_airport_data
try:
from searcher_v3 import search_multiple_routes
print("✓ Using fast-flights v3.0rc1 with SOCS cookie integration")
except ImportError:
try:
from searcher import search_multiple_routes
print("⚠️ Using legacy searcher (v2.2) - consider upgrading to v3.0rc1")
except ImportError:
print("✗ No searcher module found!")
sys.exit(1)
from formatter import format_table_single_date, format_table_seasonal, format_json, format_csv
from progress import SearchProgress
@click.command()
@click.option('--to', 'destination', help='Destination airport IATA code (e.g., JFK)')
@click.option('--to-country', 'destination_country', help='Destination country ISO code for reverse search (e.g., DE, US)')
@click.option('--country', help='Origin country ISO code (e.g., DE, US)')
@click.option('--date', help='Departure date YYYY-MM-DD. Omit for seasonal scan.')
@click.option('--window', default=SEARCH_WINDOW_MONTHS, type=int, help=f'Months to scan in seasonal mode (default: {SEARCH_WINDOW_MONTHS})')
@click.option('--daily-scan', is_flag=True, help='Scan every day (Mon-Sun) instead of just the 15th of each month')
@click.option('--start-date', help='Start date for daily scan (YYYY-MM-DD). Default: tomorrow')
@click.option('--end-date', help='End date for daily scan (YYYY-MM-DD). Default: start + window months')
@click.option('--seat', default='economy', type=click.Choice(['economy', 'premium', 'business', 'first']), help='Cabin class')
@click.option('--adults', default=1, type=int, help='Number of passengers')
@click.option('--sort', default='price', type=click.Choice(['price', 'duration']), help='Sort order')
@click.option('--from', 'from_airports', help='Comma-separated IATA codes (overrides --country)')
@click.option('--top', default=3, type=int, help='Max results per airport')
@click.option('--output', default='table', type=click.Choice(['table', 'json', 'csv']), help='Output format')
@click.option('--workers', default=5, type=int, help='Concurrency level')
@click.option('--cache-threshold', default=24, type=int, help='Cache validity in hours (default: 24)')
@click.option('--no-cache', is_flag=True, help='Disable cache, force fresh API queries')
@click.option('--dry-run', is_flag=True, help='List airports and dates without API calls')
def main(
destination: Optional[str],
destination_country: Optional[str],
country: Optional[str],
date: Optional[str],
window: int,
daily_scan: bool,
start_date: Optional[str],
end_date: Optional[str],
seat: str,
adults: int,
sort: str,
from_airports: Optional[str],
top: int,
output: str,
workers: int,
cache_threshold: int,
no_cache: bool,
dry_run: bool,
):
"""
Flight Airport Comparator - Find the best departure or arrival airport.
TWO MODES:
1. NORMAL: Multiple origins → Single destination
Compares flights from all airports in a country to one destination
2. REVERSE: Single origin → Multiple destinations
Compares flights from one airport to all airports in a country
Supports seasonal scanning to discover new routes and price trends.
Uses SQLite caching to reduce API calls and avoid rate limiting.
SCANNING STRATEGIES:
- Single date: --date YYYY-MM-DD (one specific day)
- Seasonal: Omit --date (queries 15th of each month for N months)
- Daily: --daily-scan (queries EVERY day Mon-Sun for N months)
Examples:
# NORMAL MODE: Country to single destination
python main.py --to JFK --country DE --date 2026-06-15
python main.py --to JFK --from FRA,MUC,BER --date 2026-06-15
# REVERSE MODE: Single airport to country
python main.py --from BDS --to-country DE --date 2026-06-15
python main.py --from BDS --to-country DE # Seasonal scan
# Seasonal scan (6 months, one day per month)
python main.py --to JFK --country DE
# Daily scan (every day for 3 months)
python main.py --from BDS --to DUS --daily-scan --window 3
# Daily scan with custom date range
python main.py --from BDS --to-country DE --daily-scan --start-date 2026-04-01 --end-date 2026-06-30
# Force fresh queries (ignore cache)
python main.py --to JFK --country DE --no-cache
# Use 48-hour cache threshold
python main.py --to JFK --country DE --cache-threshold 48
# Dry run to preview scan scope
python main.py --to JFK --country DE --dry-run
"""
start_time = time.time()
# Validate inputs - determine search mode
# Mode 1: Normal (many origins → single destination)
# Mode 2: Reverse (single origin → many destinations)
if destination and destination_country:
click.echo("Error: Cannot use both --to and --to-country. Choose one.", err=True)
sys.exit(1)
if not destination and not destination_country:
click.echo("Error: Either --to (single destination) or --to-country (destination country) must be provided", err=True)
sys.exit(1)
# Determine mode
reverse_mode = destination_country is not None
if reverse_mode:
# Reverse mode: single origin → multiple destinations
if not from_airports:
click.echo("Error: Reverse mode (--to-country) requires --from with a single airport", err=True)
sys.exit(1)
if ',' in from_airports:
click.echo("Error: Reverse mode requires a single origin airport in --from (no commas)", err=True)
sys.exit(1)
if country:
click.echo("Warning: --country is ignored in reverse mode (using --to-country instead)", err=True)
else:
# Normal mode: multiple origins → single destination
if not country and not from_airports:
click.echo("Error: Either --country or --from must be provided for origin airports", err=True)
sys.exit(1)
# Ensure airport data exists
try:
download_and_build_airport_data()
except Exception as e:
click.echo(f"Error building airport data: {e}", err=True)
sys.exit(1)
# Resolve airport list and routes based on mode
if reverse_mode:
# Reverse mode: single origin → multiple destinations in country
origin = from_airports # Single airport code
try:
destination_airports = resolve_airport_list(destination_country, None)
except ValueError as e:
click.echo(f"Error: {e}", err=True)
sys.exit(1)
airports = destination_airports
search_label = f"{origin} → {destination_country}"
location_label = destination_country
else:
# Normal mode: multiple origins → single destination
try:
origin_airports = resolve_airport_list(country, from_airports)
except ValueError as e:
click.echo(f"Error: {e}", err=True)
sys.exit(1)
airports = origin_airports
search_label = f"{country or 'Custom'} → {destination}"
location_label = country or 'Custom'
# Resolve dates
if date:
# Single date mode - explicit date provided
dates = [date]
elif daily_scan:
# Daily scan mode - query every day in the range
dates = resolve_dates_daily(start_date, end_date, window)
click.echo(f"Daily scan mode: {len(dates)} days from {dates[0]} to {dates[-1]}")
else:
# Seasonal mode - query one day per month (default: 15th)
dates = resolve_dates(None, window)
# Dry run mode - just show what would be scanned
if dry_run:
click.echo()
click.echo(f"Dry run: {search_label}")
click.echo(f"Mode: {'REVERSE (one → many)' if reverse_mode else 'NORMAL (many → one)'}")
click.echo()
click.echo(f"Airports to scan ({len(airports)}):")
for airport in airports[:10]:
click.echo(f" • {airport['iata']} - {airport['name']} ({airport.get('city', '')})")
if len(airports) > 10:
click.echo(f" ... and {len(airports) - 10} more")
click.echo()
click.echo(f"Dates to scan ({len(dates)}):")
for d in dates:
click.echo(f" • {d}")
click.echo()
click.echo(f"Total API calls: {len(airports)} airports × {len(dates)} dates = {len(airports) * len(dates)} requests")
click.echo(f"Estimated time: ~{(len(airports) * len(dates) * 1.0 / workers):.0f}s at {workers} workers")
click.echo()
return
# Build route list (airport × date combinations)
routes = []
if reverse_mode:
# Reverse: from single origin to each destination airport
for airport in airports:
for query_date in dates:
routes.append((from_airports, airport['iata'], query_date))
else:
# Normal: from each origin airport to single destination
for airport in airports:
for query_date in dates:
routes.append((airport['iata'], destination, query_date))
click.echo()
click.echo(f"Searching {len(routes)} routes ({len(airports)} airports × {len(dates)} dates)...")
click.echo()
# Execute searches (with caching and progress display)
use_cache = not no_cache
try:
with SearchProgress(total_routes=len(routes), show_progress=True) as progress:
def progress_callback(origin, dest, date, status, count, error=None):
progress.update(origin, dest, date, status, count, error)
results = asyncio.run(
search_multiple_routes(
routes,
seat_class=seat,
adults=adults,
max_workers=workers,
cache_threshold_hours=cache_threshold,
use_cache=use_cache,
progress_callback=progress_callback,
)
)
except Exception as e:
click.echo(f"Error during search: {e}", err=True)
sys.exit(1)
elapsed_time = time.time() - start_time
# Process results
if len(dates) == 1:
# Single-date mode
single_date = dates[0]
# Group by airport
results_by_airport = {}
if reverse_mode:
# In reverse mode, results are keyed by (origin, destination, date)
# Group by destination
for (origin, dest, query_date), flights in results.items():
if query_date == single_date and flights:
if dest not in results_by_airport:
results_by_airport[dest] = []
results_by_airport[dest].extend(flights)
# Sort and limit each destination's flights
for dest in results_by_airport:
sorted_flights = sorted(
results_by_airport[dest],
key=lambda f: f.get('price', 999999) if sort == 'price' else f.get('duration_minutes', 999999)
)
results_by_airport[dest] = sorted_flights[:top]
else:
# Normal mode: group by origin
for (origin, dest, query_date), flights in results.items():
if query_date == single_date:
if flights: # Only include if there are flights
# Take top N flights
sorted_flights = sorted(
flights,
key=lambda f: f.get('price', 999999) if sort == 'price' else f.get('duration_minutes', 999999)
)
results_by_airport[origin] = sorted_flights[:top]
else:
results_by_airport[origin] = []
# Format output
if output == 'json':
format_json(results_by_airport)
elif output == 'csv':
format_csv(results_by_airport)
else: # table
# Determine what to show in the table header
if reverse_mode:
display_destination = destination_country
display_origin = from_airports
else:
display_destination = destination
display_origin = country or 'Custom'
format_table_single_date(
results_by_airport,
display_destination if not reverse_mode else from_airports,
display_origin if not reverse_mode else destination_country,
single_date,
seat,
sort,
len(airports),
elapsed_time,
)
else:
# Seasonal mode
results_by_month = {}
if reverse_mode:
# In reverse mode, group by destination airport
for (origin, dest, query_date), flights in results.items():
month_key = query_date[:7]
if month_key not in results_by_month:
results_by_month[month_key] = {}
if flights:
if dest not in results_by_month[month_key]:
results_by_month[month_key][dest] = []
results_by_month[month_key][dest].extend(flights)
# Sort and limit flights for each destination
for month_key in results_by_month:
for dest in results_by_month[month_key]:
sorted_flights = sorted(
results_by_month[month_key][dest],
key=lambda f: f.get('price', 999999)
)
results_by_month[month_key][dest] = sorted_flights[:top]
else:
# Normal mode: group by origin
for (origin, dest, query_date), flights in results.items():
month_key = query_date[:7]
if month_key not in results_by_month:
results_by_month[month_key] = {}
if flights:
sorted_flights = sorted(flights, key=lambda f: f.get('price', 999999))
results_by_month[month_key][origin] = sorted_flights[:top]
# Detect new connections
# Convert to format expected by detect_new_connections
monthly_flights_for_detection = {}
for month_key, airports_dict in results_by_month.items():
flights_list = []
for airport_code, flights in airports_dict.items():
for flight in flights:
flights_list.append({
'origin': flight['origin'],
'destination': flight['destination'],
})
monthly_flights_for_detection[month_key] = flights_list
new_connections = detect_new_connections(monthly_flights_for_detection)
# Format output
if output == 'json':
format_json({
'results_by_month': results_by_month,
'new_connections': new_connections,
})
elif output == 'csv':
# Flatten seasonal results for CSV
flattened = {}
for month_key, airports_dict in results_by_month.items():
for airport_code, flights in airports_dict.items():
key = f"{airport_code}_{month_key}"
flattened[key] = flights
format_csv(flattened)
else: # table
# Determine what to show in the table header
if reverse_mode:
display_destination = destination_country
display_origin = from_airports
else:
display_destination = destination
display_origin = country or 'Custom'
format_table_seasonal(
results_by_month,
new_connections,
display_destination if not reverse_mode else f"from {from_airports}",
display_origin if not reverse_mode else destination_country,
seat,
len(airports),
elapsed_time,
)
if __name__ == '__main__':
main()

55
flight-comparator/nginx.conf Normal file
View File

@@ -0,0 +1,55 @@
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/json application/xml+rss;
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# API proxy
location /api/ {
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_cache_bypass $http_upgrade;
}
# Health check endpoint proxy
location /health {
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
}
# Static files with caching
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
# SPA fallback - serve index.html for all routes
location / {
try_files $uri $uri/ /index.html;
}
# Custom error pages
error_page 404 /index.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}

144
flight-comparator/progress.py Normal file
View File

@@ -0,0 +1,144 @@
"""
Live progress display for flight searches.
Shows a real-time table of search progress with cache hits, API calls, and results.
"""
from datetime import datetime
from collections import defaultdict
try:
from rich.live import Live
from rich.table import Table
from rich.console import Console
HAS_RICH = True
except ImportError:
HAS_RICH = False
class SearchProgress:
"""Track and display search progress in real-time."""
def __init__(self, total_routes: int, show_progress: bool = True):
self.total_routes = total_routes
self.show_progress = show_progress
self.completed = 0
self.cache_hits = 0
self.api_calls = 0
self.errors = 0
self.flights_found = 0
self.start_time = datetime.now()
# Track results by origin airport
self.results_by_origin = defaultdict(int)
# For live display
self.live = None
self.console = Console() if HAS_RICH else None
def __enter__(self):
"""Start live display."""
if self.show_progress and HAS_RICH:
self.live = Live(self._generate_table(), refresh_per_second=4, console=self.console)
self.live.__enter__()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""Stop live display."""
if self.live:
self.live.__exit__(exc_type, exc_val, exc_tb)
# Print final summary
self._print_summary()
def update(self, origin: str, destination: str, date: str, status: str, flights_count: int, error: str = None):
"""
Update progress with search result.
Args:
origin: Origin airport code
destination: Destination airport code
date: Search date
status: One of 'cache_hit', 'api_success', 'error'
flights_count: Number of flights found
error: Error message if status is 'error'
"""
self.completed += 1
if status == "cache_hit":
self.cache_hits += 1
elif status == "api_success":
self.api_calls += 1
elif status == "error":
self.errors += 1
if flights_count > 0:
self.flights_found += flights_count
self.results_by_origin[origin] += flights_count
# Update live display
if self.live:
self.live.update(self._generate_table())
elif self.show_progress and not HAS_RICH:
# Fallback to simple text progress
self._print_simple_progress()
def _generate_table(self) -> Table:
"""Generate Rich table with current progress."""
table = Table(title="🔍 Flight Search Progress", title_style="bold cyan")
table.add_column("Metric", style="cyan", no_wrap=True)
table.add_column("Value", style="green", justify="right")
# Progress
progress_pct = (self.completed / self.total_routes * 100) if self.total_routes > 0 else 0
table.add_row("Progress", f"{self.completed}/{self.total_routes} ({progress_pct:.1f}%)")
# Cache performance
table.add_row("💾 Cache Hits", str(self.cache_hits))
table.add_row("🌐 API Calls", str(self.api_calls))
if self.errors > 0:
table.add_row("⚠️ Errors", str(self.errors), style="yellow")
# Results
table.add_row("✈️ Flights Found", str(self.flights_found), style="bold green")
airports_with_flights = len([c for c in self.results_by_origin.values() if c > 0])
table.add_row("🛫 Airports w/ Flights", str(airports_with_flights))
# Timing
elapsed = (datetime.now() - self.start_time).total_seconds()
table.add_row("⏱️ Elapsed", f"{elapsed:.1f}s")
if self.completed > 0 and elapsed > 0:
rate = self.completed / elapsed
remaining = (self.total_routes - self.completed) / rate if rate > 0 else 0
table.add_row("⏳ Est. Remaining", f"{remaining:.0f}s")
return table
def _print_simple_progress(self):
"""Simple text progress for when Rich is not available."""
print(f"\rProgress: {self.completed}/{self.total_routes} | "
f"Cache: {self.cache_hits} | API: {self.api_calls} | "
f"Flights: {self.flights_found}", end="", flush=True)
def _print_summary(self):
"""Print final summary."""
elapsed = (datetime.now() - self.start_time).total_seconds()
print("\n")
print("="*60)
print("SEARCH SUMMARY")
print("="*60)
print(f"Total Routes: {self.total_routes}")
print(f"Completed: {self.completed}")
print(f"Cache Hits: {self.cache_hits} ({self.cache_hits/self.total_routes*100:.1f}%)")
print(f"API Calls: {self.api_calls}")
print(f"Errors: {self.errors}")
print(f"Flights Found: {self.flights_found}")
airports_with_flights = len([c for c in self.results_by_origin.values() if c > 0])
print(f"Airports w/ Flights: {airports_with_flights}")
print(f"Time Elapsed: {elapsed:.1f}s")
print("="*60)
print()

51
flight-comparator/pytest.ini Normal file
View File

@@ -0,0 +1,51 @@
[pytest]
# Pytest configuration for Flight Radar Web App
# Test discovery patterns
python_files = test_*.py
python_classes = Test*
python_functions = test_*
# Test directory
testpaths = tests
# Output options
addopts =
# Verbose output
-v
# Show summary of all test outcomes
-ra
# Show local variables in tracebacks
--showlocals
# Strict markers (fail on unknown markers)
--strict-markers
# Capture output (show print statements only on failure)
--capture=no
# Disable warnings summary
--disable-warnings
# Coverage options (for pytest-cov)
--cov=.
--cov-report=term-missing
--cov-report=html
--cov-config=.coveragerc
# Asyncio mode
asyncio_mode = auto
# Markers for categorizing tests
markers =
unit: Unit tests (fast, isolated)
integration: Integration tests (slower, multiple components)
slow: Slow tests (may take several seconds)
database: Tests that interact with database
api: Tests for API endpoints
validation: Tests for input validation
error_handling: Tests for error handling
rate_limit: Tests for rate limiting
pagination: Tests for pagination
# Ignore directories
norecursedirs = .git .venv venv env __pycache__ *.egg-info dist build
# Test output
console_output_style = progress

4
flight-comparator/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
click>=8.0.0
python-dateutil>=2.8.0
rich>=13.0.0
fast-flights>=3.0.0

224
flight-comparator/scan_discovered_routes.py Normal file
View File

@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""
Automated Daily Scans for Discovered Routes
Reads discovered_routes.json and runs targeted daily scans.
Much faster than scanning all airports because it only queries known routes.
Usage:
# First, discover routes
python discover_routes.py --from BDS --to-country DE --window 3
# Then, run targeted daily scans
python scan_discovered_routes.py discovered_routes.json --daily-scan
"""
import json
import subprocess
import sys
import click
from datetime import datetime
@click.command()
@click.argument('routes_file', type=click.Path(exists=True))
@click.option('--daily-scan', is_flag=True, help='Run daily scans (vs seasonal)')
@click.option('--start-date', help='Start date for daily scan (YYYY-MM-DD)')
@click.option('--end-date', help='End date for daily scan (YYYY-MM-DD)')
@click.option('--window', type=int, help='Override window months from discovery')
@click.option('--workers', default=5, type=int, help='Concurrency level (default: 5)')
@click.option('--output-dir', default='results', help='Directory to save results (default: results)')
@click.option('--dry-run', is_flag=True, help='Show what would be scanned without executing')
def scan_discovered(routes_file, daily_scan, start_date, end_date, window, workers, output_dir, dry_run):
"""
Run targeted scans on discovered routes.
Example:
python scan_discovered_routes.py discovered_routes.json --daily-scan
"""
# Load discovered routes
with open(routes_file, 'r') as f:
data = json.load(f)
origin = data['origin']
routes = data['routes']
default_window = data.get('window_months', 3)
if window is None:
window = default_window
print()
print("=" * 70)
print("TARGETED SCAN OF DISCOVERED ROUTES")
print("=" * 70)
print(f"Origin: {origin}")
print(f"Discovered routes: {len(routes)}")
print(f"Mode: {'Daily scan' if daily_scan else 'Seasonal scan'}")
if daily_scan and start_date and end_date:
print(f"Date range: {start_date} to {end_date}")
else:
print(f"Window: {window} months")
print(f"Workers: {workers}")
print()
if not routes:
print("⚠️ No routes to scan!")
print(f"Discovery file {routes_file} contains no routes with flights.")
sys.exit(1)
# Display routes to scan
print("Routes to scan:")
for i, route in enumerate(routes, 1):
dest = route['destination']
city = route['destination_city']
airlines = ', '.join(route['airlines'][:2])
if len(route['airlines']) > 2:
airlines += f" +{len(route['airlines']) - 2}"
print(f" {i}. {origin} → {dest} ({city}) - {airlines}")
print()
if dry_run:
print("=" * 70)
print("DRY RUN - Commands that would be executed:")
print("=" * 70)
print()
# Build and execute commands
results_summary = {
"scan_date": datetime.now().isoformat(),
"origin": origin,
"routes_scanned": len(routes),
"mode": "daily" if daily_scan else "seasonal",
"results": []
}
for i, route in enumerate(routes, 1):
dest = route['destination']
city = route['destination_city']
# Build command
cmd_parts = [
"python", "main.py",
"--from", origin,
"--to", dest,
]
if daily_scan:
cmd_parts.append("--daily-scan")
if start_date:
cmd_parts.extend(["--start-date", start_date])
if end_date:
cmd_parts.extend(["--end-date", end_date])
if not start_date and not end_date:
cmd_parts.extend(["--window", str(window)])
else:
cmd_parts.extend(["--window", str(window)])
cmd_parts.extend(["--workers", str(workers)])
# Add output file
output_file = f"{output_dir}/{origin}_{dest}_{'daily' if daily_scan else 'seasonal'}.json"
cmd_parts.extend(["--output", "json"])
command = " ".join(cmd_parts)
print(f"[{i}/{len(routes)}] Scanning {origin} → {dest} ({city})")
if dry_run:
print(f" Command: {command}")
print()
continue
try:
# Execute command
result = subprocess.run(
command,
shell=True,
capture_output=True,
text=True,
timeout=600 # 10 minute timeout per route
)
output = result.stdout + result.stderr
# Parse results (look for flight count)
import re
flights_match = re.search(r'Flights Found:\s+(\d+)', output)
flights_found = int(flights_match.group(1)) if flights_match else 0
# Save output to file
import os
os.makedirs(output_dir, exist_ok=True)
with open(output_file, 'w') as f:
f.write(output)
results_summary["results"].append({
"destination": dest,
"destination_city": city,
"flights_found": flights_found,
"output_file": output_file,
"success": result.returncode == 0
})
print(f" ✅ Complete - {flights_found} flights found")
print(f" 📄 Saved to: {output_file}")
except subprocess.TimeoutExpired:
print(f" ⏱️ Timeout - scan took too long")
results_summary["results"].append({
"destination": dest,
"destination_city": city,
"error": "timeout",
"success": False
})
except Exception as e:
print(f" ❌ Error: {e}")
results_summary["results"].append({
"destination": dest,
"destination_city": city,
"error": str(e),
"success": False
})
print()
if not dry_run:
# Save summary
summary_file = f"{output_dir}/scan_summary.json"
with open(summary_file, 'w') as f:
json.dump(results_summary, f, indent=2)
# Display summary
print("=" * 70)
print("SCAN SUMMARY")
print("=" * 70)
total_scanned = len(routes)
successful = sum(1 for r in results_summary["results"] if r.get("success", False))
total_flights = sum(r.get("flights_found", 0) for r in results_summary["results"])
print(f"Routes scanned: {total_scanned}")
print(f"Successful: {successful}/{total_scanned}")
print(f"Total flights found: {total_flights}")
print()
print(f"Results saved to: {output_dir}/")
print(f"Summary: {summary_file}")
print()
# Show top routes by flight count
sorted_results = sorted(
results_summary["results"],
key=lambda x: x.get("flights_found", 0),
reverse=True
)
print("Top routes by flight count:")
for route in sorted_results[:5]:
if route.get("flights_found", 0) > 0:
print(f" {origin} → {route['destination']}: {route['flights_found']} flights")
print()
if __name__ == '__main__':
scan_discovered()

311
flight-comparator/scan_processor.py Normal file
View File

@@ -0,0 +1,311 @@
"""
Scan Processor - Background worker for flight scans
This module processes pending flight scans by:
1. Querying flights using searcher_v3.py (with SOCS cookie integration)
2. Updating scan status and progress in real-time
3. Saving discovered routes to the database
Runs as async background tasks within the FastAPI application.
"""
import asyncio
import logging
from datetime import datetime, date, timedelta
from typing import Dict, List, Optional
import json
from database import get_connection
from airports import get_airports_for_country
from searcher_v3 import search_multiple_routes
logger = logging.getLogger(__name__)
async def process_scan(scan_id: int):
"""
Process a pending scan by querying flights and saving routes.
Args:
scan_id: The ID of the scan to process
This function:
1. Updates scan status to 'running'
2. Resolves destination airports from country
3. Queries flights for each destination
4. Saves routes to database
5. Updates progress counters in real-time
6. Sets final status to 'completed' or 'failed'
"""
conn = None
try:
logger.info(f"[Scan {scan_id}] Starting scan processing")
# Get scan details
conn = get_connection()
cursor = conn.cursor()
cursor.execute("""
SELECT origin, country, start_date, end_date, seat_class, adults
FROM scans
WHERE id = ?
""", (scan_id,))
row = cursor.fetchone()
if not row:
logger.error(f"[Scan {scan_id}] Scan not found in database")
return
origin, country_or_airports, start_date_str, end_date_str, seat_class, adults = row
logger.info(f"[Scan {scan_id}] Scan details: {origin} -> {country_or_airports}, {start_date_str} to {end_date_str}")
# Update status to 'running'
cursor.execute("""
UPDATE scans
SET status = 'running', updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (scan_id,))
conn.commit()
# Determine mode: country (2 letters) or specific airports (comma-separated)
try:
if len(country_or_airports) == 2 and country_or_airports.isalpha():
# Country mode: resolve airports from country code
logger.info(f"[Scan {scan_id}] Mode: Country search ({country_or_airports})")
destinations = get_airports_for_country(country_or_airports)
if not destinations:
raise ValueError(f"No airports found for country: {country_or_airports}")
destination_codes = [d['iata'] for d in destinations]
logger.info(f"[Scan {scan_id}] Found {len(destination_codes)} destination airports: {destination_codes}")
else:
# Specific airports mode: parse comma-separated list
destination_codes = [code.strip() for code in country_or_airports.split(',')]
destinations = [] # No pre-fetched airport details; fallback to IATA code as name
logger.info(f"[Scan {scan_id}] Mode: Specific airports ({len(destination_codes)} destinations: {destination_codes})")
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to resolve airports: {str(e)}")
cursor.execute("""
UPDATE scans
SET status = 'failed',
error_message = ?,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (f"Failed to resolve airports: {str(e)}", scan_id))
conn.commit()
return
# Note: Don't update total_routes yet - we'll set it after we know the actual number of route queries
# Generate dates to scan — every day in the window
start_date = datetime.strptime(start_date_str, '%Y-%m-%d').date()
end_date = datetime.strptime(end_date_str, '%Y-%m-%d').date()
dates = []
current = start_date
while current <= end_date:
dates.append(current.strftime('%Y-%m-%d'))
current += timedelta(days=1)
logger.info(f"[Scan {scan_id}] Will scan {len(dates)} dates: {dates}")
# Build routes list: [(origin, destination, date), ...]
routes_to_scan = []
for dest in destination_codes:
for scan_date in dates:
routes_to_scan.append((origin, dest, scan_date))
logger.info(f"[Scan {scan_id}] Total route queries: {len(routes_to_scan)}")
# Update total_routes with actual number of queries
cursor.execute("""
UPDATE scans
SET total_routes = ?,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (len(routes_to_scan), scan_id))
conn.commit()
# Progress callback to update database
# Signature: callback(origin, destination, date, status, count, error=None)
routes_scanned_count = 0
def progress_callback(origin: str, destination: str, date: str,
status: str, count: int, error: str = None):
nonlocal routes_scanned_count
# Increment counter for each route query (cache hit or API call)
if status in ('cache_hit', 'api_success', 'error'):
routes_scanned_count += 1
# Update progress in database
try:
progress_conn = get_connection()
progress_cursor = progress_conn.cursor()
progress_cursor.execute("""
UPDATE scans
SET routes_scanned = routes_scanned + 1,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (scan_id,))
progress_conn.commit()
progress_conn.close()
if routes_scanned_count % 10 == 0: # Log every 10 routes
logger.info(f"[Scan {scan_id}] Progress: {routes_scanned_count}/{len(routes_to_scan)} routes ({status}: {origin}→{destination})")
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to update progress: {str(e)}")
# Query flights using searcher_v3
logger.info(f"[Scan {scan_id}] Starting flight queries...")
results = await search_multiple_routes(
routes=routes_to_scan,
seat_class=seat_class or 'economy',
adults=adults or 1,
use_cache=True,
cache_threshold_hours=24,
max_workers=3, # Limit concurrency to avoid rate limiting
progress_callback=progress_callback
)
logger.info(f"[Scan {scan_id}] Flight queries complete. Processing results...")
# Group results by destination, preserving date per flight
# Structure: {dest: [(flight_dict, date), ...]}
routes_by_destination: Dict[str, List] = {}
total_flights = 0
for (orig, dest, scan_date), flights in results.items():
if dest not in routes_by_destination:
routes_by_destination[dest] = []
for flight in flights:
routes_by_destination[dest].append((flight, scan_date))
total_flights += len(flights)
logger.info(f"[Scan {scan_id}] Found {total_flights} total flights across {len(routes_by_destination)} destinations")
# Save routes and individual flights to database
routes_saved = 0
for destination, flight_date_pairs in routes_by_destination.items():
if not flight_date_pairs:
continue # Skip destinations with no flights
flights = [f for f, _ in flight_date_pairs]
# Get destination details (fall back to IATA code if not in DB)
dest_info = next((d for d in destinations if d['iata'] == destination), None)
dest_name = dest_info.get('name', destination) if dest_info else destination
dest_city = dest_info.get('city', '') if dest_info else ''
# Calculate statistics
prices = [f.get('price') for f in flights if f.get('price')]
airlines = list(set(f.get('airline') for f in flights if f.get('airline')))
if not prices:
logger.info(f"[Scan {scan_id}] Skipping {destination} - no prices available")
continue
min_price = min(prices)
max_price = max(prices)
avg_price = sum(prices) / len(prices)
# Insert route summary
cursor.execute("""
INSERT INTO routes (
scan_id, destination, destination_name, destination_city,
min_price, max_price, avg_price, flight_count, airlines
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id,
destination,
dest_name,
dest_city,
min_price,
max_price,
avg_price,
len(flights),
json.dumps(airlines)
))
# Insert individual flights
for flight, flight_date in flight_date_pairs:
if not flight.get('price'):
continue
cursor.execute("""
INSERT INTO flights (
scan_id, destination, date, airline,
departure_time, arrival_time, price, stops
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id,
destination,
flight_date,
flight.get('airline'),
flight.get('departure_time'),
flight.get('arrival_time'),
flight.get('price'),
flight.get('stops', 0),
))
routes_saved += 1
conn.commit()
# Update scan to completed
cursor.execute("""
UPDATE scans
SET status = 'completed',
total_flights = ?,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (total_flights, scan_id))
conn.commit()
logger.info(f"[Scan {scan_id}] ✅ Scan completed successfully! {routes_saved} routes saved with {total_flights} flights")
except Exception as e:
logger.error(f"[Scan {scan_id}] ❌ Scan failed with error: {str(e)}", exc_info=True)
# Update scan to failed
try:
if conn:
cursor = conn.cursor()
cursor.execute("""
UPDATE scans
SET status = 'failed',
error_message = ?,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (str(e), scan_id))
conn.commit()
except Exception as update_error:
logger.error(f"[Scan {scan_id}] Failed to update error status: {str(update_error)}")
finally:
if conn:
conn.close()
def start_scan_processor(scan_id: int):
"""
Start processing a scan as a background task.
Args:
scan_id: The ID of the scan to process
Returns:
asyncio.Task: The background task
"""
task = asyncio.create_task(process_scan(scan_id))
logger.info(f"[Scan {scan_id}] Background task created")
return task

240
flight-comparator/searcher.py Normal file
View File

@@ -0,0 +1,240 @@
"""
Flight search logic with concurrent queries.
Wraps fast-flights library with async concurrency and error handling.
"""
import asyncio
import random
import time
from typing import Optional
from datetime import datetime
try:
from fast_flights import FlightData, Passengers, get_flights
HAS_FAST_FLIGHTS = True
except ImportError:
HAS_FAST_FLIGHTS = False
print("⚠️ fast-flights not installed. Install with: pip install fast-flights")
async def search_direct_flights(
origin: str,
destination: str,
date: str,
seat_class: str = "economy",
adults: int = 1,
) -> list[dict]:
"""
Search for direct flights between two airports on a specific date.
Args:
origin: Origin airport IATA code
destination: Destination airport IATA code
date: Departure date in YYYY-MM-DD format
seat_class: Cabin class (economy, business, first)
adults: Number of passengers
Returns:
List of flight dicts with keys: origin, destination, airline, departure_time,
arrival_time, duration_minutes, price, currency, stops
"""
if not HAS_FAST_FLIGHTS:
return []
try:
# Add random delay to avoid rate limiting
await asyncio.sleep(random.uniform(0.5, 1.5))
# Run the synchronous get_flights in a thread pool
result = await asyncio.to_thread(
_search_flights_sync,
origin,
destination,
date,
seat_class,
adults,
)
return result
except Exception as e:
# Log but don't crash - return empty results
print(f"⚠️ {origin}->{destination} on {date}: {type(e).__name__}: {e}")
return []
def _search_flights_sync(
origin: str,
destination: str,
date: str,
seat_class: str,
adults: int,
) -> list[dict]:
"""
Synchronous flight search wrapper.
Called via asyncio.to_thread to avoid blocking the event loop.
"""
# Map seat class to fast-flights format
seat_map = {
"economy": "economy",
"premium": "premium-economy",
"business": "business",
"first": "first",
}
seat_string = seat_map.get(seat_class.lower(), "economy")
# Create flight data object
flight = FlightData(
date=date,
from_airport=origin,
to_airport=destination,
max_stops=0, # Direct flights only
)
passengers = Passengers(adults=adults)
# Query flights with common mode (tries common first, fallback if needed)
try:
result = get_flights(
flight_data=[flight], # Must be a list
trip='one-way',
passengers=passengers,
seat=seat_string,
fetch_mode='common', # Use common mode instead of fallback
)
except Exception as e:
# Retry once after a delay
time.sleep(2)
try:
result = get_flights(
flight_data=[flight],
trip='one-way',
passengers=passengers,
seat=seat_string,
fetch_mode='common',
)
except Exception as retry_error:
raise retry_error from e
# Filter to direct flights only and convert to our format
flights = []
if not result or not hasattr(result, 'flights'):
return flights
for flight_option in result.flights:
# Check if direct (0 stops)
# fast-flights may structure this differently, so we check multiple attributes
is_direct = False
# Method 1: Check stops attribute
if hasattr(flight_option, 'stops') and flight_option.stops == 0:
is_direct = True
# Method 2: Check if there's only one flight segment
if hasattr(flight_option, 'flight') and len(flight_option.flight) == 1:
is_direct = True
# Method 3: Check if departure and arrival airports match our query
# (no layovers in between)
if hasattr(flight_option, 'departure_airport') and hasattr(flight_option, 'arrival_airport'):
if (flight_option.departure_airport == origin and
flight_option.arrival_airport == destination):
is_direct = True
if not is_direct:
continue
# Extract flight details
flight_dict = {
"origin": origin,
"destination": destination,
"airline": getattr(flight_option, 'airline', 'Unknown'),
"departure_time": getattr(flight_option, 'departure_time', ''),
"arrival_time": getattr(flight_option, 'arrival_time', ''),
"duration_minutes": _parse_duration(getattr(flight_option, 'duration', '')),
"price": getattr(flight_option, 'price', 0),
"currency": getattr(flight_option, 'currency', 'USD'),
"stops": 0,
}
flights.append(flight_dict)
return flights
def _parse_duration(duration_str: str) -> int:
"""
Parse duration string to minutes.
Handles formats like "9h 30m", "9h", "90m"
Args:
duration_str: Duration string
Returns:
Total duration in minutes
"""
if not duration_str:
return 0
total_minutes = 0
# Extract hours
if 'h' in duration_str:
try:
hours_part = duration_str.split('h')[0].strip()
total_minutes += int(hours_part) * 60
except (ValueError, IndexError):
pass
# Extract minutes
if 'm' in duration_str:
try:
minutes_part = duration_str.split('h')[-1].split('m')[0].strip()
total_minutes += int(minutes_part)
except (ValueError, IndexError):
pass
return total_minutes
async def search_multiple_routes(
routes: list[tuple[str, str, str]],
seat_class: str = "economy",
adults: int = 1,
max_workers: int = 5,
) -> dict[tuple[str, str], list[dict]]:
"""
Search multiple routes concurrently.
Args:
routes: List of (origin, destination, date) tuples
seat_class: Cabin class
adults: Number of passengers
max_workers: Maximum concurrent requests
Returns:
Dict mapping (origin, date) tuples to lists of flight dicts
"""
# Create semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_workers)
async def search_with_semaphore(origin: str, destination: str, date: str):
async with semaphore:
return (origin, date), await search_direct_flights(
origin, destination, date, seat_class, adults
)
# Execute all searches concurrently (but limited by semaphore)
tasks = [
search_with_semaphore(origin, destination, date)
for origin, destination, date in routes
]
results = await asyncio.gather(*tasks)
# Convert to dict
return dict(results)

347
flight-comparator/searcher_v3.py Normal file
View File

@@ -0,0 +1,347 @@
"""
Flight search logic with concurrent queries using fast-flights v3.0rc1.
Includes SOCS cookie integration to bypass Google consent page.
Includes SQLite caching to reduce API calls and avoid rate limiting.
"""
import asyncio
import random
import time
from typing import Optional
from datetime import datetime
try:
from cache import get_cached_results, save_results
HAS_CACHE = True
except ImportError:
HAS_CACHE = False
print("⚠️ Cache module not available - all queries will hit API")
try:
from fast_flights import FlightQuery, Passengers, get_flights, create_query
from fast_flights.integrations.base import Integration
from fast_flights.querying import Query
import primp
HAS_FAST_FLIGHTS = True
except ImportError:
HAS_FAST_FLIGHTS = False
print("⚠️ fast-flights v3.0rc1 not installed.")
print(" Install with: pip install --upgrade git+https://github.com/AWeirdDev/flights.git")
class SOCSCookieIntegration(Integration):
"""
Custom integration that adds SOCS cookie to bypass Google consent page.
SOCS (Secure-1P_SameSite-Cookies) is Google's consent state cookie.
Cookie value from: https://github.com/AWeirdDev/flights/issues/46
This cookie tells Google that the user has accepted cookies/consent,
allowing us to bypass the consent page and get flight data directly.
"""
# SOCS cookie value - stores consent state for 13 months
SOCS_COOKIE = 'CAESHwgBEhJnd3NfMjAyNTAyMjctMF9SQzIaBXpoLUNOIAEaBgiAy6O-Bg'
def fetch_html(self, q: Query | str, /) -> str:
"""
Fetch flights HTML with SOCS cookie attached.
Args:
q: Query object or query string
Returns:
HTML response from Google Flights
"""
# Create client with browser impersonation
client = primp.Client(
impersonate="chrome_145",
impersonate_os="macos",
cookie_store=True, # Enable cookie persistence
)
# Prepare query parameters
if isinstance(q, Query):
params = q.params()
else:
params = {"q": q}
# Make request with SOCS cookie
response = client.get(
"https://www.google.com/travel/flights",
params=params,
cookies={'SOCS': self.SOCS_COOKIE},
headers={
'Accept-Language': 'en-US,en;q=0.9',
}
)
return response.text
async def search_direct_flights(
origin: str,
destination: str,
date: str,
seat_class: str = "economy",
adults: int = 1,
cache_threshold_hours: int = 24,
use_cache: bool = True,
progress_callback=None,
) -> list[dict]:
"""
Search for direct flights between two airports on a specific date.
Checks cache first; only queries API if cache miss or expired.
Args:
origin: Origin airport IATA code
destination: Destination airport IATA code
date: Departure date in YYYY-MM-DD format
seat_class: Cabin class (economy, premium, business, first)
adults: Number of passengers
cache_threshold_hours: Maximum age of cached results in hours
use_cache: Whether to use cache (set False to force fresh query)
Returns:
List of flight dicts with keys: origin, destination, airline, departure_time,
arrival_time, duration_minutes, price, currency, stops
"""
if not HAS_FAST_FLIGHTS:
return []
try:
# Check cache first (if enabled)
if use_cache and HAS_CACHE:
cached = get_cached_results(
origin, destination, date, seat_class, adults, cache_threshold_hours
)
if cached is not None:
if progress_callback:
progress_callback(origin, destination, date, "cache_hit", len(cached))
return cached
# Add random delay to avoid rate limiting
await asyncio.sleep(random.uniform(0.5, 1.5))
# Run the search in a thread pool (fast-flights is synchronous)
result = await asyncio.to_thread(
_search_flights_sync,
origin,
destination,
date,
seat_class,
adults,
)
# Save to cache
if use_cache and HAS_CACHE and result:
save_results(origin, destination, date, seat_class, adults, result)
# Report progress
if progress_callback:
progress_callback(origin, destination, date, "api_success", len(result))
return result
except Exception as e:
# Log but don't crash - return empty results
import traceback
print(f"\n=== SEARCH ERROR ===")
print(f"Query: {origin}→{destination} on {date}")
print(f"Error type: {type(e).__name__}")
print(f"Error message: {str(e)}")
print(f"Traceback:")
traceback.print_exc()
print("=" * 50)
if progress_callback:
progress_callback(origin, destination, date, "error", 0, str(e))
return []
def _search_flights_sync(
origin: str,
destination: str,
date: str,
seat_class: str,
adults: int,
) -> list[dict]:
"""
Synchronous flight search wrapper for v3 API.
Called via asyncio.to_thread to avoid blocking the event loop.
"""
# Create flight query
flights = [
FlightQuery(
date=date,
from_airport=origin,
to_airport=destination,
max_stops=0, # Direct flights only
)
]
# Create query with passengers and preferences
query = create_query(
flights=flights,
seat=seat_class,
trip="one-way",
passengers=Passengers(adults=adults),
)
# Create SOCS cookie integration
cookie_integration = SOCSCookieIntegration()
# Execute search with retry
try:
result = get_flights(query, integration=cookie_integration)
except Exception as e:
# Retry once after delay
time.sleep(2)
try:
result = get_flights(query, integration=cookie_integration)
except Exception as retry_error:
# Print detailed error for debugging
import traceback
print(f"\n=== FAST-FLIGHTS ERROR ===")
print(f"Query: {origin}→{destination} on {date}")
print(f"Error: {retry_error}")
print(f"Traceback:")
traceback.print_exc()
print("=" * 50)
raise retry_error from e
# Convert v3 API result to our standard format
flights_list = []
try:
if isinstance(result, list):
for flight_option in result:
# Each flight_option has: type, price, airlines, flights, etc.
price = getattr(flight_option, 'price', None)
airlines = getattr(flight_option, 'airlines', [])
flight_segments = getattr(flight_option, 'flights', [])
# Validate flight_segments is a non-empty list
if not flight_segments or price is None:
continue
# Handle case where flights attribute exists but is None
if not isinstance(flight_segments, list):
continue
if len(flight_segments) == 0:
continue
# Get first segment (should be only one for direct flights)
segment = flight_segments[0]
# Validate segment is not None
if segment is None:
continue
# Extract flight details
from_airport = getattr(segment, 'from_airport', None)
to_airport = getattr(segment, 'to_airport', None)
departure = getattr(segment, 'departure', None)
arrival = getattr(segment, 'arrival', None)
duration = getattr(segment, 'duration', 0)
plane_type = getattr(segment, 'plane_type', '')
# Parse departure and arrival times (handle both [H] and [H, M] formats)
dep_time = ""
arr_time = ""
if departure and hasattr(departure, 'time') and isinstance(departure.time, (list, tuple)) and len(departure.time) >= 1:
try:
hours = departure.time[0]
minutes = departure.time[1] if len(departure.time) > 1 else 0
dep_time = f"{hours:02d}:{minutes:02d}"
except (IndexError, TypeError, ValueError):
dep_time = ""
if arrival and hasattr(arrival, 'time') and isinstance(arrival.time, (list, tuple)) and len(arrival.time) >= 1:
try:
hours = arrival.time[0]
minutes = arrival.time[1] if len(arrival.time) > 1 else 0
arr_time = f"{hours:02d}:{minutes:02d}"
except (IndexError, TypeError, ValueError):
arr_time = ""
# Only add flight if we have essential data (price and times)
if price and price > 0 and dep_time and arr_time:
flight_dict = {
"origin": origin,
"destination": destination,
"airline": airlines[0] if airlines else "Unknown",
"departure_time": dep_time,
"arrival_time": arr_time,
"duration_minutes": duration,
"price": price,
"currency": "€", # fast-flights typically returns EUR for EU routes
"stops": 0,
"plane_type": plane_type,
}
flights_list.append(flight_dict)
except Exception as parse_error:
# Print detailed parsing error for debugging
import traceback
print(f"\n=== PARSING ERROR ===")
print(f"Query: {origin}→{destination} on {date}")
print(f"Error: {parse_error}")
print(f"Result type: {type(result)}")
print(f"Result: {result}")
print(f"Traceback:")
traceback.print_exc()
print("=" * 50)
# Return empty list instead of crashing
return []
return flights_list
async def search_multiple_routes(
routes: list[tuple[str, str, str]],
seat_class: str = "economy",
adults: int = 1,
max_workers: int = 5,
cache_threshold_hours: int = 24,
use_cache: bool = True,
progress_callback=None,
) -> dict[tuple[str, str, str], list[dict]]:
"""
Search multiple routes concurrently.
Checks cache for each route before querying API.
Args:
routes: List of (origin, destination, date) tuples
seat_class: Cabin class
adults: Number of passengers
max_workers: Maximum concurrent requests
cache_threshold_hours: Maximum age of cached results in hours
use_cache: Whether to use cache (set False to force fresh queries)
Returns:
Dict mapping (origin, destination, date) tuples to lists of flight dicts
"""
# Create semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_workers)
async def search_with_semaphore(origin: str, destination: str, date: str):
async with semaphore:
return (origin, destination, date), await search_direct_flights(
origin, destination, date, seat_class, adults,
cache_threshold_hours, use_cache, progress_callback
)
# Execute all searches concurrently (but limited by semaphore)
tasks = [
search_with_semaphore(origin, destination, date)
for origin, destination, date in routes
]
results = await asyncio.gather(*tasks)
# Convert to dict
return dict(results)

96
flight-comparator/tests/confirmed_flights.json Normal file
View File

@@ -0,0 +1,96 @@
{
"_meta": {
"description": "Confirmed real flights from live Google Flights queries, scraped via fast-flights v3 with SOCS cookie. Used as ground truth for integration tests.",
"source_scan_id": 54,
"origin": "BDS",
"window": "2026-02-26 to 2026-05-27",
"scraped_at": "2026-02-25",
"total_flights": 50
},
"routes": {
"BDS-FMM": {
"origin": "BDS",
"destination": "FMM",
"airline": "Ryanair",
"flight_count": 39,
"min_price": 15.0,
"max_price": 193.0,
"flights": [
{"date": "2026-04-01", "departure": "09:20", "arrival": "11:10", "price": 15.0},
{"date": "2026-03-30", "departure": "18:45", "arrival": "20:35", "price": 21.0},
{"date": "2026-04-22", "departure": "09:20", "arrival": "11:10", "price": 24.0},
{"date": "2026-04-02", "departure": "09:40", "arrival": "11:30", "price": 26.0},
{"date": "2026-04-17", "departure": "19:35", "arrival": "21:25", "price": 27.0},
{"date": "2026-04-24", "departure": "19:35", "arrival": "21:25", "price": 27.0},
{"date": "2026-03-29", "departure": "10:05", "arrival": "11:55", "price": 29.0},
{"date": "2026-05-11", "departure": "18:45", "arrival": "20:35", "price": 30.0},
{"date": "2026-04-15", "departure": "09:20", "arrival": "11:10", "price": 31.0},
{"date": "2026-05-07", "departure": "09:40", "arrival": "11:30", "price": 32.0},
{"date": "2026-04-23", "departure": "09:40", "arrival": "11:30", "price": 34.0},
{"date": "2026-04-16", "departure": "09:40", "arrival": "11:30", "price": 35.0},
{"date": "2026-05-20", "departure": "09:20", "arrival": "11:10", "price": 35.0},
{"date": "2026-04-27", "departure": "18:45", "arrival": "20:35", "price": 40.0},
{"date": "2026-05-06", "departure": "09:20", "arrival": "11:10", "price": 40.0},
{"date": "2026-04-20", "departure": "18:45", "arrival": "20:35", "price": 41.0},
{"date": "2026-04-29", "departure": "09:20", "arrival": "11:10", "price": 41.0},
{"date": "2026-05-13", "departure": "09:20", "arrival": "11:10", "price": 44.0},
{"date": "2026-04-26", "departure": "10:05", "arrival": "11:55", "price": 45.0},
{"date": "2026-05-21", "departure": "09:40", "arrival": "11:30", "price": 46.0},
{"date": "2026-04-13", "departure": "18:45", "arrival": "20:35", "price": 48.0},
{"date": "2026-05-14", "departure": "09:40", "arrival": "11:30", "price": 48.0},
{"date": "2026-05-27", "departure": "09:20", "arrival": "11:10", "price": 48.0},
{"date": "2026-04-19", "departure": "10:05", "arrival": "11:55", "price": 51.0},
{"date": "2026-04-03", "departure": "19:35", "arrival": "21:25", "price": 55.0},
{"date": "2026-04-30", "departure": "09:40", "arrival": "11:30", "price": 58.0},
{"date": "2026-05-10", "departure": "10:05", "arrival": "11:55", "price": 63.0},
{"date": "2026-04-05", "departure": "10:05", "arrival": "11:55", "price": 65.0},
{"date": "2026-04-10", "departure": "19:35", "arrival": "21:25", "price": 72.0},
{"date": "2026-04-09", "departure": "09:40", "arrival": "11:30", "price": 78.0},
{"date": "2026-05-25", "departure": "18:45", "arrival": "20:35", "price": 81.0},
{"date": "2026-05-04", "departure": "18:45", "arrival": "20:35", "price": 82.0},
{"date": "2026-05-18", "departure": "18:45", "arrival": "20:35", "price": 84.0},
{"date": "2026-04-08", "departure": "09:20", "arrival": "11:10", "price": 96.0},
{"date": "2026-05-24", "departure": "10:05", "arrival": "11:55", "price": 108.0},
{"date": "2026-05-03", "departure": "10:05", "arrival": "11:55", "price": 134.0},
{"date": "2026-04-06", "departure": "18:45", "arrival": "20:35", "price": 144.0},
{"date": "2026-04-12", "departure": "10:05", "arrival": "11:55", "price": 146.0},
{"date": "2026-05-17", "departure": "10:05", "arrival": "11:55", "price": 193.0}
],
"notes": "Ryanair operates ~5-6x/week. Two daily slots: morning (09:20/09:40/10:05) and evening (18:45/19:35). Season starts late March 2026."
},
"BDS-DUS": {
"origin": "BDS",
"destination": "DUS",
"airline": "Eurowings",
"flight_count": 11,
"min_price": 40.0,
"max_price": 270.0,
"flights": [
{"date": "2026-04-04", "departure": "09:20", "arrival": "11:40", "price": 40.0},
{"date": "2026-05-12", "departure": "19:45", "arrival": "22:05", "price": 90.0},
{"date": "2026-04-18", "departure": "11:20", "arrival": "13:40", "price": 120.0},
{"date": "2026-04-25", "departure": "11:20", "arrival": "13:40", "price": 120.0},
{"date": "2026-05-09", "departure": "11:20", "arrival": "13:40", "price": 120.0},
{"date": "2026-05-19", "departure": "19:45", "arrival": "22:05", "price": 140.0},
{"date": "2026-05-23", "departure": "11:20", "arrival": "13:40", "price": 160.0},
{"date": "2026-05-26", "departure": "19:45", "arrival": "22:05", "price": 160.0},
{"date": "2026-05-02", "departure": "11:20", "arrival": "13:40", "price": 240.0},
{"date": "2026-04-11", "departure": "09:20", "arrival": "11:40", "price": 270.0},
{"date": "2026-05-16", "departure": "11:20", "arrival": "13:40", "price": 270.0}
],
"notes": "Eurowings operates Saturdays only (verified: all 11 dates are Saturdays). Two time slots: morning (09:20 or 11:20) and evening (19:45). Cheapest in April."
}
},
"confirmed_dates_for_testing": {
"description": "Specific (origin, destination, date) tuples confirmed to return >=1 flight from the live API. Safe to use in integration tests without risk of flakiness due to no-service days.",
"entries": [
{"origin": "BDS", "destination": "FMM", "date": "2026-04-01", "min_flights": 1, "airline": "Ryanair", "price": 15.0},
{"origin": "BDS", "destination": "FMM", "date": "2026-04-15", "min_flights": 1, "airline": "Ryanair", "price": 31.0},
{"origin": "BDS", "destination": "FMM", "date": "2026-05-07", "min_flights": 1, "airline": "Ryanair", "price": 32.0},
{"origin": "BDS", "destination": "DUS", "date": "2026-04-04", "min_flights": 1, "airline": "Eurowings", "price": 40.0},
{"origin": "BDS", "destination": "DUS", "date": "2026-04-18", "min_flights": 1, "airline": "Eurowings", "price": 120.0},
{"origin": "BDS", "destination": "DUS", "date": "2026-05-09", "min_flights": 1, "airline": "Eurowings", "price": 120.0},
{"origin": "BDS", "destination": "DUS", "date": "2026-05-23", "min_flights": 1, "airline": "Eurowings", "price": 160.0}
]
}
}

195
flight-comparator/tests/conftest.py Normal file
View File

@@ -0,0 +1,195 @@
"""
Test fixtures and configuration for Flight Radar Web App tests.
This module provides reusable fixtures for testing the API.
"""
import pytest
import sqlite3
import os
import tempfile
from fastapi.testclient import TestClient
from typing import Generator
# Import the FastAPI app
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
from api_server import app, rate_limiter, log_buffer
from database import get_connection
@pytest.fixture(scope="session")
def test_db_path() -> Generator[str, None, None]:
"""Create a temporary database for testing."""
# Create temporary database
fd, path = tempfile.mkstemp(suffix=".db")
os.close(fd)
# Set environment variable to use test database
original_db = os.environ.get('DATABASE_PATH')
os.environ['DATABASE_PATH'] = path
yield path
# Cleanup
if original_db:
os.environ['DATABASE_PATH'] = original_db
else:
os.environ.pop('DATABASE_PATH', None)
try:
os.unlink(path)
except OSError:
pass
@pytest.fixture(scope="function")
def clean_database(test_db_path):
"""Provide a clean database for each test."""
# Initialize database with schema
conn = sqlite3.connect(test_db_path)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys
cursor = conn.cursor()
# Read and execute schema
schema_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'database', 'schema.sql')
with open(schema_path, 'r') as f:
schema_sql = f.read()
cursor.executescript(schema_sql)
conn.commit()
conn.close()
yield test_db_path
# Clean up all tables after test
conn = sqlite3.connect(test_db_path)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys for cleanup
cursor = conn.cursor()
cursor.execute("DELETE FROM routes")
cursor.execute("DELETE FROM scans")
# Note: flight_searches and flight_results are not in web app schema
conn.commit()
conn.close()
@pytest.fixture(scope="function")
def client(clean_database) -> TestClient:
"""Provide a test client for the FastAPI app."""
# Clear rate limiter for each test
rate_limiter.requests.clear()
# Clear log buffer for each test
log_buffer.clear()
with TestClient(app) as test_client:
yield test_client
@pytest.fixture
def sample_scan_data():
"""Provide sample scan data for testing."""
return {
"origin": "BDS",
"country": "DE",
"start_date": "2026-04-01",
"end_date": "2026-06-30",
"seat_class": "economy",
"adults": 2
}
@pytest.fixture
def sample_route_data():
"""Provide sample route data for testing."""
return {
"scan_id": 1,
"destination": "MUC",
"destination_name": "Munich Airport",
"destination_city": "Munich",
"flight_count": 45,
"airlines": '["Lufthansa", "Ryanair"]',
"min_price": 89.99,
"max_price": 299.99,
"avg_price": 150.50
}
@pytest.fixture
def create_test_scan(clean_database, sample_scan_data):
"""Create a test scan in the database."""
def _create_scan(**kwargs):
# Merge with defaults
data = {**sample_scan_data, **kwargs}
conn = sqlite3.connect(clean_database)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys
cursor = conn.cursor()
cursor.execute("""
INSERT INTO scans (origin, country, start_date, end_date, status, seat_class, adults)
VALUES (?, ?, ?, ?, ?, ?, ?)
""", (
data['origin'],
data['country'],
data['start_date'],
data['end_date'],
data.get('status', 'pending'),
data['seat_class'],
data['adults']
))
scan_id = cursor.lastrowid
conn.commit()
conn.close()
return scan_id
return _create_scan
@pytest.fixture
def create_test_route(clean_database, sample_route_data):
"""Create a test route in the database."""
def _create_route(**kwargs):
# Merge with defaults
data = {**sample_route_data, **kwargs}
conn = sqlite3.connect(clean_database)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys
cursor = conn.cursor()
cursor.execute("""
INSERT INTO routes (scan_id, destination, destination_name, destination_city,
flight_count, airlines, min_price, max_price, avg_price)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
data['scan_id'],
data['destination'],
data['destination_name'],
data['destination_city'],
data['flight_count'],
data['airlines'],
data['min_price'],
data['max_price'],
data['avg_price']
))
route_id = cursor.lastrowid
conn.commit()
conn.close()
return route_id
return _create_route
# Marker helpers for categorizing tests
def pytest_configure(config):
"""Configure custom pytest markers."""
config.addinivalue_line("markers", "unit: Unit tests (fast, isolated)")
config.addinivalue_line("markers", "integration: Integration tests (slower)")
config.addinivalue_line("markers", "slow: Slow tests")
config.addinivalue_line("markers", "database: Tests that interact with database")
config.addinivalue_line("markers", "api: Tests for API endpoints")

53
flight-comparator/tests/test_airports.py Normal file
View File

@@ -0,0 +1,53 @@
"""
Smoke tests for airports module.
"""
import sys
sys.path.insert(0, '..')
from airports import get_airports_for_country, resolve_airport_list
def test_get_airports_for_country():
"""Test loading airports for a country."""
de_airports = get_airports_for_country("DE")
assert len(de_airports) > 0
assert all('iata' in a for a in de_airports)
assert all('name' in a for a in de_airports)
assert all('city' in a for a in de_airports)
print(f"✓ Found {len(de_airports)} airports in Germany")
def test_resolve_airport_list_from_country():
"""Test resolving airport list from country."""
airports = resolve_airport_list("DE", None)
assert len(airports) > 0
print(f"✓ Resolved {len(airports)} airports from country DE")
def test_resolve_airport_list_from_custom():
"""Test resolving airport list from custom --from argument."""
airports = resolve_airport_list(None, "FRA,MUC,BER")
assert len(airports) == 3
assert airports[0]['iata'] == 'FRA'
assert airports[1]['iata'] == 'MUC'
assert airports[2]['iata'] == 'BER'
print(f"✓ Resolved custom airport list: {[a['iata'] for a in airports]}")
def test_invalid_country():
"""Test handling of invalid country code."""
try:
get_airports_for_country("XX")
assert False, "Should have raised ValueError"
except ValueError as e:
assert "not found" in str(e)
print("✓ Invalid country code raises appropriate error")
if __name__ == "__main__":
test_get_airports_for_country()
test_resolve_airport_list_from_country()
test_resolve_airport_list_from_custom()
test_invalid_country()
print("\n✅ All airports tests passed!")

363
flight-comparator/tests/test_api_endpoints.py Normal file
View File

@@ -0,0 +1,363 @@
"""
Unit tests for API endpoints.
Tests all API endpoints with various scenarios including success cases,
error cases, validation, pagination, and edge cases.
"""
import pytest
from fastapi.testclient import TestClient
@pytest.mark.unit
@pytest.mark.api
class TestHealthEndpoint:
"""Tests for the health check endpoint."""
def test_health_endpoint(self, client: TestClient):
"""Test health endpoint returns 200 OK."""
response = client.get("/health")
assert response.status_code == 200
assert response.json() == {"status": "healthy", "version": "2.0.0"}
def test_health_no_rate_limit(self, client: TestClient):
"""Test health endpoint is excluded from rate limiting."""
response = client.get("/health")
assert "x-ratelimit-limit" not in response.headers
assert "x-ratelimit-remaining" not in response.headers
@pytest.mark.unit
@pytest.mark.api
class TestAirportEndpoints:
"""Tests for airport search endpoints."""
def test_search_airports_valid(self, client: TestClient):
"""Test airport search with valid query."""
response = client.get("/api/v1/airports?q=MUC")
assert response.status_code == 200
data = response.json()
assert "data" in data
assert "pagination" in data
assert isinstance(data["data"], list)
assert len(data["data"]) > 0
# Check first result
airport = data["data"][0]
assert "iata" in airport
assert "name" in airport
assert "MUC" in airport["iata"]
def test_search_airports_pagination(self, client: TestClient):
"""Test airport search pagination."""
response = client.get("/api/v1/airports?q=airport&page=1&limit=5")
assert response.status_code == 200
data = response.json()
assert data["pagination"]["page"] == 1
assert data["pagination"]["limit"] == 5
assert len(data["data"]) <= 5
def test_search_airports_invalid_query_too_short(self, client: TestClient):
"""Test airport search with query too short."""
response = client.get("/api/v1/airports?q=M")
assert response.status_code == 422
error = response.json()
assert error["error"] == "validation_error"
def test_search_airports_rate_limit_headers(self, client: TestClient):
"""Test airport search includes rate limit headers."""
response = client.get("/api/v1/airports?q=MUC")
assert response.status_code == 200
assert "x-ratelimit-limit" in response.headers
assert "x-ratelimit-remaining" in response.headers
assert "x-ratelimit-reset" in response.headers
@pytest.mark.unit
@pytest.mark.api
@pytest.mark.database
class TestScanEndpoints:
"""Tests for scan management endpoints."""
def test_create_scan_valid(self, client: TestClient, sample_scan_data):
"""Test creating a scan with valid data."""
response = client.post("/api/v1/scans", json=sample_scan_data)
assert response.status_code == 200
data = response.json()
assert data["status"] == "pending"
assert data["id"] > 0
assert data["scan"]["origin"] == sample_scan_data["origin"]
assert data["scan"]["country"] == sample_scan_data["country"]
def test_create_scan_with_defaults(self, client: TestClient):
"""Test creating a scan with default dates."""
data = {
"origin": "MUC",
"country": "IT",
"window_months": 3
}
response = client.post("/api/v1/scans", json=data)
assert response.status_code == 200
scan = response.json()["scan"]
assert "start_date" in scan
assert "end_date" in scan
assert scan["seat_class"] == "economy"
assert scan["adults"] == 1
def test_create_scan_invalid_origin(self, client: TestClient):
"""Test creating a scan with invalid origin."""
data = {
"origin": "INVALID", # Too long
"country": "DE"
}
response = client.post("/api/v1/scans", json=data)
assert response.status_code == 422
error = response.json()
assert error["error"] == "validation_error"
def test_create_scan_invalid_country(self, client: TestClient):
"""Test creating a scan with invalid country."""
data = {
"origin": "BDS",
"country": "DEU" # Too long
}
response = client.post("/api/v1/scans", json=data)
assert response.status_code == 422
def test_list_scans_empty(self, client: TestClient):
"""Test listing scans when database is empty."""
response = client.get("/api/v1/scans")
assert response.status_code == 200
data = response.json()
assert data["data"] == []
assert data["pagination"]["total"] == 0
def test_list_scans_with_data(self, client: TestClient, create_test_scan):
"""Test listing scans with data."""
# Create test scans
create_test_scan(origin="BDS", country="DE")
create_test_scan(origin="MUC", country="IT")
response = client.get("/api/v1/scans")
assert response.status_code == 200
data = response.json()
assert len(data["data"]) == 2
assert data["pagination"]["total"] == 2
def test_list_scans_pagination(self, client: TestClient, create_test_scan):
"""Test scan list pagination."""
# Create 5 scans
for i in range(5):
create_test_scan(origin="BDS", country="DE")
response = client.get("/api/v1/scans?page=1&limit=2")
assert response.status_code == 200
data = response.json()
assert len(data["data"]) == 2
assert data["pagination"]["total"] == 5
assert data["pagination"]["pages"] == 3
assert data["pagination"]["has_next"] is True
def test_list_scans_filter_by_status(self, client: TestClient, create_test_scan):
"""Test filtering scans by status."""
create_test_scan(status="pending")
create_test_scan(status="completed")
create_test_scan(status="pending")
response = client.get("/api/v1/scans?status=pending")
assert response.status_code == 200
data = response.json()
assert len(data["data"]) == 2
assert all(scan["status"] == "pending" for scan in data["data"])
def test_get_scan_by_id(self, client: TestClient, create_test_scan):
"""Test getting a specific scan by ID."""
scan_id = create_test_scan(origin="FRA", country="ES")
response = client.get(f"/api/v1/scans/{scan_id}")
assert response.status_code == 200
data = response.json()
assert data["id"] == scan_id
assert data["origin"] == "FRA"
assert data["country"] == "ES"
def test_get_scan_not_found(self, client: TestClient):
"""Test getting a non-existent scan."""
response = client.get("/api/v1/scans/999")
assert response.status_code == 404
error = response.json()
assert error["error"] == "not_found"
assert "999" in error["message"]
def test_get_scan_routes_empty(self, client: TestClient, create_test_scan):
"""Test getting routes for a scan with no routes."""
scan_id = create_test_scan()
response = client.get(f"/api/v1/scans/{scan_id}/routes")
assert response.status_code == 200
data = response.json()
assert data["data"] == []
assert data["pagination"]["total"] == 0
def test_get_scan_routes_with_data(self, client: TestClient, create_test_scan, create_test_route):
"""Test getting routes for a scan with data."""
scan_id = create_test_scan()
create_test_route(scan_id=scan_id, destination="MUC", min_price=100)
create_test_route(scan_id=scan_id, destination="FRA", min_price=50)
response = client.get(f"/api/v1/scans/{scan_id}/routes")
assert response.status_code == 200
data = response.json()
assert len(data["data"]) == 2
# Routes should be ordered by price (cheapest first)
assert data["data"][0]["destination"] == "FRA"
assert data["data"][0]["min_price"] == 50
@pytest.mark.unit
@pytest.mark.api
class TestLogEndpoints:
"""Tests for log viewer endpoints."""
def test_get_logs_empty(self, client: TestClient):
"""Test getting logs when buffer is empty."""
response = client.get("/api/v1/logs")
assert response.status_code == 200
data = response.json()
# May have some startup logs
assert "data" in data
assert "pagination" in data
def test_get_logs_with_level_filter(self, client: TestClient):
"""Test filtering logs by level."""
response = client.get("/api/v1/logs?level=INFO")
assert response.status_code == 200
data = response.json()
if data["data"]:
assert all(log["level"] == "INFO" for log in data["data"])
def test_get_logs_invalid_level(self, client: TestClient):
"""Test filtering logs with invalid level."""
response = client.get("/api/v1/logs?level=INVALID")
assert response.status_code == 400
error = response.json()
assert error["error"] == "bad_request"
def test_get_logs_search(self, client: TestClient):
"""Test searching logs by text."""
response = client.get("/api/v1/logs?search=startup")
assert response.status_code == 200
data = response.json()
if data["data"]:
assert all("startup" in log["message"].lower() for log in data["data"])
@pytest.mark.unit
@pytest.mark.api
class TestErrorHandling:
"""Tests for error handling."""
def test_request_id_in_error(self, client: TestClient):
"""Test that errors include request ID."""
response = client.get("/api/v1/scans/999")
assert response.status_code == 404
error = response.json()
assert "request_id" in error
assert len(error["request_id"]) == 8 # UUID shortened to 8 chars
def test_request_id_in_headers(self, client: TestClient):
"""Test that request ID is in headers."""
response = client.get("/api/v1/scans")
assert "x-request-id" in response.headers
assert len(response.headers["x-request-id"]) == 8
def test_validation_error_format(self, client: TestClient):
"""Test validation error response format."""
response = client.post("/api/v1/scans", json={"origin": "TOOLONG", "country": "DE"})
assert response.status_code == 422
error = response.json()
assert error["error"] == "validation_error"
assert "errors" in error
assert isinstance(error["errors"], list)
assert len(error["errors"]) > 0
assert "field" in error["errors"][0]
@pytest.mark.unit
@pytest.mark.api
class TestRateLimiting:
"""Tests for rate limiting."""
def test_rate_limit_headers_present(self, client: TestClient):
"""Test that rate limit headers are present."""
response = client.get("/api/v1/airports?q=MUC")
assert "x-ratelimit-limit" in response.headers
assert "x-ratelimit-remaining" in response.headers
assert "x-ratelimit-reset" in response.headers
def test_rate_limit_decreases(self, client: TestClient):
"""Test that rate limit remaining decreases."""
response1 = client.get("/api/v1/airports?q=MUC")
remaining1 = int(response1.headers["x-ratelimit-remaining"])
response2 = client.get("/api/v1/airports?q=MUC")
remaining2 = int(response2.headers["x-ratelimit-remaining"])
assert remaining2 < remaining1
def test_rate_limit_exceeded(self, client: TestClient):
"""Test rate limit exceeded response."""
# Make requests until limit is reached (scans endpoint has limit of 10)
for i in range(12):
response = client.post("/api/v1/scans", json={"origin": "BDS", "country": "DE"})
# Should get 429 eventually
assert response.status_code == 429
error = response.json()
assert error["error"] == "rate_limit_exceeded"
assert "retry_after" in error

372
flight-comparator/tests/test_comprehensive_v3.py Executable file
View File

@@ -0,0 +1,372 @@
#!/usr/bin/env python3
"""
Comprehensive test suite for fast-flights v3.0rc1 with SOCS cookie integration.
Tests multiple routes, dates, and edge cases.
"""
import sys
import logging
import asyncio
from datetime import date, timedelta
sys.path.insert(0, '..')
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
try:
from searcher_v3 import search_direct_flights, search_multiple_routes, SOCSCookieIntegration
from fast_flights import FlightQuery, Passengers, get_flights, create_query
HAS_V3 = True
except ImportError as e:
logger.error(f"✗ Failed to import v3 modules: {e}")
logger.error(" Install with: pip install --upgrade git+https://github.com/AWeirdDev/flights.git")
HAS_V3 = False
class TestResults:
"""Track test results."""
def __init__(self):
self.total = 0
self.passed = 0
self.failed = 0
self.errors = []
def add_pass(self, test_name):
self.total += 1
self.passed += 1
logger.info(f"✓ PASS: {test_name}")
def add_fail(self, test_name, reason):
self.total += 1
self.failed += 1
self.errors.append((test_name, reason))
logger.error(f"✗ FAIL: {test_name} - {reason}")
def summary(self):
logger.info("\n" + "="*80)
logger.info("TEST SUMMARY")
logger.info("="*80)
logger.info(f"Total: {self.total}")
logger.info(f"Passed: {self.passed} ({self.passed/self.total*100:.1f}%)")
logger.info(f"Failed: {self.failed}")
if self.errors:
logger.info("\nFailed Tests:")
for name, reason in self.errors:
logger.info(f" • {name}: {reason}")
return self.failed == 0
results = TestResults()
def test_socs_integration():
"""Test that SOCS cookie integration is properly configured."""
if not HAS_V3:
results.add_fail("SOCS Integration", "v3 not installed")
return
try:
integration = SOCSCookieIntegration()
assert hasattr(integration, 'SOCS_COOKIE')
assert integration.SOCS_COOKIE.startswith('CAE')
assert hasattr(integration, 'fetch_html')
results.add_pass("SOCS Integration")
except Exception as e:
results.add_fail("SOCS Integration", str(e))
async def test_single_route_ber_bri():
"""Test BER to BRI route (known to work)."""
if not HAS_V3:
results.add_fail("BER→BRI Single Route", "v3 not installed")
return
try:
test_date = (date.today() + timedelta(days=30)).strftime('%Y-%m-%d')
flights = await search_direct_flights("BER", "BRI", test_date)
if flights and len(flights) > 0:
# Verify flight structure
f = flights[0]
assert 'origin' in f
assert 'destination' in f
assert 'price' in f
assert 'airline' in f
assert f['origin'] == 'BER'
assert f['destination'] == 'BRI'
assert f['price'] > 0
logger.info(f" Found {len(flights)} flight(s), cheapest: €{flights[0]['price']}")
results.add_pass("BER→BRI Single Route")
else:
results.add_fail("BER→BRI Single Route", "No flights found")
except Exception as e:
results.add_fail("BER→BRI Single Route", str(e))
async def test_multiple_routes():
"""Test multiple routes in one batch."""
if not HAS_V3:
results.add_fail("Multiple Routes", "v3 not installed")
return
try:
test_date = (date.today() + timedelta(days=30)).strftime('%Y-%m-%d')
routes = [
("BER", "FCO", test_date), # Berlin to Rome
("FRA", "MAD", test_date), # Frankfurt to Madrid
("MUC", "BCN", test_date), # Munich to Barcelona
]
batch_results = await search_multiple_routes(
routes,
seat_class="economy",
adults=1,
max_workers=3,
)
# Check we got results for each route
flights_found = sum(1 for flights in batch_results.values() if flights)
if flights_found >= 2: # At least 2 out of 3 should have flights
logger.info(f" Found flights for {flights_found}/3 routes")
results.add_pass("Multiple Routes")
else:
results.add_fail("Multiple Routes", f"Only {flights_found}/3 routes had flights")
except Exception as e:
results.add_fail("Multiple Routes", str(e))
async def test_different_dates():
"""Test same route with different dates."""
if not HAS_V3:
results.add_fail("Different Dates", "v3 not installed")
return
try:
dates = [
(date.today() + timedelta(days=30)).strftime('%Y-%m-%d'),
(date.today() + timedelta(days=60)).strftime('%Y-%m-%d'),
(date.today() + timedelta(days=90)).strftime('%Y-%m-%d'),
]
routes = [("BER", "BRI", d) for d in dates]
batch_results = await search_multiple_routes(
routes,
seat_class="economy",
adults=1,
max_workers=2,
)
flights_found = sum(1 for flights in batch_results.values() if flights)
if flights_found >= 2:
logger.info(f" Found flights for {flights_found}/3 dates")
results.add_pass("Different Dates")
else:
results.add_fail("Different Dates", f"Only {flights_found}/3 dates had flights")
except Exception as e:
results.add_fail("Different Dates", str(e))
async def test_no_direct_flights():
"""Test route with no direct flights (should return empty)."""
if not HAS_V3:
results.add_fail("No Direct Flights", "v3 not installed")
return
try:
test_date = (date.today() + timedelta(days=30)).strftime('%Y-%m-%d')
# BER to SYD probably has no direct flights
flights = await search_direct_flights("BER", "SYD", test_date)
# Should return empty list, not crash
assert isinstance(flights, list)
logger.info(f" Correctly handled no-direct-flights case (found {len(flights)})")
results.add_pass("No Direct Flights")
except Exception as e:
results.add_fail("No Direct Flights", str(e))
async def test_invalid_airport_code():
"""Test handling of invalid airport codes."""
if not HAS_V3:
results.add_fail("Invalid Airport", "v3 not installed")
return
try:
test_date = (date.today() + timedelta(days=30)).strftime('%Y-%m-%d')
# XXX is not a valid IATA code
flights = await search_direct_flights("XXX", "BRI", test_date)
# Should return empty or handle gracefully, not crash
assert isinstance(flights, list)
logger.info(f" Gracefully handled invalid airport code")
results.add_pass("Invalid Airport")
except Exception as e:
results.add_fail("Invalid Airport", str(e))
async def test_concurrent_requests():
"""Test that concurrent requests work properly."""
if not HAS_V3:
results.add_fail("Concurrent Requests", "v3 not installed")
return
try:
test_date = (date.today() + timedelta(days=30)).strftime('%Y-%m-%d')
# 10 concurrent requests
routes = [
("BER", "BRI", test_date),
("FRA", "FCO", test_date),
("MUC", "VIE", test_date),
("BER", "CPH", test_date),
("FRA", "AMS", test_date),
("MUC", "ZRH", test_date),
("BER", "VIE", test_date),
("FRA", "BRU", test_date),
("MUC", "CDG", test_date),
("BER", "AMS", test_date),
]
import time
start = time.time()
batch_results = await search_multiple_routes(
routes,
seat_class="economy",
adults=1,
max_workers=5,
)
elapsed = time.time() - start
flights_found = sum(1 for flights in batch_results.values() if flights)
# Should complete reasonably fast with concurrency
if flights_found >= 5 and elapsed < 60:
logger.info(f" {flights_found}/10 routes successful in {elapsed:.1f}s")
results.add_pass("Concurrent Requests")
else:
results.add_fail("Concurrent Requests", f"Only {flights_found}/10 in {elapsed:.1f}s")
except Exception as e:
results.add_fail("Concurrent Requests", str(e))
async def test_price_range():
"""Test that prices are reasonable."""
if not HAS_V3:
results.add_fail("Price Range", "v3 not installed")
return
try:
test_date = (date.today() + timedelta(days=30)).strftime('%Y-%m-%d')
flights = await search_direct_flights("BER", "BRI", test_date)
if flights:
prices = [f['price'] for f in flights if 'price' in f]
if prices:
min_price = min(prices)
max_price = max(prices)
# Sanity check: prices should be between 20 and 1000 EUR for EU routes
if 20 <= min_price <= 1000 and 20 <= max_price <= 1000:
logger.info(f" Price range: €{min_price} - €{max_price}")
results.add_pass("Price Range")
else:
results.add_fail("Price Range", f"Unreasonable prices: €{min_price} - €{max_price}")
else:
results.add_fail("Price Range", "No prices found in results")
else:
results.add_fail("Price Range", "No flights to check prices")
except Exception as e:
results.add_fail("Price Range", str(e))
async def run_all_tests():
"""Run all tests."""
logger.info("╔" + "="*78 + "╗")
logger.info("║" + " "*15 + "COMPREHENSIVE TEST SUITE - fast-flights v3.0rc1" + " "*14 + "║")
logger.info("╚" + "="*78 + "╝\n")
if not HAS_V3:
logger.error("fast-flights v3.0rc1 not installed!")
logger.error("Install with: pip install --upgrade git+https://github.com/AWeirdDev/flights.git")
return False
# Unit tests
logger.info("\n" + "-"*80)
logger.info("UNIT TESTS")
logger.info("-"*80)
test_socs_integration()
# Integration tests
logger.info("\n" + "-"*80)
logger.info("INTEGRATION TESTS")
logger.info("-"*80)
await test_single_route_ber_bri()
await asyncio.sleep(2) # Rate limiting
await test_multiple_routes()
await asyncio.sleep(2)
await test_different_dates()
await asyncio.sleep(2)
await test_no_direct_flights()
await asyncio.sleep(2)
await test_invalid_airport_code()
await asyncio.sleep(2)
# Stress tests
logger.info("\n" + "-"*80)
logger.info("STRESS TESTS")
logger.info("-"*80)
await test_concurrent_requests()
await asyncio.sleep(2)
# Validation tests
logger.info("\n" + "-"*80)
logger.info("VALIDATION TESTS")
logger.info("-"*80)
await test_price_range()
# Summary
return results.summary()
if __name__ == "__main__":
success = asyncio.run(run_all_tests())
logger.info("\n" + "="*80)
if success:
logger.info("✅ ALL TESTS PASSED!")
else:
logger.info("⚠️ SOME TESTS FAILED - See summary above")
logger.info("="*80)
sys.exit(0 if success else 1)

64
flight-comparator/tests/test_date_resolver.py Normal file
View File

@@ -0,0 +1,64 @@
"""
Smoke tests for date_resolver module.
"""
from datetime import date
from dateutil.relativedelta import relativedelta
import sys
sys.path.insert(0, '..')
from date_resolver import resolve_dates, detect_new_connections, SEARCH_WINDOW_MONTHS
def test_resolve_dates_with_specific_date():
"""Test that a specific date returns only that date."""
result = resolve_dates("2026-06-15", 6)
assert result == ["2026-06-15"]
print("✓ Specific date resolution works")
def test_resolve_dates_seasonal():
"""Test that seasonal mode generates one date per month."""
result = resolve_dates(None, 3)
assert len(result) == 3
# All should be valid date strings
for date_str in result:
assert len(date_str) == 10 # YYYY-MM-DD format
assert date_str.count('-') == 2
print(f"✓ Seasonal resolution works: {result}")
def test_detect_new_connections():
"""Test new connection detection logic."""
monthly_results = {
"2026-03": [
{"origin": "FRA", "destination": "JFK"},
{"origin": "MUC", "destination": "JFK"},
],
"2026-04": [
{"origin": "FRA", "destination": "JFK"},
{"origin": "MUC", "destination": "JFK"},
{"origin": "BER", "destination": "JFK"}, # NEW
],
"2026-05": [
{"origin": "FRA", "destination": "JFK"},
{"origin": "BER", "destination": "JFK"},
{"origin": "HAM", "destination": "JFK"}, # NEW
],
}
new = detect_new_connections(monthly_results)
assert "BER->JFK" in new
assert new["BER->JFK"] == "2026-04"
assert "HAM->JFK" in new
assert new["HAM->JFK"] == "2026-05"
assert "FRA->JFK" not in new # Was in first month
assert "MUC->JFK" not in new # Was in first month
print(f"✓ New connection detection works: {new}")
if __name__ == "__main__":
test_resolve_dates_with_specific_date()
test_resolve_dates_seasonal()
test_detect_new_connections()
print("\n✅ All date_resolver tests passed!")

23
flight-comparator/tests/test_formatter.py Normal file
View File

@@ -0,0 +1,23 @@
"""
Smoke tests for formatter module.
"""
import sys
sys.path.insert(0, '..')
from formatter import format_duration
def test_format_duration():
"""Test duration formatting."""
assert format_duration(0) == "—"
assert format_duration(60) == "1h"
assert format_duration(90) == "1h 30m"
assert format_duration(570) == "9h 30m"
assert format_duration(615) == "10h 15m"
print("✓ Duration formatting works")
if __name__ == "__main__":
test_format_duration()
print("\n✅ All formatter tests passed!")

309
flight-comparator/tests/test_integration.py Normal file
View File

@@ -0,0 +1,309 @@
"""
Integration tests for Flight Radar Web App.
Tests that verify multiple components working together, including
database operations, full workflows, and system behavior.
"""
import pytest
import sqlite3
import time
from fastapi.testclient import TestClient
@pytest.mark.integration
@pytest.mark.database
class TestScanWorkflow:
"""Integration tests for complete scan workflow."""
def test_create_and_retrieve_scan(self, client: TestClient):
"""Test creating a scan and retrieving it."""
# Create scan
create_data = {
"origin": "BDS",
"country": "DE",
"start_date": "2026-04-01",
"end_date": "2026-06-30",
"adults": 2
}
create_response = client.post("/api/v1/scans", json=create_data)
assert create_response.status_code == 200
scan_id = create_response.json()["id"]
# Retrieve scan
get_response = client.get(f"/api/v1/scans/{scan_id}")
assert get_response.status_code == 200
scan = get_response.json()
assert scan["id"] == scan_id
assert scan["origin"] == create_data["origin"]
assert scan["country"] == create_data["country"]
assert scan["status"] == "pending"
def test_scan_appears_in_list(self, client: TestClient):
"""Test that created scan appears in list."""
# Create scan
create_response = client.post("/api/v1/scans", json={
"origin": "MUC",
"country": "IT"
})
scan_id = create_response.json()["id"]
# List scans
list_response = client.get("/api/v1/scans")
scans = list_response.json()["data"]
# Find our scan
found = any(scan["id"] == scan_id for scan in scans)
assert found
def test_scan_with_routes_workflow(self, client: TestClient, create_test_route):
"""Test creating scan and adding routes."""
# Create scan
create_response = client.post("/api/v1/scans", json={
"origin": "BDS",
"country": "DE"
})
scan_id = create_response.json()["id"]
# Add routes
create_test_route(scan_id=scan_id, destination="MUC", min_price=100)
create_test_route(scan_id=scan_id, destination="FRA", min_price=50)
create_test_route(scan_id=scan_id, destination="BER", min_price=75)
# Get routes
routes_response = client.get(f"/api/v1/scans/{scan_id}/routes")
assert routes_response.status_code == 200
routes = routes_response.json()["data"]
assert len(routes) == 3
# Check ordering (by price)
prices = [r["min_price"] for r in routes]
assert prices == sorted(prices)
@pytest.mark.integration
@pytest.mark.database
class TestDatabaseOperations:
"""Integration tests for database operations."""
def test_foreign_key_constraints(self, client: TestClient, clean_database):
"""Test that foreign key constraints are enforced."""
# Try to create route for non-existent scan
conn = sqlite3.connect(clean_database)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys
cursor = conn.cursor()
with pytest.raises(sqlite3.IntegrityError):
cursor.execute("""
INSERT INTO routes (scan_id, destination, destination_name,
destination_city, flight_count, airlines)
VALUES (999, 'MUC', 'Munich', 'Munich', 10, '[]')
""")
conn.commit()
conn.close()
def test_cascade_delete(self, client: TestClient, create_test_scan, create_test_route, clean_database):
"""Test that deleting scan cascades to routes."""
# Create scan and routes
scan_id = create_test_scan()
create_test_route(scan_id=scan_id, destination="MUC")
create_test_route(scan_id=scan_id, destination="FRA")
# Delete scan
conn = sqlite3.connect(clean_database)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys for cascade
cursor = conn.cursor()
cursor.execute("DELETE FROM scans WHERE id = ?", (scan_id,))
conn.commit()
# Check routes are deleted
cursor.execute("SELECT COUNT(*) FROM routes WHERE scan_id = ?", (scan_id,))
count = cursor.fetchone()[0]
conn.close()
assert count == 0
def test_timestamp_triggers(self, client: TestClient, create_test_scan, clean_database):
"""Test that timestamp triggers work."""
scan_id = create_test_scan()
# Get original timestamp
conn = sqlite3.connect(clean_database)
conn.execute("PRAGMA foreign_keys = ON") # Enable foreign keys
cursor = conn.cursor()
cursor.execute("SELECT updated_at FROM scans WHERE id = ?", (scan_id,))
original_time = cursor.fetchone()[0]
# Wait a moment (SQLite CURRENT_TIMESTAMP has 1-second precision)
time.sleep(1.1)
# Update scan
cursor.execute("UPDATE scans SET status = 'running' WHERE id = ?", (scan_id,))
conn.commit()
# Get new timestamp
cursor.execute("SELECT updated_at FROM scans WHERE id = ?", (scan_id,))
new_time = cursor.fetchone()[0]
conn.close()
assert new_time != original_time
@pytest.mark.integration
class TestPaginationAcrossEndpoints:
"""Integration tests for pagination consistency."""
def test_pagination_metadata_consistency(self, client: TestClient, create_test_scan):
"""Test pagination metadata is consistent across endpoints."""
# Create 10 scans
for i in range(10):
create_test_scan()
# Test scans pagination
response = client.get("/api/v1/scans?page=1&limit=3")
data = response.json()
assert data["pagination"]["page"] == 1
assert data["pagination"]["limit"] == 3
assert data["pagination"]["total"] == 10
assert data["pagination"]["pages"] == 4
assert data["pagination"]["has_next"] is True
assert data["pagination"]["has_prev"] is False
def test_pagination_last_page(self, client: TestClient, create_test_scan):
"""Test pagination on last page."""
# Create 7 scans
for i in range(7):
create_test_scan()
# Get last page
response = client.get("/api/v1/scans?page=2&limit=5")
data = response.json()
assert data["pagination"]["page"] == 2
assert data["pagination"]["has_next"] is False
assert data["pagination"]["has_prev"] is True
assert len(data["data"]) == 2 # Only 2 items on last page
@pytest.mark.integration
class TestErrorHandlingIntegration:
"""Integration tests for error handling across the system."""
def test_error_logging(self, client: TestClient):
"""Test that errors are logged."""
# Trigger error
client.get("/api/v1/scans/999")
# Check logs contain error (would need to check log buffer)
# This is a basic integration test
response = client.get("/api/v1/logs?search=not+found")
# Just verify we can get logs, specific content may vary
assert response.status_code == 200
def test_request_id_consistency(self, client: TestClient):
"""Test that request ID is consistent in error response and headers."""
response = client.get("/api/v1/scans/999")
request_id_header = response.headers.get("x-request-id")
request_id_body = response.json().get("request_id")
assert request_id_header == request_id_body
@pytest.mark.integration
@pytest.mark.slow
class TestRateLimitingIntegration:
"""Integration tests for rate limiting system."""
def test_rate_limit_per_endpoint(self, client: TestClient):
"""Test that different endpoints have different rate limits."""
# Airports endpoint (100/min)
airport_response = client.get("/api/v1/airports?q=MUC")
airport_limit = int(airport_response.headers["x-ratelimit-limit"])
# Scans endpoint (10/min)
scan_response = client.post("/api/v1/scans", json={"origin": "BDS", "country": "DE"})
scan_limit = int(scan_response.headers["x-ratelimit-limit"])
# Different limits
assert airport_limit > scan_limit
assert airport_limit == 100
assert scan_limit == 10
def test_rate_limit_recovery(self, client: TestClient):
"""Test that rate limit counter is per-IP and independent."""
# Make some requests to airports
for i in range(3):
client.get("/api/v1/airports?q=MUC")
# Scans endpoint should have independent counter
response = client.post("/api/v1/scans", json={"origin": "BDS", "country": "DE"})
remaining = int(response.headers["x-ratelimit-remaining"])
# Should still have most of scan limit available (10 total, used 1)
assert remaining >= 8
@pytest.mark.integration
class TestStartupCleanup:
"""Integration tests for startup cleanup behavior."""
def test_stuck_scans_detection(self, client: TestClient, create_test_scan, clean_database):
"""Test that stuck scans are detected."""
# Create stuck scan
scan_id = create_test_scan(status="running")
# Verify it's in running state
conn = sqlite3.connect(clean_database)
cursor = conn.cursor()
cursor.execute("SELECT status FROM scans WHERE id = ?", (scan_id,))
status = cursor.fetchone()[0]
conn.close()
assert status == "running"
# Note: Actual cleanup happens on server restart, tested manually
@pytest.mark.integration
class TestValidationIntegration:
"""Integration tests for validation across the system."""
def test_validation_consistency(self, client: TestClient):
"""Test that validation is consistent across endpoints."""
# Invalid IATA code
response1 = client.post("/api/v1/scans", json={"origin": "TOOLONG", "country": "DE"})
assert response1.status_code == 422
# Invalid date format
response2 = client.post("/api/v1/scans", json={
"origin": "BDS",
"country": "DE",
"start_date": "01-04-2026" # Wrong format
})
assert response2.status_code == 422
def test_auto_normalization(self, client: TestClient):
"""Test that IATA codes are auto-normalized to uppercase."""
response = client.post("/api/v1/scans", json={
"origin": "bds", # lowercase
"country": "de" # lowercase
})
assert response.status_code == 200
scan = response.json()["scan"]
assert scan["origin"] == "BDS" # Uppercased
assert scan["country"] == "DE" # Uppercased

296
flight-comparator/tests/test_scan_pipeline.py Normal file
View File

@@ -0,0 +1,296 @@
"""
Integration tests for the full scan pipeline: searcher → processor → database.
Confirmed flight data is stored in confirmed_flights.json (generated 2026-02-25
from a live scan of BDS→FMM,DUS across the full Feb 26 – May 27 2026 window).
Key confirmed routes:
BDS → FMM 39 flights Mar–May 2026 Ryanair ~5-6x/week, two daily slots
BDS → DUS 11 flights Apr–May 2026 Eurowings Saturdays only, two time slots
These tests make real network calls to Google Flights via fast-flights.
Mark: integration, slow
"""
import asyncio
import json
import os
import sqlite3
import sys
import tempfile
from pathlib import Path
import pytest
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
from searcher_v3 import search_multiple_routes
from scan_processor import process_scan
from database import initialize_database
# ---------------------------------------------------------------------------
# Load confirmed flight data from JSON fixture
# ---------------------------------------------------------------------------
_FIXTURE_PATH = Path(__file__).parent / "confirmed_flights.json"
with open(_FIXTURE_PATH) as _f:
CONFIRMED = json.load(_f)
# (origin, destination, date, min_expected_flights, description)
# Built from confirmed_dates_for_testing — each entry is a specific (route, date)
# pair that returned ≥1 real flight from the live API.
KNOWN_ROUTES = [
(
e["origin"],
e["destination"],
e["date"],
e["min_flights"],
f"{e['origin']}→{e['destination']} {e['airline']} on {e['date']} (confirmed €{e['price']:.0f})",
)
for e in CONFIRMED["confirmed_dates_for_testing"]["entries"]
]
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(scope="module")
def tmp_db():
"""Isolated SQLite database for pipeline tests."""
fd, path = tempfile.mkstemp(suffix=".db")
os.close(fd)
os.environ["DATABASE_PATH"] = path
initialize_database(db_path=Path(path), verbose=False)
yield path
os.environ.pop("DATABASE_PATH", None)
try:
os.unlink(path)
except OSError:
pass
def _insert_scan(db_path, origin, country, start_date, end_date,
seat_class="economy", adults=1):
"""Insert a pending scan and return its ID."""
conn = sqlite3.connect(db_path)
conn.execute("PRAGMA foreign_keys = ON")
cur = conn.cursor()
cur.execute(
"""INSERT INTO scans (origin, country, start_date, end_date, status, seat_class, adults)
VALUES (?, ?, ?, ?, 'pending', ?, ?)""",
(origin, country, start_date, end_date, seat_class, adults),
)
scan_id = cur.lastrowid
conn.commit()
conn.close()
return scan_id
def _get_scan(db_path, scan_id):
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
row = conn.execute("SELECT * FROM scans WHERE id=?", (scan_id,)).fetchone()
conn.close()
return dict(row) if row else None
def _get_routes(db_path, scan_id):
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
rows = conn.execute(
"SELECT * FROM routes WHERE scan_id=?", (scan_id,)
).fetchall()
conn.close()
return [dict(r) for r in rows]
# ---------------------------------------------------------------------------
# Searcher tests — verify live data comes back for confirmed routes
# ---------------------------------------------------------------------------
class TestSearcherKnownRoutes:
"""
Directly test search_multiple_routes() against confirmed real routes.
Each test uses a date/route pair we know has flights from our earlier scans.
"""
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.parametrize("origin,dest,date,min_flights,desc", KNOWN_ROUTES)
def test_returns_flights_for_confirmed_route(self, origin, dest, date, min_flights, desc):
"""Searcher returns ≥min_flights for a confirmed live route."""
results = asyncio.run(
search_multiple_routes(
routes=[(origin, dest, date)],
seat_class="economy",
adults=1,
use_cache=False,
max_workers=1,
)
)
flights = results.get((origin, dest, date), [])
assert len(flights) >= min_flights, (
f"{desc}: expected ≥{min_flights} flight(s) on {origin}→{dest} {date}, "
f"got {len(flights)}"
)
@pytest.mark.integration
@pytest.mark.slow
def test_flight_has_required_fields(self):
"""Every returned flight dict has the mandatory fields."""
origin, dest, date = "BDS", "FMM", "2026-04-05"
results = asyncio.run(
search_multiple_routes(
routes=[(origin, dest, date)],
seat_class="economy",
adults=1,
use_cache=False,
max_workers=1,
)
)
flights = results.get((origin, dest, date), [])
assert flights, f"No flights returned for {origin}→{dest} {date}"
required = {"origin", "destination", "airline", "departure_time",
"arrival_time", "price", "stops"}
for flight in flights:
missing = required - flight.keys()
assert not missing, f"Flight missing fields: {missing}. Got: {flight}"
assert flight["stops"] == 0, "Expected direct flight only"
assert flight["price"] > 0, "Price must be positive"
@pytest.mark.integration
@pytest.mark.slow
def test_no_results_for_unknown_route(self):
"""Routes with no service return an empty list, not an error."""
# BDS → JFK: no direct flight exists
results = asyncio.run(
search_multiple_routes(
routes=[("BDS", "JFK", "2026-04-05")],
seat_class="economy",
adults=1,
use_cache=False,
max_workers=1,
)
)
# Should complete without raising; result may be empty or have 0 flights
assert ("BDS", "JFK", "2026-04-05") in results
# ---------------------------------------------------------------------------
# Pipeline tests — scan processor saves flights to the database
# ---------------------------------------------------------------------------
class TestScanProcessorSavesRoutes:
"""
Test that process_scan() correctly saves discovered flights into the
routes table. These tests catch the regression where dest_info lookup
silently discarded all results.
"""
@pytest.mark.integration
@pytest.mark.slow
def test_airports_mode_saves_routes(self, tmp_db):
"""
Airports mode (comma-separated in country field) must save routes.
Regression: after removing get_airport_data() call, destinations=[]
caused dest_info to always be None → all routes silently skipped.
"""
scan_id = _insert_scan(
tmp_db,
origin="BDS",
country="FMM", # single airport in destinations-mode format
start_date="2026-04-05",
end_date="2026-04-06",
)
asyncio.run(process_scan(scan_id))
scan = _get_scan(tmp_db, scan_id)
assert scan["status"] == "completed", (
f"Scan failed: {scan.get('error_message')}"
)
routes = _get_routes(tmp_db, scan_id)
assert len(routes) >= 1, (
"No routes saved for BDS→FMM even though Ryanair flies this route"
)
fmm_route = next(r for r in routes if r["destination"] == "FMM")
assert fmm_route["flight_count"] >= 1
assert fmm_route["min_price"] > 0
@pytest.mark.integration
@pytest.mark.slow
def test_airports_mode_unknown_airport_uses_iata_fallback(self, tmp_db):
"""
When an airport code is not in airports_by_country.json, the route
is still saved with the IATA code as its name (not silently dropped).
"""
scan_id = _insert_scan(
tmp_db,
origin="BDS",
country="FMM",
start_date="2026-04-05",
end_date="2026-04-06",
)
asyncio.run(process_scan(scan_id))
routes = _get_routes(tmp_db, scan_id)
for route in routes:
# name must be set (IATA code at minimum, not empty/None)
assert route["destination_name"], (
f"destination_name is empty for route to {route['destination']}"
)
@pytest.mark.integration
@pytest.mark.slow
def test_country_mode_includes_fmm(self, tmp_db):
"""
Country mode must scan ALL airports, not just the first 20.
Regression: [:20] alphabetical cut-off excluded FMM (#72 in DE list)
and STR (#21), which are among the most active BDS routes.
"""
scan_id = _insert_scan(
tmp_db,
origin="BDS",
country="DE",
start_date="2026-04-05",
end_date="2026-04-06",
)
asyncio.run(process_scan(scan_id))
scan = _get_scan(tmp_db, scan_id)
assert scan["status"] == "completed", scan.get("error_message")
routes = _get_routes(tmp_db, scan_id)
destinations_found = {r["destination"] for r in routes}
# FMM and DUS must appear — they have confirmed flights on 2026-04-05
assert "FMM" in destinations_found, (
f"FMM (Ryanair BDS→FMM) missing from results. Found: {destinations_found}"
)
@pytest.mark.integration
@pytest.mark.slow
def test_multi_airport_mode_saves_all_routes(self, tmp_db):
"""
Comma-separated destinations: all airports with flights must be saved.
"""
scan_id = _insert_scan(
tmp_db,
origin="BDS",
country="FMM,DUS", # two confirmed routes
start_date="2026-04-04", # Saturday (DUS) — range extends to Apr 15 (FMM mid-week)
end_date="2026-04-16", # captures 2026-04-04 (Sat) AND 2026-04-15 (Wed)
)
asyncio.run(process_scan(scan_id))
scan = _get_scan(tmp_db, scan_id)
assert scan["status"] == "completed", scan.get("error_message")
routes = _get_routes(tmp_db, scan_id)
destinations_found = {r["destination"] for r in routes}
assert "FMM" in destinations_found, "FMM route not saved"
assert "DUS" in destinations_found, "DUS route not saved (Saturday flight)"

33
flight-comparator/tests/test_searcher.py Normal file
View File

@@ -0,0 +1,33 @@
"""
Smoke tests for searcher module.
"""
import sys
sys.path.insert(0, '..')
from searcher import _parse_duration
def test_parse_duration():
"""Test duration parsing logic."""
assert _parse_duration("9h 30m") == 570
assert _parse_duration("9h") == 540
assert _parse_duration("90m") == 90
assert _parse_duration("10h 15m") == 615
assert _parse_duration("") == 0
print("✓ Duration parsing works")
def test_parse_duration_edge_cases():
"""Test edge cases in duration parsing."""
assert _parse_duration("0h 0m") == 0
assert _parse_duration("1h 1m") == 61
assert _parse_duration("24h") == 1440
print("✓ Duration parsing edge cases work")
if __name__ == "__main__":
test_parse_duration()
test_parse_duration_edge_cases()
print("\n✅ All searcher tests passed!")
print("ℹ️ Note: Full API integration tests require fast-flights and live network")