Compare commits

...

31 Commits

Author SHA1 Message Date
5c65e4d2ee fix: add BER to airports.py so scans include Berlin Brandenburg
All checks were successful
Deploy / deploy (push) Successful in 21s
OpenFlights dataset predates BER's 2020 opening. The patch already
existed in api_server.py for the search UI, but scan_processor.py
uses airports.py directly, so Germany scans silently skipped BER.

Added _MISSING_AIRPORTS to airports.py, patched both
get_airports_for_country() and _all_airports_by_iata() to inject
the extras, making BER available to scans and lookups.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 21:45:07 +01:00
cf40736f0e feat: add date range option to create scan dialog
All checks were successful
Deploy / deploy (push) Successful in 49s
Adds a Rolling Window / Date Range toggle to the Parameters section.
Rolling Window mode (default) keeps the existing window_months stepper.
Date Range mode shows From/To date pickers and sends start_date/end_date
to the API instead of window_months. Validation covers empty dates,
past start dates, end ≤ start, and ranges exceeding 12 months.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 21:38:02 +01:00
4f4f7e86d1 fix: make migrations resilient to orphaned scans_new tables
All checks were successful
Deploy / deploy (push) Successful in 12s
- Add _recover_orphaned_new_tables() called at DB init startup: if scans_new
  exists but scans doesn't (from a previously aborted migration), rename it back
- Drop VIEW recent_scans/active_scans before dropping the scans table in both
  _migrate_add_pause_cancel_status and _migrate_add_reverse_scan_support, so
  executescript can cleanly recreate them
- Add DROP TABLE IF EXISTS scans_new / scheduled_scans_new at the start of each
  table-recreation migration to prevent 'table scans_new already exists' errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 18:18:22 +01:00
77d2a46264 feat: implement reverse scan (country → specific airports)
All checks were successful
Deploy / deploy (push) Successful in 30s
- DB schema: relaxed origin CHECK to >=2 chars, added scan_mode column to
  scans and scheduled_scans, added origin_airport to routes and flights,
  updated unique index to (scan_id, COALESCE(origin_airport,''), destination)
- Migrations: init_db.py recreates tables and adds columns via guarded ALTERs
- API: scan_mode field on ScanRequest/Scan; Route/Flight expose origin_airport;
  GET /scans/{id}/flights accepts origin_airport filter; CreateScheduleRequest
  and Schedule carry scan_mode; scheduler and run-now pass scan_mode through
- scan_processor: _write_route_incremental accepts origin_airport; process_scan
  branches on scan_mode=reverse (country → airports × destinations × dates)
- Frontend: new CountrySelect component (populated from GET /api/v1/countries);
  Scans page adds Direction toggle + CountrySelect for both modes; ScanDetails
  shows Origin column for reverse scans and uses composite route keys; Re-run
  preserves scan_mode

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 17:58:55 +01:00
7ece1f9b45 fix: replace faint sort arrows with ChevronsUpDown indicator on inactive columns
All checks were successful
Deploy / deploy (push) Successful in 35s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 15:39:36 +01:00
69c2ddae29 feat: sort flights by date and add sortable date/price columns
All checks were successful
Deploy / deploy (push) Successful in 36s
- Change API ORDER BY from price/date to date/price (chronological default)
- Add flightSortField/flightSortDir state to ScanDetails
- Make Date and Price sub-table headers clickable with sort icons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 15:27:21 +01:00
3cad8a8447 fix: commit missing ScanTimer component and useScanTimer hook
All checks were successful
Deploy / deploy (push) Successful in 31s
These files were referenced by ScanDetails.tsx but never committed,
breaking the Docker build (tsc could not resolve the imports).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:12:47 +01:00
9a76d7af82 feat: add cancel, pause, and resume flow control for scans
Some checks failed
Deploy / deploy (push) Failing after 18s
Users running large scans can now pause (keep partial results, resume
later), cancel (stop permanently, partial results preserved), or resume
a paused scan which races through cache hits before continuing.

Backend:
- Extend scans.status CHECK to include 'paused' and 'cancelled'
- Add _migrate_add_pause_cancel_status() table-recreation migration
- scan_processor: _running_tasks/_cancel_reasons registries,
  cancel_scan_task/pause_scan_task/stop_scan_task helpers,
  CancelledError handler in process_scan(), start_resume_processor()
- api_server: POST /scans/{id}/pause|cancel|resume endpoints with
  rate limits (30/min pause+cancel, 10/min resume); list_scans now
  accepts paused/cancelled as status filter values

Frontend:
- Scan.status type extended with 'paused' | 'cancelled'
- scanApi.pause/cancel/resume added
- StatusChip: amber PauseCircle chip for paused, grey Ban for cancelled
- ScanDetails: context-aware action row with inline-confirm for
  Pause and Cancel; Resume button for paused scans

Tests: 129 total (58 new) across test_scan_control.py,
test_scan_processor_control.py, and additions to existing suites

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:11:23 +01:00
d494e80ff7 fix: force Traefik to use domverse network for routing
All checks were successful
Deploy / deploy (push) Successful in 11s
When a container is attached to multiple networks, Traefik picks an IP
from any of them. The frontend is on both domverse and flight-radar_default;
Traefik isn't on the latter, so ~50% of requests 504 after 30s.

Adding traefik.docker.network=domverse pins routing to the shared network.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 16:05:17 +01:00
cde496ad48 infra: fix CORS, add monitoring and dashboard labels
All checks were successful
Deploy / deploy (push) Successful in 11s
- Set ALLOWED_ORIGINS to production domain (fixes CORS for the web app)
- Add LOG_LEVEL=INFO to backend
- Add AutoKuma monitoring labels for Uptime Kuma auto-discovery
- Add Homepage dashboard labels (Productivity group)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 15:59:28 +01:00
7b07775845 fix: replace wget with curl in frontend healthcheck
All checks were successful
Deploy / deploy (push) Successful in 12s
nginx:alpine is a minimal image that does not include wget.
Install curl explicitly and use curl -f for the healthcheck.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 14:17:56 +01:00
6c1cffbdd4 fix: add missing vite-env.d.ts for CSS import type declarations
All checks were successful
Deploy / deploy (push) Successful in 36s
Without this file TypeScript errors on plain CSS side-effect imports
(e.g. import './index.css') because it lacks Vite's built-in type shims.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 14:06:10 +01:00
442e300457 fix: add missing tsconfig files and restore npm run build
Some checks failed
Deploy / deploy (push) Failing after 19s
tsconfig.json, tsconfig.app.json, and tsconfig.node.json were never
committed because *.json was gitignored without exceptions for them.
Added whitelist entries and restored Dockerfile to use npm run build
(tsc -b + vite) now that the config files are present.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 14:02:21 +01:00
8eeb774d4e fix: replace wget --spider with wget -qO /dev/null for healthcheck
All checks were successful
Deploy / deploy (push) Successful in 11s
--spider sends a HEAD request which newer BusyBox builds in nginx:alpine
may not handle consistently. A plain GET to /dev/null is more reliable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:40:31 +01:00
000391f7fc fix: guard migrations against fresh-database installs
All checks were successful
Deploy / deploy (push) Successful in 36s
On a fresh DB, migrations ran before the schema was applied and tried to
operate on tables that didn't exist yet (routes, scans), causing:
  "no such table: routes" on _migrate_add_routes_unique_index
  "no such table: scans" on _migrate_add_scheduled_scan_id_to_scans

Added table-existence checks so both migrations bail out when the table
isn't there yet. The schema's CREATE TABLE IF NOT EXISTS handles fresh
installs correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:35:45 +01:00
9b982ad9a5 fix: bypass tsc -b in Docker build, use vite directly
Some checks failed
Deploy / deploy (push) Failing after 54s
tsc -b with project references fails in the Alpine Docker environment
(TypeScript 5.9 + no composite:true on referenced configs). Vite uses
esbuild for TS compilation anyway, so tsc -b only served as a type-check
which is redundant in a production build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:32:40 +01:00
de491dbb1f Update .gitea/workflows/deploy.yml
Some checks failed
Deploy / deploy (push) Failing after 37s
2026-02-28 12:22:54 +01:00
7c125dbaeb fix: add domverse network to backend service
Some checks failed
Deploy / deploy (push) Failing after 37s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:07:54 +01:00
65b0d48f9d feat: add Traefik + Authentik integration to docker-compose
- Route https://flights.domverse-berlin.eu via Traefik on the domverse network
- Protect with Authentik (authentik@docker ForwardAuth middleware)
- Remove host port bindings (80, 8000) — Traefik handles all ingress
- Frontend joins both default compose network (nginx→backend) and domverse (Traefik)
- Backend stays internal-only, no external exposure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:07:31 +01:00
cdb8c20e82 ci: switch from GitLab CI to Gitea Actions, fix Dockerfile.backend
- Replace .gitlab-ci.yml with .gitea/workflows/deploy.yml
- Fix Dockerfile.backend: add scan_processor.py and searcher_v3.py to
  COPY command (they were missing, would cause runtime ImportError)
- Update docker-compose.yml comment to reference Gitea workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 11:42:02 +01:00
717b976293 ci: add GitLab CI/CD pipeline for Docker deploy
On every push to main: builds both Docker images on the server via
docker compose up --build -d, prunes dangling images, and prints the
running container list. No registry required — shell executor runner
on the deployment server is all that's needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 11:32:18 +01:00
836c8474eb feat: add scheduled scans (cron-like recurring scans)
- New `scheduled_scans` table with daily/weekly/monthly frequencies
- asyncio background scheduler loop checks for due schedules every 60s
- 6 REST endpoints: CRUD + toggle enabled + run-now
- `scheduled_scan_id` FK added to scans table; migrated automatically
- Frontend: Schedules page (list + create form), Schedules nav link,
  "Scheduled" badge on ScanDetails when scan was triggered by a schedule

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 10:48:43 +01:00
ef5a27097d fix: enrich route destination names from airport DB when not stored
Specific-airports mode scans never resolved full airport names — they
stored the IATA code as destination_name. Fixed in two places:

- airports.py: add lookup_airport(iata) cached helper
- api_server.py: enrich destination_name/city on the fly in the routes
  endpoint when the stored value equals the IATA code (fixes all past scans)
- scan_processor.py: resolve airport names at scan time in specific-airports
  mode using lookup_airport (fixes future scans at the DB level)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 21:04:46 +01:00
0a2fed7465 feat: add info icon tooltip for airport names in routes table
Replaces the non-functional title attribute with a small Info icon
next to the IATA code badge. Hovering shows a dark tooltip with the
full airport name and city. Only rendered when useful name data exists.
Clicking the icon stops propagation so it doesn't expand the flights row.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 20:59:37 +01:00
ce1cf667d2 feat: write routes live during scan instead of bulk-insert at completion
Routes and individual flights are now written to the database as each
query result arrives, rather than after all queries finish. The frontend
already polls /scans/:id/routes while status=running, so routes appear
progressively with no frontend changes needed.

Changes:
- database/schema.sql: UNIQUE INDEX uq_routes_scan_dest(scan_id, destination)
- database/init_db.py: _migrate_add_routes_unique_index() migration
- scan_processor.py: _write_route_incremental() helper; progress_callback
  now writes routes/flights immediately; Phase 2 bulk-write replaced with
  a lightweight totals query
- searcher_v3.py: pass flights= kwarg to progress_callback on cache_hit
  and api_success paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 20:53:04 +01:00
4926e89e46 feat: re-run and delete scan from detail page
Backend:
- DELETE /api/v1/scans/{id} — 204 on success, 404 if missing,
  409 if pending/running; CASCADE removes routes and flights

Frontend (api.ts):
- scanApi.delete(id)

Frontend (ScanDetails.tsx):
- Re-run button: derives window_months from stored dates, detects
  country vs airports mode via comma in scan.country, creates new
  scan and navigates to it; disabled while scan is active
- Delete button: inline two-step confirm (no modal), navigates to
  dashboard on success; disabled while scan is active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 16:33:45 +01:00
f9411edd3c remove: docker-compose.dev.yml — develop locally, deploy with docker compose
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 16:24:12 +01:00
06e6ae700f feat: add docker-compose.dev.yml for local development
- docker-compose.dev.yml: backend on 8000, frontend (Vite) on 5173
- Backend mounts source files + uvicorn --reload for hot reload
- Frontend uses node:20-alpine, mounts ./frontend, runs npm run dev --host
- vite.config.ts: proxy target reads from API_TARGET env var
  (defaults to localhost:8000 for plain npm run dev,
   set to http://backend:8000 by docker-compose.dev.yml)

Usage: docker compose -f docker-compose.dev.yml up

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 16:08:53 +01:00
6d168652d4 refactor: split back into two containers (backend + frontend)
Single-container supervisord approach added unnecessary complexity.
Two containers is simpler and more standard:

- Dockerfile.backend: python:3.11-slim, uvicorn on port 8000
- Dockerfile.frontend: node build → nginx:alpine on port 80
- nginx.conf: proxy_pass restored to http://backend:8000
- docker-compose.yml: two services with depends_on
- Removed combined Dockerfile and supervisord.conf

Each container does one thing; logs are separate; either can be
restarted independently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 16:06:26 +01:00
8bd47ac43a fix: improve airport search ranking and add missing modern airports
- Rewrite airport search to use priority buckets instead of simple
  append: exact IATA → IATA prefix → city prefix → city contains →
  name prefix → name contains → country match. This ensures BER
  appears before Berlin-Schönefeld when typing "BER".
- Add _MISSING_AIRPORTS patch list to get_airport_data() so airports
  absent from the OpenFlights dataset (e.g. BER opened Nov 2020,
  IST new Istanbul airport) are included at runtime.
- Deduplicate results via seen-set to avoid duplicate entries.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 15:48:43 +01:00
260f3aa196 fix: add web app deps and git to Docker build
- Added fastapi, uvicorn, pydantic, requests to requirements.txt
  (were missing — only CLI deps were present)
- Changed fast-flights entry to git+GitHub URL (v3 not on PyPI)
- Added git to apt-get install in Dockerfile (needed for pip git+ URL)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 15:42:11 +01:00
37 changed files with 5269 additions and 503 deletions

View File

@@ -0,0 +1,53 @@
# ──────────────────────────────────────────────────────────────────────────────
# Flight Radar — Gitea Actions CI/CD
#
# PREREQUISITES (one-time setup — see README for full instructions):
#
# 1. Add the act_runner service to your Gitea Portainer stack.
#
# 2. Pre-create the runner config file on the host:
# /srv/docker/traefik/stacks/gitea/volumes/act_runner/config.yaml
# (see content in the README / deployment docs)
#
# 3. Start the runner, then grab the registration token from:
# Gitea → Site Administration → Runners → Create Runner
# Add ACT_RUNNER_TOKEN to Portainer stack environment variables.
#
# 4. Give the runner access to Docker (socket mounted via config.yaml).
#
# PIPELINE BEHAVIOUR:
# • Triggers on every push to the default branch (main).
# • Builds both Docker images on the server (no registry needed).
# • Brings the app up with docker compose; only changed services restart.
# • If the build fails the old containers keep running — no downtime.
# • Prunes dangling images after a successful deploy.
# ──────────────────────────────────────────────────────────────────────────────
name: Deploy
on:
push:
branches:
- main
workflow_dispatch:
env:
COMPOSE_PROJECT: flight-radar
COMPOSE_FILE: flight-comparator/docker-compose.yml
jobs:
deploy:
runs-on: ubuntu-latest # resolved to catthehacker/ubuntu:act-22.04 by runner config
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy with docker compose
run: |
echo "=== Deploying commit ${{ gitea.sha }} to ${{ gitea.ref_name }} ==="
docker compose -f "$COMPOSE_FILE" -p "$COMPOSE_PROJECT" up --build -d --remove-orphans
- name: Prune dangling images
run: docker image prune -f
- name: Show running containers
run: docker compose -f "$COMPOSE_FILE" -p "$COMPOSE_PROJECT" ps

321
PRD-reverse-scan.md Normal file
View File

@@ -0,0 +1,321 @@
# PRD: Reverse Scan — Country → Specific Airport(s)
**Status:** Ready for implementation
**Date:** 2026-03-01
**Target:** Web App (api_server.py + scan_processor.py + database + frontend)
---
## 1. Problem
All scans today are fixed-origin, variable-destination:
> *"From BDS — which German airports can I fly to directly?"*
The opposite question is equally common but unsupported:
> *"I want to fly to BRI or BDS — which German airport should I depart from?"*
This is the natural perspective of a traveller based in Germany looking for the cheapest access point to Puglia, not a traveller already in Puglia wondering where to go.
---
## 2. Goal
Let users create a scan where:
- **Origin** = all airports in a country (e.g. DE)
- **Destination** = one or more specific airports (e.g. BRI, BDS)
The results table for a reverse scan shows one row per **origin airport × destination** pair, so the user can directly compare which German airport is cheapest for each destination.
---
## 3. User Stories
- **As a user**, I want to search DE → BRI,BDS so I can see which German airport has the cheapest direct flight to Puglia.
- **As a user**, I want to schedule a reverse scan to run weekly.
- **As a user**, I want to mix both directions in my scan history.
- **As a user**, I want the destination country in forward scans to be a dropdown, not free text.
---
## 4. Scope
### In scope
- `scan_mode` field on scans and scheduled scans: `"forward"` (default) or `"reverse"`
- Reverse scan: origin = ISO country (dropdown), destinations = specific airport(s)
- Forward scan: destination country changed from text input to dropdown
- New `GET /api/v1/countries` endpoint (powers both dropdowns)
- `routes` table: add `origin_airport` column (nullable; populated for reverse scans)
- `flights` table: add `origin_airport` column (nullable; populated for reverse scans)
- ScanDetails routes table: show Origin column for reverse scans
- Scheduled scans: support `scan_mode`
### Out of scope
- Multi-country origin (e.g. DE + IT → BRI)
- Round-trip reverse scans
---
## 5. Data Model Changes
### 5.1 `scans` table
```sql
ALTER TABLE scans ADD COLUMN scan_mode TEXT NOT NULL DEFAULT 'forward'
CHECK (scan_mode IN ('forward', 'reverse'));
```
Column semantics by mode:
| Column | Forward (existing) | Reverse (new) |
|--------|--------------------|---------------|
| `origin` | Single IATA (e.g. `BDS`) | ISO country code (e.g. `DE`) |
| `country` | ISO destination country OR comma-separated destination IATAs | Comma-separated destination IATAs (e.g. `BRI,BDS`) |
| `scan_mode` | `forward` | `reverse` |
All existing rows default to `forward`. No data migration needed.
### 5.2 `routes` table
```sql
ALTER TABLE routes ADD COLUMN origin_airport TEXT;
```
- **Forward scans**: `origin_airport` = NULL (origin is always the scan-level `origin` IATA, already known)
- **Reverse scans**: `origin_airport` = the specific origin IATA for this route (e.g. `BER`, `FRA`)
The unique constraint on routes changes from `(scan_id, destination)` to `(scan_id, COALESCE(origin_airport, ''), destination)`.
**Reverse scan routes example** for `DE → BRI, BDS`:
| origin_airport | destination | flight_count | min_price |
|----------------|-------------|--------------|-----------|
| BER | BRI | 14 | €15 |
| FMM | BRI | 8 | €17 |
| DUS | BRI | 6 | €22 |
| BER | BDS | 3 | €28 |
| MUC | BDS | 2 | €34 |
### 5.3 `flights` table
```sql
ALTER TABLE flights ADD COLUMN origin_airport TEXT;
```
- **Forward scans**: `origin_airport` = NULL (origin is the scan-level `origin` IATA)
- **Reverse scans**: `origin_airport` = the specific IATA the flight departs from
### 5.4 `scheduled_scans` table
```sql
ALTER TABLE scheduled_scans ADD COLUMN scan_mode TEXT NOT NULL DEFAULT 'forward'
CHECK (scan_mode IN ('forward', 'reverse'));
```
Same column semantics as `scans`. When the scheduler fires a reverse scheduled scan, it creates a reverse scan child.
---
## 6. New API Endpoint
### `GET /api/v1/countries`
Returns the list of countries that have at least one airport in the database. Powers both the forward-mode destination country dropdown and the reverse-mode origin country dropdown.
**Response:**
```json
{
"data": [
{ "code": "DE", "name": "Germany", "airport_count": 23 },
{ "code": "IT", "name": "Italy", "airport_count": 41 },
...
]
}
```
Sorted alphabetically by name. Derived from `data/airports_by_country.json` at startup (same source as airport resolution). Rate limit: 100/min.
Country names are resolved from a static ISO → name mapping in `api_server.py` (same `COUNTRY_NAME_TO_ISO` dict already in `airports.py`, inverted).
---
## 7. Backend Changes
### 7.1 `ScanRequest` model
Add `scan_mode: Literal['forward', 'reverse'] = 'forward'`.
Root validator enforces:
**Forward mode** (unchanged):
- `origin`: required, 3-char IATA
- `country` OR `destinations`: required
**Reverse mode**:
- `origin`: required, 2-char ISO country code
- `destinations`: required, 110 IATAs
- `country` must be absent or null
### 7.2 `scan_processor.py` — `process_scan()`
**Forward** (unchanged):
```
fixed origin IATA × [airports resolved from destination country / destination IATAs]
```
**Reverse** (new):
```
[airports resolved from origin country] × fixed destination IATA(s)
```
Steps for reverse:
1. Read `scan.origin` as ISO country code → resolve airport list via `airports.py`
2. Read `scan.country` as comma-separated destination IATAs
3. Build pairs: `[(orig, dest) for orig in origin_airports for dest in destination_iatas]`
4. For each pair: call `search_multiple_routes(orig, dest)` as usual
5. Save route with `origin_airport=orig`, `destination=dest`
6. Save each flight with `origin_airport=orig`
`total_routes` = `len(origin_airports) × len(destination_iatas)`.
### 7.3 `GET /scans/{id}/routes` response
Add `origin_airport: str | null` to the `Route` response model. Frontend uses this to show/hide the Origin column.
### 7.4 `GET /scans/{id}/flights` response
Add `origin_airport: str | null` to the `Flight` response model.
### 7.5 Scheduled scans
`ScheduleRequest` and `Schedule` models gain `scan_mode`. The scheduler's `_fire_due_scans()` function passes `scan_mode` when inserting child scans into the `scans` table.
---
## 8. Frontend Changes
### 8.1 New `countriesApi` client (`api.ts`)
```typescript
export interface Country { code: string; name: string; airport_count: number; }
export const countriesApi = {
list: () => api.get<{ data: Country[] }>('/countries'),
};
```
### 8.2 New `CountrySelect` component
A standard `<select>` dropdown, populated from `GET /api/v1/countries` on mount. Shows `"Germany (DE)"` style labels, emits the ISO code. Reused in two places:
- Forward scan form: destination country field (replaces current text input)
- Reverse scan form: origin country field
### 8.3 Scan creation form (`/scans` page)
Add a **Direction toggle** at the top of the form (same segmented button style as "By Country / By Airports"):
```
[ → Forward ] [ ← Reverse ]
```
**Forward mode** (current layout, destination country now uses `CountrySelect`):
```
Origin airport: [AirportSearch]
Destination: [By Country ▼] [By Airports]
→ Country: [CountrySelect dropdown]
→ Airports: [comma-sep IATA input]
```
**Reverse mode** (new):
```
Origin country: [CountrySelect dropdown]
Destination airports: [AirportSearch / comma-sep IATA input]
```
### 8.4 ScanDetails — routes table
For reverse scans (`scan_mode === 'reverse'`), prepend an **Origin** column to the routes table:
| Origin | Destination | Flights | Airlines | Min | Avg | Max |
|--------|-------------|---------|----------|-----|-----|-----|
| BER Berlin | BRI Bari | 14 | Ryanair | €15 | €22 | €45 |
| FMM Memmingen | BRI Bari | 8 | Ryanair | €17 | €25 | €50 |
The existing click-to-expand flights sub-table still works — shows individual flight dates/times/prices for that specific origin→destination pair.
For forward scans: routes table unchanged (no Origin column).
### 8.5 ScanDetails — header
```
Forward: BDS → DE
Reverse: DE → BRI, BDS
```
### 8.6 `CreateScanRequest` type update (`api.ts`)
```typescript
export interface CreateScanRequest {
scan_mode?: 'forward' | 'reverse';
origin: string; // IATA (forward) or ISO country code (reverse)
country?: string; // forward only
destinations?: string[]; // forward (by airports) or reverse
window_months?: number;
seat_class?: 'economy' | 'premium' | 'business' | 'first';
adults?: number;
}
```
---
## 9. API Contract
### `POST /api/v1/scans`
**Forward — by country (existing, unchanged):**
```json
{ "origin": "BDS", "country": "DE", "window_months": 6, "seat_class": "economy", "adults": 1 }
```
**Forward — by airports (existing, unchanged):**
```json
{ "origin": "BDS", "destinations": ["FRA", "MUC"], "window_months": 6, "seat_class": "economy", "adults": 1 }
```
**Reverse (new):**
```json
{ "scan_mode": "reverse", "origin": "DE", "destinations": ["BRI", "BDS"], "window_months": 6, "seat_class": "economy", "adults": 1 }
```
---
## 10. Migration (`init_db.py`)
Four `ALTER TABLE … ADD COLUMN` statements, each guarded by `try/except OperationalError`:
```python
migrations = [
"ALTER TABLE scans ADD COLUMN scan_mode TEXT NOT NULL DEFAULT 'forward' CHECK (scan_mode IN ('forward', 'reverse'))",
"ALTER TABLE routes ADD COLUMN origin_airport TEXT",
"ALTER TABLE flights ADD COLUMN origin_airport TEXT",
"ALTER TABLE scheduled_scans ADD COLUMN scan_mode TEXT NOT NULL DEFAULT 'forward' CHECK (scan_mode IN ('forward', 'reverse'))",
]
for sql in migrations:
try:
conn.execute(sql)
except sqlite3.OperationalError:
pass # column already exists
conn.commit()
```
---
## 11. Acceptance Criteria
1. `GET /api/v1/countries` returns a sorted list of countries with airport counts.
2. Forward scan destination country field is a dropdown populated from that endpoint.
3. A reverse scan `DE → BRI, BDS` can be created via the form and the API.
4. The processor iterates all German airports × [BRI, BDS], storing `origin_airport` on each route and flight row.
5. ScanDetails for a reverse scan shows the Origin column in the routes table and `DE → BRI, BDS` in the header.
6. Scheduled scans accept `scan_mode`; the scheduler passes it through to child scans.
7. All existing forward scans continue to work — no regressions.
8. All four DB columns default correctly after migration with no data loss.

View File

@@ -52,6 +52,9 @@ htmlcov/
!tests/confirmed_flights.json
!frontend/package.json
!frontend/package-lock.json
!frontend/tsconfig.json
!frontend/tsconfig.app.json
!frontend/tsconfig.node.json
# Database files
*.db

View File

@@ -1,54 +0,0 @@
# ── Stage 1: Build React frontend ─────────────────────────────────────────
FROM node:20-alpine AS frontend-builder
WORKDIR /app
COPY frontend/package*.json ./
RUN npm ci
COPY frontend/ .
RUN npm run build
# ── Stage 2: Single runtime image ─────────────────────────────────────────
FROM python:3.11-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1
# Install nginx, supervisor, and gcc (for some pip packages)
RUN apt-get update && apt-get install -y --no-install-recommends \
nginx \
supervisor \
gcc \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Backend source
COPY api_server.py airports.py cache.py ./
COPY database/ ./database/
# Frontend build output
COPY --from=frontend-builder /app/dist /usr/share/nginx/html
# Config files
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY supervisord.conf /etc/supervisor/conf.d/app.conf
# Remove the default nginx site
RUN rm -f /etc/nginx/sites-enabled/default
# Pre-fetch airport data and initialise the database at build time
RUN mkdir -p data && \
python -c "from airports import download_and_build_airport_data; download_and_build_airport_data()" && \
python database/init_db.py
EXPOSE 80
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
CMD wget -q --spider http://localhost/ || exit 1
CMD ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisor/conf.d/app.conf"]

View File

@@ -0,0 +1,30 @@
FROM python:3.11-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
git \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY api_server.py airports.py cache.py scan_processor.py searcher_v3.py ./
COPY database/ ./database/
# Pre-fetch airport data and initialise the database at build time
RUN mkdir -p data && \
python -c "from airports import download_and_build_airport_data; download_and_build_airport_data()" && \
python database/init_db.py
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
CMD ["python", "api_server.py"]

View File

@@ -0,0 +1,21 @@
# ── Stage 1: Build React frontend ─────────────────────────────────────────
FROM node:20-alpine AS builder
WORKDIR /app
COPY frontend/package*.json ./
RUN npm ci
COPY frontend/ .
RUN npm run build
# ── Stage 2: Serve with nginx ──────────────────────────────────────────────
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
RUN apk add --no-cache curl
EXPOSE 80
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
CMD curl -f http://localhost/ || exit 1

View File

@@ -0,0 +1,427 @@
# PRD: Live Routes During Active Scan
**Status:** Draft
**Date:** 2026-02-27
**Scope:** Backend only — no frontend or API contract changes required
---
## 1. Problem
Routes and flights are only visible in the UI **after a scan fully completes**. For large scans (e.g., 30 destinations × 14 days = 420 queries), users stare at a progress bar with zero actionable data for potentially many minutes.
The root cause is a two-phase waterfall in `scan_processor.py`:
- **Phase 1 (line 170):** `await search_multiple_routes(...)``asyncio.gather()` waits for **all** queries to finish before returning anything.
- **Phase 2 (line 197):** Bulk `INSERT INTO routes` and `INSERT INTO flights` only after Phase 1 completes.
---
## 2. Goal
Write each destination's route row and its individual flight rows to the database **as soon as that destination's results arrive**, rather than after all queries finish.
The frontend already polls `/scans/:id/routes` on an interval while status is `running`. No frontend changes are needed — routes will simply appear progressively.
---
## 3. Desired Behavior
| Moment | Before fix | After fix |
|--------|-----------|-----------|
| Query 1 completes (BER → MUC) | Nothing visible | MUC route + flights appear in UI |
| 50% through scan | 0 routes in UI | ~50% of routes visible |
| Scan completes | All routes appear at once | No change (already visible) |
---
## 4. Architecture Analysis
### 4.1 Event Loop Threading
`progress_callback` is called from within `search_direct_flights` (`searcher_v3.py`):
- **Cache hit** (line 121): called directly in the async coroutine → **event loop thread**
- **API success** (line 143): called after `await asyncio.to_thread()` returns → **event loop thread**
- **Error** (line 159): same → **event loop thread**
All callback invocations happen on the **single asyncio event loop thread**. No locking is needed. The existing `progress_callback` in `scan_processor.py` already opens and closes a fresh `get_connection()` per call — we reuse the same pattern.
### 4.2 Why Multiple Callbacks Fire Per Destination
For a scan covering 14 dates, a single destination (e.g., MUC) gets 14 separate queries:
- `(BER, MUC, 2026-03-01)`, `(BER, MUC, 2026-03-02)`, ... `(BER, MUC, 2026-03-14)`
Each query completes independently and fires `progress_callback`. The incremental write logic must **merge** successive results for the same destination into a single route row.
### 4.3 Required Schema Change
`routes` has no UNIQUE constraint on `(scan_id, destination)`. We need one to support upsert semantics (detect "route already started" vs "first result for this destination").
Current schema (no uniqueness):
```sql
CREATE INDEX IF NOT EXISTS idx_routes_scan_id ON routes(scan_id);
```
Required addition:
```sql
CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_dest ON routes(scan_id, destination);
```
---
## 5. Changes Required
### 5.1 `database/schema.sql`
Add a unique index at the end of the routes indexes block:
```sql
-- Unique constraint: one route row per (scan, destination)
CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_dest
ON routes(scan_id, destination);
```
### 5.2 `database/init_db.py`
Add a migration function to create the index on existing databases where it doesn't yet exist. Called before `executescript(schema_sql)`.
```python
def _migrate_add_routes_unique_index(conn, verbose=True):
"""
Migration: Add UNIQUE index on routes(scan_id, destination).
Required for incremental route writes during active scans.
The index enables upsert (INSERT + UPDATE on conflict) semantics.
Safe to run on existing data: if any (scan_id, destination) duplicates
exist, we collapse them first (keep the row with more flights).
"""
cursor = conn.execute("""
SELECT name FROM sqlite_master
WHERE type='index' AND name='uq_routes_scan_dest'
""")
if cursor.fetchone():
return # Already migrated
if verbose:
print(" 🔄 Migrating routes table: adding unique index on (scan_id, destination)...")
# Collapse any existing duplicates (completed scans may have none,
# but guard against edge cases).
conn.execute("""
DELETE FROM routes
WHERE id NOT IN (
SELECT MIN(id)
FROM routes
GROUP BY scan_id, destination
)
""")
# Create the unique index
conn.execute("""
CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_dest
ON routes(scan_id, destination)
""")
conn.commit()
if verbose:
print(" ✅ Migration complete: uq_routes_scan_dest index created")
```
Wire it into `initialize_database()` before `executescript`:
```python
# Apply migrations before running schema
_migrate_relax_country_constraint(conn, verbose)
_migrate_add_routes_unique_index(conn, verbose) # ← ADD THIS
# Load and execute schema
schema_sql = load_schema()
```
### 5.3 `scan_processor.py` — Extend `progress_callback`
#### Current signature (line 138):
```python
def progress_callback(origin: str, destination: str, date: str,
status: str, count: int, error: str = None):
```
#### New signature:
```python
def progress_callback(origin: str, destination: str, date: str,
status: str, count: int, error: str = None,
flights: list = None):
```
#### New helper function (add before `process_scan`):
```python
def _write_route_incremental(scan_id: int, destination: str,
dest_name: str, dest_city: str,
new_flights: list):
"""
Write or update a route row and its flight rows incrementally.
Called from progress_callback each time a query for (scan_id, destination)
returns results. Merges into the existing route row if one already exists.
Uses a read-then-write pattern so airlines JSON arrays can be merged in
Python rather than in SQLite (SQLite has no native JSON array merge).
Safe to call from the event loop thread — opens its own connection.
"""
if not new_flights:
return
prices = [f.get('price') for f in new_flights if f.get('price')]
if not prices:
return # No priced flights — nothing to aggregate
new_airlines = list({f.get('airline') for f in new_flights if f.get('airline')})
new_count = len(new_flights)
new_min = min(prices)
new_max = max(prices)
new_avg = sum(prices) / len(prices)
try:
conn = get_connection()
cursor = conn.cursor()
# Check if a route row already exists for this destination
cursor.execute("""
SELECT id, flight_count, min_price, max_price, avg_price, airlines
FROM routes
WHERE scan_id = ? AND destination = ?
""", (scan_id, destination))
existing = cursor.fetchone()
if existing is None:
# First result for this destination — INSERT
cursor.execute("""
INSERT INTO routes (
scan_id, destination, destination_name, destination_city,
flight_count, airlines, min_price, max_price, avg_price
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id, destination, dest_name, dest_city,
new_count, json.dumps(new_airlines),
new_min, new_max, new_avg,
))
else:
# Subsequent result — merge stats
old_count = existing['flight_count'] or 0
old_min = existing['min_price']
old_max = existing['max_price']
old_avg = existing['avg_price'] or 0.0
old_airlines = json.loads(existing['airlines']) if existing['airlines'] else []
merged_count = old_count + new_count
merged_min = min(old_min, new_min) if old_min is not None else new_min
merged_max = max(old_max, new_max) if old_max is not None else new_max
# Running weighted average
merged_avg = (old_avg * old_count + new_avg * new_count) / merged_count
merged_airlines = json.dumps(list(set(old_airlines) | set(new_airlines)))
cursor.execute("""
UPDATE routes
SET flight_count = ?,
min_price = ?,
max_price = ?,
avg_price = ?,
airlines = ?
WHERE scan_id = ? AND destination = ?
""", (
merged_count, merged_min, merged_max, merged_avg, merged_airlines,
scan_id, destination,
))
# INSERT individual flight rows (always new rows; duplicates not expected
# because each (scan_id, destination, date) query fires exactly once)
for flight in new_flights:
if not flight.get('price'):
continue
cursor.execute("""
INSERT INTO flights (
scan_id, destination, date, airline,
departure_time, arrival_time, price, stops
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id,
destination,
flight.get('date', ''), # date passed from caller
flight.get('airline'),
flight.get('departure_time'),
flight.get('arrival_time'),
flight.get('price'),
flight.get('stops', 0),
))
conn.commit()
conn.close()
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to write incremental route {destination}: {e}")
```
**Note on flight date:** Individual flights must carry their `date` when passed to `_write_route_incremental`. The callback receives `date` as a parameter, so attach it to each flight dict before passing: `f['date'] = date`.
#### Updated `progress_callback` inside `process_scan`:
```python
def progress_callback(origin: str, destination: str, date: str,
status: str, count: int, error: str = None,
flights: list = None):
nonlocal routes_scanned_count
if status in ('cache_hit', 'api_success', 'error'):
routes_scanned_count += 1
# Write route + flights immediately if we have results
if flights and status in ('cache_hit', 'api_success'):
# Annotate each flight with its query date
for f in flights:
f['date'] = date
dest_info = next((d for d in destinations if d['iata'] == destination), None)
dest_name = dest_info.get('name', destination) if dest_info else destination
dest_city = dest_info.get('city', '') if dest_info else ''
_write_route_incremental(scan_id, destination, dest_name, dest_city, flights)
# Update progress counter
try:
progress_conn = get_connection()
progress_cursor = progress_conn.cursor()
progress_cursor.execute("""
UPDATE scans
SET routes_scanned = routes_scanned + 1,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (scan_id,))
progress_conn.commit()
progress_conn.close()
if routes_scanned_count % 10 == 0:
logger.info(f"[Scan {scan_id}] Progress: {routes_scanned_count}/{len(routes_to_scan)} routes ({status}: {origin}{destination})")
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to update progress: {str(e)}")
```
#### Replace Phase 2 (lines 182262 in `process_scan`):
Remove the existing bulk-write block. Replace with a lightweight totals-only block:
```python
# Wait for all queries to complete
results = await search_multiple_routes(
routes=routes_to_scan,
seat_class=seat_class or 'economy',
adults=adults or 1,
use_cache=True,
cache_threshold_hours=24,
max_workers=3,
progress_callback=progress_callback
)
logger.info(f"[Scan {scan_id}] All queries complete. Finalizing scan...")
# Count total flights (routes already written by callback)
total_flights = sum(len(flights) for flights in results.values())
routes_saved = cursor.execute(
"SELECT COUNT(*) FROM routes WHERE scan_id = ?", (scan_id,)
).fetchone()[0]
logger.info(f"[Scan {scan_id}] ✅ {routes_saved} routes, {total_flights} flights")
```
### 5.4 `searcher_v3.py` — Pass `flights` to callback
In `search_direct_flights`, update both callback calls to pass `flights=`:
**Cache hit (line 121):**
```python
if progress_callback:
progress_callback(origin, destination, date, "cache_hit", len(cached), flights=cached)
return cached
```
**API success (line 143):**
```python
if progress_callback:
progress_callback(origin, destination, date, "api_success", len(result), flights=result)
```
The `error` callback (line 159) does not need `flights=` since there are no results.
---
## 6. Concurrency & Safety
| Concern | Analysis | Verdict |
|---------|----------|---------|
| Simultaneous writes to same route row | Cannot happen: the event loop is single-threaded; callbacks are not concurrent. `asyncio.gather` runs tasks concurrently but yields at `await` points, and callbacks fire synchronously after each `await to_thread` returns. | ✅ Safe |
| Two callbacks for same destination arriving "simultaneously" | Impossible in single-threaded event loop. Second callback blocks until first completes (no `await` in callback). | ✅ Safe |
| Duplicate flight rows | Each `(scan_id, destination, date)` query fires exactly once; its flights are written exactly once. | ✅ No duplicates |
| `total_flights` trigger still fires correctly | SQLite triggers on `INSERT INTO routes` and `UPDATE OF flight_count ON routes` fire for each incremental write — counts stay accurate. | ✅ Works |
| Scan completion `total_flights` update | Still set explicitly at completion from `results` dict count — redundant but harmless. | ✅ OK |
---
## 7. Edge Cases
| Case | Handling |
|------|----------|
| Destination returns 0 flights | `_write_route_incremental` returns early — no row created. Route only appears if at least one priced flight found. |
| Scan is deleted mid-run | `DELETE CASCADE` on `scans` removes routes/flights automatically. Progress callback write will fail with FK error, caught by `except` block and logged. |
| Scan fails mid-run | Routes written so far remain in DB. Status set to `failed`. UI will show partial results with `failed` badge — acceptable. |
| DB write error in callback | Logged, does not crash the scan. Query continues, flight data lost for that callback. |
| Existing scans (pre-feature) | No impact. Migration adds index but doesn't change old data (all complete scans already have 1 row per destination). |
---
## 8. Migration Plan
1. **`database/schema.sql`**: Add `CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_dest`.
2. **`database/init_db.py`**: Add `_migrate_add_routes_unique_index()` + call it in `initialize_database()`.
3. **`scan_processor.py`**: Add `_write_route_incremental()` helper; update `progress_callback` closure; remove bulk-write Phase 2.
4. **`searcher_v3.py`**: Pass `flights=` kwarg to both successful callback invocations.
**Migration is backward-safe:** The UNIQUE index is added with `IF NOT EXISTS`. Existing `completed` scans already have at most 1 route row per destination — the index creation will succeed without errors.
**No API changes:** `/scans/:id/routes` endpoint already returns live data from the `routes` table. The frontend polling already works.
---
## 9. Rollback
To revert: remove the `flights=` kwarg from `searcher_v3.py` callbacks, restore the bulk-write Phase 2 in `scan_processor.py`, and remove `_write_route_incremental`. The UNIQUE index can remain — it only adds a constraint that is naturally satisfied by the bulk-write approach anyway.
---
## 10. Testing Plan
### Unit Tests (new)
1. `test_write_route_incremental_new` — first call creates route row
2. `test_write_route_incremental_merge` — second call updates stats correctly
3. `test_write_route_incremental_no_prices` — empty-price flights produce no row
4. `test_write_route_incremental_airlines_merge` — duplicate airlines deduplicated
5. `test_weighted_average_formula` — verify avg formula with known numbers
### Integration Tests (extend existing)
6. Extend `test_scan_lifecycle` — poll routes every 0.1s during mock scan, verify routes appear before completion
7. `test_incremental_writes_idempotent` — simulate same callback called twice for same destination
8. `test_unique_index_exists` — verify migration creates index
9. `test_migration_collapses_duplicates` — seed duplicate route rows, run migration, verify collapsed
---
## 11. Out of Scope
- WebSocket or SSE for push-based updates (polling already works)
- Frontend changes (none needed)
- Real-time price charts
- Partial scan resume after crash
- `total_flights` trigger removal (keep for consistency)

View File

@@ -0,0 +1,409 @@
# PRD: Scheduled Scans
**Status:** Draft
**Date:** 2026-02-27
**Verdict:** Fully feasible — no new dependencies required
---
## 1. Problem
Every scan is triggered manually. If you want to track prices for a route over time (e.g. BDS → Germany every Monday) you have to remember to click "Re-run" yourself. Price trends are only discoverable by comparing scan history manually.
---
## 2. Goal
Let users define a recurring schedule for any scan configuration. The server runs the scan automatically at the defined cadence, building a historical record of price data over time.
---
## 3. User Stories
- **As a user**, I want to schedule a weekly scan of BDS → Germany so I can see how prices change without manually re-running it.
- **As a user**, I want to enable/disable a schedule without deleting it.
- **As a user**, I want to see which scans were created by a schedule and navigate to that schedule from a scan.
- **As a user**, I want to trigger a scheduled scan immediately without waiting for the next interval.
---
## 4. Scheduling Options
Three frequencies are sufficient for flight price tracking:
| Frequency | Parameters | Example |
|-----------|-----------|---------|
| `daily` | hour, minute | Every day at 06:00 |
| `weekly` | day_of_week (0=Mon6=Sun), hour, minute | Every Monday at 06:00 |
| `monthly` | day_of_month (128), hour, minute | 1st of every month at 06:00 |
Day of month capped at 28 to avoid Feb 29/30/31 edge cases. All times stored and executed in UTC.
---
## 5. Architecture
### 5.1 Scheduler Design
No new dependencies. A simple asyncio background task wakes every 60 seconds, queries the DB for due schedules, and fires a scan for each.
```
lifespan startup
└── asyncio.create_task(_scheduler_loop())
└── while True:
_check_and_run_due_schedules() # queries DB
await asyncio.sleep(60)
```
`_check_and_run_due_schedules()`:
1. `SELECT * FROM scheduled_scans WHERE enabled=1 AND next_run_at <= NOW()`
2. For each result, skip if previous scan for this schedule is still `pending` or `running`
3. Create a new scan row (same INSERT as `POST /scans`)
4. Call `start_scan_processor(scan_id)`
5. Update `last_run_at = NOW()` and compute + store `next_run_at`
### 5.2 `next_run_at` Computation
Precomputed in Python after every run (and on create/update). Stored as a TIMESTAMP column with an index — scheduler lookup is a single indexed range query.
```python
def compute_next_run(frequency, hour, minute,
day_of_week=None, day_of_month=None,
after=None) -> datetime:
now = after or datetime.utcnow()
base = now.replace(hour=hour, minute=minute, second=0, microsecond=0)
if frequency == 'daily':
return base if base > now else base + timedelta(days=1)
elif frequency == 'weekly':
days_ahead = (day_of_week - now.weekday()) % 7
if days_ahead == 0 and base <= now:
days_ahead = 7
return (now + timedelta(days=days_ahead)).replace(
hour=hour, minute=minute, second=0, microsecond=0)
elif frequency == 'monthly':
candidate = now.replace(day=day_of_month, hour=hour, minute=minute, second=0, microsecond=0)
if candidate <= now:
m, y = (now.month % 12) + 1, now.year + (1 if now.month == 12 else 0)
candidate = candidate.replace(year=y, month=m)
return candidate
```
---
## 6. Schema Changes
### 6.1 New table: `scheduled_scans`
```sql
CREATE TABLE IF NOT EXISTS scheduled_scans (
id INTEGER PRIMARY KEY AUTOINCREMENT,
-- Scan parameters
origin TEXT NOT NULL CHECK(length(origin) = 3),
country TEXT NOT NULL CHECK(length(country) >= 2),
window_months INTEGER NOT NULL DEFAULT 1
CHECK(window_months >= 1 AND window_months <= 12),
seat_class TEXT NOT NULL DEFAULT 'economy',
adults INTEGER NOT NULL DEFAULT 1
CHECK(adults > 0 AND adults <= 9),
-- Schedule definition
frequency TEXT NOT NULL
CHECK(frequency IN ('daily', 'weekly', 'monthly')),
hour INTEGER NOT NULL DEFAULT 6
CHECK(hour >= 0 AND hour <= 23),
minute INTEGER NOT NULL DEFAULT 0
CHECK(minute >= 0 AND minute <= 59),
day_of_week INTEGER CHECK(day_of_week >= 0 AND day_of_week <= 6),
day_of_month INTEGER CHECK(day_of_month >= 1 AND day_of_month <= 28),
-- State
enabled INTEGER NOT NULL DEFAULT 1,
label TEXT,
last_run_at TIMESTAMP,
next_run_at TIMESTAMP NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- Frequency-specific constraints
CHECK(
(frequency = 'weekly' AND day_of_week IS NOT NULL) OR
(frequency = 'monthly' AND day_of_month IS NOT NULL) OR
(frequency = 'daily')
)
);
-- Fast lookup of due schedules
CREATE UNIQUE INDEX IF NOT EXISTS uq_scheduled_scans_id
ON scheduled_scans(id);
CREATE INDEX IF NOT EXISTS idx_scheduled_scans_next_run
ON scheduled_scans(next_run_at)
WHERE enabled = 1;
-- Auto-update updated_at
CREATE TRIGGER IF NOT EXISTS update_scheduled_scans_timestamp
AFTER UPDATE ON scheduled_scans
FOR EACH ROW BEGIN
UPDATE scheduled_scans SET updated_at = CURRENT_TIMESTAMP WHERE id = NEW.id;
END;
-- Insert schema version bump
INSERT OR IGNORE INTO schema_version (version, description)
VALUES (2, 'Add scheduled_scans table');
```
### 6.2 Add FK column to `scans`
```sql
-- Migration: add scheduled_scan_id to scans
ALTER TABLE scans ADD COLUMN scheduled_scan_id INTEGER
REFERENCES scheduled_scans(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_scans_scheduled_scan_id
ON scans(scheduled_scan_id)
WHERE scheduled_scan_id IS NOT NULL;
```
---
## 7. Migration (`database/init_db.py`)
Add two migration functions, called before `executescript(schema_sql)`:
```python
def _migrate_add_scheduled_scans(conn, verbose=True):
"""Migration: create scheduled_scans table and add FK to scans."""
cursor = conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='scheduled_scans'"
)
if cursor.fetchone():
return # Already exists
if verbose:
print(" 🔄 Migrating: adding scheduled_scans table...")
conn.execute("""
CREATE TABLE scheduled_scans (
id INTEGER PRIMARY KEY AUTOINCREMENT, ...
)
""")
# Add scheduled_scan_id to existing scans table
try:
conn.execute("ALTER TABLE scans ADD COLUMN scheduled_scan_id INTEGER REFERENCES scheduled_scans(id) ON DELETE SET NULL")
except sqlite3.OperationalError:
pass # Column already exists
conn.execute("CREATE INDEX IF NOT EXISTS idx_scans_scheduled_scan_id ON scans(scheduled_scan_id) WHERE scheduled_scan_id IS NOT NULL")
conn.commit()
if verbose:
print(" ✅ Migration complete: scheduled_scans table created")
```
---
## 8. API Endpoints
All under `/api/v1/schedules`. Rate limit: 30 req/min per IP (same as scans list).
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/schedules` | List all schedules (paginated) |
| `POST` | `/schedules` | Create a schedule |
| `GET` | `/schedules/{id}` | Schedule details + last 5 scan IDs |
| `PATCH` | `/schedules/{id}` | Update (enable/disable, change frequency/params) |
| `DELETE` | `/schedules/{id}` | Delete schedule (scans are kept, FK set to NULL) |
| `POST` | `/schedules/{id}/run-now` | Trigger immediately (ignores next_run_at) |
### Request model: `CreateScheduleRequest`
```python
class CreateScheduleRequest(BaseModel):
origin: str # 3-char IATA
country: Optional[str] # 2-letter ISO country code
destinations: Optional[List[str]] # Alternative: list of IATA codes
window_months: int = 1 # Weeks of data per scan run
seat_class: str = 'economy'
adults: int = 1
label: Optional[str] # Human-readable name
frequency: str # 'daily' | 'weekly' | 'monthly'
hour: int = 6 # UTC hour (023)
minute: int = 0 # UTC minute (059)
day_of_week: Optional[int] # Required when frequency='weekly' (0=Mon)
day_of_month: Optional[int] # Required when frequency='monthly' (128)
```
### Response model: `Schedule`
```python
class Schedule(BaseModel):
id: int
origin: str
country: str
window_months: int
seat_class: str
adults: int
label: Optional[str]
frequency: str
hour: int
minute: int
day_of_week: Optional[int]
day_of_month: Optional[int]
enabled: bool
last_run_at: Optional[str]
next_run_at: str
created_at: str
recent_scan_ids: List[int] # Last 5 scans created by this schedule
```
---
## 9. Scheduler Lifecycle (`api_server.py`)
### 9.1 Startup
In the existing `lifespan()` context manager, after existing startup code:
```python
scheduler_task = asyncio.create_task(_scheduler_loop())
logger.info("Scheduled scan background task started")
yield
scheduler_task.cancel()
try:
await scheduler_task
except asyncio.CancelledError:
pass
```
### 9.2 Missed runs on restart
When the server starts, `_check_and_run_due_schedules()` fires immediately (before the 60-second sleep), catching any schedules that were due while the server was down. Each overdue schedule runs exactly once — `next_run_at` is then advanced to the next future interval. Multiple missed intervals are not caught up.
### 9.3 Concurrency guard
Before firing a scan for a schedule, check:
```python
running = conn.execute("""
SELECT id FROM scans
WHERE scheduled_scan_id = ? AND status IN ('pending', 'running')
""", (schedule_id,)).fetchone()
if running:
logger.info(f"Schedule {schedule_id}: previous scan {running[0]} still active, skipping this run")
# Still advance next_run_at so we try again next interval
continue
```
---
## 10. Frontend Changes
### 10.1 New page: `Schedules.tsx`
**List view:**
- Table of all schedules: label, origin → country, frequency, next run (local time), last run, enabled toggle
- "New Schedule" button opens create form (same airport search component as Scans)
- Inline enable/disable toggle (PATCH request, optimistic update)
- "Run now" button per row
**Create form fields (below existing scan form fields):**
- Frequency selector: Daily / Weekly / Monthly (segmented button)
- Time of day: hour:minute picker (UTC, with note)
- Day of week (shown only for Weekly): MonSun selector
- Day of month (shown only for Monthly): 128 number input
- Optional label field
### 10.2 Modified: `ScanDetails.tsx`
When a scan has `scheduled_scan_id`, show a small "Scheduled" chip in the header with a link to `/schedules/{scheduled_scan_id}`.
### 10.3 Navigation (`Layout.tsx`)
Add "Schedules" link to sidebar between Scans and Airports.
### 10.4 API client (`api.ts`)
```typescript
export interface Schedule {
id: number;
origin: string;
country: string;
window_months: number;
seat_class: string;
adults: number;
label?: string;
frequency: 'daily' | 'weekly' | 'monthly';
hour: number;
minute: number;
day_of_week?: number;
day_of_month?: number;
enabled: boolean;
last_run_at?: string;
next_run_at: string;
created_at: string;
recent_scan_ids: number[];
}
export const scheduleApi = {
list: (page = 1, limit = 20) =>
api.get<PaginatedResponse<Schedule>>('/schedules', { params: { page, limit } }),
get: (id: number) =>
api.get<Schedule>(`/schedules/${id}`),
create: (data: CreateScheduleRequest) =>
api.post<Schedule>('/schedules', data),
update: (id: number, data: Partial<CreateScheduleRequest> & { enabled?: boolean }) =>
api.patch<Schedule>(`/schedules/${id}`, data),
delete: (id: number) =>
api.delete(`/schedules/${id}`),
runNow: (id: number) =>
api.post<{ scan_id: number }>(`/schedules/${id}/run-now`),
};
```
---
## 11. Edge Cases
| Case | Handling |
|------|----------|
| Previous scan still running at next interval | Skip this interval's run, advance `next_run_at`, log warning |
| Server down when schedule is due | On startup, runs any overdue schedule once; does not catch up multiple missed intervals |
| Schedule deleted while scan is running | `ON DELETE SET NULL` on FK — scan continues, `scheduled_scan_id` becomes NULL |
| `window_months` covers past dates | Scan start date is always "tomorrow" at creation time, same as manual scans |
| Monthly with day_of_month=29..31 | Capped at 28 in validation — avoids invalid dates in all months |
| Simultaneous due schedules | Each creates an independent asyncio task; existing `max_workers=3` semaphore in scan_processor limits total API concurrency across all running scans |
| Schedule created at 05:59, fires at 06:00 UTC | `next_run_at` is computed at creation time — if 06:00 today already passed, fires tomorrow |
---
## 12. Files Changed
| File | Change |
|------|--------|
| `database/schema.sql` | Add `scheduled_scans` table, trigger, indexes, schema_version bump |
| `database/init_db.py` | `_migrate_add_scheduled_scans()` + call in `initialize_database()` |
| `api_server.py` | `compute_next_run()`, `_scheduler_loop()`, `_check_and_run_due_schedules()`, 6 new endpoints, lifespan update, new Pydantic models |
| `frontend/src/api.ts` | `Schedule` type, `CreateScheduleRequest` type, `scheduleApi` object |
| `frontend/src/pages/Schedules.tsx` | New page (list + inline create form) |
| `frontend/src/pages/ScanDetails.tsx` | "Scheduled" badge + link when `scheduled_scan_id` present |
| `frontend/src/components/Layout.tsx` | Schedules nav link |
Total: 7 files. Estimated ~500 new lines (backend ~250, frontend ~250).
---
## 13. Out of Scope
- Notifications / alerts when a scheduled scan completes (email, webhook)
- Per-schedule price change detection / diffing between runs
- Timezone-aware scheduling (all times UTC for now)
- Pause/resume of scheduled scans (separate PRD)
- Rate limiting across simultaneous scheduled scans (existing semaphore provides soft protection)
- Dashboard widgets for upcoming scheduled runs

View File

@@ -6,6 +6,7 @@ Handles loading and filtering airport data from OpenFlights dataset.
import json
import csv
from functools import lru_cache
from pathlib import Path
from typing import Optional
import urllib.request
@@ -68,6 +69,13 @@ COUNTRY_NAME_TO_ISO = {
}
# Airports missing from the OpenFlights dataset (opened or renamed after dataset was last updated).
# Keyed by ISO country code; dicts match the airports_by_country.json schema (iata/name/city/icao).
_MISSING_AIRPORTS: dict[str, list[dict]] = {
'DE': [{'iata': 'BER', 'name': 'Berlin Brandenburg Airport', 'city': 'Berlin', 'icao': 'EDDB'}],
}
def country_name_to_iso_code(country_name: str) -> Optional[str]:
"""
Convert country name to ISO 2-letter code.
@@ -196,7 +204,12 @@ def get_airports_for_country(country_code: str) -> list[dict]:
f"Available codes (sample): {', '.join(available)}..."
)
return airports_by_country[country_code]
result = list(airports_by_country[country_code])
existing_iatas = {a['iata'] for a in result}
for extra in _MISSING_AIRPORTS.get(country_code, []):
if extra['iata'] not in existing_iatas:
result.append(extra)
return result
def resolve_airport_list(country: Optional[str], from_airports: Optional[str]) -> list[dict]:
@@ -225,6 +238,30 @@ def resolve_airport_list(country: Optional[str], from_airports: Optional[str]) -
raise ValueError("Either --country or --from must be provided")
@lru_cache(maxsize=1)
def _all_airports_by_iata() -> dict:
"""Return {iata: airport_dict} for every airport. Cached after first load."""
if not AIRPORTS_JSON_PATH.exists():
download_and_build_airport_data()
with open(AIRPORTS_JSON_PATH, 'r', encoding='utf-8') as f:
airports_by_country = json.load(f)
result = {
a['iata']: a
for airports in airports_by_country.values()
for a in airports
}
for extras in _MISSING_AIRPORTS.values():
for extra in extras:
if extra['iata'] not in result:
result[extra['iata']] = extra
return result
def lookup_airport(iata: str) -> dict | None:
"""Look up a single airport by IATA code. Returns None if not found."""
return _all_airports_by_iata().get(iata.upper())
if __name__ == "__main__":
# Build the dataset if run directly
download_and_build_airport_data(force_rebuild=True)

File diff suppressed because it is too large Load Diff

View File

@@ -130,6 +130,373 @@ def _migrate_relax_country_constraint(conn, verbose=True):
print(" ✅ Migration complete: country column now accepts >= 2 chars")
def _migrate_add_routes_unique_index(conn, verbose=True):
"""
Migration: Add UNIQUE index on routes(scan_id, destination).
Required for incremental route writes during active scans.
Collapses any pre-existing duplicate (scan_id, destination) rows first
(keeps the row with the lowest id) before creating the index.
"""
# Fresh install: routes table doesn't exist yet — schema will create the index
cursor = conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='routes'"
)
if not cursor.fetchone():
return
cursor = conn.execute(
"SELECT name FROM sqlite_master WHERE type='index' AND name='uq_routes_scan_dest'"
)
if cursor.fetchone():
return # Already migrated
if verbose:
print(" 🔄 Migrating routes table: adding UNIQUE index on (scan_id, destination)...")
# Collapse any existing duplicates (guard against edge cases)
conn.execute("""
DELETE FROM routes
WHERE id NOT IN (
SELECT MIN(id)
FROM routes
GROUP BY scan_id, destination
)
""")
conn.execute("""
CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_dest
ON routes(scan_id, destination)
""")
conn.commit()
if verbose:
print(" ✅ Migration complete: uq_routes_scan_dest index created")
def _migrate_add_scheduled_scan_id_to_scans(conn, verbose=True):
"""
Migration: add scheduled_scan_id column to scans table.
Existing rows get NULL (manual scans). New column has no inline FK
declaration because SQLite's ALTER TABLE ADD COLUMN doesn't support it;
the relationship is enforced at the application level.
"""
cursor = conn.execute("PRAGMA table_info(scans)")
columns = [row[1] for row in cursor.fetchall()]
if not columns:
return # Fresh install: scans table doesn't exist yet — schema will create the column
if 'scheduled_scan_id' in columns:
return # Already migrated
if verbose:
print(" 🔄 Migrating scans table: adding scheduled_scan_id column...")
conn.execute("ALTER TABLE scans ADD COLUMN scheduled_scan_id INTEGER")
conn.commit()
if verbose:
print(" ✅ Migration complete: scheduled_scan_id column added to scans")
def _migrate_add_timing_columns_to_scans(conn, verbose=True):
"""
Migration: add started_at and completed_at columns to the scans table.
started_at — set when status transitions to 'running'
completed_at — set when status transitions to 'completed' or 'failed'
Both are nullable so existing rows are unaffected.
"""
cursor = conn.execute("PRAGMA table_info(scans)")
columns = [row[1] for row in cursor.fetchall()]
if not columns:
return # Fresh install: scans table doesn't exist yet — schema will create the columns
if 'started_at' in columns and 'completed_at' in columns:
return # Already migrated
if verbose:
print(" 🔄 Migrating scans table: adding started_at and completed_at columns...")
if 'started_at' not in columns:
conn.execute("ALTER TABLE scans ADD COLUMN started_at TIMESTAMP")
if 'completed_at' not in columns:
conn.execute("ALTER TABLE scans ADD COLUMN completed_at TIMESTAMP")
conn.commit()
if verbose:
print(" ✅ Migration complete: started_at and completed_at columns added to scans")
def _recover_orphaned_new_tables(conn, verbose=True):
"""
Recovery: if a previous migration left behind scans_new or scheduled_scans_new
(e.g. after a crash between DROP TABLE scans and RENAME), restore them.
"""
tables = [r[0] for r in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()]
if 'scans_new' in tables and 'scans' not in tables:
if verbose:
print(" 🔧 Recovering: renaming orphaned scans_new → scans")
conn.execute("ALTER TABLE scans_new RENAME TO scans")
conn.commit()
if 'scheduled_scans_new' in tables and 'scheduled_scans' not in tables:
if verbose:
print(" 🔧 Recovering: renaming orphaned scheduled_scans_new → scheduled_scans")
conn.execute("ALTER TABLE scheduled_scans_new RENAME TO scheduled_scans")
conn.commit()
def _migrate_add_pause_cancel_status(conn, verbose=True):
"""
Migration: Extend status CHECK constraint to include 'paused' and 'cancelled'.
Needed for cancel/pause/resume scan flow control feature.
Uses the same table-recreation pattern as _migrate_relax_country_constraint
because SQLite doesn't support modifying CHECK constraints in-place.
"""
cursor = conn.execute(
"SELECT sql FROM sqlite_master WHERE type='table' AND name='scans'"
)
row = cursor.fetchone()
if not row or 'paused' in row[0]:
return # Table doesn't exist yet (fresh install) or already migrated
if verbose:
print(" 🔄 Migrating scans table: adding 'paused' and 'cancelled' status values...")
# SQLite doesn't support ALTER TABLE MODIFY COLUMN, so recreate the table.
# Use PRAGMA foreign_keys = OFF to avoid FK errors during the swap.
conn.execute("PRAGMA foreign_keys = OFF")
# Drop views that reference scans so they can be cleanly recreated by executescript.
conn.execute("DROP VIEW IF EXISTS recent_scans")
conn.execute("DROP VIEW IF EXISTS active_scans")
# Drop any leftover _new table from a previously aborted migration.
conn.execute("DROP TABLE IF EXISTS scans_new")
# Drop triggers that reference scans (they are recreated by executescript below).
conn.execute("DROP TRIGGER IF EXISTS update_scans_timestamp")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_insert")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_update")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_delete")
conn.execute("""
CREATE TABLE scans_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
origin TEXT NOT NULL CHECK(length(origin) = 3),
country TEXT NOT NULL CHECK(length(country) >= 2),
start_date TEXT NOT NULL,
end_date TEXT NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
started_at TIMESTAMP,
completed_at TIMESTAMP,
status TEXT NOT NULL DEFAULT 'pending'
CHECK(status IN ('pending', 'running', 'completed', 'failed', 'cancelled', 'paused')),
total_routes INTEGER NOT NULL DEFAULT 0 CHECK(total_routes >= 0),
routes_scanned INTEGER NOT NULL DEFAULT 0 CHECK(routes_scanned >= 0),
total_flights INTEGER NOT NULL DEFAULT 0 CHECK(total_flights >= 0),
error_message TEXT,
seat_class TEXT DEFAULT 'economy',
adults INTEGER DEFAULT 1 CHECK(adults > 0 AND adults <= 9),
scheduled_scan_id INTEGER,
CHECK(end_date >= start_date),
CHECK(routes_scanned <= total_routes OR total_routes = 0)
)
""")
# Use named columns to handle different column orderings (ALTER TABLE vs fresh schema).
conn.execute("""
INSERT INTO scans_new (
id, origin, country, start_date, end_date,
created_at, updated_at, started_at, completed_at,
status, total_routes, routes_scanned, total_flights,
error_message, seat_class, adults, scheduled_scan_id
)
SELECT
id, origin, country, start_date, end_date,
created_at, updated_at, started_at, completed_at,
status, total_routes, routes_scanned, total_flights,
error_message, seat_class, adults, scheduled_scan_id
FROM scans
""")
conn.execute("DROP TABLE scans")
conn.execute("ALTER TABLE scans_new RENAME TO scans")
conn.execute("PRAGMA foreign_keys = ON")
conn.commit()
if verbose:
print(" ✅ Migration complete: status now accepts 'paused' and 'cancelled'")
def _migrate_add_reverse_scan_support(conn, verbose=True):
"""
Migration: Add reverse scan support across all affected tables.
Changes:
- scans: relax origin CHECK (3→>=2), add scan_mode column
- routes: add origin_airport column, replace unique index
- flights: add origin_airport column
- scheduled_scans: relax origin CHECK (3→>=2), add scan_mode column
"""
# ── scans table ──────────────────────────────────────────────────────────
cursor = conn.execute("PRAGMA table_info(scans)")
scans_cols = [row[1] for row in cursor.fetchall()]
if scans_cols and 'scan_mode' not in scans_cols:
if verbose:
print(" 🔄 Migrating scans table: relaxing origin constraint, adding scan_mode…")
conn.execute("PRAGMA foreign_keys = OFF")
conn.execute("DROP VIEW IF EXISTS recent_scans")
conn.execute("DROP VIEW IF EXISTS active_scans")
conn.execute("DROP TABLE IF EXISTS scans_new")
conn.execute("DROP TRIGGER IF EXISTS update_scans_timestamp")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_insert")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_update")
conn.execute("DROP TRIGGER IF EXISTS update_scan_flight_count_delete")
conn.execute("""
CREATE TABLE scans_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
origin TEXT NOT NULL CHECK(length(origin) >= 2),
country TEXT NOT NULL CHECK(length(country) >= 2),
scan_mode TEXT NOT NULL DEFAULT 'forward'
CHECK(scan_mode IN ('forward', 'reverse')),
start_date TEXT NOT NULL,
end_date TEXT NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
started_at TIMESTAMP,
completed_at TIMESTAMP,
status TEXT NOT NULL DEFAULT 'pending'
CHECK(status IN ('pending', 'running', 'completed', 'failed', 'cancelled', 'paused')),
total_routes INTEGER NOT NULL DEFAULT 0 CHECK(total_routes >= 0),
routes_scanned INTEGER NOT NULL DEFAULT 0 CHECK(routes_scanned >= 0),
total_flights INTEGER NOT NULL DEFAULT 0 CHECK(total_flights >= 0),
error_message TEXT,
seat_class TEXT DEFAULT 'economy',
adults INTEGER DEFAULT 1 CHECK(adults > 0 AND adults <= 9),
scheduled_scan_id INTEGER,
CHECK(end_date >= start_date),
CHECK(routes_scanned <= total_routes OR total_routes = 0)
)
""")
conn.execute("""
INSERT INTO scans_new (
id, origin, country, scan_mode, start_date, end_date,
created_at, updated_at, started_at, completed_at,
status, total_routes, routes_scanned, total_flights,
error_message, seat_class, adults, scheduled_scan_id
)
SELECT
id, origin, country, 'forward', start_date, end_date,
created_at, updated_at, started_at, completed_at,
status, total_routes, routes_scanned, total_flights,
error_message, seat_class, adults, scheduled_scan_id
FROM scans
""")
conn.execute("DROP TABLE scans")
conn.execute("ALTER TABLE scans_new RENAME TO scans")
conn.execute("PRAGMA foreign_keys = ON")
conn.commit()
if verbose:
print(" ✅ scans table migrated")
# ── routes: add origin_airport column ────────────────────────────────────
cursor = conn.execute("PRAGMA table_info(routes)")
routes_cols = [row[1] for row in cursor.fetchall()]
if routes_cols and 'origin_airport' not in routes_cols:
if verbose:
print(" 🔄 Migrating routes table: adding origin_airport column…")
conn.execute("ALTER TABLE routes ADD COLUMN origin_airport TEXT")
conn.commit()
if verbose:
print(" ✅ routes.origin_airport column added")
# ── routes: replace unique index ─────────────────────────────────────────
cursor = conn.execute(
"SELECT name FROM sqlite_master WHERE type='index' AND name='uq_routes_scan_dest'"
)
if cursor.fetchone():
if verbose:
print(" 🔄 Replacing routes unique index…")
conn.execute("DROP INDEX IF EXISTS uq_routes_scan_dest")
conn.execute("""
CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_origin_dest
ON routes(scan_id, COALESCE(origin_airport, ''), destination)
""")
conn.commit()
if verbose:
print(" ✅ Routes unique index replaced")
# ── flights: add origin_airport column ───────────────────────────────────
cursor = conn.execute("PRAGMA table_info(flights)")
flights_cols = [row[1] for row in cursor.fetchall()]
if flights_cols and 'origin_airport' not in flights_cols:
if verbose:
print(" 🔄 Migrating flights table: adding origin_airport column…")
conn.execute("ALTER TABLE flights ADD COLUMN origin_airport TEXT")
conn.commit()
if verbose:
print(" ✅ flights.origin_airport column added")
# ── scheduled_scans: relax origin + add scan_mode ────────────────────────
cursor = conn.execute("PRAGMA table_info(scheduled_scans)")
sched_cols = [row[1] for row in cursor.fetchall()]
if sched_cols and 'scan_mode' not in sched_cols:
if verbose:
print(" 🔄 Migrating scheduled_scans table: relaxing origin constraint, adding scan_mode…")
conn.execute("DROP TABLE IF EXISTS scheduled_scans_new")
conn.execute("DROP TRIGGER IF EXISTS update_scheduled_scans_timestamp")
conn.execute("""
CREATE TABLE scheduled_scans_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
origin TEXT NOT NULL CHECK(length(origin) >= 2),
country TEXT NOT NULL CHECK(length(country) >= 2),
scan_mode TEXT NOT NULL DEFAULT 'forward'
CHECK(scan_mode IN ('forward', 'reverse')),
window_months INTEGER NOT NULL DEFAULT 1
CHECK(window_months >= 1 AND window_months <= 12),
seat_class TEXT NOT NULL DEFAULT 'economy',
adults INTEGER NOT NULL DEFAULT 1
CHECK(adults > 0 AND adults <= 9),
frequency TEXT NOT NULL
CHECK(frequency IN ('daily', 'weekly', 'monthly')),
hour INTEGER NOT NULL DEFAULT 6
CHECK(hour >= 0 AND hour <= 23),
minute INTEGER NOT NULL DEFAULT 0
CHECK(minute >= 0 AND minute <= 59),
day_of_week INTEGER CHECK(day_of_week >= 0 AND day_of_week <= 6),
day_of_month INTEGER CHECK(day_of_month >= 1 AND day_of_month <= 28),
enabled INTEGER NOT NULL DEFAULT 1,
label TEXT,
last_run_at TIMESTAMP,
next_run_at TIMESTAMP NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
CHECK(
(frequency = 'weekly' AND day_of_week IS NOT NULL) OR
(frequency = 'monthly' AND day_of_month IS NOT NULL) OR
(frequency = 'daily')
)
)
""")
conn.execute("""
INSERT INTO scheduled_scans_new (
id, origin, country, scan_mode, window_months, seat_class, adults,
frequency, hour, minute, day_of_week, day_of_month,
enabled, label, last_run_at, next_run_at, created_at, updated_at
)
SELECT
id, origin, country, 'forward', window_months, seat_class, adults,
frequency, hour, minute, day_of_week, day_of_month,
enabled, label, last_run_at, next_run_at, created_at, updated_at
FROM scheduled_scans
""")
conn.execute("DROP TABLE scheduled_scans")
conn.execute("ALTER TABLE scheduled_scans_new RENAME TO scheduled_scans")
conn.commit()
if verbose:
print(" ✅ scheduled_scans table migrated")
def initialize_database(db_path=None, verbose=True):
"""
Initialize or migrate the database.
@@ -172,8 +539,16 @@ def initialize_database(db_path=None, verbose=True):
else:
print(" No existing tables found")
# Recover any orphaned _new tables left by previously aborted migrations
_recover_orphaned_new_tables(conn, verbose)
# Apply migrations before running schema
_migrate_relax_country_constraint(conn, verbose)
_migrate_add_routes_unique_index(conn, verbose)
_migrate_add_scheduled_scan_id_to_scans(conn, verbose)
_migrate_add_timing_columns_to_scans(conn, verbose)
_migrate_add_pause_cancel_status(conn, verbose)
_migrate_add_reverse_scan_support(conn, verbose)
# Load and execute schema
schema_sql = load_schema()

View File

@@ -20,18 +20,23 @@ CREATE TABLE IF NOT EXISTS scans (
id INTEGER PRIMARY KEY AUTOINCREMENT,
-- Search parameters (validated by CHECK constraints)
origin TEXT NOT NULL CHECK(length(origin) = 3),
-- origin stores IATA code (forward scans) or ISO country code (reverse scans)
origin TEXT NOT NULL CHECK(length(origin) >= 2),
country TEXT NOT NULL CHECK(length(country) >= 2),
scan_mode TEXT NOT NULL DEFAULT 'forward'
CHECK(scan_mode IN ('forward', 'reverse')),
start_date TEXT NOT NULL, -- ISO 8601: YYYY-MM-DD
end_date TEXT NOT NULL,
-- Timestamps (auto-managed)
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
started_at TIMESTAMP, -- Set when status transitions to 'running'
completed_at TIMESTAMP, -- Set when status transitions to 'completed' or 'failed'
-- Scan status (enforced enum via CHECK)
status TEXT NOT NULL DEFAULT 'pending'
CHECK(status IN ('pending', 'running', 'completed', 'failed')),
CHECK(status IN ('pending', 'running', 'completed', 'failed', 'cancelled', 'paused')),
-- Progress tracking
total_routes INTEGER NOT NULL DEFAULT 0 CHECK(total_routes >= 0),
@@ -45,6 +50,9 @@ CREATE TABLE IF NOT EXISTS scans (
seat_class TEXT DEFAULT 'economy',
adults INTEGER DEFAULT 1 CHECK(adults > 0 AND adults <= 9),
-- FK to scheduled_scans (NULL for manual scans)
scheduled_scan_id INTEGER,
-- Constraints across columns
CHECK(end_date >= start_date),
CHECK(routes_scanned <= total_routes OR total_routes = 0)
@@ -61,6 +69,10 @@ CREATE INDEX IF NOT EXISTS idx_scans_status
CREATE INDEX IF NOT EXISTS idx_scans_created_at
ON scans(created_at DESC); -- For recent scans query
CREATE INDEX IF NOT EXISTS idx_scans_scheduled_scan_id
ON scans(scheduled_scan_id)
WHERE scheduled_scan_id IS NOT NULL;
-- ============================================================================
-- Table: routes
-- Purpose: Store discovered routes with flight statistics
@@ -72,7 +84,10 @@ CREATE TABLE IF NOT EXISTS routes (
-- Foreign key to scans (cascade delete)
scan_id INTEGER NOT NULL,
-- Destination airport
-- Route airports
-- For forward scans: origin_airport is NULL (implicit from scan.origin)
-- For reverse scans: origin_airport is the variable origin IATA
origin_airport TEXT,
destination TEXT NOT NULL CHECK(length(destination) = 3),
destination_name TEXT NOT NULL,
destination_city TEXT,
@@ -111,6 +126,10 @@ CREATE INDEX IF NOT EXISTS idx_routes_min_price
ON routes(min_price)
WHERE min_price IS NOT NULL; -- Partial index for routes with prices
-- One route row per (scan, origin_airport, destination) — supports both forward and reverse scans
CREATE UNIQUE INDEX IF NOT EXISTS uq_routes_scan_origin_dest
ON routes(scan_id, COALESCE(origin_airport, ''), destination);
-- ============================================================================
-- Triggers: Auto-update timestamps and aggregates
-- ============================================================================
@@ -178,6 +197,8 @@ CREATE TABLE IF NOT EXISTS flights (
scan_id INTEGER NOT NULL,
-- Route
-- origin_airport: NULL for forward scans, specific IATA for reverse scans
origin_airport TEXT,
destination TEXT NOT NULL CHECK(length(destination) = 3),
date TEXT NOT NULL, -- ISO 8601: YYYY-MM-DD
@@ -240,7 +261,9 @@ ORDER BY created_at ASC;
-- Initial Data: None (tables start empty)
-- ============================================================================
-- ============================================================================
-- Schema version tracking (for future migrations)
-- ============================================================================
CREATE TABLE IF NOT EXISTS schema_version (
version INTEGER PRIMARY KEY,
applied_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
@@ -250,6 +273,66 @@ CREATE TABLE IF NOT EXISTS schema_version (
INSERT OR IGNORE INTO schema_version (version, description)
VALUES (1, 'Initial web app schema with scans and routes tables');
-- ============================================================================
-- Table: scheduled_scans
-- Purpose: Define recurring scan schedules (daily / weekly / monthly)
-- ============================================================================
CREATE TABLE IF NOT EXISTS scheduled_scans (
id INTEGER PRIMARY KEY AUTOINCREMENT,
-- Scan parameters (same as scans table)
origin TEXT NOT NULL CHECK(length(origin) >= 2),
country TEXT NOT NULL CHECK(length(country) >= 2),
scan_mode TEXT NOT NULL DEFAULT 'forward'
CHECK(scan_mode IN ('forward', 'reverse')),
window_months INTEGER NOT NULL DEFAULT 1
CHECK(window_months >= 1 AND window_months <= 12),
seat_class TEXT NOT NULL DEFAULT 'economy',
adults INTEGER NOT NULL DEFAULT 1
CHECK(adults > 0 AND adults <= 9),
-- Schedule definition
frequency TEXT NOT NULL
CHECK(frequency IN ('daily', 'weekly', 'monthly')),
hour INTEGER NOT NULL DEFAULT 6
CHECK(hour >= 0 AND hour <= 23),
minute INTEGER NOT NULL DEFAULT 0
CHECK(minute >= 0 AND minute <= 59),
day_of_week INTEGER CHECK(day_of_week >= 0 AND day_of_week <= 6),
day_of_month INTEGER CHECK(day_of_month >= 1 AND day_of_month <= 28),
-- State
enabled INTEGER NOT NULL DEFAULT 1,
label TEXT,
last_run_at TIMESTAMP,
next_run_at TIMESTAMP NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- Frequency-specific field requirements
CHECK(
(frequency = 'weekly' AND day_of_week IS NOT NULL) OR
(frequency = 'monthly' AND day_of_month IS NOT NULL) OR
(frequency = 'daily')
)
);
-- Fast lookup of due schedules (partial index on enabled rows only)
CREATE INDEX IF NOT EXISTS idx_scheduled_scans_next_run
ON scheduled_scans(next_run_at)
WHERE enabled = 1;
-- Auto-update updated_at on every PATCH
CREATE TRIGGER IF NOT EXISTS update_scheduled_scans_timestamp
AFTER UPDATE ON scheduled_scans
FOR EACH ROW BEGIN
UPDATE scheduled_scans SET updated_at = CURRENT_TIMESTAMP WHERE id = NEW.id;
END;
INSERT OR IGNORE INTO schema_version (version, description)
VALUES (2, 'Add scheduled_scans table');
-- ============================================================================
-- Verification Queries (for testing)
-- ============================================================================

View File

@@ -1,15 +1,64 @@
name: flight-radar # pins the project name — must match COMPOSE_PROJECT in .gitea/workflows/deploy.yml
services:
app:
build: .
container_name: flight-radar
backend:
build:
context: .
dockerfile: Dockerfile.backend
container_name: flight-radar-backend
restart: unless-stopped
ports:
- "80:80"
environment:
- DATABASE_PATH=/app/data/cache.db
- ALLOWED_ORIGINS=https://flights.domverse-berlin.eu
- LOG_LEVEL=INFO
volumes:
- flight-radar-data:/app/data
networks:
- default
- domverse
# No ports exposed — only reachable by the frontend via nginx proxy
frontend:
build:
context: .
dockerfile: Dockerfile.frontend
container_name: flight-radar-frontend
restart: unless-stopped
depends_on:
- backend
networks:
- default # shares default compose network with backend (nginx → http://backend:8000)
- domverse # Traefik discovers the container on this network
labels:
# Traefik routing
- "traefik.docker.network=domverse"
- "traefik.enable=true"
- "traefik.http.routers.flight-radar.rule=Host(`flights.domverse-berlin.eu`)"
- "traefik.http.routers.flight-radar.entrypoints=https"
- "traefik.http.routers.flight-radar.tls.certresolver=http"
- "traefik.http.routers.flight-radar.middlewares=authentik@docker"
- "traefik.http.services.flight-radar.loadbalancer.server.port=80"
# AutoKuma monitoring
- "kuma.flight-radar.http.name=Flight Radar"
- "kuma.flight-radar.http.url=https://flights.domverse-berlin.eu"
- "kuma.flight-radar.http.interval=60"
- "kuma.flight-radar.http.max_retries=2"
- "kuma.flight-radar.http.retry_interval=60"
# Homepage dashboard
- "homepage.group=Productivity"
- "homepage.name=Flight Radar"
- "homepage.icon=mdi-airplane"
- "homepage.href=https://flights.domverse-berlin.eu"
- "homepage.description=Flight price comparison tool"
- "homepage.weight=20"
volumes:
flight-radar-data:
driver: local
networks:
default: {} # explicit declaration required when any service has a custom networks block
domverse:
external: true

View File

@@ -3,6 +3,7 @@ import Layout from './components/Layout';
import Dashboard from './pages/Dashboard';
import Scans from './pages/Scans';
import ScanDetails from './pages/ScanDetails';
import Schedules from './pages/Schedules';
import Airports from './pages/Airports';
import Logs from './pages/Logs';
import ErrorBoundary from './components/ErrorBoundary';
@@ -16,6 +17,7 @@ function App() {
<Route index element={<Dashboard />} />
<Route path="scans" element={<Scans />} />
<Route path="scans/:id" element={<ScanDetails />} />
<Route path="schedules" element={<Schedules />} />
<Route path="airports" element={<Airports />} />
<Route path="logs" element={<Logs />} />
</Route>

View File

@@ -10,11 +10,12 @@ const api = axios.create({
// Types
export interface Scan {
id: number;
scan_mode: 'forward' | 'reverse';
origin: string;
country: string;
start_date: string;
end_date: string;
status: 'pending' | 'running' | 'completed' | 'failed';
status: 'pending' | 'running' | 'completed' | 'failed' | 'paused' | 'cancelled';
created_at: string;
updated_at: string;
total_routes: number;
@@ -23,11 +24,51 @@ export interface Scan {
error_message?: string;
seat_class: string;
adults: number;
scheduled_scan_id?: number;
started_at?: string; // ISO-8601 UTC — set when status transitions to 'running'
completed_at?: string; // ISO-8601 UTC — set when status transitions to 'completed' or 'failed'
}
export interface Schedule {
id: number;
scan_mode: 'forward' | 'reverse';
origin: string;
country: string;
window_months: number;
seat_class: string;
adults: number;
label?: string;
frequency: 'daily' | 'weekly' | 'monthly';
hour: number;
minute: number;
day_of_week?: number;
day_of_month?: number;
enabled: boolean;
last_run_at?: string;
next_run_at: string;
created_at: string;
recent_scan_ids: number[];
}
export interface CreateScheduleRequest {
scan_mode?: 'forward' | 'reverse';
origin: string;
country: string;
window_months?: number;
seat_class?: string;
adults?: number;
label?: string;
frequency: 'daily' | 'weekly' | 'monthly';
hour?: number;
minute?: number;
day_of_week?: number;
day_of_month?: number;
}
export interface Route {
id: number;
scan_id: number;
origin_airport?: string;
destination: string;
destination_name: string;
destination_city?: string;
@@ -42,6 +83,7 @@ export interface Route {
export interface Flight {
id: number;
scan_id: number;
origin_airport?: string;
destination: string;
date: string;
airline?: string;
@@ -79,7 +121,14 @@ export interface PaginatedResponse<T> {
};
}
export interface Country {
code: string;
name: string;
airport_count: number;
}
export interface CreateScanRequest {
scan_mode?: 'forward' | 'reverse';
origin: string;
country?: string; // Optional: provide either country or destinations
destinations?: string[]; // Optional: provide either country or destinations
@@ -118,11 +167,18 @@ export const scanApi = {
});
},
getFlights: (id: number, destination?: string, page = 1, limit = 50) => {
getFlights: (id: number, destination?: string, originAirport?: string, page = 1, limit = 50) => {
const params: Record<string, unknown> = { page, limit };
if (destination) params.destination = destination;
if (originAirport) params.origin_airport = originAirport;
return api.get<PaginatedResponse<Flight>>(`/scans/${id}/flights`, { params });
},
delete: (id: number) => api.delete(`/scans/${id}`),
pause: (id: number) => api.post(`/scans/${id}/pause`),
cancel: (id: number) => api.post(`/scans/${id}/cancel`),
resume: (id: number) => api.post(`/scans/${id}/resume`),
};
export const airportApi = {
@@ -133,6 +189,30 @@ export const airportApi = {
},
};
export const countriesApi = {
list: () => api.get<Country[]>('/countries'),
};
export const scheduleApi = {
list: (page = 1, limit = 20) =>
api.get<PaginatedResponse<Schedule>>('/schedules', { params: { page, limit } }),
get: (id: number) =>
api.get<Schedule>(`/schedules/${id}`),
create: (data: CreateScheduleRequest) =>
api.post<Schedule>('/schedules', data),
update: (id: number, data: Partial<CreateScheduleRequest> & { enabled?: boolean }) =>
api.patch<Schedule>(`/schedules/${id}`, data),
delete: (id: number) =>
api.delete(`/schedules/${id}`),
runNow: (id: number) =>
api.post<{ scan_id: number }>(`/schedules/${id}/run-now`),
};
export const logApi = {
list: (page = 1, limit = 50, level?: string, search?: string) => {
const params: any = { page, limit };

View File

@@ -0,0 +1,51 @@
import { useEffect, useState } from 'react';
import { countriesApi } from '../api';
import type { Country } from '../api';
interface CountrySelectProps {
value: string;
onChange: (code: string) => void;
placeholder?: string;
hasError?: boolean;
className?: string;
}
export default function CountrySelect({
value,
onChange,
placeholder = 'Select a country…',
hasError = false,
className,
}: CountrySelectProps) {
const [countries, setCountries] = useState<Country[]>([]);
const [loading, setLoading] = useState(true);
useEffect(() => {
countriesApi.list()
.then(resp => setCountries(resp.data))
.catch(() => setCountries([]))
.finally(() => setLoading(false));
}, []);
const baseCls =
'w-full h-12 px-3 border rounded-xs bg-surface text-on-surface text-sm outline-none transition-colors ' +
(hasError
? 'border-error focus:border-error focus:ring-2 focus:ring-error/20'
: 'border-outline focus:border-primary focus:ring-2 focus:ring-primary/20');
return (
<select
value={value}
onChange={e => onChange(e.target.value)}
disabled={loading}
className={`${baseCls} ${className ?? ''}`}
>
<option value="">{loading ? 'Loading countries…' : placeholder}</option>
{countries.map(c => (
<option key={c.code} value={c.code}>
{c.name} ({c.code}) {c.airport_count} airport{c.airport_count !== 1 ? 's' : ''}
</option>
))}
</select>
);
}

View File

@@ -7,6 +7,7 @@ import {
ScrollText,
PlaneTakeoff,
Plus,
CalendarClock,
} from 'lucide-react';
import { cn } from '../lib/utils';
@@ -18,8 +19,9 @@ type NavItem = {
const PRIMARY_NAV: NavItem[] = [
{ icon: LayoutDashboard, label: 'Dashboard', path: '/' },
{ icon: ScanSearch, label: 'Scans', path: '/scans' },
{ icon: MapPin, label: 'Airports', path: '/airports' },
{ icon: ScanSearch, label: 'Scans', path: '/scans' },
{ icon: CalendarClock, label: 'Schedules', path: '/schedules' },
{ icon: MapPin, label: 'Airports', path: '/airports' },
];
const SECONDARY_NAV: NavItem[] = [
@@ -32,6 +34,7 @@ function getPageTitle(pathname: string): string {
if (pathname === '/') return 'Dashboard';
if (pathname.startsWith('/scans/')) return 'Scan Details';
if (pathname === '/scans') return 'New Scan';
if (pathname === '/schedules') return 'Schedules';
if (pathname === '/airports') return 'Airports';
if (pathname === '/logs') return 'Logs';
return 'Flight Radar';

View File

@@ -0,0 +1,51 @@
import type { ScanTimerResult } from '../hooks/useScanTimer';
/** Format a non-negative number of seconds into a human-readable string. */
export function formatDuration(totalSeconds: number): string {
const s = Math.floor(totalSeconds);
if (s < 60) return `${s}s`;
const m = Math.floor(s / 60);
const rem = s % 60;
if (m < 60) return rem > 0 ? `${m}m ${rem}s` : `${m}m`;
const h = Math.floor(m / 60);
const remM = m % 60;
return remM > 0 ? `${h}h ${remM}m` : `${h}h`;
}
interface ScanTimerProps extends ScanTimerResult {
/** When true, renders a compact single-line format for the completed stat card. */
compact?: boolean;
}
/**
* Displays elapsed time and ETA for an active scan, or final duration for a
* completed/failed scan.
*/
export default function ScanTimer({ elapsedSeconds, remainingSeconds, isEstimating, compact }: ScanTimerProps) {
if (compact) {
return <span>{formatDuration(elapsedSeconds)}</span>;
}
const remainingLabel = isEstimating
? 'Estimating…'
: remainingSeconds !== null
? `~${formatDuration(remainingSeconds)}`
: null;
return (
<div className="mt-3 grid grid-cols-2 gap-x-4 gap-y-0.5 text-xs">
<span className="text-on-surface-variant">Elapsed</span>
<span className="font-mono text-on-surface tabular-nums">
{formatDuration(elapsedSeconds)}
</span>
{remainingLabel !== null && (
<>
<span className="text-on-surface-variant">Remaining</span>
<span className="font-mono text-on-surface tabular-nums">
{remainingLabel}
</span>
</>
)}
</div>
);
}

View File

@@ -1,8 +1,8 @@
import { CheckCircle2, Loader2, Clock, XCircle } from 'lucide-react';
import { CheckCircle2, Loader2, Clock, XCircle, PauseCircle, Ban } from 'lucide-react';
import type { LucideIcon } from 'lucide-react';
import { cn } from '../lib/utils';
export type ScanStatus = 'completed' | 'running' | 'pending' | 'failed';
export type ScanStatus = 'completed' | 'running' | 'pending' | 'failed' | 'paused' | 'cancelled';
interface StatusConfig {
icon: LucideIcon;
@@ -38,6 +38,18 @@ const CONFIGS: Record<ScanStatus, StatusConfig> = {
chipClass: 'bg-[#FDECEA] text-[#A50E0E] border border-[#F5C6C6]',
iconClass: 'text-[#A50E0E]',
},
paused: {
icon: PauseCircle,
label: 'paused',
chipClass: 'bg-[#FEF7E0] text-[#7A5200] border border-[#F9D659]',
iconClass: 'text-[#7A5200]',
},
cancelled: {
icon: Ban,
label: 'cancelled',
chipClass: 'bg-[#F3F3F3] text-[#5F6368] border border-[#DADCE0]',
iconClass: 'text-[#5F6368]',
},
};
interface StatusChipProps {

View File

@@ -0,0 +1,88 @@
import { useEffect, useRef, useState } from 'react';
import type { Scan } from '../api';
export interface ScanTimerResult {
/** Seconds elapsed since the scan started processing. */
elapsedSeconds: number;
/**
* Estimated seconds remaining, or null when not enough data yet
* (fewer than 5 routes scanned or elapsed time is 0).
*/
remainingSeconds: number | null;
/** True while the estimate is still too early to be reliable. */
isEstimating: boolean;
}
const MIN_ROUTES_FOR_ESTIMATE = 5;
function calcElapsed(startedAt: string): number {
return Math.max(0, (Date.now() - new Date(startedAt).getTime()) / 1000);
}
function calcRemaining(
elapsed: number,
routesScanned: number,
totalRoutes: number,
): number | null {
if (elapsed <= 0 || routesScanned < MIN_ROUTES_FOR_ESTIMATE || totalRoutes <= 0) {
return null;
}
const rate = routesScanned / elapsed; // routes per second
const remaining = (totalRoutes - routesScanned) / rate;
return Math.max(0, remaining);
}
export function useScanTimer(scan: Scan | null): ScanTimerResult {
const [elapsedSeconds, setElapsedSeconds] = useState(0);
const [remainingSeconds, setRemainingSeconds] = useState<number | null>(null);
const intervalRef = useRef<ReturnType<typeof setInterval> | undefined>(undefined);
useEffect(() => {
if (!scan) return;
// For completed / failed scans with both timestamps: compute static duration.
if (
(scan.status === 'completed' || scan.status === 'failed') &&
scan.started_at &&
scan.completed_at
) {
const duration = Math.max(
0,
(new Date(scan.completed_at).getTime() - new Date(scan.started_at).getTime()) / 1000,
);
setElapsedSeconds(duration);
setRemainingSeconds(0);
return;
}
// For running scans with a start time: run a live 1-second timer.
if (scan.status === 'running' && scan.started_at) {
const tick = () => {
const elapsed = calcElapsed(scan.started_at!);
const remaining = calcRemaining(elapsed, scan.routes_scanned, scan.total_routes);
setElapsedSeconds(elapsed);
setRemainingSeconds(remaining);
};
tick(); // run immediately
intervalRef.current = setInterval(tick, 1000);
return () => {
if (intervalRef.current !== undefined) {
clearInterval(intervalRef.current);
intervalRef.current = undefined;
}
};
}
// Pending or no started_at: reset
setElapsedSeconds(0);
setRemainingSeconds(null);
}, [scan?.status, scan?.started_at, scan?.completed_at, scan?.routes_scanned, scan?.total_routes]);
const isEstimating =
scan?.status === 'running' &&
(scan.routes_scanned < MIN_ROUTES_FOR_ESTIMATE || scan.total_routes <= 0);
return { elapsedSeconds, remainingSeconds, isEstimating };
}

View File

@@ -1,18 +1,27 @@
import { Fragment, useEffect, useState } from 'react';
import { useParams, useNavigate } from 'react-router-dom';
import { useParams, useNavigate, Link } from 'react-router-dom';
import {
ArrowLeft,
PlaneTakeoff,
Calendar,
CalendarClock,
Users,
Armchair,
Clock,
Timer,
ChevronRight,
ChevronUp,
ChevronDown,
ChevronsUpDown,
MapPin,
AlertCircle,
Loader2,
RotateCcw,
Trash2,
Info,
Pause,
Play,
X,
} from 'lucide-react';
import { scanApi } from '../api';
import type { Scan, Route, Flight } from '../api';
@@ -21,6 +30,8 @@ import type { ScanStatus } from '../components/StatusChip';
import StatCard from '../components/StatCard';
import EmptyState from '../components/EmptyState';
import { SkeletonStatCard, SkeletonTableRow } from '../components/SkeletonCard';
import ScanTimer, { formatDuration } from '../components/ScanTimer';
import { useScanTimer } from '../hooks/useScanTimer';
import { cn } from '../lib/utils';
const formatPrice = (price?: number) =>
@@ -42,9 +53,21 @@ export default function ScanDetails() {
const [totalPages, setTotalPages] = useState(1);
const [sortField, setSortField] = useState<'min_price' | 'destination' | 'flight_count'>('min_price');
const [sortDirection, setSortDirection] = useState<'asc' | 'desc'>('asc');
const [expandedRoute, setExpandedRoute] = useState<string | null>(null);
const [flightsByDest, setFlightsByDest] = useState<Record<string, Flight[]>>({});
const [expandedRoute, setExpandedRoute] = useState<string | null>(null);
const [flightsByDest, setFlightsByDest] = useState<Record<string, Flight[]>>({});
const [flightSortField, setFlightSortField] = useState<'date' | 'price'>('date');
const [flightSortDir, setFlightSortDir] = useState<'asc' | 'desc'>('asc');
const [loadingFlights, setLoadingFlights] = useState<string | null>(null);
const [rerunning, setRerunning] = useState(false);
const [confirmDelete, setConfirmDelete] = useState(false);
const [deleting, setDeleting] = useState(false);
const [confirmPause, setConfirmPause] = useState(false);
const [confirmCancel, setConfirmCancel] = useState(false);
const [stopping, setStopping] = useState(false);
const [resuming, setResuming] = useState(false);
// Must be called unconditionally before any early returns (Rules of Hooks)
const timer = useScanTimer(scan);
useEffect(() => {
if (id) loadScanDetails();
@@ -95,28 +118,147 @@ export default function ScanDetails() {
}
};
const toggleFlights = async (destination: string) => {
if (expandedRoute === destination) { setExpandedRoute(null); return; }
setExpandedRoute(destination);
if (flightsByDest[destination]) return;
setLoadingFlights(destination);
const handleFlightSort = (field: 'date' | 'price') => {
if (flightSortField === field) {
setFlightSortDir(d => d === 'asc' ? 'desc' : 'asc');
} else {
setFlightSortField(field);
setFlightSortDir('asc');
}
};
const sortedFlights = (flights: Flight[]) =>
[...flights].sort((a, b) => {
const aVal = flightSortField === 'date' ? a.date : (a.price ?? Infinity);
const bVal = flightSortField === 'date' ? b.date : (b.price ?? Infinity);
if (aVal < bVal) return flightSortDir === 'asc' ? -1 : 1;
if (aVal > bVal) return flightSortDir === 'asc' ? 1 : -1;
return 0;
});
// For reverse scans, route key = "ORIG:DEST"; for forward scans = "DEST"
const routeKey = (route: Route) =>
route.origin_airport ? `${route.origin_airport}:${route.destination}` : route.destination;
const toggleFlights = async (route: Route) => {
const key = routeKey(route);
if (expandedRoute === key) { setExpandedRoute(null); return; }
setExpandedRoute(key);
if (flightsByDest[key]) return;
setLoadingFlights(key);
try {
const resp = await scanApi.getFlights(Number(id), destination, 1, 200);
setFlightsByDest(prev => ({ ...prev, [destination]: resp.data.data }));
const resp = await scanApi.getFlights(Number(id), route.destination, route.origin_airport, 1, 200);
setFlightsByDest(prev => ({ ...prev, [key]: resp.data.data }));
} catch {
setFlightsByDest(prev => ({ ...prev, [destination]: [] }));
setFlightsByDest(prev => ({ ...prev, [key]: [] }));
} finally {
setLoadingFlights(null);
}
};
const handleRerun = async () => {
if (!scan) return;
setRerunning(true);
try {
const ms = new Date(scan.end_date).getTime() - new Date(scan.start_date).getTime();
const window_months = Math.max(1, Math.round(ms / (1000 * 60 * 60 * 24 * 30)));
const base = {
scan_mode: (scan.scan_mode ?? 'forward') as 'forward' | 'reverse',
origin: scan.origin,
window_months,
seat_class: scan.seat_class as 'economy' | 'premium' | 'business' | 'first',
adults: scan.adults,
};
let extra: Record<string, unknown>;
if (scan.scan_mode === 'reverse') {
// For reverse: country column holds comma-separated dest IATAs
extra = { destinations: scan.country.split(',') };
} else {
// For forward: country column holds ISO code or comma-separated IATAs
const isAirports = scan.country.includes(',');
extra = isAirports
? { destinations: scan.country.split(',') }
: { country: scan.country };
}
const resp = await scanApi.create({ ...base, ...extra });
navigate(`/scans/${resp.data.id}`);
} catch {
// silently fall through — the navigate won't happen
} finally {
setRerunning(false);
}
};
const handleDelete = async () => {
if (!scan) return;
setDeleting(true);
try {
await scanApi.delete(scan.id);
navigate('/');
} catch {
setDeleting(false);
setConfirmDelete(false);
}
};
const handlePause = async () => {
if (!scan) return;
setStopping(true);
try {
await scanApi.pause(scan.id);
await loadScanDetails();
} catch {
// fall through
} finally {
setStopping(false);
setConfirmPause(false);
}
};
const handleCancel = async () => {
if (!scan) return;
setStopping(true);
try {
await scanApi.cancel(scan.id);
await loadScanDetails();
} catch {
// fall through
} finally {
setStopping(false);
setConfirmCancel(false);
}
};
const handleResume = async () => {
if (!scan) return;
setResuming(true);
try {
await scanApi.resume(scan.id);
await loadScanDetails();
} catch {
// fall through
} finally {
setResuming(false);
}
};
const SortIcon = ({ field }: { field: typeof sortField }) => {
if (sortField !== field) return <ChevronUp size={14} className="opacity-30" />;
if (sortField !== field) return <ChevronsUpDown size={14} className="opacity-50" />;
return sortDirection === 'asc'
? <ChevronUp size={14} className="text-primary" />
: <ChevronDown size={14} className="text-primary" />;
};
const FlightSortIcon = ({ field }: { field: 'date' | 'price' }) => {
if (flightSortField !== field) return <ChevronsUpDown size={12} className="opacity-50" />;
return flightSortDir === 'asc'
? <ChevronUp size={12} className="text-secondary" />
: <ChevronDown size={12} className="text-secondary" />;
};
const thCls = (field?: typeof sortField) => cn(
'px-4 py-3 text-left text-xs font-semibold uppercase tracking-wider select-none',
field
@@ -174,8 +316,20 @@ export default function ScanDetails() {
<div className="flex items-center gap-2 flex-wrap min-w-0">
<PlaneTakeoff size={20} className="text-primary shrink-0" aria-hidden="true" />
<h1 className="text-xl font-semibold text-on-surface">
{scan.origin} {scan.country}
{scan.scan_mode === 'reverse'
? `${scan.origin}${scan.country.split(',').join(', ')}`
: `${scan.origin}${scan.country}`}
</h1>
{scan.scheduled_scan_id != null && (
<Link
to={`/schedules`}
className="inline-flex items-center gap-1 px-2 py-0.5 rounded-full text-xs font-medium bg-primary-container text-on-primary-container hover:opacity-80 transition-opacity"
title={`Scheduled scan #${scan.scheduled_scan_id}`}
>
<CalendarClock size={11} aria-hidden="true" />
Scheduled
</Link>
)}
</div>
<StatusChip status={scan.status as ScanStatus} />
</div>
@@ -203,10 +357,170 @@ export default function ScanDetails() {
Created {formatDate(scan.created_at)}
</p>
)}
{/* Row 4: actions */}
<div className="mt-4 pt-4 border-t border-outline flex items-center justify-end gap-2 flex-wrap">
{/* ── Active (pending / running): Pause + Cancel ── */}
{isActive && (
<>
{/* Pause — inline confirm */}
{confirmPause ? (
<div className="inline-flex items-center gap-1.5">
<span className="text-sm text-on-surface-variant">Pause this scan?</span>
<button
onClick={handlePause}
disabled={stopping}
className="inline-flex items-center gap-1 px-3 py-1.5 text-sm font-medium rounded-xs bg-[#7A5200] text-white hover:bg-[#5C3D00] disabled:opacity-60 transition-colors"
>
{stopping ? 'Pausing…' : 'Yes, pause'}
</button>
<button
onClick={() => setConfirmPause(false)}
disabled={stopping}
className="px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 transition-colors"
>
No
</button>
</div>
) : (
<button
onClick={() => setConfirmPause(true)}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 transition-colors"
>
<Pause size={14} />
Pause
</button>
)}
{/* Cancel — inline confirm */}
{confirmCancel ? (
<div className="inline-flex items-center gap-1.5">
<span className="text-sm text-on-surface-variant">Cancel this scan?</span>
<button
onClick={handleCancel}
disabled={stopping}
className="inline-flex items-center gap-1 px-3 py-1.5 text-sm font-medium rounded-xs bg-error text-white hover:bg-error/90 disabled:opacity-60 transition-colors"
>
{stopping ? 'Cancelling…' : 'Yes, cancel'}
</button>
<button
onClick={() => setConfirmCancel(false)}
disabled={stopping}
className="px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 transition-colors"
>
No
</button>
</div>
) : (
<button
onClick={() => setConfirmCancel(true)}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-error/40 text-error hover:bg-error/5 transition-colors"
>
<X size={14} />
Cancel
</button>
)}
</>
)}
{/* ── Paused: Resume + Re-run + Delete ── */}
{scan.status === 'paused' && (
<>
<button
onClick={handleResume}
disabled={resuming}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
>
<Play size={14} className={resuming ? 'animate-pulse' : ''} />
{resuming ? 'Resuming…' : 'Resume'}
</button>
<button
onClick={handleRerun}
disabled={rerunning}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
>
<RotateCcw size={14} className={rerunning ? 'animate-spin' : ''} />
{rerunning ? 'Starting…' : 'Re-run'}
</button>
{confirmDelete ? (
<div className="inline-flex items-center gap-1.5">
<span className="text-sm text-on-surface-variant">Delete this scan?</span>
<button
onClick={handleDelete}
disabled={deleting}
className="inline-flex items-center gap-1 px-3 py-1.5 text-sm font-medium rounded-xs bg-error text-white hover:bg-error/90 disabled:opacity-60 transition-colors"
>
{deleting ? 'Deleting…' : 'Yes, delete'}
</button>
<button
onClick={() => setConfirmDelete(false)}
disabled={deleting}
className="px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 transition-colors"
>
Cancel
</button>
</div>
) : (
<button
onClick={() => setConfirmDelete(true)}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-error/40 text-error hover:bg-error/5 transition-colors"
>
<Trash2 size={14} />
Delete
</button>
)}
</>
)}
{/* ── Completed / Failed / Cancelled: Re-run + Delete ── */}
{!isActive && scan.status !== 'paused' && (
<>
<button
onClick={handleRerun}
disabled={rerunning}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
>
<RotateCcw size={14} className={rerunning ? 'animate-spin' : ''} />
{rerunning ? 'Starting…' : 'Re-run'}
</button>
{confirmDelete ? (
<div className="inline-flex items-center gap-1.5">
<span className="text-sm text-on-surface-variant">Delete this scan?</span>
<button
onClick={handleDelete}
disabled={deleting}
className="inline-flex items-center gap-1 px-3 py-1.5 text-sm font-medium rounded-xs bg-error text-white hover:bg-error/90 disabled:opacity-60 transition-colors"
>
{deleting ? 'Deleting…' : 'Yes, delete'}
</button>
<button
onClick={() => setConfirmDelete(false)}
disabled={deleting}
className="px-3 py-1.5 text-sm font-medium rounded-xs border border-outline text-on-surface hover:bg-surface-2 transition-colors"
>
Cancel
</button>
</div>
) : (
<button
onClick={() => setConfirmDelete(true)}
className="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium rounded-xs border border-error/40 text-error hover:bg-error/5 transition-colors"
>
<Trash2 size={14} />
Delete
</button>
)}
</>
)}
</div>
</div>
{/* ── Stat cards ────────────────────────────────────────────── */}
<div className="grid grid-cols-3 gap-3">
<div className={`grid gap-3 ${!isActive && scan.started_at && scan.completed_at ? 'grid-cols-4' : 'grid-cols-3'}`}>
{loading ? (
[0, 1, 2].map(i => <SkeletonStatCard key={i} />)
) : (
@@ -214,6 +528,14 @@ export default function ScanDetails() {
<StatCard label="Total Routes" value={scan.total_routes} icon={MapPin} variant="primary" />
<StatCard label="Routes Scanned" value={scan.routes_scanned} icon={ChevronDown} variant="secondary" />
<StatCard label="Flights Found" value={scan.total_flights} icon={PlaneTakeoff} variant="primary" />
{!isActive && scan.started_at && scan.completed_at && (
<StatCard
label="Scan Duration"
value={formatDuration(timer.elapsedSeconds)}
icon={Timer}
variant="secondary"
/>
)}
</>
)}
</div>
@@ -241,6 +563,9 @@ export default function ScanDetails() {
<p className="mt-2 text-xs text-on-surface-variant">
{scan.routes_scanned} of {scan.total_routes > 0 ? scan.total_routes : '?'} routes · auto-refreshing every 3 s
</p>
{scan.status === 'running' && scan.started_at && (
<ScanTimer {...timer} />
)}
</div>
)}
@@ -277,6 +602,9 @@ export default function ScanDetails() {
<table className="w-full">
<thead className="bg-surface-2 border-b border-outline">
<tr>
{scan.scan_mode === 'reverse' && (
<th className={thCls()}>Origin</th>
)}
<th
className={thCls('destination')}
onClick={() => handleSort('destination')}
@@ -308,13 +636,23 @@ export default function ScanDetails() {
</thead>
<tbody className="divide-y divide-outline">
{routes.map((route) => {
const isExpanded = expandedRoute === route.destination;
const key = routeKey(route);
const isExpanded = expandedRoute === key;
const colSpan = scan.scan_mode === 'reverse' ? 7 : 6;
return (
<Fragment key={route.id}>
<tr
className="hover:bg-surface-2 cursor-pointer transition-colors duration-150"
onClick={() => toggleFlights(route.destination)}
onClick={() => toggleFlights(route)}
>
{/* Origin (reverse scans only) */}
{scan.scan_mode === 'reverse' && (
<td className="px-4 py-4">
<span className="font-mono text-secondary bg-surface-2 px-2 py-0.5 rounded-sm text-sm font-medium">
{route.origin_airport ?? '—'}
</span>
</td>
)}
{/* Destination */}
<td className="px-4 py-4">
<div className="flex items-center gap-2">
@@ -332,6 +670,19 @@ export default function ScanDetails() {
<span className="text-sm text-on-surface-variant truncate max-w-[180px]">
{route.destination_name || route.destination_city || ''}
</span>
{/* Info icon + tooltip — only when useful name data exists */}
{(route.destination_name && route.destination_name !== route.destination) || route.destination_city ? (
<span
className="relative group/tip inline-flex shrink-0"
onClick={e => e.stopPropagation()}
>
<Info size={12} className="text-on-surface-variant/40 hover:text-on-surface-variant/70 cursor-help transition-colors" />
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 z-50 invisible group-hover/tip:visible bg-gray-900 text-white text-xs rounded px-2 py-1 whitespace-nowrap shadow-lg">
{[route.destination_name, route.destination_city].filter(s => s && s !== route.destination).join(', ')}
<span className="absolute top-full left-1/2 -translate-x-1/2 border-4 border-transparent border-t-gray-900" />
</span>
</span>
) : null}
</div>
</td>
{/* Flights */}
@@ -358,13 +709,13 @@ export default function ScanDetails() {
{/* Expanded flights sub-row */}
<tr key={`${route.id}-flights`}>
<td colSpan={6} className="p-0">
<td colSpan={colSpan} className="p-0">
<div
className="overflow-hidden transition-all duration-250 ease-in-out"
style={{ maxHeight: isExpanded ? '600px' : '0' }}
>
<div className="bg-[#F8FDF9]">
{loadingFlights === route.destination ? (
{loadingFlights === key ? (
<table className="w-full">
<tbody>
<SkeletonTableRow />
@@ -376,15 +727,29 @@ export default function ScanDetails() {
<table className="w-full">
<thead className="bg-[#EEF7F0]">
<tr>
<th className="pl-12 pr-4 py-2 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Date</th>
<th
className="pl-12 pr-4 py-2 text-left text-xs font-semibold uppercase tracking-wider select-none cursor-pointer hover:bg-[#D4EDDA] transition-colors"
onClick={() => handleFlightSort('date')}
>
<span className={cn('inline-flex items-center gap-1', flightSortField === 'date' ? 'text-secondary' : 'text-on-surface-variant')}>
Date <FlightSortIcon field="date" />
</span>
</th>
<th className="px-4 py-2 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Airline</th>
<th className="px-4 py-2 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Departure</th>
<th className="px-4 py-2 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Arrival</th>
<th className="px-4 py-2 text-right text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Price</th>
<th
className="px-4 py-2 text-right text-xs font-semibold uppercase tracking-wider select-none cursor-pointer hover:bg-[#D4EDDA] transition-colors"
onClick={() => handleFlightSort('price')}
>
<span className={cn('inline-flex items-center justify-end gap-1', flightSortField === 'price' ? 'text-secondary' : 'text-on-surface-variant')}>
Price <FlightSortIcon field="price" />
</span>
</th>
</tr>
</thead>
<tbody className="divide-y divide-[#D4EDDA]">
{(flightsByDest[route.destination] || []).map((f) => (
{sortedFlights(flightsByDest[key] || []).map((f) => (
<tr key={f.id} className="hover:bg-[#EEF7F0] transition-colors">
<td className="pl-12 pr-4 py-2.5 text-sm text-on-surface">
<span className="font-mono text-xs font-semibold text-on-surface-variant mr-2">{weekday(f.date)}</span>
@@ -398,7 +763,7 @@ export default function ScanDetails() {
</td>
</tr>
))}
{(flightsByDest[route.destination] || []).length === 0 && (
{(flightsByDest[key] || []).length === 0 && (
<tr>
<td colSpan={5} className="pl-12 py-4 text-sm text-on-surface-variant">
No flight details available

View File

@@ -1,11 +1,12 @@
import { useState } from 'react';
import { useNavigate } from 'react-router-dom';
import { Globe, PlaneTakeoff, Minus, Plus } from 'lucide-react';
import { Globe, PlaneTakeoff, Minus, Plus, ArrowRight, ArrowLeft, CalendarDays, CalendarRange } from 'lucide-react';
import { scanApi } from '../api';
import type { CreateScanRequest } from '../api';
import AirportSearch from '../components/AirportSearch';
import SegmentedButton from '../components/SegmentedButton';
import AirportChip from '../components/AirportChip';
import CountrySelect from '../components/CountrySelect';
import Button from '../components/Button';
import Toast from '../components/Toast';
@@ -15,10 +16,17 @@ interface FormErrors {
airports?: string;
window_months?: string;
adults?: string;
start_date?: string;
end_date?: string;
}
export default function Scans() {
const navigate = useNavigate();
// Direction: forward (fixed origin → variable destinations) or reverse (variable origins → fixed destinations)
const [scanMode, setScanMode] = useState<'forward' | 'reverse'>('forward');
// Forward mode state
const [destinationMode, setDestinationMode] = useState<'country' | 'airports'>('country');
const [formData, setFormData] = useState<CreateScanRequest>({
origin: '',
@@ -27,22 +35,62 @@ export default function Scans() {
seat_class: 'economy',
adults: 1,
});
// Window mode: rolling N months or specific date range
const [windowMode, setWindowMode] = useState<'window' | 'range'>('window');
const [startDate, setStartDate] = useState('');
const [endDate, setEndDate] = useState('');
// Shared state
const [selectedAirports, setSelectedAirports] = useState<string[]>([]);
const [selectedOriginCountry, setSelectedOriginCountry] = useState('');
const [loading, setLoading] = useState(false);
const [errors, setErrors] = useState<FormErrors>({});
const [toast, setToast] = useState<{ message: string; type: 'success' | 'error' } | null>(null);
const validate = (): boolean => {
const next: FormErrors = {};
if (!formData.origin || formData.origin.length !== 3) {
next.origin = 'Enter a valid 3-letter IATA code';
if (scanMode === 'reverse') {
if (!selectedOriginCountry) {
next.country = 'Select an origin country';
}
if (selectedAirports.length === 0) {
next.airports = 'Add at least one destination airport';
}
} else {
if (!formData.origin || formData.origin.length !== 3) {
next.origin = 'Enter a valid 3-letter IATA code';
}
if (destinationMode === 'country' && !formData.country) {
next.country = 'Select a destination country';
}
if (destinationMode === 'airports' && selectedAirports.length === 0) {
next.airports = 'Add at least one destination airport';
}
}
if (destinationMode === 'country' && (!formData.country || formData.country.length !== 2)) {
next.country = 'Enter a valid 2-letter country code';
}
if (destinationMode === 'airports' && selectedAirports.length === 0) {
next.airports = 'Add at least one destination airport';
if (windowMode === 'range') {
const today = new Date();
today.setHours(0, 0, 0, 0);
if (!startDate) {
next.start_date = 'Select a start date';
} else if (new Date(startDate) <= today) {
next.start_date = 'Start date must be in the future';
}
if (!endDate) {
next.end_date = 'Select an end date';
} else if (startDate && endDate <= startDate) {
next.end_date = 'End date must be after start date';
} else if (startDate && endDate) {
const s = new Date(startDate);
const e = new Date(endDate);
const months = (e.getFullYear() - s.getFullYear()) * 12 + (e.getMonth() - s.getMonth());
if (months > 12) next.end_date = 'Date range cannot exceed 12 months';
}
}
setErrors(next);
return Object.keys(next).length === 0;
};
@@ -53,17 +101,32 @@ export default function Scans() {
setLoading(true);
try {
const requestData: any = {
origin: formData.origin,
window_months: formData.window_months,
seat_class: formData.seat_class,
adults: formData.adults,
};
let requestData: CreateScanRequest;
if (destinationMode === 'country') {
requestData.country = formData.country;
const windowParams = windowMode === 'range'
? { start_date: startDate, end_date: endDate }
: { window_months: formData.window_months };
if (scanMode === 'reverse') {
requestData = {
scan_mode: 'reverse',
origin: selectedOriginCountry,
destinations: selectedAirports,
seat_class: formData.seat_class,
adults: formData.adults,
...windowParams,
};
} else {
requestData.destinations = selectedAirports;
requestData = {
scan_mode: 'forward',
origin: formData.origin,
seat_class: formData.seat_class,
adults: formData.adults,
...windowParams,
...(destinationMode === 'country'
? { country: formData.country }
: { destinations: selectedAirports }),
};
}
const response = await scanApi.create(requestData);
@@ -86,42 +149,97 @@ export default function Scans() {
}));
};
// Shared input class
const inputCls = (hasError?: boolean) =>
`w-full h-12 px-3 border rounded-xs bg-surface text-on-surface text-sm outline-none transition-colors ` +
(hasError
? 'border-error focus:border-error focus:ring-2 focus:ring-error/20'
: 'border-outline focus:border-primary focus:ring-2 focus:ring-primary/20');
const tomorrowStr = (() => {
const d = new Date();
d.setDate(d.getDate() + 1);
return d.toISOString().split('T')[0];
})();
const minEndDate = startDate
? (() => { const d = new Date(startDate); d.setDate(d.getDate() + 1); return d.toISOString().split('T')[0]; })()
: tomorrowStr;
return (
<>
<form onSubmit={handleSubmit} className="space-y-4 max-w-2xl">
{/* ── Section: Direction ───────────────────────────────────── */}
<div className="bg-surface rounded-lg shadow-level-1 p-6">
<p className="text-xs font-semibold uppercase tracking-wider text-on-surface-variant mb-4">
Direction
</p>
<SegmentedButton
options={[
{ value: 'forward', label: 'Forward', icon: ArrowRight },
{ value: 'reverse', label: 'Reverse', icon: ArrowLeft },
]}
value={scanMode}
onChange={(v) => {
setScanMode(v as 'forward' | 'reverse');
setErrors({});
setSelectedAirports([]);
}}
/>
<p className="mt-2 text-xs text-on-surface-variant">
{scanMode === 'forward'
? 'Fixed origin airport → all airports in a country (or specific airports)'
: 'All airports in a country → specific destination airport(s)'}
</p>
</div>
{/* ── Section: Origin ─────────────────────────────────────── */}
<div className="bg-surface rounded-lg shadow-level-1 p-6">
<p className="text-xs font-semibold uppercase tracking-wider text-on-surface-variant mb-4">
Origin
</p>
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Origin Airport
</label>
<AirportSearch
value={formData.origin}
onChange={(value) => {
setFormData(prev => ({ ...prev, origin: value }));
if (errors.origin) setErrors(prev => ({ ...prev, origin: undefined }));
}}
placeholder="e.g. BDS, MUC, FRA"
hasError={!!errors.origin}
/>
{errors.origin ? (
<p className="mt-1 text-xs text-error">{errors.origin}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">3-letter IATA code (e.g. BDS for Brindisi)</p>
)}
</div>
{scanMode === 'reverse' ? (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Origin Country
</label>
<CountrySelect
value={selectedOriginCountry}
onChange={(code) => {
setSelectedOriginCountry(code);
if (errors.country) setErrors(prev => ({ ...prev, country: undefined }));
}}
placeholder="Select origin country…"
hasError={!!errors.country}
/>
{errors.country ? (
<p className="mt-1 text-xs text-error">{errors.country}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">All airports in this country will be checked</p>
)}
</div>
) : (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Origin Airport
</label>
<AirportSearch
value={formData.origin}
onChange={(value) => {
setFormData(prev => ({ ...prev, origin: value }));
if (errors.origin) setErrors(prev => ({ ...prev, origin: undefined }));
}}
placeholder="e.g. BDS, MUC, FRA"
hasError={!!errors.origin}
/>
{errors.origin ? (
<p className="mt-1 text-xs text-error">{errors.origin}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">3-letter IATA code (e.g. BDS for Brindisi)</p>
)}
</div>
)}
</div>
{/* ── Section: Destination ────────────────────────────────── */}
@@ -130,42 +248,8 @@ export default function Scans() {
Destination
</p>
<SegmentedButton
options={[
{ value: 'country', label: 'By Country', icon: Globe },
{ value: 'airports', label: 'By Airports', icon: PlaneTakeoff },
]}
value={destinationMode}
onChange={(v) => {
setDestinationMode(v as 'country' | 'airports');
setErrors(prev => ({ ...prev, country: undefined, airports: undefined }));
}}
className="mb-4"
/>
{destinationMode === 'country' ? (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Destination Country
</label>
<input
type="text"
value={formData.country}
onChange={(e) => {
setFormData(prev => ({ ...prev, country: e.target.value.toUpperCase() }));
if (errors.country) setErrors(prev => ({ ...prev, country: undefined }));
}}
maxLength={2}
placeholder="e.g. DE, IT, ES"
className={inputCls(!!errors.country)}
/>
{errors.country ? (
<p className="mt-1 text-xs text-error">{errors.country}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">ISO 2-letter country code (e.g. DE for Germany)</p>
)}
</div>
) : (
{scanMode === 'reverse' ? (
/* Reverse: specific destination airports */
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Destination Airports
@@ -203,6 +287,82 @@ export default function Scans() {
</p>
)}
</div>
) : (
/* Forward: by country or by specific airports */
<>
<SegmentedButton
options={[
{ value: 'country', label: 'By Country', icon: Globe },
{ value: 'airports', label: 'By Airports', icon: PlaneTakeoff },
]}
value={destinationMode}
onChange={(v) => {
setDestinationMode(v as 'country' | 'airports');
setErrors(prev => ({ ...prev, country: undefined, airports: undefined }));
}}
className="mb-4"
/>
{destinationMode === 'country' ? (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Destination Country
</label>
<CountrySelect
value={formData.country ?? ''}
onChange={(code) => {
setFormData(prev => ({ ...prev, country: code }));
if (errors.country) setErrors(prev => ({ ...prev, country: undefined }));
}}
placeholder="Select destination country…"
hasError={!!errors.country}
/>
{errors.country ? (
<p className="mt-1 text-xs text-error">{errors.country}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">All airports in this country will be searched</p>
)}
</div>
) : (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Destination Airports
</label>
<AirportSearch
value=""
onChange={(code) => {
if (code && code.length === 3 && !selectedAirports.includes(code)) {
setSelectedAirports(prev => [...prev, code]);
if (errors.airports) setErrors(prev => ({ ...prev, airports: undefined }));
}
}}
clearAfterSelect
placeholder="Search and add airports…"
hasError={!!errors.airports}
/>
{selectedAirports.length > 0 && (
<div className="flex flex-wrap gap-2 mt-3">
{selectedAirports.map((code) => (
<AirportChip
key={code}
code={code}
onRemove={() => setSelectedAirports(prev => prev.filter(c => c !== code))}
/>
))}
</div>
)}
{errors.airports ? (
<p className="mt-1 text-xs text-error">{errors.airports}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">
{selectedAirports.length === 0
? 'Search and add destination airports (up to 50)'
: `${selectedAirports.length} airport${selectedAirports.length !== 1 ? 's' : ''} selected`}
</p>
)}
</div>
)}
</>
)}
</div>
@@ -212,54 +372,141 @@ export default function Scans() {
Parameters
</p>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
{/* Search Window */}
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Search Window
</label>
<div className="flex items-center gap-2">
<button
type="button"
onClick={() => adjustNumber('window_months', -1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors"
aria-label="Decrease months"
>
<Minus size={14} />
</button>
<div className="flex-1 h-12 flex items-center justify-center border border-outline rounded-xs bg-surface text-on-surface text-sm font-medium">
{formData.window_months} {formData.window_months === 1 ? 'month' : 'months'}
</div>
<button
type="button"
onClick={() => adjustNumber('window_months', 1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors"
aria-label="Increase months"
>
<Plus size={14} />
</button>
</div>
<p className="mt-1 text-xs text-on-surface-variant">Months to look ahead (112)</p>
</div>
{/* Seat Class */}
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Seat Class
</label>
<select
value={formData.seat_class}
onChange={(e) => setFormData(prev => ({ ...prev, seat_class: e.target.value as 'economy' | 'premium' | 'business' | 'first' }))}
className={inputCls()}
>
<option value="economy">Economy</option>
<option value="premium">Premium Economy</option>
<option value="business">Business</option>
<option value="first">First Class</option>
</select>
</div>
{/* Search Window toggle */}
<div className="mb-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Search Window
</label>
<SegmentedButton
options={[
{ value: 'window', label: 'Rolling Window', icon: CalendarDays },
{ value: 'range', label: 'Date Range', icon: CalendarRange },
]}
value={windowMode}
onChange={(v) => {
setWindowMode(v as 'window' | 'range');
setStartDate('');
setEndDate('');
setErrors(prev => ({ ...prev, start_date: undefined, end_date: undefined }));
}}
/>
</div>
{windowMode === 'window' ? (
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
{/* Stepper */}
<div>
<div className="flex items-center gap-2">
<button
type="button"
onClick={() => adjustNumber('window_months', -1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors"
aria-label="Decrease months"
>
<Minus size={14} />
</button>
<div className="flex-1 h-12 flex items-center justify-center border border-outline rounded-xs bg-surface text-on-surface text-sm font-medium">
{formData.window_months} {formData.window_months === 1 ? 'month' : 'months'}
</div>
<button
type="button"
onClick={() => adjustNumber('window_months', 1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors"
aria-label="Increase months"
>
<Plus size={14} />
</button>
</div>
<p className="mt-1 text-xs text-on-surface-variant">Months to look ahead (112)</p>
</div>
{/* Seat Class */}
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Seat Class
</label>
<select
value={formData.seat_class}
onChange={(e) => setFormData(prev => ({ ...prev, seat_class: e.target.value as 'economy' | 'premium' | 'business' | 'first' }))}
className={inputCls()}
>
<option value="economy">Economy</option>
<option value="premium">Premium Economy</option>
<option value="business">Business</option>
<option value="first">First Class</option>
</select>
</div>
</div>
) : (
<>
{/* Date pickers */}
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
From
</label>
<input
type="date"
value={startDate}
min={tomorrowStr}
onChange={(e) => {
setStartDate(e.target.value);
if (errors.start_date) setErrors(prev => ({ ...prev, start_date: undefined }));
}}
className={inputCls(!!errors.start_date)}
/>
{errors.start_date ? (
<p className="mt-1 text-xs text-error">{errors.start_date}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">First date to scan</p>
)}
</div>
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
To
</label>
<input
type="date"
value={endDate}
min={minEndDate}
onChange={(e) => {
setEndDate(e.target.value);
if (errors.end_date) setErrors(prev => ({ ...prev, end_date: undefined }));
}}
className={inputCls(!!errors.end_date)}
/>
{errors.end_date ? (
<p className="mt-1 text-xs text-error">{errors.end_date}</p>
) : (
<p className="mt-1 text-xs text-on-surface-variant">Last date to scan</p>
)}
</div>
</div>
<p className="mt-2 text-xs text-on-surface-variant">
Flights will be sampled on the 15th of each month within this range
</p>
{/* Seat Class */}
<div className="mt-4 grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Seat Class
</label>
<select
value={formData.seat_class}
onChange={(e) => setFormData(prev => ({ ...prev, seat_class: e.target.value as 'economy' | 'premium' | 'business' | 'first' }))}
className={inputCls()}
>
<option value="economy">Economy</option>
<option value="premium">Premium Economy</option>
<option value="business">Business</option>
<option value="first">First Class</option>
</select>
</div>
</div>
</>
)}
{/* Adults — full width below */}
<div className="mt-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">

View File

@@ -0,0 +1,609 @@
import { useEffect, useState } from 'react';
import {
Globe,
PlaneTakeoff,
Minus,
Plus,
Play,
Trash2,
CalendarClock,
} from 'lucide-react';
import { scheduleApi } from '../api';
import type { Schedule, CreateScheduleRequest } from '../api';
import AirportSearch from '../components/AirportSearch';
import SegmentedButton from '../components/SegmentedButton';
import AirportChip from '../components/AirportChip';
import Button from '../components/Button';
import Toast from '../components/Toast';
import EmptyState from '../components/EmptyState';
import { cn } from '../lib/utils';
// ── Helpers ──────────────────────────────────────────────────────────────────
const DAYS = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'];
function formatNextRun(utcStr: string): string {
// utcStr is like "2026-03-01 06:00:00" (no Z suffix from SQLite)
const d = new Date(utcStr.replace(' ', 'T') + 'Z');
if (isNaN(d.getTime())) return utcStr;
return d.toLocaleString(undefined, {
month: 'short', day: 'numeric',
hour: '2-digit', minute: '2-digit',
});
}
function formatLastRun(utcStr?: string): string {
if (!utcStr) return '—';
const d = new Date(utcStr.replace(' ', 'T') + 'Z');
if (isNaN(d.getTime())) return utcStr;
return d.toLocaleString(undefined, {
month: 'short', day: 'numeric',
hour: '2-digit', minute: '2-digit',
});
}
function describeSchedule(s: Schedule): string {
const pad = (n: number) => String(n).padStart(2, '0');
const time = `${pad(s.hour)}:${pad(s.minute)} UTC`;
if (s.frequency === 'daily') return `Every day at ${time}`;
if (s.frequency === 'weekly') return `Every ${DAYS[s.day_of_week ?? 0]} at ${time}`;
if (s.frequency === 'monthly') return `${s.day_of_month}th of month at ${time}`;
return s.frequency;
}
// ── Form state ────────────────────────────────────────────────────────────────
interface FormState {
origin: string;
country: string;
window_months: number;
seat_class: string;
adults: number;
label: string;
frequency: 'daily' | 'weekly' | 'monthly';
hour: number;
minute: number;
day_of_week: number;
day_of_month: number;
}
const defaultForm = (): FormState => ({
origin: '',
country: '',
window_months: 1,
seat_class: 'economy',
adults: 1,
label: '',
frequency: 'weekly',
hour: 6,
minute: 0,
day_of_week: 0,
day_of_month: 1,
});
interface FormErrors {
origin?: string;
country?: string;
airports?: string;
hour?: string;
minute?: string;
}
// ── Component ─────────────────────────────────────────────────────────────────
export default function Schedules() {
const [schedules, setSchedules] = useState<Schedule[]>([]);
const [loading, setLoading] = useState(true);
const [showForm, setShowForm] = useState(false);
const [destinationMode, setDestinationMode] = useState<'country' | 'airports'>('country');
const [selectedAirports, setSelectedAirports] = useState<string[]>([]);
const [form, setForm] = useState<FormState>(defaultForm);
const [errors, setErrors] = useState<FormErrors>({});
const [saving, setSaving] = useState(false);
const [runningId, setRunningId] = useState<number | null>(null);
const [deletingId, setDeletingId] = useState<number | null>(null);
const [toast, setToast] = useState<{ message: string; type: 'success' | 'error' } | null>(null);
useEffect(() => { loadSchedules(); }, []);
const loadSchedules = async () => {
try {
setLoading(true);
const res = await scheduleApi.list(1, 100);
setSchedules(res.data.data);
} catch {
setToast({ message: 'Failed to load schedules', type: 'error' });
} finally {
setLoading(false);
}
};
const validate = (): boolean => {
const next: FormErrors = {};
if (!form.origin || form.origin.length !== 3)
next.origin = 'Enter a valid 3-letter IATA code';
if (destinationMode === 'country' && (!form.country || form.country.length < 2))
next.country = 'Enter a valid 2-letter country code';
if (destinationMode === 'airports' && selectedAirports.length === 0)
next.airports = 'Add at least one destination airport';
if (form.hour < 0 || form.hour > 23)
next.hour = '023';
if (form.minute < 0 || form.minute > 59)
next.minute = '059';
setErrors(next);
return Object.keys(next).length === 0;
};
const handleCreate = async (e: React.FormEvent) => {
e.preventDefault();
if (!validate()) return;
setSaving(true);
try {
const req: CreateScheduleRequest = {
origin: form.origin,
country: destinationMode === 'country'
? form.country
: selectedAirports.join(','),
window_months: form.window_months,
seat_class: form.seat_class,
adults: form.adults,
label: form.label || undefined,
frequency: form.frequency,
hour: form.hour,
minute: form.minute,
...(form.frequency === 'weekly' ? { day_of_week: form.day_of_week } : {}),
...(form.frequency === 'monthly' ? { day_of_month: form.day_of_month } : {}),
};
await scheduleApi.create(req);
setToast({ message: 'Schedule created', type: 'success' });
setShowForm(false);
setForm(defaultForm());
setSelectedAirports([]);
loadSchedules();
} catch (err: any) {
const msg = err.response?.data?.detail || 'Failed to create schedule';
setToast({ message: typeof msg === 'string' ? msg : JSON.stringify(msg), type: 'error' });
} finally {
setSaving(false);
}
};
const toggleEnabled = async (s: Schedule) => {
try {
const updated = await scheduleApi.update(s.id, { enabled: !s.enabled });
setSchedules(prev => prev.map(x => x.id === s.id ? updated.data : x));
} catch {
setToast({ message: 'Failed to update schedule', type: 'error' });
}
};
const handleRunNow = async (s: Schedule) => {
setRunningId(s.id);
try {
const res = await scheduleApi.runNow(s.id);
setToast({ message: `Scan #${res.data.scan_id} started`, type: 'success' });
loadSchedules();
} catch {
setToast({ message: 'Failed to trigger scan', type: 'error' });
} finally {
setRunningId(null);
}
};
const handleDelete = async (s: Schedule) => {
if (!confirm(`Delete schedule "${s.label || `${s.origin}${s.country}`}"?`)) return;
setDeletingId(s.id);
try {
await scheduleApi.delete(s.id);
setSchedules(prev => prev.filter(x => x.id !== s.id));
setToast({ message: 'Schedule deleted', type: 'success' });
} catch {
setToast({ message: 'Failed to delete schedule', type: 'error' });
} finally {
setDeletingId(null);
}
};
const adjustNumber = (field: 'window_months' | 'adults', delta: number) => {
const limits: Record<string, [number, number]> = { window_months: [1, 12], adults: [1, 9] };
const [min, max] = limits[field];
setForm(prev => ({ ...prev, [field]: Math.min(max, Math.max(min, prev[field] + delta)) }));
};
const inputCls = (hasError?: boolean) =>
`w-full h-12 px-3 border rounded-xs bg-surface text-on-surface text-sm outline-none transition-colors ` +
(hasError
? 'border-error focus:border-error focus:ring-2 focus:ring-error/20'
: 'border-outline focus:border-primary focus:ring-2 focus:ring-primary/20');
// ── Render ────────────────────────────────────────────────────────────────
return (
<>
<div className="space-y-4 max-w-4xl">
{/* Header actions */}
<div className="flex items-center justify-between">
<p className="text-sm text-on-surface-variant">
{loading ? 'Loading…' : `${schedules.length} schedule${schedules.length !== 1 ? 's' : ''}`}
</p>
{!showForm && (
<Button variant="filled" onClick={() => setShowForm(true)}>
New Schedule
</Button>
)}
</div>
{/* ── Create Form ─────────────────────────────────────────────── */}
{showForm && (
<form onSubmit={handleCreate} className="space-y-4">
{/* Origin */}
<div className="bg-surface rounded-lg shadow-level-1 p-6">
<p className="text-xs font-semibold uppercase tracking-wider text-on-surface-variant mb-4">Origin</p>
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Origin Airport
</label>
<AirportSearch
value={form.origin}
onChange={(v) => {
setForm(prev => ({ ...prev, origin: v }));
if (errors.origin) setErrors(prev => ({ ...prev, origin: undefined }));
}}
placeholder="e.g. BDS, MUC, FRA"
hasError={!!errors.origin}
/>
{errors.origin
? <p className="mt-1 text-xs text-error">{errors.origin}</p>
: <p className="mt-1 text-xs text-on-surface-variant">3-letter IATA code</p>}
</div>
</div>
{/* Destination */}
<div className="bg-surface rounded-lg shadow-level-1 p-6">
<p className="text-xs font-semibold uppercase tracking-wider text-on-surface-variant mb-4">Destination</p>
<SegmentedButton
options={[
{ value: 'country', label: 'By Country', icon: Globe },
{ value: 'airports', label: 'By Airports', icon: PlaneTakeoff },
]}
value={destinationMode}
onChange={(v) => {
setDestinationMode(v as 'country' | 'airports');
setErrors(prev => ({ ...prev, country: undefined, airports: undefined }));
}}
className="mb-4"
/>
{destinationMode === 'country' ? (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Destination Country
</label>
<input
type="text"
value={form.country}
onChange={(e) => {
setForm(prev => ({ ...prev, country: e.target.value.toUpperCase() }));
if (errors.country) setErrors(prev => ({ ...prev, country: undefined }));
}}
maxLength={2}
placeholder="e.g. DE, IT, ES"
className={inputCls(!!errors.country)}
/>
{errors.country
? <p className="mt-1 text-xs text-error">{errors.country}</p>
: <p className="mt-1 text-xs text-on-surface-variant">ISO 2-letter country code</p>}
</div>
) : (
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Destination Airports
</label>
<AirportSearch
value=""
onChange={(code) => {
if (code && code.length === 3 && !selectedAirports.includes(code)) {
setSelectedAirports(prev => [...prev, code]);
if (errors.airports) setErrors(prev => ({ ...prev, airports: undefined }));
}
}}
clearAfterSelect
placeholder="Search and add airports…"
hasError={!!errors.airports}
/>
{selectedAirports.length > 0 && (
<div className="flex flex-wrap gap-2 mt-3">
{selectedAirports.map(code => (
<AirportChip
key={code}
code={code}
onRemove={() => setSelectedAirports(prev => prev.filter(c => c !== code))}
/>
))}
</div>
)}
{errors.airports
? <p className="mt-1 text-xs text-error">{errors.airports}</p>
: <p className="mt-1 text-xs text-on-surface-variant">
{selectedAirports.length === 0
? 'Search and add destination airports'
: `${selectedAirports.length} airport${selectedAirports.length !== 1 ? 's' : ''} selected`}
</p>}
</div>
)}
</div>
{/* Parameters */}
<div className="bg-surface rounded-lg shadow-level-1 p-6">
<p className="text-xs font-semibold uppercase tracking-wider text-on-surface-variant mb-4">Parameters</p>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">Search Window</label>
<div className="flex items-center gap-2">
<button type="button" onClick={() => adjustNumber('window_months', -1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors">
<Minus size={14} />
</button>
<div className="flex-1 h-12 flex items-center justify-center border border-outline rounded-xs bg-surface text-on-surface text-sm font-medium">
{form.window_months} {form.window_months === 1 ? 'month' : 'months'}
</div>
<button type="button" onClick={() => adjustNumber('window_months', 1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors">
<Plus size={14} />
</button>
</div>
</div>
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">Seat Class</label>
<select value={form.seat_class}
onChange={(e) => setForm(prev => ({ ...prev, seat_class: e.target.value }))}
className={inputCls()}>
<option value="economy">Economy</option>
<option value="premium">Premium Economy</option>
<option value="business">Business</option>
<option value="first">First Class</option>
</select>
</div>
</div>
<div className="mt-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">Passengers</label>
<div className="flex items-center gap-2">
<button type="button" onClick={() => adjustNumber('adults', -1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors">
<Minus size={14} />
</button>
<div className="w-32 h-12 flex items-center justify-center border border-outline rounded-xs bg-surface text-on-surface text-sm font-medium">
{form.adults} {form.adults === 1 ? 'adult' : 'adults'}
</div>
<button type="button" onClick={() => adjustNumber('adults', 1)}
className="w-9 h-9 flex items-center justify-center rounded-full border border-outline text-on-surface-variant hover:bg-surface-2 transition-colors">
<Plus size={14} />
</button>
</div>
</div>
</div>
{/* Schedule */}
<div className="bg-surface rounded-lg shadow-level-1 p-6">
<p className="text-xs font-semibold uppercase tracking-wider text-on-surface-variant mb-4">Schedule</p>
{/* Optional label */}
<div className="mb-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Label <span className="font-normal opacity-60">(optional)</span>
</label>
<input
type="text"
value={form.label}
onChange={(e) => setForm(prev => ({ ...prev, label: e.target.value }))}
placeholder="e.g. Weekly BDS → Germany"
className={inputCls()}
maxLength={100}
/>
</div>
{/* Frequency */}
<div className="mb-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">Frequency</label>
<div className="flex gap-2">
{(['daily', 'weekly', 'monthly'] as const).map(f => (
<button
key={f}
type="button"
onClick={() => setForm(prev => ({ ...prev, frequency: f }))}
className={cn(
'flex-1 h-10 rounded-xs border text-sm font-medium transition-colors',
form.frequency === f
? 'bg-primary text-on-primary border-primary'
: 'border-outline text-on-surface hover:bg-surface-2',
)}
>
{f.charAt(0).toUpperCase() + f.slice(1)}
</button>
))}
</div>
</div>
{/* Day of week (weekly only) */}
{form.frequency === 'weekly' && (
<div className="mb-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">Day of week</label>
<div className="flex gap-1.5 flex-wrap">
{DAYS.map((d, i) => (
<button
key={d}
type="button"
onClick={() => setForm(prev => ({ ...prev, day_of_week: i }))}
className={cn(
'w-12 h-10 rounded-xs border text-sm font-medium transition-colors',
form.day_of_week === i
? 'bg-primary text-on-primary border-primary'
: 'border-outline text-on-surface hover:bg-surface-2',
)}
>
{d}
</button>
))}
</div>
</div>
)}
{/* Day of month (monthly only) */}
{form.frequency === 'monthly' && (
<div className="mb-4">
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Day of month <span className="font-normal opacity-60">(128)</span>
</label>
<input
type="number"
value={form.day_of_month}
onChange={(e) => setForm(prev => ({ ...prev, day_of_month: Number(e.target.value) }))}
min={1} max={28}
className={cn(inputCls(), 'w-28')}
/>
</div>
)}
{/* Time */}
<div>
<label className="block text-sm font-medium text-on-surface-variant mb-1.5">
Time <span className="font-normal opacity-60">(UTC)</span>
</label>
<div className="flex items-center gap-2">
<input
type="number"
value={form.hour}
onChange={(e) => setForm(prev => ({ ...prev, hour: Number(e.target.value) }))}
min={0} max={23}
className={cn(inputCls(!!errors.hour), 'w-20 text-center')}
placeholder="HH"
/>
<span className="text-on-surface-variant font-bold">:</span>
<input
type="number"
value={form.minute}
onChange={(e) => setForm(prev => ({ ...prev, minute: Number(e.target.value) }))}
min={0} max={59}
className={cn(inputCls(!!errors.minute), 'w-20 text-center')}
placeholder="MM"
/>
</div>
{(errors.hour || errors.minute) && (
<p className="mt-1 text-xs text-error">Hour: 023, Minute: 059</p>
)}
</div>
</div>
{/* Actions */}
<div className="flex justify-end gap-3 pb-4">
<Button variant="outlined" type="button"
onClick={() => { setShowForm(false); setForm(defaultForm()); setErrors({}); }}>
Cancel
</Button>
<Button variant="filled" type="submit" loading={saving}>
Create Schedule
</Button>
</div>
</form>
)}
{/* ── Schedules list ───────────────────────────────────────────── */}
{!loading && schedules.length === 0 && !showForm && (
<EmptyState
icon={CalendarClock}
title="No schedules yet"
description="Create a schedule to automatically run scans at a regular interval."
action={{ label: 'New Schedule', onClick: () => setShowForm(true) }}
/>
)}
{schedules.length > 0 && (
<div className="bg-surface rounded-lg shadow-level-1 overflow-hidden">
<table className="w-full text-sm">
<thead className="border-b border-outline">
<tr>
<th className="px-4 py-3 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Route</th>
<th className="px-4 py-3 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Cadence</th>
<th className="px-4 py-3 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant hidden md:table-cell">Next Run</th>
<th className="px-4 py-3 text-left text-xs font-semibold uppercase tracking-wider text-on-surface-variant hidden lg:table-cell">Last Run</th>
<th className="px-4 py-3 text-center text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Active</th>
<th className="px-4 py-3 text-right text-xs font-semibold uppercase tracking-wider text-on-surface-variant">Actions</th>
</tr>
</thead>
<tbody className="divide-y divide-outline">
{schedules.map(s => (
<tr key={s.id} className="hover:bg-surface-2 transition-colors">
<td className="px-4 py-3">
<div className="font-medium text-on-surface">
{s.label || `${s.origin}${s.country}`}
</div>
{s.label && (
<div className="text-xs text-on-surface-variant mt-0.5">
{s.origin} {s.country}
</div>
)}
</td>
<td className="px-4 py-3 text-on-surface-variant">
{describeSchedule(s)}
</td>
<td className="px-4 py-3 text-on-surface-variant hidden md:table-cell">
{formatNextRun(s.next_run_at)}
</td>
<td className="px-4 py-3 text-on-surface-variant hidden lg:table-cell">
{formatLastRun(s.last_run_at)}
</td>
<td className="px-4 py-3 text-center">
{/* Toggle switch */}
<button
type="button"
onClick={() => toggleEnabled(s)}
className={cn(
'relative inline-flex h-6 w-11 items-center rounded-full transition-colors focus:outline-none',
s.enabled ? 'bg-primary' : 'bg-outline',
)}
aria-label={s.enabled ? 'Disable schedule' : 'Enable schedule'}
>
<span className={cn(
'inline-block h-4 w-4 transform rounded-full bg-white transition-transform',
s.enabled ? 'translate-x-6' : 'translate-x-1',
)} />
</button>
</td>
<td className="px-4 py-3">
<div className="flex items-center justify-end gap-1">
<button
type="button"
onClick={() => handleRunNow(s)}
disabled={runningId === s.id}
className="p-2 rounded-xs text-on-surface-variant hover:bg-surface-2 hover:text-primary disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
title="Run now"
>
<Play size={15} />
</button>
<button
type="button"
onClick={() => handleDelete(s)}
disabled={deletingId === s.id}
className="p-2 rounded-xs text-on-surface-variant hover:bg-surface-2 hover:text-error disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
title="Delete schedule"
>
<Trash2 size={15} />
</button>
</div>
</td>
</tr>
))}
</tbody>
</table>
</div>
)}
</div>
{toast && (
<Toast message={toast.message} type={toast.type} onClose={() => setToast(null)} />
)}
</>
);
}

View File

@@ -0,0 +1 @@
/// <reference types="vite/client" />

View File

@@ -0,0 +1,22 @@
{
"compilerOptions": {
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"noUncheckedSideEffectImports": true
},
"include": ["src"]
}

View File

@@ -0,0 +1,7 @@
{
"files": [],
"references": [
{ "path": "./tsconfig.app.json" },
{ "path": "./tsconfig.node.json" }
]
}

View File

@@ -0,0 +1,20 @@
{
"compilerOptions": {
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
"target": "ES2022",
"lib": ["ES2023"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"noUncheckedSideEffectImports": true
},
"include": ["vite.config.ts"]
}

View File

@@ -1,20 +1,18 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
// When running inside Docker the backend is reachable via its service name.
// Outside Docker (plain `npm run dev`) it falls back to localhost.
const apiTarget = process.env.API_TARGET ?? 'http://localhost:8000'
// https://vite.dev/config/
export default defineConfig({
plugins: [react()],
server: {
port: 5173,
proxy: {
'/api': {
target: 'http://localhost:8000',
changeOrigin: true,
},
'/health': {
target: 'http://localhost:8000',
changeOrigin: true,
}
'/api': { target: apiTarget, changeOrigin: true },
'/health': { target: apiTarget, changeOrigin: true },
}
}
})

View File

@@ -17,7 +17,7 @@ server {
# API proxy
location /api/ {
proxy_pass http://localhost:8000;
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
@@ -30,7 +30,7 @@ server {
# Health check endpoint proxy
location /health {
proxy_pass http://localhost:8000;
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
}

View File

@@ -1,4 +1,15 @@
# Web API
fastapi>=0.104.0
uvicorn[standard]>=0.24.0
pydantic>=2.0.0
# CLI tool
click>=8.0.0
python-dateutil>=2.8.0
rich>=13.0.0
fast-flights>=3.0.0
# Shared utilities
requests>=2.31.0
# Flight search (GitHub only — not on PyPI as a stable release)
git+https://github.com/AWeirdDev/flights.git

View File

@@ -12,16 +12,165 @@ Runs as async background tasks within the FastAPI application.
import asyncio
import logging
from datetime import datetime, date, timedelta
from typing import Dict, List, Optional
import json
from database import get_connection
from airports import get_airports_for_country
from airports import get_airports_for_country, lookup_airport
from searcher_v3 import search_multiple_routes
logger = logging.getLogger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# Task registry — tracks running asyncio tasks so they can be cancelled.
# ─────────────────────────────────────────────────────────────────────────────
_running_tasks: dict[int, asyncio.Task] = {}
_cancel_reasons: dict[int, str] = {}
def cancel_scan_task(scan_id: int) -> bool:
"""Cancel the background task for a scan. Returns True if a task was found and cancelled."""
task = _running_tasks.get(scan_id)
if task and not task.done():
task.cancel()
return True
return False
def pause_scan_task(scan_id: int) -> bool:
"""Signal the running task to stop with status='paused'. Returns True if task was found."""
_cancel_reasons[scan_id] = 'paused'
return cancel_scan_task(scan_id)
def stop_scan_task(scan_id: int) -> bool:
"""Signal the running task to stop with status='cancelled'. Returns True if task was found."""
_cancel_reasons[scan_id] = 'cancelled'
return cancel_scan_task(scan_id)
def _write_route_incremental(scan_id: int, destination: str,
dest_name: str, dest_city: str,
new_flights: list, origin_airport: str = None):
"""
Write or update a route row and its individual flight rows immediately.
Called from progress_callback each time a (scan_id, destination, date)
query returns results. Merges into the existing route row if one already
exists, using a running weighted average for avg_price.
For reverse scans, origin_airport is the variable origin IATA code.
For forward scans, origin_airport is None.
Opens its own DB connection — safe to call from the event loop thread.
"""
prices = [f.get('price') for f in new_flights if f.get('price')]
if not prices:
return
new_airlines = list({f.get('airline') for f in new_flights if f.get('airline')})
new_count = len(prices)
new_min = min(prices)
new_max = max(prices)
new_avg = sum(prices) / new_count
try:
conn = get_connection()
cursor = conn.cursor()
# Fetch existing route row (key: scan_id + origin_airport + destination)
if origin_airport is None:
cursor.execute("""
SELECT id, flight_count, min_price, max_price, avg_price, airlines
FROM routes
WHERE scan_id = ? AND origin_airport IS NULL AND destination = ?
""", (scan_id, destination))
else:
cursor.execute("""
SELECT id, flight_count, min_price, max_price, avg_price, airlines
FROM routes
WHERE scan_id = ? AND origin_airport = ? AND destination = ?
""", (scan_id, origin_airport, destination))
existing = cursor.fetchone()
if existing is None:
cursor.execute("""
INSERT INTO routes (
scan_id, origin_airport, destination, destination_name, destination_city,
flight_count, airlines, min_price, max_price, avg_price
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id, origin_airport, destination, dest_name, dest_city,
new_count, json.dumps(new_airlines),
new_min, new_max, new_avg,
))
else:
old_count = existing['flight_count'] or 0
old_min = existing['min_price']
old_max = existing['max_price']
old_avg = existing['avg_price'] or 0.0
old_airlines = json.loads(existing['airlines']) if existing['airlines'] else []
merged_count = old_count + new_count
merged_min = min(old_min, new_min) if old_min is not None else new_min
merged_max = max(old_max, new_max) if old_max is not None else new_max
merged_avg = (old_avg * old_count + new_avg * new_count) / merged_count
merged_airlines = json.dumps(list(set(old_airlines) | set(new_airlines)))
if origin_airport is None:
cursor.execute("""
UPDATE routes
SET flight_count = ?,
min_price = ?,
max_price = ?,
avg_price = ?,
airlines = ?
WHERE scan_id = ? AND origin_airport IS NULL AND destination = ?
""", (
merged_count, merged_min, merged_max, merged_avg, merged_airlines,
scan_id, destination,
))
else:
cursor.execute("""
UPDATE routes
SET flight_count = ?,
min_price = ?,
max_price = ?,
avg_price = ?,
airlines = ?
WHERE scan_id = ? AND origin_airport = ? AND destination = ?
""", (
merged_count, merged_min, merged_max, merged_avg, merged_airlines,
scan_id, origin_airport, destination,
))
for flight in new_flights:
if not flight.get('price'):
continue
cursor.execute("""
INSERT INTO flights (
scan_id, origin_airport, destination, date, airline,
departure_time, arrival_time, price, stops
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id,
origin_airport,
destination,
flight.get('date', ''),
flight.get('airline'),
flight.get('departure_time'),
flight.get('arrival_time'),
flight.get('price'),
flight.get('stops', 0),
))
conn.commit()
conn.close()
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to write incremental route {destination}: {e}")
async def process_scan(scan_id: int):
"""
@@ -47,7 +196,7 @@ async def process_scan(scan_id: int):
cursor = conn.cursor()
cursor.execute("""
SELECT origin, country, start_date, end_date, seat_class, adults
SELECT origin, country, scan_mode, start_date, end_date, seat_class, adults
FROM scans
WHERE id = ?
""", (scan_id,))
@@ -57,36 +206,52 @@ async def process_scan(scan_id: int):
logger.error(f"[Scan {scan_id}] Scan not found in database")
return
origin, country_or_airports, start_date_str, end_date_str, seat_class, adults = row
origin, country_or_airports, scan_mode, start_date_str, end_date_str, seat_class, adults = row
scan_mode = scan_mode or 'forward'
logger.info(f"[Scan {scan_id}] Scan details: {origin} -> {country_or_airports}, {start_date_str} to {end_date_str}")
logger.info(f"[Scan {scan_id}] Scan details: mode={scan_mode}, {origin} -> {country_or_airports}, {start_date_str} to {end_date_str}")
# Update status to 'running'
# Update status to 'running' and record when processing started
cursor.execute("""
UPDATE scans
SET status = 'running', updated_at = CURRENT_TIMESTAMP
SET status = 'running', started_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (scan_id,))
conn.commit()
# Determine mode: country (2 letters) or specific airports (comma-separated)
# Resolve airports based on scan_mode
try:
if len(country_or_airports) == 2 and country_or_airports.isalpha():
# Country mode: resolve airports from country code
logger.info(f"[Scan {scan_id}] Mode: Country search ({country_or_airports})")
destinations = get_airports_for_country(country_or_airports)
if not destinations:
raise ValueError(f"No airports found for country: {country_or_airports}")
if scan_mode == 'reverse':
# Reverse scan: origin = ISO country, country_or_airports = comma-separated dest IATAs
logger.info(f"[Scan {scan_id}] Mode: Reverse scan ({origin} country → {country_or_airports})")
origin_airports = get_airports_for_country(origin)
if not origin_airports:
raise ValueError(f"No airports found for origin country: {origin}")
origin_iatas = [a['iata'] for a in origin_airports]
destination_codes = [d['iata'] for d in destinations]
logger.info(f"[Scan {scan_id}] Found {len(destination_codes)} destination airports: {destination_codes}")
destination_codes = [code.strip() for code in country_or_airports.split(',')]
dest_infos = {
code: lookup_airport(code) or {'iata': code, 'name': code, 'city': ''}
for code in destination_codes
}
logger.info(f"[Scan {scan_id}] {len(origin_iatas)} origins × {len(destination_codes)} destinations")
else:
# Specific airports mode: parse comma-separated list
destination_codes = [code.strip() for code in country_or_airports.split(',')]
destinations = [] # No pre-fetched airport details; fallback to IATA code as name
logger.info(f"[Scan {scan_id}] Mode: Specific airports ({len(destination_codes)} destinations: {destination_codes})")
# Forward scan: origin = fixed IATA, country_or_airports = country code or dest IATAs
if len(country_or_airports) == 2 and country_or_airports.isalpha():
logger.info(f"[Scan {scan_id}] Mode: Forward country search ({country_or_airports})")
dest_list = get_airports_for_country(country_or_airports)
if not dest_list:
raise ValueError(f"No airports found for country: {country_or_airports}")
destination_codes = [d['iata'] for d in dest_list]
dest_infos = {d['iata']: d for d in dest_list}
else:
destination_codes = [code.strip() for code in country_or_airports.split(',')]
dest_infos = {
code: lookup_airport(code) or {'iata': code, 'name': code, 'city': ''}
for code in destination_codes
}
logger.info(f"[Scan {scan_id}] Mode: Forward specific airports ({destination_codes})")
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to resolve airports: {str(e)}")
@@ -94,14 +259,13 @@ async def process_scan(scan_id: int):
UPDATE scans
SET status = 'failed',
error_message = ?,
completed_at = CURRENT_TIMESTAMP,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (f"Failed to resolve airports: {str(e)}", scan_id))
conn.commit()
return
# Note: Don't update total_routes yet - we'll set it after we know the actual number of route queries
# Generate dates to scan — every day in the window
start_date = datetime.strptime(start_date_str, '%Y-%m-%d').date()
end_date = datetime.strptime(end_date_str, '%Y-%m-%d').date()
@@ -114,11 +278,17 @@ async def process_scan(scan_id: int):
logger.info(f"[Scan {scan_id}] Will scan {len(dates)} dates: {dates}")
# Build routes list: [(origin, destination, date), ...]
# Build routes list: [(origin_iata, destination_iata, date), ...]
routes_to_scan = []
for dest in destination_codes:
for scan_date in dates:
routes_to_scan.append((origin, dest, scan_date))
if scan_mode == 'reverse':
for orig_iata in origin_iatas:
for dest_code in destination_codes:
for scan_date in dates:
routes_to_scan.append((orig_iata, dest_code, scan_date))
else:
for dest_code in destination_codes:
for scan_date in dates:
routes_to_scan.append((origin, dest_code, scan_date))
logger.info(f"[Scan {scan_id}] Total route queries: {len(routes_to_scan)}")
@@ -131,19 +301,33 @@ async def process_scan(scan_id: int):
""", (len(routes_to_scan), scan_id))
conn.commit()
# Progress callback to update database
# Signature: callback(origin, destination, date, status, count, error=None)
# Progress callback updates DB progress counter and writes routes live
# Signature: callback(origin, destination, date, status, count, error=None, flights=None)
routes_scanned_count = 0
def progress_callback(origin: str, destination: str, date: str,
status: str, count: int, error: str = None):
def progress_callback(cb_origin: str, destination: str, date: str,
status: str, count: int, error: str = None,
flights: list = None):
nonlocal routes_scanned_count
# Increment counter for each route query (cache hit or API call)
if status in ('cache_hit', 'api_success', 'error'):
routes_scanned_count += 1
# Update progress in database
# Write route + flights to DB immediately if results available
if flights and status in ('cache_hit', 'api_success'):
for f in flights:
f['date'] = date
dest_info = dest_infos.get(destination) or {'iata': destination, 'name': destination, 'city': ''}
dest_name = dest_info.get('name', destination)
dest_city = dest_info.get('city', '')
# For reverse scans, cb_origin is the variable origin airport IATA
route_origin = cb_origin if scan_mode == 'reverse' else None
_write_route_incremental(
scan_id, destination, dest_name, dest_city, flights,
origin_airport=route_origin
)
# Update progress counter
try:
progress_conn = get_connection()
progress_cursor = progress_conn.cursor()
@@ -158,8 +342,8 @@ async def process_scan(scan_id: int):
progress_conn.commit()
progress_conn.close()
if routes_scanned_count % 10 == 0: # Log every 10 routes
logger.info(f"[Scan {scan_id}] Progress: {routes_scanned_count}/{len(routes_to_scan)} routes ({status}: {origin}{destination})")
if routes_scanned_count % 10 == 0:
logger.info(f"[Scan {scan_id}] Progress: {routes_scanned_count}/{len(routes_to_scan)} routes ({status}: {cb_origin}{destination})")
except Exception as e:
logger.error(f"[Scan {scan_id}] Failed to update progress: {str(e)}")
@@ -177,101 +361,46 @@ async def process_scan(scan_id: int):
progress_callback=progress_callback
)
logger.info(f"[Scan {scan_id}] Flight queries complete. Processing results...")
logger.info(f"[Scan {scan_id}] Flight queries complete.")
# Group results by destination, preserving date per flight
# Structure: {dest: [(flight_dict, date), ...]}
routes_by_destination: Dict[str, List] = {}
total_flights = 0
# Routes and flights were written incrementally by progress_callback.
routes_saved = cursor.execute(
"SELECT COUNT(*) FROM routes WHERE scan_id = ?", (scan_id,)
).fetchone()[0]
total_flights_saved = cursor.execute(
"SELECT COALESCE(SUM(flight_count), 0) FROM routes WHERE scan_id = ?", (scan_id,)
).fetchone()[0]
for (orig, dest, scan_date), flights in results.items():
if dest not in routes_by_destination:
routes_by_destination[dest] = []
for flight in flights:
routes_by_destination[dest].append((flight, scan_date))
total_flights += len(flights)
logger.info(f"[Scan {scan_id}] Found {total_flights} total flights across {len(routes_by_destination)} destinations")
# Save routes and individual flights to database
routes_saved = 0
for destination, flight_date_pairs in routes_by_destination.items():
if not flight_date_pairs:
continue # Skip destinations with no flights
flights = [f for f, _ in flight_date_pairs]
# Get destination details (fall back to IATA code if not in DB)
dest_info = next((d for d in destinations if d['iata'] == destination), None)
dest_name = dest_info.get('name', destination) if dest_info else destination
dest_city = dest_info.get('city', '') if dest_info else ''
# Calculate statistics
prices = [f.get('price') for f in flights if f.get('price')]
airlines = list(set(f.get('airline') for f in flights if f.get('airline')))
if not prices:
logger.info(f"[Scan {scan_id}] Skipping {destination} - no prices available")
continue
min_price = min(prices)
max_price = max(prices)
avg_price = sum(prices) / len(prices)
# Insert route summary
cursor.execute("""
INSERT INTO routes (
scan_id, destination, destination_name, destination_city,
min_price, max_price, avg_price, flight_count, airlines
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id,
destination,
dest_name,
dest_city,
min_price,
max_price,
avg_price,
len(flights),
json.dumps(airlines)
))
# Insert individual flights
for flight, flight_date in flight_date_pairs:
if not flight.get('price'):
continue
cursor.execute("""
INSERT INTO flights (
scan_id, destination, date, airline,
departure_time, arrival_time, price, stops
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""", (
scan_id,
destination,
flight_date,
flight.get('airline'),
flight.get('departure_time'),
flight.get('arrival_time'),
flight.get('price'),
flight.get('stops', 0),
))
routes_saved += 1
conn.commit()
# Update scan to completed
# Update scan to completed and record finish time
cursor.execute("""
UPDATE scans
SET status = 'completed',
total_flights = ?,
completed_at = CURRENT_TIMESTAMP,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (total_flights, scan_id))
""", (total_flights_saved, scan_id))
conn.commit()
logger.info(f"[Scan {scan_id}] ✅ Scan completed successfully! {routes_saved} routes saved with {total_flights} flights")
logger.info(f"[Scan {scan_id}] ✅ Scan completed successfully! {routes_saved} routes saved with {total_flights_saved} flights")
except asyncio.CancelledError:
reason = _cancel_reasons.pop(scan_id, 'cancelled')
logger.info(f"[Scan {scan_id}] Scan {reason} by user request")
try:
if conn:
cursor = conn.cursor()
cursor.execute("""
UPDATE scans
SET status = ?,
completed_at = CURRENT_TIMESTAMP,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (reason, scan_id))
conn.commit()
except Exception as update_error:
logger.error(f"[Scan {scan_id}] Failed to update {reason} status: {str(update_error)}")
raise # must re-raise so asyncio marks the task as cancelled
except Exception as e:
logger.error(f"[Scan {scan_id}] ❌ Scan failed with error: {str(e)}", exc_info=True)
@@ -284,6 +413,7 @@ async def process_scan(scan_id: int):
UPDATE scans
SET status = 'failed',
error_message = ?,
completed_at = CURRENT_TIMESTAMP,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
""", (str(e), scan_id))
@@ -307,5 +437,28 @@ def start_scan_processor(scan_id: int):
asyncio.Task: The background task
"""
task = asyncio.create_task(process_scan(scan_id))
_running_tasks[scan_id] = task
task.add_done_callback(lambda _: _running_tasks.pop(scan_id, None))
logger.info(f"[Scan {scan_id}] Background task created")
return task
def start_resume_processor(scan_id: int):
"""
Resume processing a paused scan as a background task.
The API endpoint has already reset status to 'pending' and cleared counters.
process_scan() will transition the status to 'running' and re-run all routes,
getting instant cache hits for already-queried routes.
Args:
scan_id: The ID of the paused scan to resume
Returns:
asyncio.Task: The background task
"""
task = asyncio.create_task(process_scan(scan_id))
_running_tasks[scan_id] = task
task.add_done_callback(lambda _: _running_tasks.pop(scan_id, None))
logger.info(f"[Scan {scan_id}] Resume task created")
return task

View File

@@ -118,7 +118,7 @@ async def search_direct_flights(
)
if cached is not None:
if progress_callback:
progress_callback(origin, destination, date, "cache_hit", len(cached))
progress_callback(origin, destination, date, "cache_hit", len(cached), flights=cached)
return cached
# Add random delay to avoid rate limiting
@@ -140,7 +140,7 @@ async def search_direct_flights(
# Report progress
if progress_callback:
progress_callback(origin, destination, date, "api_success", len(result))
progress_callback(origin, destination, date, "api_success", len(result), flights=result)
return result

View File

@@ -1,24 +0,0 @@
[supervisord]
nodaemon=true
logfile=/dev/null
logfile_maxbytes=0
pidfile=/tmp/supervisord.pid
[program:api]
command=python /app/api_server.py
directory=/app
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:nginx]
command=nginx -g "daemon off;"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

View File

@@ -245,6 +245,45 @@ class TestScanEndpoints:
assert data["data"][0]["destination"] == "FRA"
assert data["data"][0]["min_price"] == 50
def test_get_scan_paused_status(self, client: TestClient, create_test_scan):
"""Test that GET /scans/{id} returns paused status correctly."""
scan_id = create_test_scan(status='paused')
response = client.get(f"/api/v1/scans/{scan_id}")
assert response.status_code == 200
assert response.json()["status"] == "paused"
def test_get_scan_cancelled_status(self, client: TestClient, create_test_scan):
"""Test that GET /scans/{id} returns cancelled status correctly."""
scan_id = create_test_scan(status='cancelled')
response = client.get(f"/api/v1/scans/{scan_id}")
assert response.status_code == 200
assert response.json()["status"] == "cancelled"
def test_list_scans_filter_paused(self, client: TestClient, create_test_scan):
"""Test filtering scans by paused status."""
create_test_scan(status='paused')
create_test_scan(status='completed')
create_test_scan(status='running')
response = client.get("/api/v1/scans?status=paused")
assert response.status_code == 200
data = response.json()
assert len(data["data"]) == 1
assert data["data"][0]["status"] == "paused"
def test_list_scans_filter_cancelled(self, client: TestClient, create_test_scan):
"""Test filtering scans by cancelled status."""
create_test_scan(status='cancelled')
create_test_scan(status='pending')
response = client.get("/api/v1/scans?status=cancelled")
assert response.status_code == 200
data = response.json()
assert len(data["data"]) == 1
assert data["data"][0]["status"] == "cancelled"
@pytest.mark.unit
@pytest.mark.api

View File

@@ -86,6 +86,25 @@ class TestScanWorkflow:
prices = [r["min_price"] for r in routes]
assert prices == sorted(prices)
def test_pause_and_resume_preserves_scan_id(self, client: TestClient, create_test_scan):
"""Resume returns the same scan id, not a new one (unlike Re-run)."""
scan_id = create_test_scan(status='running')
# Pause
pause_resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert pause_resp.status_code == 200
assert pause_resp.json()["id"] == scan_id
# Resume
resume_resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert resume_resp.status_code == 200
assert resume_resp.json()["id"] == scan_id
# Confirm scan still exists with same id
get_resp = client.get(f"/api/v1/scans/{scan_id}")
assert get_resp.status_code == 200
assert get_resp.json()["id"] == scan_id
@pytest.mark.integration
@pytest.mark.database

View File

@@ -0,0 +1,370 @@
"""
Tests for scan control endpoints: pause, cancel, resume.
Covers API behaviour, DB state, status transitions, rate limit headers,
and schema-level acceptance of the new 'paused' and 'cancelled' values.
"""
import pytest
import sqlite3
from fastapi.testclient import TestClient
# =============================================================================
# TestScanControlEndpoints — API unit tests
# =============================================================================
@pytest.mark.unit
@pytest.mark.api
class TestScanControlEndpoints:
"""Tests for pause, cancel, and resume endpoints in isolation."""
# ── Pause ──────────────────────────────────────────────────────────────
def test_pause_running_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert resp.status_code == 200
body = resp.json()
assert body["status"] == "paused"
assert body["id"] == scan_id
def test_pause_pending_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='pending')
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert resp.status_code == 200
assert resp.json()["status"] == "paused"
def test_pause_nonexistent_scan(self, client: TestClient):
resp = client.post("/api/v1/scans/99999/pause")
assert resp.status_code == 404
def test_pause_completed_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='completed')
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert resp.status_code == 409
def test_pause_already_paused_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='paused')
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert resp.status_code == 409
def test_pause_cancelled_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='cancelled')
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert resp.status_code == 409
# ── Cancel ─────────────────────────────────────────────────────────────
def test_cancel_running_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.post(f"/api/v1/scans/{scan_id}/cancel")
assert resp.status_code == 200
assert resp.json()["status"] == "cancelled"
def test_cancel_pending_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='pending')
resp = client.post(f"/api/v1/scans/{scan_id}/cancel")
assert resp.status_code == 200
assert resp.json()["status"] == "cancelled"
def test_cancel_nonexistent_scan(self, client: TestClient):
resp = client.post("/api/v1/scans/99999/cancel")
assert resp.status_code == 404
def test_cancel_completed_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='completed')
resp = client.post(f"/api/v1/scans/{scan_id}/cancel")
assert resp.status_code == 409
def test_cancel_already_cancelled_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='cancelled')
resp = client.post(f"/api/v1/scans/{scan_id}/cancel")
assert resp.status_code == 409
# ── Resume ─────────────────────────────────────────────────────────────
def test_resume_paused_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='paused')
resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert resp.status_code == 200
body = resp.json()
assert body["status"] == "pending"
assert body["id"] == scan_id
def test_resume_nonexistent_scan(self, client: TestClient):
resp = client.post("/api/v1/scans/99999/resume")
assert resp.status_code == 404
def test_resume_running_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert resp.status_code == 409
def test_resume_cancelled_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='cancelled')
resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert resp.status_code == 409
def test_resume_completed_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='completed')
resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert resp.status_code == 409
# ── Response shape ──────────────────────────────────────────────────────
def test_pause_response_shape(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
body = client.post(f"/api/v1/scans/{scan_id}/pause").json()
assert "id" in body
assert "status" in body
def test_cancel_response_shape(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
body = client.post(f"/api/v1/scans/{scan_id}/cancel").json()
assert "id" in body
assert "status" in body
def test_resume_response_shape(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='paused')
body = client.post(f"/api/v1/scans/{scan_id}/resume").json()
assert "id" in body
assert "status" in body
# =============================================================================
# TestScanControlDatabaseState — verify DB state after operations
# =============================================================================
@pytest.mark.database
class TestScanControlDatabaseState:
"""Tests that verify SQLite state after pause/cancel/resume operations."""
def test_pause_sets_completed_at(self, client: TestClient, create_test_scan, clean_database):
scan_id = create_test_scan(status='running')
client.post(f"/api/v1/scans/{scan_id}/pause")
conn = sqlite3.connect(clean_database)
row = conn.execute("SELECT completed_at FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
assert row[0] is not None
def test_cancel_sets_completed_at(self, client: TestClient, create_test_scan, clean_database):
scan_id = create_test_scan(status='running')
client.post(f"/api/v1/scans/{scan_id}/cancel")
conn = sqlite3.connect(clean_database)
row = conn.execute("SELECT completed_at FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
assert row[0] is not None
def test_resume_clears_completed_at(self, client: TestClient, create_test_scan, clean_database):
scan_id = create_test_scan(status='paused')
client.post(f"/api/v1/scans/{scan_id}/resume")
conn = sqlite3.connect(clean_database)
row = conn.execute("SELECT completed_at FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
assert row[0] is None
def test_resume_resets_started_at_from_old_value(self, client: TestClient, create_test_scan, clean_database):
"""After resume, started_at is no longer the old seeded timestamp.
The endpoint clears started_at; the background processor may then
set a new timestamp immediately. Either way, the old value is gone.
"""
old_timestamp = '2026-01-01 10:00:00'
scan_id = create_test_scan(status='paused')
conn = sqlite3.connect(clean_database)
conn.execute("UPDATE scans SET started_at = ? WHERE id = ?", (old_timestamp, scan_id))
conn.commit()
conn.close()
client.post(f"/api/v1/scans/{scan_id}/resume")
conn = sqlite3.connect(clean_database)
row = conn.execute("SELECT started_at FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
# The endpoint cleared the old timestamp; the processor may have set a new one
assert row[0] != old_timestamp
def test_resume_resets_routes_scanned(self, client: TestClient, create_test_scan, clean_database):
scan_id = create_test_scan(status='paused')
conn = sqlite3.connect(clean_database)
conn.execute("UPDATE scans SET routes_scanned = 50, total_routes = 100 WHERE id = ?", (scan_id,))
conn.commit()
conn.close()
client.post(f"/api/v1/scans/{scan_id}/resume")
conn = sqlite3.connect(clean_database)
row = conn.execute("SELECT routes_scanned FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
assert row[0] == 0
def test_pause_preserves_routes(
self, client: TestClient, create_test_scan, create_test_route, clean_database
):
scan_id = create_test_scan(status='running')
create_test_route(scan_id=scan_id, destination='MUC')
client.post(f"/api/v1/scans/{scan_id}/pause")
conn = sqlite3.connect(clean_database)
count = conn.execute(
"SELECT COUNT(*) FROM routes WHERE scan_id = ?", (scan_id,)
).fetchone()[0]
conn.close()
assert count == 1
def test_cancel_preserves_routes(
self, client: TestClient, create_test_scan, create_test_route, clean_database
):
scan_id = create_test_scan(status='running')
create_test_route(scan_id=scan_id, destination='MUC')
client.post(f"/api/v1/scans/{scan_id}/cancel")
conn = sqlite3.connect(clean_database)
count = conn.execute(
"SELECT COUNT(*) FROM routes WHERE scan_id = ?", (scan_id,)
).fetchone()[0]
conn.close()
assert count == 1
# =============================================================================
# TestScanControlStatusTransitions — full workflow integration tests
# =============================================================================
@pytest.mark.integration
@pytest.mark.database
class TestScanControlStatusTransitions:
"""Full workflow tests across multiple API calls."""
def test_running_to_paused_to_pending(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
# Pause it
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert resp.json()["status"] == "paused"
# Verify persisted
assert client.get(f"/api/v1/scans/{scan_id}").json()["status"] == "paused"
# Resume → pending (background processor moves to running)
resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert resp.json()["status"] == "pending"
def test_running_to_cancelled(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.post(f"/api/v1/scans/{scan_id}/cancel")
assert resp.json()["status"] == "cancelled"
assert client.get(f"/api/v1/scans/{scan_id}").json()["status"] == "cancelled"
def test_pause_then_delete(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='paused')
resp = client.delete(f"/api/v1/scans/{scan_id}")
assert resp.status_code == 204
def test_cancel_then_delete(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='cancelled')
resp = client.delete(f"/api/v1/scans/{scan_id}")
assert resp.status_code == 204
def test_cannot_delete_running_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.delete(f"/api/v1/scans/{scan_id}")
assert resp.status_code == 409
def test_cannot_delete_pending_scan(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='pending')
resp = client.delete(f"/api/v1/scans/{scan_id}")
assert resp.status_code == 409
def test_list_scans_filter_paused(self, client: TestClient, create_test_scan):
paused_id = create_test_scan(status='paused')
create_test_scan(status='running')
create_test_scan(status='completed')
resp = client.get("/api/v1/scans?status=paused")
assert resp.status_code == 200
scans = resp.json()["data"]
assert len(scans) >= 1
assert all(s["status"] == "paused" for s in scans)
assert any(s["id"] == paused_id for s in scans)
def test_list_scans_filter_cancelled(self, client: TestClient, create_test_scan):
cancelled_id = create_test_scan(status='cancelled')
create_test_scan(status='running')
resp = client.get("/api/v1/scans?status=cancelled")
assert resp.status_code == 200
scans = resp.json()["data"]
assert len(scans) >= 1
assert all(s["status"] == "cancelled" for s in scans)
assert any(s["id"] == cancelled_id for s in scans)
# =============================================================================
# TestScanControlRateLimits — rate limit headers on control endpoints
# =============================================================================
@pytest.mark.api
class TestScanControlRateLimits:
"""Verify that rate limit response headers are present on control endpoints."""
def test_pause_rate_limit_headers(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.post(f"/api/v1/scans/{scan_id}/pause")
assert "x-ratelimit-limit" in resp.headers
assert "x-ratelimit-remaining" in resp.headers
def test_cancel_rate_limit_headers(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='running')
resp = client.post(f"/api/v1/scans/{scan_id}/cancel")
assert "x-ratelimit-limit" in resp.headers
assert "x-ratelimit-remaining" in resp.headers
def test_resume_rate_limit_headers(self, client: TestClient, create_test_scan):
scan_id = create_test_scan(status='paused')
resp = client.post(f"/api/v1/scans/{scan_id}/resume")
assert "x-ratelimit-limit" in resp.headers
assert "x-ratelimit-remaining" in resp.headers
# =============================================================================
# TestScanControlNewStatuses — schema-level acceptance of new status values
# =============================================================================
@pytest.mark.database
class TestScanControlNewStatuses:
"""Verify the new status values are accepted/rejected at the SQLite level."""
def test_paused_status_accepted_by_schema(self, clean_database, create_test_scan):
scan_id = create_test_scan(status='pending')
conn = sqlite3.connect(clean_database)
conn.execute("UPDATE scans SET status='paused' WHERE id = ?", (scan_id,))
conn.commit()
row = conn.execute("SELECT status FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
assert row[0] == 'paused'
def test_cancelled_status_accepted_by_schema(self, clean_database, create_test_scan):
scan_id = create_test_scan(status='pending')
conn = sqlite3.connect(clean_database)
conn.execute("UPDATE scans SET status='cancelled' WHERE id = ?", (scan_id,))
conn.commit()
row = conn.execute("SELECT status FROM scans WHERE id = ?", (scan_id,)).fetchone()
conn.close()
assert row[0] == 'cancelled'
def test_invalid_status_rejected_by_schema(self, clean_database, create_test_scan):
scan_id = create_test_scan(status='pending')
conn = sqlite3.connect(clean_database)
with pytest.raises(sqlite3.IntegrityError):
conn.execute("UPDATE scans SET status='stopped' WHERE id = ?", (scan_id,))
conn.commit()
conn.close()
def test_filter_active_scans_excludes_paused(self, clean_database, create_test_scan):
paused_id = create_test_scan(status='paused')
conn = sqlite3.connect(clean_database)
rows = conn.execute("SELECT id FROM active_scans").fetchall()
conn.close()
ids = [r[0] for r in rows]
assert paused_id not in ids
def test_filter_active_scans_excludes_cancelled(self, clean_database, create_test_scan):
cancelled_id = create_test_scan(status='cancelled')
conn = sqlite3.connect(clean_database)
rows = conn.execute("SELECT id FROM active_scans").fetchall()
conn.close()
ids = [r[0] for r in rows]
assert cancelled_id not in ids

View File

@@ -0,0 +1,127 @@
"""
Tests for scan_processor task registry and control functions.
Tests cancel_scan_task, pause_scan_task, stop_scan_task, and the
done-callback that removes tasks from the registry on completion.
"""
import asyncio
import pytest
import sys
import os
from unittest.mock import MagicMock
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
from scan_processor import (
_running_tasks,
_cancel_reasons,
cancel_scan_task,
pause_scan_task,
stop_scan_task,
)
class TestScanProcessorControl:
"""Tests for task registry and cancel/pause/stop functions."""
def teardown_method(self, _method):
"""Clean up any test state from _running_tasks and _cancel_reasons."""
for key in [9001, 8001, 8002, 7001]:
_running_tasks.pop(key, None)
_cancel_reasons.pop(key, None)
# ── cancel_scan_task ───────────────────────────────────────────────────
def test_cancel_scan_task_returns_false_when_no_task(self):
"""Returns False when no task is registered for the given scan id."""
result = cancel_scan_task(99999)
assert result is False
def test_cancel_scan_task_returns_true_when_task_exists(self):
"""Returns True and calls task.cancel() when a live task is registered."""
mock_task = MagicMock()
mock_task.done.return_value = False
_running_tasks[9001] = mock_task
result = cancel_scan_task(9001)
assert result is True
mock_task.cancel.assert_called_once()
def test_cancel_scan_task_returns_false_for_completed_task(self):
"""Returns False when the registered task is already done."""
mock_task = MagicMock()
mock_task.done.return_value = True
_running_tasks[9001] = mock_task
result = cancel_scan_task(9001)
assert result is False
mock_task.cancel.assert_not_called()
# ── pause_scan_task ────────────────────────────────────────────────────
def test_pause_sets_cancel_reason_paused(self):
"""pause_scan_task sets _cancel_reasons[id] = 'paused'."""
mock_task = MagicMock()
mock_task.done.return_value = False
_running_tasks[8001] = mock_task
pause_scan_task(8001)
assert _cancel_reasons.get(8001) == 'paused'
def test_pause_calls_cancel_on_task(self):
"""pause_scan_task triggers cancellation of the underlying task."""
mock_task = MagicMock()
mock_task.done.return_value = False
_running_tasks[8001] = mock_task
result = pause_scan_task(8001)
assert result is True
mock_task.cancel.assert_called_once()
# ── stop_scan_task ─────────────────────────────────────────────────────
def test_stop_sets_cancel_reason_cancelled(self):
"""stop_scan_task sets _cancel_reasons[id] = 'cancelled'."""
mock_task = MagicMock()
mock_task.done.return_value = False
_running_tasks[8002] = mock_task
stop_scan_task(8002)
assert _cancel_reasons.get(8002) == 'cancelled'
def test_stop_calls_cancel_on_task(self):
"""stop_scan_task triggers cancellation of the underlying task."""
mock_task = MagicMock()
mock_task.done.return_value = False
_running_tasks[8002] = mock_task
result = stop_scan_task(8002)
assert result is True
mock_task.cancel.assert_called_once()
# ── done callback ──────────────────────────────────────────────────────
def test_task_removed_from_registry_on_completion(self):
"""The done-callback registered by start_scan_processor removes the task."""
async def run():
async def quick():
return
task = asyncio.create_task(quick())
_running_tasks[7001] = task
task.add_done_callback(lambda _: _running_tasks.pop(7001, None))
await task
# Yield to let done callbacks fire
await asyncio.sleep(0)
return 7001 not in _running_tasks
result = asyncio.run(run())
assert result is True