Full-stack flight price scanner built on fast-flights v3 (SOCS cookie bypass): Backend (FastAPI + SQLite): - REST API with rate limiting, Pydantic v2 validation, paginated responses - Scan pipeline: resolves airports, queries every day in the window, saves individual flights + aggregate route stats to SQLite - Background async scan processor with real-time progress tracking - Airport search endpoint backed by OpenFlights dataset - Daily scan window (all dates, not monthly samples) Frontend (React 19 + TypeScript + Tailwind CSS v4): - Dashboard with live scan status and recent scans - Create scan form: country mode or specific airports (searchable dropdown) - Scan detail page with expandable route rows showing individual flights (date, airline, departure, arrival, price) loaded on demand - AirportSearch component with debounced live search and multi-select Database: - scans → routes → flights schema with FK cascade and auto-update triggers - Migrations for schema evolution (relaxed country constraint) Tests: - 74 tests: unit + integration, isolated per-test SQLite DB - Confirmed flight fixtures in tests/confirmed_flights.json (50 real flights, BDS→FMM Ryanair + BDS→DUS Eurowings, scraped Feb 2026) - Integration tests parametrized from confirmed routes Docker: - Multi-stage builds, Compose orchestration, Nginx reverse proxy Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
235 lines
5.6 KiB
Markdown
235 lines
5.6 KiB
Markdown
# Migration Guide: fast-flights v3.0rc1 with SOCS Cookie
|
|
|
|
## What Changed
|
|
|
|
The Flight Airport Comparator now uses **fast-flights v3.0rc1** with **SOCS cookie integration** to successfully bypass Google's consent page and retrieve real flight data.
|
|
|
|
## Quick Start
|
|
|
|
### 1. Install fast-flights v3.0rc1
|
|
|
|
```bash
|
|
pip install --upgrade git+https://github.com/AWeirdDev/flights.git
|
|
```
|
|
|
|
### 2. Verify Installation
|
|
|
|
```bash
|
|
python -c "import fast_flights; print('✓ v3.0rc1 installed')"
|
|
```
|
|
|
|
### 3. Test It Works
|
|
|
|
```bash
|
|
cd flight-comparator
|
|
python test_v3_with_cookies.py
|
|
```
|
|
|
|
You should see:
|
|
```
|
|
✅ SUCCESS! Found 1 flight option(s):
|
|
1. Ryanair
|
|
Price: €89
|
|
BER → BRI
|
|
...
|
|
```
|
|
|
|
## What's New
|
|
|
|
### ✅ SOCS Cookie Integration
|
|
|
|
The breakthrough solution! A custom `Integration` class injects Google's SOCS (consent) cookie into every request:
|
|
|
|
```python
|
|
class SOCSCookieIntegration(Integration):
|
|
SOCS_COOKIE = 'CAESHwgBEhJnd3NfMjAyNTAyMjctMF9SQzIaBXpoLUNOIAEaBgiAy6O-Bg'
|
|
|
|
def fetch_html(self, q: Query | str, /) -> str:
|
|
client = primp.Client(...)
|
|
response = client.get(
|
|
"https://www.google.com/travel/flights",
|
|
params=params,
|
|
cookies={'SOCS': self.SOCS_COOKIE}, # ← Magic happens here
|
|
)
|
|
return response.text
|
|
```
|
|
|
|
This tells Google the user has accepted cookies, bypassing the consent page entirely.
|
|
|
|
### ✅ v3 API Changes
|
|
|
|
**Old (v2.2):**
|
|
```python
|
|
from fast_flights import FlightData, get_flights
|
|
|
|
flight = FlightData(
|
|
date="2026-03-23",
|
|
from_airport="BER",
|
|
to_airport="BRI"
|
|
)
|
|
|
|
result = get_flights(
|
|
flight,
|
|
passengers=Passengers(adults=1),
|
|
seat=1,
|
|
fetch_mode='fallback'
|
|
)
|
|
```
|
|
|
|
**New (v3.0rc1):**
|
|
```python
|
|
from fast_flights import FlightQuery, create_query, get_flights
|
|
|
|
flights = [FlightQuery(
|
|
date="2026-03-23",
|
|
from_airport="BER",
|
|
to_airport="BRI",
|
|
max_stops=0
|
|
)]
|
|
|
|
query = create_query(
|
|
flights=flights,
|
|
seat="economy", # String, not number
|
|
trip="one-way",
|
|
passengers=Passengers(adults=1) # Keyword argument
|
|
)
|
|
|
|
result = get_flights(query, integration=cookie_integration)
|
|
```
|
|
|
|
### ✅ Automatic Fallback
|
|
|
|
The tool automatically uses `searcher_v3.py` if v3.0rc1 is installed, otherwise falls back to the legacy searcher:
|
|
|
|
```python
|
|
try:
|
|
from searcher_v3 import search_multiple_routes
|
|
print("✓ Using fast-flights v3.0rc1 with SOCS cookie integration")
|
|
except ImportError:
|
|
from searcher import search_multiple_routes
|
|
print("⚠️ Using legacy searcher (v2.2)")
|
|
```
|
|
|
|
## File Structure
|
|
|
|
```
|
|
flight-comparator/
|
|
├── searcher_v3.py # NEW: v3 searcher with SOCS cookie
|
|
├── searcher.py # OLD: v2 searcher (kept for fallback)
|
|
├── main.py # UPDATED: Auto-detects v3 or v2
|
|
├── test_v3_with_cookies.py # NEW: v3 cookie integration test
|
|
├── tests/
|
|
│ └── test_comprehensive_v3.py # NEW: Full test suite
|
|
├── MIGRATION_V3.md # This file
|
|
└── FAST_FLIGHTS_TEST_REPORT.md # Research findings
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "fast-flights not found"
|
|
|
|
```bash
|
|
pip install --upgrade git+https://github.com/AWeirdDev/flights.git
|
|
```
|
|
|
|
### "Cannot import FlightQuery"
|
|
|
|
You have v2.2 installed. Uninstall and reinstall v3:
|
|
|
|
```bash
|
|
pip uninstall fast-flights
|
|
pip install git+https://github.com/AWeirdDev/flights.git
|
|
```
|
|
|
|
### "Still getting consent page"
|
|
|
|
The SOCS cookie may have expired (13-month lifetime). Get a fresh one:
|
|
|
|
1. Open Google Flights in your browser
|
|
2. Accept cookies
|
|
3. Check browser dev tools → Application → Cookies → `SOCS`
|
|
4. Copy the value
|
|
5. Update `SOCS_COOKIE` in `searcher_v3.py`
|
|
|
|
### "Protobuf version conflict"
|
|
|
|
v3.0rc1 requires protobuf >= 5.27.0, which may conflict with other packages:
|
|
|
|
```bash
|
|
pip install --upgrade protobuf
|
|
# OR
|
|
pip install protobuf==5.27.0 --force-reinstall
|
|
```
|
|
|
|
If conflicts persist, use a virtual environment:
|
|
|
|
```bash
|
|
python -m venv venv
|
|
source venv/bin/activate # or `venv\Scripts\activate` on Windows
|
|
pip install -r requirements.txt
|
|
pip install git+https://github.com/AWeirdDev/flights.git
|
|
```
|
|
|
|
## Testing
|
|
|
|
### Run Full Test Suite
|
|
|
|
```bash
|
|
cd tests
|
|
python test_comprehensive_v3.py
|
|
```
|
|
|
|
This tests:
|
|
- ✅ SOCS cookie integration
|
|
- ✅ Single route queries
|
|
- ✅ Multiple routes batch processing
|
|
- ✅ Different dates
|
|
- ✅ No direct flights handling
|
|
- ✅ Invalid airport codes
|
|
- ✅ Concurrent requests (10 routes)
|
|
- ✅ Price validation
|
|
|
|
### Quick Smoke Test
|
|
|
|
```bash
|
|
python test_v3_with_cookies.py
|
|
```
|
|
|
|
### Test Your Tool End-to-End
|
|
|
|
```bash
|
|
python main.py --to BDS --from BER,FRA,MUC --date 2026-06-15
|
|
```
|
|
|
|
## Performance
|
|
|
|
With v3.0rc1 + SOCS cookie:
|
|
|
|
| Metric | Performance |
|
|
|--------|-------------|
|
|
| Single query | ~3-5s |
|
|
| 10 concurrent routes | ~20-30s |
|
|
| Success rate | ~80-90% (some routes have no direct flights) |
|
|
| Consent page bypass | ✅ 100% |
|
|
|
|
## What's Next
|
|
|
|
1. **Monitor SOCS cookie validity** - May need refresh after 13 months
|
|
2. **Consider caching** - Save results to avoid repeated API calls
|
|
3. **Add retry logic** - For transient network errors
|
|
4. **Rate limiting awareness** - Google may still throttle excessive requests
|
|
|
|
## Credits
|
|
|
|
- Solution based on [GitHub Issue #46](https://github.com/AWeirdDev/flights/issues/46)
|
|
- SOCS cookie research from [Cookie Library](https://cookielibrary.org/cookie_consent/socs/)
|
|
- fast-flights by [@AWeirdDev](https://github.com/AWeirdDev/flights)
|
|
|
|
## Support
|
|
|
|
If you encounter issues:
|
|
|
|
1. Check [FAST_FLIGHTS_TEST_REPORT.md](./FAST_FLIGHTS_TEST_REPORT.md) for detailed findings
|
|
2. Review [GitHub Issues](https://github.com/AWeirdDev/flights/issues)
|
|
3. Ensure you're on v3.0rc1: `python -c "import fast_flights; print(dir(fast_flights))"`
|