Indian Domestic Flight 'Buy vs. Wait' Advisory Platform

The Indian Domestic Flight 'Buy vs. Wait' Advisory Platform is an end-to-end, production-grade MLOps system designed to analyze domestic airline booking curves, predict fair market ticket prices, and provide deterministic advisory signals (BUY_NOW, WAIT, or FAIR_PRICE).

The live interactive application is deployed and accessible at flight-advisor.streamlit.app.

1. Architecture Overview

The system spans the full MLOps lifecycle from data ingestion to automated continuous training:

[ Developer Commit / PR ]
        │
        ▼
┌─────────────────────────────────────────────────────────────┐
│ STAGE 1: CONTINUOUS INTEGRATION (CI)                        │
│ 1. Data Quality Gate (Great Expectations - 55 assertions)   │
│ 2. Automated Tests (pytest - /health, schemas, latency)     │
│ 3. Model Training & Metric Gating (MLflow - R² > 0.95)      │
└──────────────────────────────┬──────────────────────────────┘
                               │ (CI Passed ✅)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│ STAGE 2: CONTINUOUS DEPLOYMENT (CD)                         │
│ 1. Multi-Arch Docker Build (--platform linux/amd64)         │
│ 2. Push Image to Amazon ECR with Commit SHA + :latest       │
│ 3. Rolling Zero-Downtime Deployment on AWS ECS Fargate      │
└──────────────────────────────┬──────────────────────────────┘
                               │ (Live in Production 🚀)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│ STAGE 3: CONTINUOUS MONITORING & RETRAINING (CT)            │
│ 1. Live inference payloads logged to S3/Postgres            │
│ 2. Evidently AI evaluates Drift (K-S test & PSI)            │
│ 3. If PSI > 0.20 ➔ Webhook triggers CI/CD retraining job    │
└─────────────────────────────────────────────────────────────┘

2. Key Architecture Decisions (ADRs)

ADR 1: Feast as the Unified Feature Store

• Problem: Computing historical route metrics (volatility, min/max/average price) inside ad-hoc training scripts introduces train-serve skew if the inference API calculates them differently.
• Decision: Implemented Feast with a unified route_features entity view.
• Result: The training pipeline reads point-in-time correct historical features from the offline Parquet store, while the FastAPI inference service queries the online Redis store with sub-10ms latency.

ADR 2: Decoupled Model Loading (models/model.joblib)

• Problem: In MLflow 2.14+, fetching models strictly via remote tracking URIs inside Docker containers requires database connectivity and introduces cold-boot latency.
• Decision: train.py registers the model in the MLflow Model Registry for governance, while exporting a standalone models/model.joblib artifact for the container.
• Result: The production container boots in under 1 second without runtime dependencies on the tracking database.

ADR 3: AWS ECS Fargate over AWS Lambda

• Problem: Choosing between serverless containers (ECS Fargate) and serverless functions (AWS Lambda).
• Decision: Deployed the API microservice to AWS ECS Fargate.
• Result: Eliminated 2–4s container cold starts, maintained persistent connection pooling to Redis, and achieved consistent sub-25ms latency.

ADR 4: Multi-Arch --platform linux/amd64 Build Pipeline

• Problem: Building containers on Apple Silicon Macs creates linux/arm64 binaries, resulting in CannotPullContainerError on standard AMD64 Fargate clusters.
• Decision: Enforced docker build --platform linux/amd64 in the build scripts and CI/CD pipelines.

3. Dataset, Training & Metric Tracking

• Dataset: 300,153 Indian domestic records covering top metro hubs (Delhi, Mumbai, Bengaluru, Kolkata, Hyderabad, Chennai) versioned with DVC and AWS S3.
• Quality Gate: 55 automated assertions in Great Expectations covering schema integrity, zero-null constraints, and valid fare bounds (₹1,000 to ₹150,000).
• Model: LightGBM Regressor with hyperparameters tuned for early-stopping validation.
• MLflow Tracking Results:
  • $R^2$ Score: 0.9827 (98.3% variance explained)
  • Mean Absolute Percentage Error (MAPE): 12.89%
  • Mean Absolute Error (MAE): ₹1,646.80
  • Root Mean Squared Error (RMSE): ₹2,990.21

4. Algorithmic Advisory Engine

The inference layer compares live booking prices against the model-predicted fair value and route volatility:

1. Urgency Trigger (days_left <= 7): Recommends BUY_NOW due to steep departure-window surge curves.
2. Deal Trigger (listed_price <= predicted_price * 0.92): Recommends BUY_NOW when fares are discounted >8% below model fair value.
3. Inflation Trigger (listed_price >= predicted_price * 1.10 and days_left > 14): Recommends WAIT due to high probability of fare stabilization.
4. Fair Zone: Recommends FAIR_PRICE.

5. Drift Monitoring & Continuous Training Loop

To handle dynamic airfare inflation and seasonal festival surges:

• Evidently AI evaluates incoming production payloads against the baseline training distribution using Kolmogorov-Smirnov tests and the Population Stability Index (PSI).
• When PSI exceeds 0.20, a webhook triggers a repository_dispatch event on GitHub Actions to automatically pull fresh DVC data, retrain the LightGBM model, validate metrics against production baselines, and execute a zero-downtime rolling update on AWS ECS.