Goal (Sprint outcome)

Make the pipeline repeatable, scheduled, and observable: automated refresh for Library + History, dbt jobs triggered reliably, fail-fast on tests, plus monitoring + clear docs for demo/portfolio.

User value

• Reliability: “I can rerun this anytime and get consistent outputs.”
• Speed of delivery: one-click / scheduled runs instead of manual steps.
• Professional polish: monitoring page + documented recovery steps + presentation-ready story.

Automation flow (Sprint 3)

n8n schedules → run Python extraction/load → trigger dbt Cloud jobs → run dbt tests (fail fast) → write run logs to BigQuery → update monitoring dashboard + send notifications.

Epic C1 — n8n Workflow Setup

Objective

Set up n8n as the orchestrator to run Library/History refresh workflows with secure credentials + notifications.

Tasks (checklist)

• [ ] C1.1 Install/configure n8n (cloud or self-hosted) + secure credentials (YouTube Music / Spotify / GCP).
• [ ] C1.2 Build a monthly workflow for Library refresh (Python → raw load → trigger dbt).
• [ ] C1.3 Build a bi-weekly workflow for History refresh (new Takeout / incremental updates → dbt).
• [ ] C1.4 Add success/failure notifications (email or messaging).

Definition of Done (C1)

• Library and History workflows run end-to-end on schedule
• Notifications received on success/failure