GCP Serverless Data Pipeline Case Study

Infrastructure fully provisioned and managed with Terraform (GCS, Cloud Functions, Cloud Run, BigQuery, IAM)
Files are uploaded to a Cloud Storage bucket
A Cloud Function is triggered on object finalization events
Validated data is forwarded to a Cloud Run service
Processed records are loaded into BigQuery

Serverless data pipeline architecture diagram