If you use BigQuery Fresh Daily exports, you might see Data Not Available for
some traffic_source fields. This guide helps you automatically backfill most
of the missing traffic source data in existing exports as soon as it's
available, typically by 5 AM each day.
Here are the steps to automate the backfill:
- Listen for the daily completeness signal from BigQuery.
- Identify the events with missing traffic source data in your BigQuery export.
- Query the complete data for those events from Google Ads.
- Join the complete event data with your BigQuery export.
Create a Pub/Sub topic
- Open Pub/Sub in the left navigation menu of the Google Cloud console. If you don't see Pub/Sub, search for it in the
Google Cloud console search bar:
- Click + CREATE TOPIC in the Topics tab:
- Enter a name in the Topic ID field.
- Select Add a default subscription, leave the other options blank:
- Click Create.
Create a Log Router sink
- Open Log router in the Google Cloud console:
- Click Create sink:
- Enter a name and description for your sink, then click Next.
- Choose Cloud Pub/Sub topic as the sink service.
- Choose the topic you created, then click Next.
Enter the following code in Build inclusion filter:
logName="projects/YOUR-PROJECT-ID/logs/analyticsdata.googleapis.com%2Ffresh_bigquery_export_status"Replace YOUR-PROJECT-ID with the ID for your Google Cloud console project.
Click Next, then click Create sink. You don't need to filter out any logs.
Verify the sink is now listed under Log Router Sinks.
Join the missing data
Use a Cloud Run function to automatically execute the code to backfill traffic source data when Pub/Sub detects the completeness signal:
- Open Cloud Run functions:
- Click CREATE FUNCTION:
- Choose Cloud Run function for the Environment.
- Enter a name for your function.
- Choose Cloud Pub/Sub as the Trigger type, and the topic you created as the Cloud Pub/Sub topic.
- Click Next, then enter your code to join the Google Ads attribution data with your BigQuery export in the box.