ウェブサイトが進化するにつれ、新しいページの追加や古いページの削除が行われるため、 リンクされていない可能性が高まります。Google 広告の広告キャンペーンと ウェブサイトへのアクセスは多くの広告主様にとって難しい場合があります。ライブ広告は 存在しないページを参照していると、ユーザー エクスペリエンスが低下したり精度が下がったりします Google 広告の統計情報
リンク チェッカーは、すべての広告を反復処理して、 キーワード、サイトリンクなど）を組み込んで、URL によって「ページが 見つかりました」その他の種類のエラー レスポンスに対応し、エラー レスポンスが その分析結果をスプレッドシートに保存します。
アカウントによっては、URL の数が多いため、すべての URL を 1 回の実行時間が失敗する 制限または 割り当てこのスクリプトは、 複数の実行にわたって進捗状況を追跡し、 チェックされていない URL のみをチェックします。スクリプトでは、 「分析」すべての URL を 1 回のフルパスで参照できます。たとえば、 完了するまで複数回実行します
構成
スクリプトのメイン オプションはスプレッドシートで設定できます。
- 範囲
- スクリプトが広告、キーワード、サイトリンクをチェックするかどうかを決定します。また、 広告が一時停止されていてもチェックされます。ほとんどのユーザーには、 すべての URL がチェックされるようにしますが、通常は必須ではありません。 一時停止されているエンティティを確認できます。
- 有効なレスポンス コード
- 有効とみなされる HTTP レスポンス コードのリスト。ほとんどのユーザーは、
200のみを有効なものとみなす レスポンス コード。 - スクリプトが実行されるたびにメールを受信する
- このオプションを有効にすると、スクリプトから URL の概要がメールで通知されます。 チェックされますこれにより、特定の脅威に関する早期のアラートが すべての URL の検証を待つことなく、URL エラーが発生する （複数回の実行が必要になる場合があります）。
- すべての分析終了後のメール
- このオプションを有効にすると、終了後にスクリプトから統合サマリーがメールで送信されるようになります。 すべての URL のチェックが完了
- エラーが見つからない場合でもメールを受信する
- このオプションを有効にすると、前述の 2 つの通知の内容に基づいて、スクリプトから オプション）が表示されます。ほとんどのユーザーはメールでの エラーがない場合でもメールを受信している場合にのみ スケジュールどおりにスクリプトが実行されるようにするための便利な手段です。
- エラーのない URL をスプレッドシートに保存する
- このオプションを有効にすると、スクリプトはチェックするすべての URL を エラーのあるものだけでなくほとんどのユーザーは、 URL 全体を参照したい場合もありますが、 確認されます。
- 分析間の日数
- このオプションを使用すると、スクリプトが新しい分析を開始する頻度をコントロールできます。 URL リスト全体を削除します。なお、このオプションでは、ラベルの最小数を 日数のことです。実際の数は、分析に より長い時間がかかります（URL が多数ある場合など）。詳しくは、 スケジュール設定をご覧ください。
高度なリンク検証
ユースケースによっては、URL が正規 URL であることを知らせるだけでは 返すことができます。たとえば商品ページが 「product discontinued」というテキストが含まれています。そのような場合は 表示されます。
このスクリプトには、次の 2 つの検証オプションがあります。
- 失敗文字列
[単純な失敗文字列マッチングを使用する] を
Yesに設定して、オカレンスを検索します。 エラー文字列で定義されている文字列のリスト。箇所が ウェブページが壊れていると マークします- カスタム検証
柔軟性をさらに高めるには、カスタムの JavaScript 検証関数を使用します。 レスポンスを記録します。たとえば、スペースのタイトルが ウェブページに次のようなブランド名が含まれています。
- [カスタム検証関数を使用する] を
Yesに設定します。 カスタム検証ロジックを スクリプト:
function isValidResponse(url, response, options, entityDetails) { // Your custom logic here, returning true for valid links, false otherwise. // ... }
基本的な実装はすでに行われており、アクセス方法の例も示されています。 返されます。
- [カスタム検証関数を使用する] を
スケジュール
スクリプトを実行するたびに、スクリプトを再開すべきかどうかが自動的に検出され、 または、最後の分析が終了し、終了したかどうかが 新しい分析を開始するタイミング（分析間の日数に基づく） オプション）。そのため、新規リリースを行う頻度に関係なく、 スクリプトが毎時間実行されるようスケジュールを設定し、 できるだけ速く URL を精査する必要があります。
たとえば、Days while analyses（分析の間の日数）を 1 に設定すると、スクリプトが起動されます。
新しい分析を 1 日に 1 回以下に抑えます。スクリプトの実行がスケジュールされている場合
毎時間: すべての URL のチェックを 1 日以内に完了します。
その後の実行は直ちに終了し、翌日の
新しい分析を開始できます
仕組み
処理の進捗状況をトラッキングするために、スクリプトによってラベルが作成され、広告に適用されます。 サイトリンク、キーワード、サイトリンクが 自動的に配信されますすると、スクリプトは チェックされます。分析が完了すると（すべての URL が チェックされている場合、スクリプトは新しい分析のためにラベルを消去します。
このスクリプトでは UrlFetchApp を使用します。
を使って、最終ページ URL とモバイルの最終ページ URL をチェックできます。
セットアップ
下のボタンをクリックすると、スプレッドシート ベースのスクリプトを Google Google 広告アカウント。
下のボタンをクリックすると、テンプレート スプレッドシートのコピーを作成できます。
スクリプトの
spreadsheet_urlとrecipient_emailsを更新します。スクリプトが毎時間実行されるようにスケジュール設定します。
ソースコード
// Copyright 2016, Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/**
* @name Link Checker
*
* @overview The Link Checker script iterates through the ads, keywords, and
* sitelinks in your account and makes sure their URLs do not produce "Page
* not found" or other types of error responses. See
* https://developers.google.com/google-ads/scripts/docs/solutions/link-checker
* for more details.
*
* @author Google Ads Scripts Team [adwords-scripts@googlegroups.com]
*
* @version 3.1
*
* @changelog
* - version 4.0
* - Refactored for readability and efficiency - particularly for sitelinks.
* - version 3.1
* - Split into info, config, and code.
* - version 3.0
* - Updated to use new Google Ads scripts features.
* - version 2.2
* - Added support for failure strings and custom validation functions.
* - version 2.1
* - Added expansion of conditional ValueTrack parameters (e.g. ifmobile).
* - Added expanded text ad and other ad format support.
* - version 2.0.3
* - Added validation for external spreadsheet setup.
* - version 2.0.2
* - Allow the custom tracking label to include spaces.
* - version 2.0.1
* - Catch and output all UrlFetchApp exceptions.
* - version 2.0
* - Completely revised the script to work on larger accounts.
* - Check URLs in campaign and ad group sitelinks.
* - version 1.2
* - Released initial version.
*/
/**
* Configuration to be used for the Link Checker.
*/
CONFIG = {
// URL of the spreadsheet template.
// This should be a copy of https://docs.google.com/spreadsheets/d/1iO1iEGwlbe510qo3Li-j4KgyCeVSmodxU6J7M756ppk/copy.
'spreadsheet_url': 'YOUR_SPREADSHEET_URL',
// Array of addresses to be alerted via email if issues are found.
'recipient_emails': [
'YOUR_EMAIL_HERE'
],
// Label to use when a link has been checked. Label will be created if it doesn't exist.
'label': 'LinkChecker_Done',
// Number of seconds to sleep after each URL request. If your URLs are
// all on one or a few domains, use this throttle to reduce the load that the
// script imposes on your web server(s).
'throttle': 0,
// Number of seconds before timeout that the script should stop checking URLs
// to make sure it has time to output its findings.
'timeout_buffer': 120,
'advanced_options': {
/**
* Parameters controlling the script's behavior after hitting a UrlFetchApp
* QPS quota limit.
*/
'quota_config': {
'INIT_SLEEP_TIME': 250,
'BACKOFF_FACTOR': 2,
'MAX_TRIES': 5
}
}
};
/**
* Performs custom validation on a URL, with access to details such as the URL,
* the response from the server, configuration options and entity Details.
*
* To use, the "Use Custom Validation" option in the configuration spreadsheet
* must be set to "Yes", and your custom validation code implemented within the
* below function.
*
* See the documentation for this solution for further details.
*
* @param {string} url The URL being checked.
* @param {!Object} response The response object for the request.
* @param {!Object} options Configuration options.
* @param {!Object} entityDetails Details of the associated Ad / Keywords etc.
* @return {boolean} Return true if the URL and response are deemed valid.
*/
function isValidResponse(url, response, options, entityDetails) {
/*
Some examples of data that can be used in determining the validity of this
URL. This is not exhaustive and there are further properties available.
*/
// The HTTP status code, e.g. 200, 404
// const responseCode = response.getResponseCode();
// The HTTP response body, e.g. HTML for web pages:
// const responseText = response.getContentText();
// The failure strings from the configuration spreadsheet, as an array:
// const failureStrings = options.failureStrings;
// The type of the entity associated with the URL, e.g. Ad, Keyword, Sitelink.
// const entityType = entityDetails.entityType;
// The campaign name
// const campaignName = entityDetails.campaign;
// The ad group name, if applicable
// const adGroupName = entityDetails.adGroup;
// The ad text, if applicable
// const adText = entityDetails.ad;
// The keyword text, if applicable
// const keywordText = entityDetails.keyword;
// The sitelink link text, if applicable
// const sitelinkText = entityDetails.sitelink;
/*
Remove comments and insert custom logic to determine whether this URL and
response are valid, using the data obtained above.
If valid, return true. If invalid, return false.
*/
// Placeholder implementation treats all URLs as valid
return true;
}
const QUOTA_CONFIG = CONFIG.advanced_options.quota_config;
/**
* Exceptions that prevent the script from finishing checking all URLs in an
* account but allow it to resume next time.
*/
const EXCEPTIONS = {
QPS: 'Reached UrlFetchApp QPS limit',
LIMIT: 'Reached UrlFetchApp daily quota',
TIMEOUT: 'Approached script execution time limit'
};
/**
* Named ranges in the spreadsheet.
*/
const NAMES = {
CHECK_AD_URLS: 'checkAdUrls',
CHECK_KEYWORD_URLS: 'checkKeywordUrls',
CHECK_SITELINK_URLS: 'checkSitelinkUrls',
CHECK_PAUSED_ADS: 'checkPausedAds',
CHECK_PAUSED_KEYWORDS: 'checkPausedKeywords',
CHECK_PAUSED_SITELINKS: 'checkPausedSitelinks',
VALID_CODES: 'validCodes',
EMAIL_EACH_RUN: 'emailEachRun',
EMAIL_NON_ERRORS: 'emailNonErrors',
EMAIL_ON_COMPLETION: 'emailOnCompletion',
FAILURE_STRINGS: 'failureStrings',
SAVE_ALL_URLS: 'saveAllUrls',
FREQUENCY: 'frequency',
DATE_STARTED: 'dateStarted',
DATE_COMPLETED: 'dateCompleted',
DATE_EMAILED: 'dateEmailed',
NUM_ERRORS: 'numErrors',
RESULT_HEADERS: 'resultHeaders',
ARCHIVE_HEADERS: 'archiveHeaders',
USE_SIMPLE_FAILURE_STRINGS: 'useSimpleFailureStrings',
USE_CUSTOM_VALIDATION: 'useCustomValidation'
};
const MILLISECONDS_PER_SECOND = 1000;
const SPREADSHEET_URL = CONFIG.spreadsheet_url;
const RECIPIENT_EMAILS = CONFIG.recipient_emails;
const LABEL = CONFIG.label;
const THROTTLE = CONFIG.throttle;
const TIMEOUT_BUFFER = CONFIG.timeout_buffer;
/**
* The Link Checker script iterates through the ads, keywords, and sitelinks
* in your account and makes sure their URLs do not produce "Page not found"
* or other types of error responses.
*/
function main() {
const spreadsheet = validateAndGetSpreadsheet(SPREADSHEET_URL);
validateEmailAddresses(RECIPIENT_EMAILS);
spreadsheet.setSpreadsheetTimeZone(AdsApp.currentAccount().getTimeZone());
const options = loadOptions(spreadsheet);
const status = loadStatus(spreadsheet);
if (!status.dateStarted) {
// This is the very first execution of the script.
console.log(`First time analyzing the account.`);
startNewAnalysis(spreadsheet);
} else if (status.dateStarted > status.dateCompleted) {
console.log('Resuming work from a previous execution.');
} else if (
dayDifference(status.dateStarted, new Date()) < options.frequency) {
console.log(
`Waiting until ${options.frequency} ` +
`days have elapsed since the start of the last analysis.`);
return;
} else {
// Enough time has passed since the last analysis to start a new one.
console.log(`Restarting analysis`);
removeLabel(CONFIG.label);
startNewAnalysis(spreadsheet);
}
const results = analyzeAccount(options);
console.log(`Completed analysis, outputting results.`);
outputResults(results, options);
}
/**
* Checks as many new URLs as possible that have not previously been checked,
* subject to quota and time limits.
*
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {!Object} An object with fields for the URLs checked and an
* indication if the analysis was completed (no remaining URLs to check).
*/
function analyzeAccount(options) {
// Ensure the label exists before attempting to retrieve already checked URLs.
ensureLabel(CONFIG.label);
const labelResourceName = loadLabelResourceName(CONFIG.label);
const urlChecks = [];
const checkedUrls = getAlreadyCheckedUrls(labelResourceName, options);
console.log(`Already checked ${checkedUrls.size} urls`);
let didComplete = false;
try {
didComplete = checkUrls(labelResourceName, checkedUrls, urlChecks, options);
} catch (e) {
if (e === EXCEPTIONS.QPS || e === EXCEPTIONS.LIMIT ||
e === EXCEPTIONS.TIMEOUT) {
console.warn(`Stopped checking URLs early because: ${e}`);
console.log('Checked URLs will still be output.');
} else {
throw e;
}
}
return {urlChecks, didComplete};
}
/**
* Given a label name, load the resource name for that label. There will only be
* one label with the specified name which is active (there may be many with
* this name which are removed). This label is used to identify entities which
* have already been processed and to annotate entities which have just been
* processed.
*
* @param {string} labelName The name of the label used to annotate entities
* which have been checked.
* @return {string} The resource name of the relevant label.
*/
function loadLabelResourceName(labelName) {
const query =
`SELECT label.resource_name, label.name, label.status from label where label.name='${
labelName}' and label.status = 'ENABLED'`;
const rows = AdsApp.search(query);
if (rows.totalNumEntities() > 1) {
throw new Error(`Found ${rows.length} labels with name '${
labelName}' when there should only be 1.`);
}
if (rows.totalNumEntities() === 0) {
throw new Error(`Could not find label with name '${labelName}'.`);
}
return rows.next().label.resourceName;
}
/**
* Outputs the results to a spreadsheet and sends emails if appropriate.
*
* @param {!Object} results An object with fields for the URLs checked and an
* indication if the analysis was completed (no remaining URLs to check).
* @param {!Object} options Dictionary of options.
*/
function outputResults(results, options) {
const spreadsheet = SpreadsheetApp.openByUrl(SPREADSHEET_URL);
const numErrors = countErrors(results.urlChecks, options);
console.log(`Found ${numErrors} invalid urls this execution.`);
saveUrlsToSpreadsheet(spreadsheet, results.urlChecks, options);
// Reload the status to get the total number of errors for the entire
// analysis, which is calculated by the spreadsheet.
const status = loadStatus(spreadsheet);
if (results.didComplete) {
spreadsheet.getRangeByName(NAMES.DATE_COMPLETED).setValue(new Date());
console.log(
`Found ${status.numErrors} invalid urls across the entire analysis.`);
}
if (RECIPIENT_EMAILS) {
if (!results.didComplete && options.emailEachRun &&
(options.emailNonErrors || numErrors > 0)) {
sendIntermediateEmail(spreadsheet, numErrors);
}
if (results.didComplete &&
(options.emailEachRun || options.emailOnCompletion) &&
(options.emailNonErrors || status.numErrors > 0)) {
sendFinalEmail(spreadsheet, status.numErrors);
}
}
}
/**
* Loads data from a spreadsheet based on named ranges. Strings 'Yes' and 'No'
* are converted to booleans. One-dimensional ranges are converted to arrays
* with blank cells omitted. Assumes each named range exists.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {!Array.<string>} names A list of named ranges that should be loaded.
* @return {!Object} A dictionary with the names as keys and the values
* as the cell values from the spreadsheet.
*/
function loadDatabyName(spreadsheet, names) {
const data = {};
for (const name of names) {
const range = spreadsheet.getRangeByName(name);
if (range.getNumRows() > 1 && range.getNumColumns() > 1) {
// Name refers to a 2d range, so load it as a 2d array.
data[name] = range.getValues();
} else if (range.getNumRows() === 1 && range.getNumColumns() === 1) {
// Name refers to a single cell, so load it as a value and replace
// Yes/No with boolean true/false.
data[name] = range.getValue();
data[name] = data[name] === 'Yes' ? true : data[name];
data[name] = data[name] === 'No' ? false : data[name];
} else {
// Name refers to a 1d range, so load it as an array (regardless of
// whether the 1d range is oriented horizontally or vertically).
const isByRow = range.getNumRows() > 1;
const limit = isByRow ? range.getNumRows() : range.getNumColumns();
const cellValues = range.getValues();
data[name] = [];
for (let j = 0; j < limit; j++) {
const cellValue = isByRow ? cellValues[j][0] : cellValues[0][j];
if (cellValue) {
data[name].push(cellValue);
}
}
}
}
return data;
}
/**
* Loads options from the spreadsheet.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @return {!Object} A dictionary of options.
*/
function loadOptions(spreadsheet) {
return loadDatabyName(spreadsheet,
[NAMES.CHECK_AD_URLS, NAMES.CHECK_KEYWORD_URLS,
NAMES.CHECK_SITELINK_URLS, NAMES.CHECK_PAUSED_ADS,
NAMES.CHECK_PAUSED_KEYWORDS, NAMES.CHECK_PAUSED_SITELINKS,
NAMES.VALID_CODES, NAMES.EMAIL_EACH_RUN,
NAMES.EMAIL_NON_ERRORS, NAMES.EMAIL_ON_COMPLETION,
NAMES.SAVE_ALL_URLS, NAMES.FREQUENCY,
NAMES.FAILURE_STRINGS, NAMES.USE_SIMPLE_FAILURE_STRINGS,
NAMES.USE_CUSTOM_VALIDATION]);
}
/**
* Loads state information from the spreadsheet.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @return {!Object} A dictionary of status information.
*/
function loadStatus(spreadsheet) {
return loadDatabyName(spreadsheet,
[NAMES.DATE_STARTED, NAMES.DATE_COMPLETED,
NAMES.DATE_EMAILED, NAMES.NUM_ERRORS]);
}
/**
* Saves the start date to the spreadsheet and archives results of the last
* analysis to a separate sheet.
*
* @param {!Object} spreadsheet The spreadsheet object.
*/
function startNewAnalysis(spreadsheet) {
console.log('Starting a new analysis.');
spreadsheet.getRangeByName(NAMES.DATE_STARTED).setValue(new Date());
// Helper method to get the output area on the results or archive sheets.
function getOutputRange(rangeName) {
const headers = spreadsheet.getRangeByName(rangeName);
return headers.offset(1, 0, headers.getSheet().getDataRange().getLastRow());
}
getOutputRange(NAMES.ARCHIVE_HEADERS).clearContent();
const results = getOutputRange(NAMES.RESULT_HEADERS);
results.copyTo(getOutputRange(NAMES.ARCHIVE_HEADERS));
getOutputRange(NAMES.RESULT_HEADERS).clearContent();
}
/**
* Counts the number of errors in the results.
*
* @param {!Array.<!Object>} urlChecks A list of URL check results.
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {number} The number of errors in the results.
*/
function countErrors(urlChecks, options) {
let numErrors = 0;
for (const urlCheck of urlChecks) {
if (options.validCodes.indexOf(urlCheck.responseCode) === -1) {
numErrors++;
}
}
return numErrors;
}
/**
* Saves URLs for a particular account to the spreadsheet starting at the first
* unused row.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {!Array.<!Object>} urlChecks A list of URL check results.
* @param {!Map<string, !Object>} options options Dictionary of options.
*/
function saveUrlsToSpreadsheet(spreadsheet, urlChecks, options) {
// Build each row of output values in the order of the columns.
const outputValues = [];
for (const urlCheck of urlChecks) {
if (options.saveAllUrls ||
options.validCodes.indexOf(urlCheck.responseCode) === -1) {
outputValues.push([
urlCheck.customerId,
new Date(urlCheck.timestamp),
urlCheck.url,
urlCheck.responseCode,
urlCheck.entityType,
urlCheck.campaign,
urlCheck.adGroup,
urlCheck.ad,
urlCheck.keyword,
urlCheck.sitelink
]);
}
}
if (outputValues.length > 0) {
// Find the first open row on the Results tab below the headers and create a
// range large enough to hold all of the output, one per row.
const headers = spreadsheet.getRangeByName(NAMES.RESULT_HEADERS);
const lastRow = headers.getSheet().getDataRange().getLastRow();
const outputRange = headers.offset(lastRow - headers.getRow() + 1,
0, outputValues.length);
outputRange.setValues(outputValues);
}
for (const email of RECIPIENT_EMAILS) {
spreadsheet.addEditor(email);
}
}
/**
* Sends an email to a list of email addresses with a link to the spreadsheet
* and the results of this execution of the script.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {boolean} numErrors The number of errors found in this execution.
*/
function sendIntermediateEmail(spreadsheet, numErrors) {
spreadsheet.getRangeByName(NAMES.DATE_EMAILED).setValue(new Date());
MailApp.sendEmail(RECIPIENT_EMAILS.join(','),
'Link Checker Results',
`The Link Checker script found ${numErrors} URLs with errors in ` +
`an execution that just finished. See ${spreadsheet.getUrl()} for details.`
);
}
/**
* Sends an email to a list of email addresses with a link to the spreadsheet
* and the results across the entire account.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {boolean} numErrors The number of errors found in the entire account.
*/
function sendFinalEmail(spreadsheet, numErrors) {
spreadsheet.getRangeByName(NAMES.DATE_EMAILED).setValue(new Date());
MailApp.sendEmail(RECIPIENT_EMAILS.join(','),
'Link Checker Results',
`The Link Checker script found ${numErrors} URLs with errors ` +
`across its entire analysis. See ${spreadsheet.getUrl()} for details.`);
}
/**
* Retrieves all final URLs and mobile final URLs in the account across ads,
* keywords, and sitelinks that were checked in a previous run, as indicated by
* them having been labeled.
*
* @param {string} labelResourceName The resource name of the label applied to
* entities that have already been checked.
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {!Object} A map of previously checked URLs with the URL as the key.
*/
function getAlreadyCheckedUrls(labelResourceName, options) {
const urls = new Set();
if (options.checkAdUrls) {
const adUrls = getCheckedAdUrls(options.checkPausedAds, labelResourceName);
for (const url of adUrls.values()) {
urls.add(url);
}
}
if (options.checkKeywordUrls) {
const keywordUrls =
getCheckedKeywordUrls(options.checkPausedKeywords, labelResourceName);
for (const url of keywordUrls.values()) {
urls.add(url);
}
}
if (options.checkSitelinkUrls) {
const sitelinkUrls =
getCheckedSitelinkUrls(options.checkPausedSitelinks, labelResourceName);
for (const url of sitelinkUrls.values()) {
urls.add(url);
}
}
return urls;
}
/**
* Returns a set of ad URLs that have already been processed. This ensures we
* only process unique URLs.
*
* @param {boolean} checkPaused Whether to include ads that are paused.
* @param {string} labelResourceName The label to identify processed/unprocessed
* ads.
* @return {!Set<string>} A set of all URLs that have already been checked.
*/
function getCheckedAdUrls(checkPaused, labelResourceName) {
const urls = new Set();
let conditions = [];
if (checkPaused) {
conditions.push(`ad_group_ad.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`ad_group_ad.status = 'ENABLED'`);
conditions.push(`campaign.status = 'ENABLED'`);
conditions.push(`ad_group.status = 'ENABLED'`);
}
conditions.push(`ad_group_ad.labels CONTAINS ALL ('${labelResourceName}')`);
const query =
`SELECT ad_group_ad.ad.final_urls, ad_group_ad.ad.final_mobile_urls from ad_group_ad where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
for (const row of rows) {
if (row.adGroupAd.ad.finalUrls) {
urls.add(row.adGroupAd.ad.finalUrls[0]);
}
if (row.adGroupAd.ad.finalMobileUrls) {
urls.add(row.adGroupAd.ad.finalMobileUrls[0]);
}
}
return urls;
}
/**
* Returns a set of keyword URLs that have already been processed. This ensures
* we only process unique URLs.
*
* @param {boolean} checkPaused Whether to include keywords that are paused.
* @param {string} labelResourceName The label to identify processed/unprocessed
* keywords.
* @return {!Set<string>} A set of all URLs that have already been checked.
*/
function getCheckedKeywordUrls(checkPaused, labelResourceName) {
const urls = new Set();
let conditions = [];
if (checkPaused) {
conditions.push(`ad_group_criterion.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`ad_group_criterion.status = 'ENABLED'`);
conditions.push(`campaign.status = 'ENABLED'`);
conditions.push(`ad_group.status = 'ENABLED'`);
}
conditions.push(
`ad_group_criterion.labels CONTAINS ALL ('${labelResourceName}')`);
conditions.push(`ad_group_criterion.type = 'KEYWORD'`);
conditions.push(`ad_group_criterion.negative = false`);
const query =
`SELECT ad_group_criterion.final_urls, ad_group_criterion.final_mobile_urls from ad_group_criterion where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
for (const row of rows) {
if (row.adGroupCriterion.finalUrls) {
urls.add(row.adGroupCriterion.finalUrls[0]);
}
if (row.adGroupCriterion.finalMobileUrls) {
urls.add(row.adGroupCriterion.finalMobileUrls[0]);
}
}
return urls;
}
/**
* Retrieves all final URLs and mobile final URLs for campaign and ad group
* sitelinks that have already been checked.
*
* @param {boolean} checkPaused Whether to include paused sitelinks or not.
* @param {string} labelResourceName The resource name of the label applied to
* campaigns and ad groups whose sitelinks have already been checked.
* @return {!Set.<string>} A set of URLs that have already been checked.
*/
function getCheckedSitelinkUrls(checkPaused, labelResourceName) {
const urls = new Set();
const campaignResourceNames =
getCheckedCampaignResourceNames(checkPaused, labelResourceName);
while (campaignResourceNames.length > 0) {
// Load campaigns 10k at a time.
let someCampaignResourceNames = campaignResourceNames.splice(0, 10000);
let query = `SELECT campaign_asset.campaign, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from campaign_asset where campaign_asset.campaign IN (${
someCampaignResourceNames.join(
', ')}) and campaign_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
let rows = AdsApp.search(query);
for (const row of rows) {
if (row.asset.finalUrls) {
urls.add(row.asset.finalUrls[0]);
}
if (row.asset.finalMobileUrls) {
urls.add(row.asset.finalMobileUrls[0]);
}
}
}
const adGroupResourceNames =
getCheckedAdGroupResourceNames(checkPaused, labelResourceName);
while (adGroupResourceNames.length > 0) {
// Load ad groups 10k at a time.
let someAdGroupResourceNames = adGroupResourceNames.splice(0, 10000);
query = `SELECT ad_group_asset.ad_group, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from ad_group_asset where ad_group_asset.ad_group IN (${
someAdGroupResourceNames.join(
', ')}) and ad_group_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
rows = AdsApp.search(query);
for (const row of rows) {
if (row.asset.finalUrls) {
urls.add(row.asset.finalUrls[0]);
}
if (row.asset.finalMobileUrls) {
urls.add(row.asset.finalMobileUrls[0]);
}
}
}
return urls;
}
/**
* A helper function that returns a list of resource names for campaigns that
* have already been processed. See getCampaignResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused campaigns.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of campaign resource names.
*/
function getCheckedCampaignResourceNames(checkPaused, labelResourceName) {
return getCampaignResourceNames(true, checkPaused, labelResourceName);
}
/**
* A helper function that returns a list of resource names for campaigns that
* have not been processed yet. See getCampaignResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused campaigns.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of campaign resource names.
*/
function getUncheckedCampaignResourceNames(checkPaused, labelResourceName) {
return getCampaignResourceNames(false, checkPaused, labelResourceName);
}
/**
* Returns a list of campaign resource names that match the specified criteria.
* In order to efficiently fetch campaign sitelinks, we need to first get a list
* of relevant campaigns. This is because we cannot filter campaign sitelinks by
* campaign attributes (only campaign resource names).
*
* NOTE: The resource names are returned quoted. This simplifies usage as
* resource names need to be quoted for use in queries.
*
* @param {boolean} alreadyChecked True if we want campaigns that have been
* processed already.
* @param {boolean} checkPaused True if we want to include paused campaigns.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of campaign resource names.
*/
function getCampaignResourceNames(
alreadyChecked, checkPaused, labelResourceName) {
const conditions = [];
// If we want campaigns we already processed, we want to see if the label is
// applied.
if (alreadyChecked) {
conditions.push(`campaign.labels CONTAINS ALL ('${labelResourceName}')`);
} else {
// If we want campaigns we haven't processed, we want to see if the label
// was not applied.
conditions.push(`campaign.labels CONTAINS NONE ('${labelResourceName}')`);
}
if (checkPaused) {
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`campaign.status = 'ENABLED'`);
}
const query = `SELECT campaign.resource_name from campaign where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
const resourceNames = [];
for (const row of rows) {
resourceNames.push(`"${row.campaign.resourceName}"`);
}
return resourceNames;
}
/**
* A helper function that returns a list of resource names for ad groups that
* have already been processed. See getAdGroupResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused ad groups.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of ad group resource names.
*/
function getCheckedAdGroupResourceNames(checkPaused, labelResourceName) {
return getAdGroupResourceNames(true, checkPaused, labelResourceName);
}
/**
* A helper function that returns a list of resource names for ad groups that
* have not been processed yet. See getAdGroupResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused ad groups.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of ad group resource names.
*/
function getUncheckedAdGroupResourceNames(checkPaused, labelResourceName) {
return getAdGroupResourceNames(false, checkPaused, labelResourceName);
}
/**
* Returns a list of ad group resource names that match the specified criteria.
* In order to efficiently fetch ad group sitelinks, we need to first get a list
* of relevant ad groups. This is because we cannot filter ad group sitelinks by
* ad group attributes (only ad group resource names).
*
* NOTE: The resource names are returned quoted. This simplifies usage as
* resource names need to be quoted for use in queries.
*
* @param {boolean} alreadyChecked True if we want ad groups that have been
* processed already.
* @param {boolean} checkPaused True if we want to include paused ad groups.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of ad group resource names.
*/
function getAdGroupResourceNames(
alreadyChecked, checkPaused, labelResourceName) {
const conditions = [];
// If we want ad groups we already processed, we want to see if the label is
// applied.
if (alreadyChecked) {
conditions.push(`ad_group.labels CONTAINS ALL ('${labelResourceName}')`);
} else {
// If we want ad groups we haven't processed, we want to see if the label
// was not applied.
conditions.push(`ad_group.labels CONTAINS NONE ('${labelResourceName}')`);
}
if (checkPaused) {
conditions.push(`ad_group.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`ad_group.status = 'ENABLED'`);
conditions.push(`campaign.status = 'ENABLED'`);
}
const query = `SELECT ad_group.resource_name from ad_group where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
const resourceNames = [];
for (const row of rows) {
resourceNames.push(`"${row.adGroup.resourceName}"`);
}
return resourceNames;
}
/**
* Retrieves all final URLs and mobile final URLs in the account across ads,
* keywords, and sitelinks, and checks their response code. Does not check
* previously checked URLs.
*
* @param {string} labelResourceName The resource name of the label to check.
* @param {Set.<string>!} checkedUrls A set of previously checked URLs.
* @param {!Array.<!Object>} urlChecks An array into which the results of each
* URL check will be inserted.
* @param {!Object} options Dictionary of options.
* @return {boolean} True if all URLs were checked.
*/
function checkUrls(labelResourceName, checkedUrls, urlChecks, options) {
let completed = true;
if (options.checkAdUrls) {
completed = completed &&
checkAdUrls(labelResourceName, checkedUrls, urlChecks, options);
if (!completed) return completed;
}
if (options.checkKeywordUrls) {
completed = completed &&
checkKeywordUrls(labelResourceName, checkedUrls, urlChecks, options);
if (!completed) return completed;
}
if (options.checkSitelinkUrls) {
completed = completed &&
checkSitelinkUrls(labelResourceName, checkedUrls, urlChecks, options);
}
return completed;
}
/**
* Retrieves all final URLs and mobile final URLs in a selector and checks them
* for a valid response code. Does not check previously checked URLs. Labels the
* entity that it was checked, if possible.
*
* @param {string} labelResourceName The resource name of the label for
* annotating
* ads when they are processed.
* @param {Set<string>!} checkedUrls A map of previously checked URLs with the
* URL as the key.
* @param {!Array.<!Object>} urlChecks An array into which the results of each
* URL check will be inserted.
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {boolean} True if all URLs were checked.
*/
function checkAdUrls(labelResourceName, checkedUrls, urlChecks, options) {
const customerId = AdsApp.currentAccount().getCustomerId();
const selector =
AdsApp.ads()
.withCondition(
`ad_group_ad.labels CONTAINS NONE ('${labelResourceName}')`);
if (options.checkPausedAds) {
selector.withCondition(`ad_group_ad.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`campaign.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
selector.withCondition(`ad_group_ad.status = 'ENABLED'`);
selector.withCondition(`campaign.status = 'ENABLED'`);
selector.withCondition(`ad_group.status = 'ENABLED'`);
}
let urlsChecked = 0;
let urlsSkipped = 0;
const ads = selector.get();
console.log(`Processing ${ads.totalNumEntities()} ads.`);
for (const ad of ads) {
const url = ad.urls().getFinalUrl();
const mobileUrl = ad.urls().getMobileFinalUrl();
const entityDetails = {
entityType: 'Ad',
campaign: ad.getCampaign().getName(),
adGroup: ad.getAdGroup().getName(),
ad: getAdAsText(ad),
keyword: '',
sitelink: ''
};
if (url && !checkedUrls.has(url)) {
urlsChecked++;
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'Ad',
campaign: ad.getCampaign().getName(),
adGroup: ad.getAdGroup().getName(),
ad: getAdAsText(ad),
keyword: '',
sitelink: ''
});
checkedUrls.add(url);
} else if (url) {
urlsSkipped++;
}
if (mobileUrl && !checkedUrls.has(mobileUrl)) {
urlsChecked++;
const responseCode = requestUrl(mobileUrl, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: mobileUrl,
responseCode: responseCode,
entityType: 'Ad',
campaign: ad.getCampaign().getName(),
adGroup: ad.getAdGroup().getName(),
ad: getAdAsText(ad),
keyword: '',
sitelink: ''
});
checkedUrls.add(mobileUrl);
} else if (mobileUrl) {
urlsSkipped++;
}
ad.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${urlsChecked} ad urls. Skipped ${
urlsSkipped} urls (duplicate).`);
return false;
}
}
console.log(`Checked ${urlsChecked} ad urls. Skipped ${
urlsSkipped} urls (duplicate).`);
return true;
}
/**
* Helper function to grab relevant text for an ad. The relevant text is
* different for each type of ad.
*
* @param {Ad!} ad The ad we need text for.
* @return {string} Relevant text to describe the ad.
*/
function getAdAsText(ad) {
// There is no AdTypeSpace method for textAd
if (ad.getType() === 'TEXT_AD') {
return ad.getHeadline();
} else if (ad.isType().expandedTextAd()) {
const eta = ad.asType().expandedTextAd();
return eta.getHeadlinePart1() + ' - ' + eta.getHeadlinePart2();
} else if (ad.isType().gmailImageAd()) {
return ad.asType().gmailImageAd().getName();
} else if (ad.isType().gmailMultiProductAd()) {
return ad.asType().gmailMultiProductAd().getHeadline();
} else if (ad.isType().gmailSinglePromotionAd()) {
return ad.asType().gmailSinglePromotionAd().getHeadline();
} else if (ad.isType().html5Ad()) {
return ad.asType().html5Ad().getName();
} else if (ad.isType().imageAd()) {
return ad.asType().imageAd().getName();
} else if (ad.isType().responsiveDisplayAd()) {
return ad.asType().responsiveDisplayAd().getLongHeadline();
} else if (ad.isType().responsiveSearchAd()) {
return ad.asType().responsiveSearchAd().getHeadlines().map(h => h.text).join(', ');
}
return 'N/A';
}
/**
* Check URLs for all keywords in the account. Skip keywords that were already
* processed. Skip URLs that were already checked. The results of each URL that
* is checked is stored in urlChecks.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* keywords.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining keywords were processed. Returns
* false otherwise.
*/
function checkKeywordUrls(labelResourceName, checkedUrls, urlChecks, options) {
const customerId = AdsApp.currentAccount().getCustomerId();
const selector =
AdsApp.keywords()
.withCondition(`ad_group_criterion.labels CONTAINS NONE ('${
labelResourceName}')`);
if (options.checkPausedKeywords) {
selector.withCondition(
`ad_group_criterion.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`campaign.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
selector.withCondition(`ad_group_criterion.status = 'ENABLED'`);
selector.withCondition(`campaign.status = 'ENABLED'`);
selector.withCondition(`ad_group.status = 'ENABLED'`);
}
let urlsSkipped = 0;
let urlsChecked = 0;
let keywordsWithNoUrls = 0;
const keywords = selector.get();
console.log(`Processing ${keywords.totalNumEntities()} keywords.`);
for (const keyword of keywords) {
const url = keyword.urls().getFinalUrl();
const mobileUrl = keyword.urls().getMobileFinalUrl();
const entityDetails = {
entityType: 'Keyword',
campaign: keyword.getCampaign().getName(),
adGroup: keyword.getAdGroup().getName(),
ad: '',
keyword: keyword.getText(),
sitelink: ''
};
if (url && !checkedUrls.has(url)) {
urlsChecked++;
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'Keyword',
campaign: keyword.getCampaign().getName(),
adGroup: keyword.getAdGroup().getName(),
ad: '',
keyword: keyword.getText(),
sitelink: ''
});
checkedUrls.add(url);
} else if (url) {
urlsSkipped++;
}
if (mobileUrl && !checkedUrls.has(mobileUrl)) {
urlsChecked++;
const responseCode = requestUrl(mobileUrl, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: mobileUrl,
responseCode: responseCode,
entityType: 'Keyword',
campaign: keyword.getCampaign().getName(),
adGroup: keyword.getAdGroup().getName(),
ad: '',
keyword: keyword.getText(),
sitelink: ''
});
checkedUrls.add(mobileUrl);
} else if (mobileUrl) {
urlsSkipped++;
}
if (!url && !mobileUrl) {
keywordsWithNoUrls++;
}
keyword.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${urlsChecked} keyword urls. Skipped ${keywordsWithNoUrls} keywords (no urls). Skipped ${urlsSkipped} urls (duplicate).`);
return false;
}
}
console.log(`Checked ${urlsChecked} keyword urls. Skipped ${keywordsWithNoUrls} keywords (no urls). Skipped ${urlsSkipped} urls (duplicate).`);
return true;
}
/**
* Helper function that checks all sitelink URLs in the account. See
* checkCampaignSitelinks and checkAdGroupSitelinks for more details.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* campaigns or ad groups.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining campaigns and ad groups were
* processed. Returns false otherwise.
*/
function checkSitelinkUrls(labelResourceName, checkedUrls, urlChecks, options) {
const uncheckedUrls = loadUncheckedSitelinkUrls(
options.checkPausedSitelinks, labelResourceName);
const checkedCampaigns = checkCampaignSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options);
if (!checkedCampaigns) {
return false;
}
const checkedAdGroups = checkAdGroupSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options);
return checkedAdGroups;
}
/**
* Check URLs for all campaign sitelinks in the account. Skip campaigns that
* were already processed. Skip URLs that were already checked. The results of
* each URL that is checked is stored in urlChecks.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* campaigns.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Map<string, !Array.<!Object>>} uncheckedUrls A map of resource name
* to URLs to check for all campaigns
* that have not been processed yet.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining campaigns were processed. Returns
* false otherwise.
*/
function checkCampaignSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options) {
const campaignResourceNames = getUncheckedCampaignResourceNames(
options.checkPausedSitelinks, labelResourceName);
let urlsSkipped = 0;
let urlsChecked = 0;
const customerId = AdsApp.currentAccount().getCustomerId();
while (campaignResourceNames.length > 0) {
// Load campaigns 10k at a time.
let someCampaignResourceNames = campaignResourceNames.splice(0, 10000);
let campaigns = AdsApp.campaigns()
.withCondition(`campaign.resource_name IN (${
someCampaignResourceNames.join(', ')})`)
.get();
for (const campaign of campaigns) {
const resourceName = campaign.getResourceName();
if (!uncheckedUrls.has(resourceName)) {
continue;
}
const urlsToCheck = uncheckedUrls.get(resourceName);
for (const urlInfo of urlsToCheck) {
const url = urlInfo.url;
const linkText = urlInfo.linkText;
if (checkedUrls.has(url)) {
// already checked this url
urlsSkipped++;
} else {
urlsChecked++;
const entityDetails = {
entityType: 'CampaignSitelink',
campaign: campaign.getName(),
adGroup: '',
ad: '',
keyword: '',
sitelink: linkText
};
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'CampaignSitelink',
campaign: campaign.getName(),
adGroup: '',
ad: '',
keyword: '',
sitelink: linkText
});
checkedUrls.add(url);
}
}
campaign.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${
urlsChecked} campaign sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return false;
}
}
}
console.log(`Checked ${urlsChecked} campaign sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return true;
}
/**
* Check URLs for all ad group sitelinks in the account. Skip ad groups that
* were already processed. Skip URLs that were already checked. The results of
* each URL that is checked is stored in urlChecks.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* ad groups.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Map<string, !Array.<!Object>>} uncheckedUrls A map of resource name
* to URLs to check for all ad groups
* that have not been processed yet.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining ad groups were processed. Returns
* false otherwise.
*/
function checkAdGroupSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options) {
const adGroupResourceNames = getUncheckedAdGroupResourceNames(
options.checkPausedSitelinks, labelResourceName);
let urlsSkipped = 0;
let urlsChecked = 0;
const customerId = AdsApp.currentAccount().getCustomerId();
while (adGroupResourceNames.length > 0) {
// Load ad groups 10k at a time.
let someAdGroupResourceNames = adGroupResourceNames.splice(0, 10000);
let adGroups = AdsApp.adGroups()
.withCondition(`ad_group.resource_name IN (${
someAdGroupResourceNames.join(', ')})`)
.get();
for (const adGroup of adGroups) {
const resourceName = adGroup.getResourceName();
if (!uncheckedUrls.has(resourceName)) {
continue;
}
const urlsToCheck = uncheckedUrls.get(resourceName);
for (const urlInfo of urlsToCheck) {
const url = urlInfo.url;
const linkText = urlInfo.linkText;
if (checkedUrls.has(url)) {
// already checked this url
urlsSkipped++;
} else {
urlsChecked++;
const entityDetails = {
entityType: 'AdGroupSitelink',
campaign: adGroup.getCampaign().getName(),
adGroup: adGroup.getName(),
ad: '',
keyword: '',
sitelink: linkText
};
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'AdGroupSitelink',
campaign: adGroup.getCampaign().getName(),
adGroup: adGroup.getName(),
ad: '',
keyword: '',
sitelink: linkText
});
checkedUrls.add(url);
}
}
adGroup.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${
urlsChecked} adgroup sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return false;
}
}
}
console.log(`Checked ${urlsChecked} adgroup sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return true;
}
/**
* Returns a map of resource name (campaign or ad group) to a list of sitelink
* URLs to check. For each URL, we also include the sitelink link text for
* reporting reasons. These URLs may have already been tested (when checking
* other URLs). The logic to ensure URLs are only checked once is handled
* elsewhere.
*
* @param {boolean} checkPaused Whether to include sitelinks that are "paused"
* (whether the campaign
* or ad group is paused)
* @param {string} labelResourceName The label to identify processed/unprocessed
* campaigns/ad groups.
* @return {!Map<string, !Array.<!Object>>} A map of resource name to an array
* of objects ({url, linkText}).
*/
function loadUncheckedSitelinkUrls(checkPaused, labelResourceName) {
const resourceToUrlMap = new Map();
const campaignResourceNames =
getUncheckedCampaignResourceNames(checkPaused, labelResourceName);
while (campaignResourceNames.length > 0) {
// Load campaigns 10k at a time.
let someCampaignResourceNames = campaignResourceNames.splice(0, 10000);
let query = `SELECT campaign_asset.campaign, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from campaign_asset where campaign_asset.campaign IN (${
someCampaignResourceNames.join(
', ')}) and campaign_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
let rows = AdsApp.search(query);
for (const row of rows) {
const campaign = row.campaignAsset.campaign;
const finalUrl = row.asset.finalUrls[0];
const linkText = row.asset.sitelinkAsset.linkText;
if (!resourceToUrlMap.has(campaign)) {
resourceToUrlMap.set(campaign, []);
}
const urls = resourceToUrlMap.get(campaign);
// Sitelinks always have final url but might not have final mobile url
urls.push({url: finalUrl, linkText: linkText});
if (row.asset.finalMobileUrls) {
urls.push({url: row.asset.finalMobileUrls[0], linkText: linkText});
}
resourceToUrlMap.set(campaign, urls);
}
}
const adGroupResourceNames =
getUncheckedAdGroupResourceNames(checkPaused, labelResourceName);
while (adGroupResourceNames.length > 0) {
// Load ad groups 10k at a time.
let someAdGroupResourceNames = adGroupResourceNames.splice(0, 10000);
query = `SELECT ad_group_asset.ad_group, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from ad_group_asset where ad_group_asset.ad_group IN (${
someAdGroupResourceNames.join(
', ')}) and ad_group_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
rows = AdsApp.search(query);
for (const row of rows) {
const adGroup = row.adGroupAsset.adGroup;
const finalUrl = row.asset.finalUrls[0];
const linkText = row.asset.sitelinkAsset.linkText;
if (!resourceToUrlMap.has(adGroup)) {
resourceToUrlMap.set(adGroup, []);
}
const urls = resourceToUrlMap.get(adGroup);
urls.push({url: finalUrl, linkText: linkText});
if (row.asset.finalMobileUrls) {
urls.push({url: row.asset.finalMobileUrls[0], linkText: linkText});
}
resourceToUrlMap.set(adGroup, urls);
}
}
return resourceToUrlMap;
}
/**
* Expands a URL that contains ValueTrack parameters such as {ifmobile:mobile}
* to all the combinations, and returns as an array. The following pairs of
* ValueTrack parameters are currently expanded:
* 1. {ifmobile:<...>} and {ifnotmobile:<...>} to produce URLs simulating
* clicks from either mobile or non-mobile devices.
* 2. {ifsearch:<...>} and {ifcontent:<...>} to produce URLs simulating
* clicks on either the search or display networks.
* Any other ValueTrack parameters or customer parameters are stripped out from
* the URL entirely.
*
* @param {string} url The URL which may contain ValueTrack parameters.
* @return {!Array.<string>} An array of one or more expanded URLs.
*/
function expandUrlModifiers(url) {
const ifRegex = /({(if\w+):([^}]+)})/gi;
const modifiers = {};
let matches;
let modifiedUrls;
while (matches = ifRegex.exec(url)) {
// Tags are case-insensitive, e.g. IfMobile is valid.
modifiers[matches[2].toLowerCase()] = {
substitute: matches[0],
replacement: matches[3]
};
}
if (Object.keys(modifiers).length) {
let mobileCombinations;
if (modifiers.ifmobile || modifiers.ifnotmobile) {
mobileCombinations =
pairedUrlModifierReplace(modifiers, 'ifmobile', 'ifnotmobile', url);
} else {
mobileCombinations = [url];
}
// Store in a map on the offchance that there are duplicates.
const combinations = {};
for (const url of mobileCombinations) {
if (modifiers.ifsearch || modifiers.ifcontent) {
for (const modifiedUrl of
pairedUrlModifierReplace(modifiers, 'ifsearch', 'ifcontent', url)) {
combinations[modifiedUrl] = true;
}
} else {
combinations[url] = true;
}
}
modifiedUrls = Object.keys(combinations);
} else {
modifiedUrls = [url];
}
// Remove any custom parameters
return modifiedUrls.map(url => url.replace(/{[\w_+:]+}/g, ''));
}
/**
* Return a pair of URLs, where each of the two modifiers is mutually exclusive,
* one for each combination. e.g. Evaluating ifmobile and ifnotmobile for a
* mobile and a non-mobile scenario.
*
* @param {!Object} modifiers A map of ValueTrack modifiers.
* @param {string} modifier1 The modifier to honour in the URL.
* @param {string} modifier2 The modifier to remove from the URL.
* @param {string} url The URL potentially containing ValueTrack parameters.
* @return {!Array.<string>} A pair of URLs, as a list.
*/
function pairedUrlModifierReplace(modifiers, modifier1, modifier2, url) {
return [
urlModifierReplace(modifiers, modifier1, modifier2, url),
urlModifierReplace(modifiers, modifier2, modifier1, url)
];
}
/**
* Produces a URL where the first {if...} modifier is set, and the second is
* deleted.
*
* @param {!Object} mods A map of ValueTrack modifiers.
* @param {string} mod1 The modifier to honour in the URL.
* @param {string} mod2 The modifier to remove from the URL.
* @param {string} url The URL potentially containing ValueTrack parameters.
* @return {string} The resulting URL with substitutions.
*/
function urlModifierReplace(mods, mod1, mod2, url) {
const modUrl = mods[mod1] ?
url.replace(mods[mod1].substitute, mods[mod1].replacement) :
url;
return mods[mod2] ? modUrl.replace(mods[mod2].substitute, '') : modUrl;
}
/**
* Requests a given URL. Retries if the UrlFetchApp QPS limit was reached,
* exponentially backing off on each retry. Throws an exception if it reaches
* the maximum number of retries. Throws an exception if the UrlFetchApp daily
* quota limit was reached.
*
* @param {string} url The URL to test.
* @param {!Object} options The options loaded from the configuration sheet.
* @param {!Object} entityDetails Details of the entity, e.g. type, name etc.
* @return {number|string} The response code received when requesting the URL,
* or an error message.
*/
function requestUrl(url, options, entityDetails) {
const expandedUrls = expandUrlModifiers(url);
let responseCode;
let sleepTime = QUOTA_CONFIG.INIT_SLEEP_TIME;
let numTries = 0;
for (const expandedUrl of expandedUrls) {
while (numTries < QUOTA_CONFIG.MAX_TRIES && !responseCode) {
try {
// If UrlFetchApp.fetch() throws an exception, responseCode will remain
// undefined.
const response =
UrlFetchApp.fetch(expandedUrl, {muteHttpExceptions: true});
responseCode = response.getResponseCode();
if (options.validCodes.indexOf(responseCode) !== -1) {
if (options.useSimpleFailureStrings &&
bodyContainsFailureStrings(response, options.failureStrings)) {
return 'Failure string detected';
} else if (
options.useCustomValidation &&
!isValidResponse(url, response, options, entityDetails)) {
return 'Custom validation failed';
}
}
if (THROTTLE > 0) {
Utilities.sleep(THROTTLE * MILLISECONDS_PER_SECOND);
}
} catch (e) {
if (e.message.indexOf(
'Service invoked too many times in a short time:') !== -1) {
Utilities.sleep(sleepTime);
sleepTime *= QUOTA_CONFIG.BACKOFF_FACTOR;
} else if (
e.message.indexOf('Service invoked too many times:') !== -1) {
throw EXCEPTIONS.LIMIT;
} else {
return e.message;
}
}
numTries++;
}
}
if (!responseCode) {
throw EXCEPTIONS.QPS;
} else {
return responseCode;
}
}
/**
* Searches the body of a HTTP response for any occurrence of a "failure string"
* as defined in the configuration spreadsheet. For example, "Out of stock".
*
* @param {!Object} response The response from the UrlFetchApp request.
* @param {!Array.<string>} failureStrings A list of failure strings.
* @return {boolean} Returns true if at least one failure string found.
*/
function bodyContainsFailureStrings(response, failureStrings) {
const contentText = response.getContentText() || '';
// Whilst searching for each separate failure string across the body text
// separately may not be the most efficient, it is simple, and tests suggest
// it is not overly poor performance-wise.
return failureStrings.some(
failureString => contentText.indexOf(failureString) !== -1);
}
/**
* @return {boolean} True iff there is less than TIMEOUT_BUFFER seconds left in
* the execution.
*/
function aboutToTimeout() {
return AdsApp.getExecutionInfo().getRemainingTime() < TIMEOUT_BUFFER;
}
/**
* Returns the number of days between two dates.
*
* @param {!Object} from The older Date object.
* @param {!Object} to The newer (more recent) Date object.
* @return {number} The number of days between the given dates (possibly
* fractional).
*/
function dayDifference(from, to) {
return (to.getTime() - from.getTime()) / (24 * 3600 * 1000);
}
/**
* Given a label name, ensure the label is already in the account. If the label
* is not present *and* this is a live execution, go ahead and create the label.
* If this is a preview, throw an error instructing the user to create the
* label.
*
* @param {string} labelName The name of the label to annotate processed
* entities with.
*/
function ensureLabel(labelName) {
const labels = AdsApp.labels()
.withCondition(`label.name = '${labelName}'`)
.withCondition(`label.status != 'REMOVED'`)
.get();
if (!labels.hasNext()) {
if (!AdsApp.getExecutionInfo().isPreview()) {
AdsApp.createLabel(labelName);
} else {
throw `Label ${labelName} is missing and cannot be created in ` +
`preview mode. Please run the script or create the label manually.`;
}
}
}
/**
* Removes the label from the account. Since labels cannot be removed
* in preview mode, throws an exception in preview mode.
*
* @param {string} labelName The name of the label to remove.
*/
function removeLabel(labelName) {
if (AdsApp.getExecutionInfo().isPreview()) {
throw 'Cannot remove labels in preview mode. Please run the script or ' +
'remove the label manually.';
}
const labels = AdsApp.labels()
.withCondition(`label.name = '${labelName}'`)
.withCondition(`label.status != 'REMOVED'`)
.get();
if (labels.hasNext()) {
labels.next().remove();
}
}
/**
* Validates the provided spreadsheet URL to make sure that it's set up
* properly. Throws a descriptive error message if validation fails.
*
* @param {string} spreadsheeturl The URL of the spreadsheet to open.
* @return {!Object} The spreadsheet object itself, fetched from the URL.
* @throws {Error} If the spreadsheet URL hasn't been set
*/
function validateAndGetSpreadsheet(spreadsheeturl) {
if (spreadsheeturl === 'YOUR_SPREADSHEET_URL') {
throw new Error('Please specify a valid Spreadsheet URL. You can find' +
' a link to a template in the associated guide for this script.');
}
return SpreadsheetApp.openByUrl(spreadsheeturl);
}
/**
* Validates the provided email addresses to make sure it's not the default.
* Throws a descriptive error message if validation fails.
*
* @param {!Array.<string>} recipientEmails The list of email addresses.
* @throws {Error} If the list of email addresses is still the default
*/
function validateEmailAddresses(recipientEmails) {
if (recipientEmails && recipientEmails[0] === 'YOUR_EMAIL_HERE') {
throw new Error('Please either specify a valid email address or clear' +
' the RECIPIENT_EMAILS field.');
}
}