По мере развития веб-сайта добавляются новые страницы, а старые удаляются, что приводит к возможным неработающим ссылкам. Синхронизация кампании Google Ads с веб-сайтом может оказаться сложной задачей для многих рекламодателей. Живая реклама может указывать на несуществующие страницы, что приводит к ухудшению пользовательского опыта и неточной статистике Google Рекламы.
Link Checker решает эту проблему, перебирая все ваши объявления, ключевые слова и дополнительные ссылки, проверяя, чтобы их URL-адреса не выдавали сообщения «Страница не найдена» или другие типы ответов об ошибках, отправляя вам электронное письмо при обнаружении ответов об ошибках и сохраняя результаты. его анализ в электронную таблицу.
Некоторые учетные записи могут иметь большое количество URL-адресов, которые невозможно проверить за один раз из-за ограничений по времени выполнения или квот . Сценарий предназначен для обработки этого случая, отслеживая его ход при нескольких запусках и проверяя только те URL-адреса, которые он не проверял ранее. Скрипт использует термин «анализ» для обозначения одного полного прохода по всем вашим URL-адресам, даже если для его завершения требуется несколько прогонов.
Конфигурация
Основные параметры скрипта можно настроить в электронной таблице.
- Объем
- Определяет, проверяет ли скрипт объявления, ключевые слова и дополнительные ссылки; и проверяет ли их сценарий, даже если они приостановлены. Большинству пользователей следует включить все три, чтобы убедиться, что все их URL-адреса проверены, но обычно нет необходимости проверять приостановленные объекты.
- Действительные коды ответа
- Список кодов ответов HTTP, которые считаются действительными. Большинству пользователей следует считать действительным код ответа только
200. - Электронная почта после каждого выполнения скрипта
- Если вы включите эту опцию, сценарий отправит вам по электронной почте сводку URL-адресов, которые он проверял, после каждого запуска. Это позволит вам заранее получать оповещения об ошибках URL-адресов, вместо того, чтобы ждать, пока проверятся все ваши URL-адреса (что может занять несколько запусков).
- Электронная почта после завершения всего анализа
- Если вы включите эту опцию, сценарий отправит вам сводную сводку по электронной почте после завершения проверки всех URL-адресов.
- Отправьте электронное письмо, даже если ошибок не обнаружено
- Если вы включите эту опцию, скрипт отправит вам электронное письмо (на основе двух предыдущих опций), даже если он не обнаружил никаких ошибок. Большинство пользователей предпочитают получать электронные письма только в случае ошибок, но получение электронного письма, даже если ошибок нет, может быть полезным способом убедиться, что сценарий работает по расписанию.
- Сохранить URL-адреса OK в электронную таблицу
- Если вы включите эту опцию, скрипт сохранит в электронной таблице каждый URL-адрес, который он проверяет, а не только те, которые содержат ошибки. Большинство пользователей предпочитают сохранять только неработающие URL-адреса, но некоторым может быть интересно просмотреть все URL-адреса, проверенные сценарием.
- Дни между анализами
- Используйте эту опцию, чтобы контролировать, как часто сценарий запускает новый анализ всего списка URL-адресов. Обратите внимание, что этот параметр определяет минимальное количество дней между анализами. Фактическое число может быть выше, если анализ занимает больше времени (например, если у вас большое количество URL-адресов). Более подробную информацию см. в разделе Планирование .
Расширенная проверка ссылки
В некоторых случаях может быть недостаточно просто знать, что URL-адрес возвращает действительную страницу. Например, рассмотрим случай, когда страница продукта содержит текст «продукт снят с производства». В этом случае вы можете пометить страницу как неработающую ссылку.
Скрипт предлагает два варианта проверки:
- Строки отказа
Установите для параметра Использовать простое соответствие строки сбоя значение
Yesчтобы искать вхождения в список строк, определенных в строках сбоя . Любое событие, обнаруженное на веб-странице, помечает ее как неработающую.- Пользовательская проверка
Для еще большей гибкости для каждого URL-адреса и ответа можно использовать специальную функцию проверки JavaScript. Например, вы можете подтвердить, что заголовок веб-страницы содержит название вашего бренда следующим образом:
- Установите для параметра Использовать пользовательскую функцию проверки значение
Yes. Вставьте свою собственную логику проверки в функцию в верхней части скрипта:
function isValidResponse(url, response, options, entityDetails) { // Your custom logic here, returning true for valid links, false otherwise. // ... }
Базовая реализация уже имеется, с примерами доступа к тексту ответа.
- Установите для параметра Использовать пользовательскую функцию проверки значение
Планирование
Каждый раз при запуске сценария он автоматически определяет, следует ли возобновить уже выполняемый анализ или последний анализ завершен и пришло время начать новый (на основе параметра «Дни между анализами» ). В результате, независимо от того, как часто вы хотите запускать новый анализ, запланируйте запуск сценария ежечасно , чтобы каждый анализ выполнялся по вашим URL-адресам как можно быстрее.
Например, установите «Дни между анализами» на 1 , чтобы скрипт запускал новый анализ не чаще одного раза в день. Если сценарий запланирован на ежечасный запуск и он завершит проверку всех ваших URL-адресов менее чем за день, последующие запуски будут немедленно прекращены до следующего дня, когда придет время начать новый анализ.
Как это работает
Чтобы отслеживать ход выполнения, скрипт создает метку и применяет ее к вашим объявлениям, ключевым словам и дополнительным ссылкам после их проверки. Затем сценарий сможет идентифицировать проверенные URL-адреса при следующем запуске. После завершения анализа (все URL-адреса проверены) сценарий очищает метку для нового анализа.
Скрипт использует UrlFetchApp для проверки конечных URL-адресов и конечных URL-адресов мобильных устройств.
Настраивать
Нажмите кнопку ниже, чтобы создать сценарий на основе электронной таблицы в своем аккаунте Google Рекламы.
Нажмите кнопку ниже, чтобы сделать копию таблицы шаблона.
Обновите
spreadsheet_urlиrecipient_emailsв вашем скрипте.Запланируйте запуск сценария ежечасно .
Исходный код
// Copyright 2016, Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/**
* @name Link Checker
*
* @overview The Link Checker script iterates through the ads, keywords, and
* sitelinks in your account and makes sure their URLs do not produce "Page
* not found" or other types of error responses. See
* https://developers.google.com/google-ads/scripts/docs/solutions/link-checker
* for more details.
*
* @author Google Ads Scripts Team [adwords-scripts@googlegroups.com]
*
* @version 3.1
*
* @changelog
* - version 4.0
* - Refactored for readability and efficiency - particularly for sitelinks.
* - version 3.1
* - Split into info, config, and code.
* - version 3.0
* - Updated to use new Google Ads scripts features.
* - version 2.2
* - Added support for failure strings and custom validation functions.
* - version 2.1
* - Added expansion of conditional ValueTrack parameters (e.g. ifmobile).
* - Added expanded text ad and other ad format support.
* - version 2.0.3
* - Added validation for external spreadsheet setup.
* - version 2.0.2
* - Allow the custom tracking label to include spaces.
* - version 2.0.1
* - Catch and output all UrlFetchApp exceptions.
* - version 2.0
* - Completely revised the script to work on larger accounts.
* - Check URLs in campaign and ad group sitelinks.
* - version 1.2
* - Released initial version.
*/
/**
* Configuration to be used for the Link Checker.
*/
CONFIG = {
// URL of the spreadsheet template.
// This should be a copy of https://docs.google.com/spreadsheets/d/1iO1iEGwlbe510qo3Li-j4KgyCeVSmodxU6J7M756ppk/copy.
'spreadsheet_url': 'YOUR_SPREADSHEET_URL',
// Array of addresses to be alerted via email if issues are found.
'recipient_emails': [
'YOUR_EMAIL_HERE'
],
// Label to use when a link has been checked. Label will be created if it doesn't exist.
'label': 'LinkChecker_Done',
// Number of seconds to sleep after each URL request. If your URLs are
// all on one or a few domains, use this throttle to reduce the load that the
// script imposes on your web server(s).
'throttle': 0,
// Number of seconds before timeout that the script should stop checking URLs
// to make sure it has time to output its findings.
'timeout_buffer': 120,
'advanced_options': {
/**
* Parameters controlling the script's behavior after hitting a UrlFetchApp
* QPS quota limit.
*/
'quota_config': {
'INIT_SLEEP_TIME': 250,
'BACKOFF_FACTOR': 2,
'MAX_TRIES': 5
}
}
};
/**
* Performs custom validation on a URL, with access to details such as the URL,
* the response from the server, configuration options and entity Details.
*
* To use, the "Use Custom Validation" option in the configuration spreadsheet
* must be set to "Yes", and your custom validation code implemented within the
* below function.
*
* See the documentation for this solution for further details.
*
* @param {string} url The URL being checked.
* @param {!Object} response The response object for the request.
* @param {!Object} options Configuration options.
* @param {!Object} entityDetails Details of the associated Ad / Keywords etc.
* @return {boolean} Return true if the URL and response are deemed valid.
*/
function isValidResponse(url, response, options, entityDetails) {
/*
Some examples of data that can be used in determining the validity of this
URL. This is not exhaustive and there are further properties available.
*/
// The HTTP status code, e.g. 200, 404
// const responseCode = response.getResponseCode();
// The HTTP response body, e.g. HTML for web pages:
// const responseText = response.getContentText();
// The failure strings from the configuration spreadsheet, as an array:
// const failureStrings = options.failureStrings;
// The type of the entity associated with the URL, e.g. Ad, Keyword, Sitelink.
// const entityType = entityDetails.entityType;
// The campaign name
// const campaignName = entityDetails.campaign;
// The ad group name, if applicable
// const adGroupName = entityDetails.adGroup;
// The ad text, if applicable
// const adText = entityDetails.ad;
// The keyword text, if applicable
// const keywordText = entityDetails.keyword;
// The sitelink link text, if applicable
// const sitelinkText = entityDetails.sitelink;
/*
Remove comments and insert custom logic to determine whether this URL and
response are valid, using the data obtained above.
If valid, return true. If invalid, return false.
*/
// Placeholder implementation treats all URLs as valid
return true;
}
const QUOTA_CONFIG = CONFIG.advanced_options.quota_config;
/**
* Exceptions that prevent the script from finishing checking all URLs in an
* account but allow it to resume next time.
*/
const EXCEPTIONS = {
QPS: 'Reached UrlFetchApp QPS limit',
LIMIT: 'Reached UrlFetchApp daily quota',
TIMEOUT: 'Approached script execution time limit'
};
/**
* Named ranges in the spreadsheet.
*/
const NAMES = {
CHECK_AD_URLS: 'checkAdUrls',
CHECK_KEYWORD_URLS: 'checkKeywordUrls',
CHECK_SITELINK_URLS: 'checkSitelinkUrls',
CHECK_PAUSED_ADS: 'checkPausedAds',
CHECK_PAUSED_KEYWORDS: 'checkPausedKeywords',
CHECK_PAUSED_SITELINKS: 'checkPausedSitelinks',
VALID_CODES: 'validCodes',
EMAIL_EACH_RUN: 'emailEachRun',
EMAIL_NON_ERRORS: 'emailNonErrors',
EMAIL_ON_COMPLETION: 'emailOnCompletion',
FAILURE_STRINGS: 'failureStrings',
SAVE_ALL_URLS: 'saveAllUrls',
FREQUENCY: 'frequency',
DATE_STARTED: 'dateStarted',
DATE_COMPLETED: 'dateCompleted',
DATE_EMAILED: 'dateEmailed',
NUM_ERRORS: 'numErrors',
RESULT_HEADERS: 'resultHeaders',
ARCHIVE_HEADERS: 'archiveHeaders',
USE_SIMPLE_FAILURE_STRINGS: 'useSimpleFailureStrings',
USE_CUSTOM_VALIDATION: 'useCustomValidation'
};
const MILLISECONDS_PER_SECOND = 1000;
const SPREADSHEET_URL = CONFIG.spreadsheet_url;
const RECIPIENT_EMAILS = CONFIG.recipient_emails;
const LABEL = CONFIG.label;
const THROTTLE = CONFIG.throttle;
const TIMEOUT_BUFFER = CONFIG.timeout_buffer;
/**
* The Link Checker script iterates through the ads, keywords, and sitelinks
* in your account and makes sure their URLs do not produce "Page not found"
* or other types of error responses.
*/
function main() {
const spreadsheet = validateAndGetSpreadsheet(SPREADSHEET_URL);
validateEmailAddresses(RECIPIENT_EMAILS);
spreadsheet.setSpreadsheetTimeZone(AdsApp.currentAccount().getTimeZone());
const options = loadOptions(spreadsheet);
const status = loadStatus(spreadsheet);
if (!status.dateStarted) {
// This is the very first execution of the script.
console.log(`First time analyzing the account.`);
startNewAnalysis(spreadsheet);
} else if (status.dateStarted > status.dateCompleted) {
console.log('Resuming work from a previous execution.');
} else if (
dayDifference(status.dateStarted, new Date()) < options.frequency) {
console.log(
`Waiting until ${options.frequency} ` +
`days have elapsed since the start of the last analysis.`);
return;
} else {
// Enough time has passed since the last analysis to start a new one.
console.log(`Restarting analysis`);
removeLabel(CONFIG.label);
startNewAnalysis(spreadsheet);
}
const results = analyzeAccount(options);
console.log(`Completed analysis, outputting results.`);
outputResults(results, options);
}
/**
* Checks as many new URLs as possible that have not previously been checked,
* subject to quota and time limits.
*
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {!Object} An object with fields for the URLs checked and an
* indication if the analysis was completed (no remaining URLs to check).
*/
function analyzeAccount(options) {
// Ensure the label exists before attempting to retrieve already checked URLs.
ensureLabel(CONFIG.label);
const labelResourceName = loadLabelResourceName(CONFIG.label);
const urlChecks = [];
const checkedUrls = getAlreadyCheckedUrls(labelResourceName, options);
console.log(`Already checked ${checkedUrls.size} urls`);
let didComplete = false;
try {
didComplete = checkUrls(labelResourceName, checkedUrls, urlChecks, options);
} catch (e) {
if (e === EXCEPTIONS.QPS || e === EXCEPTIONS.LIMIT ||
e === EXCEPTIONS.TIMEOUT) {
console.warn(`Stopped checking URLs early because: ${e}`);
console.log('Checked URLs will still be output.');
} else {
throw e;
}
}
return {urlChecks, didComplete};
}
/**
* Given a label name, load the resource name for that label. There will only be
* one label with the specified name which is active (there may be many with
* this name which are removed). This label is used to identify entities which
* have already been processed and to annotate entities which have just been
* processed.
*
* @param {string} labelName The name of the label used to annotate entities
* which have been checked.
* @return {string} The resource name of the relevant label.
*/
function loadLabelResourceName(labelName) {
const query =
`SELECT label.resource_name, label.name, label.status from label where label.name='${
labelName}' and label.status = 'ENABLED'`;
const rows = AdsApp.search(query);
if (rows.totalNumEntities() > 1) {
throw new Error(`Found ${rows.length} labels with name '${
labelName}' when there should only be 1.`);
}
if (rows.totalNumEntities() === 0) {
throw new Error(`Could not find label with name '${labelName}'.`);
}
return rows.next().label.resourceName;
}
/**
* Outputs the results to a spreadsheet and sends emails if appropriate.
*
* @param {!Object} results An object with fields for the URLs checked and an
* indication if the analysis was completed (no remaining URLs to check).
* @param {!Object} options Dictionary of options.
*/
function outputResults(results, options) {
const spreadsheet = SpreadsheetApp.openByUrl(SPREADSHEET_URL);
const numErrors = countErrors(results.urlChecks, options);
console.log(`Found ${numErrors} invalid urls this execution.`);
saveUrlsToSpreadsheet(spreadsheet, results.urlChecks, options);
// Reload the status to get the total number of errors for the entire
// analysis, which is calculated by the spreadsheet.
const status = loadStatus(spreadsheet);
if (results.didComplete) {
spreadsheet.getRangeByName(NAMES.DATE_COMPLETED).setValue(new Date());
console.log(
`Found ${status.numErrors} invalid urls across the entire analysis.`);
}
if (RECIPIENT_EMAILS) {
if (!results.didComplete && options.emailEachRun &&
(options.emailNonErrors || numErrors > 0)) {
sendIntermediateEmail(spreadsheet, numErrors);
}
if (results.didComplete &&
(options.emailEachRun || options.emailOnCompletion) &&
(options.emailNonErrors || status.numErrors > 0)) {
sendFinalEmail(spreadsheet, status.numErrors);
}
}
}
/**
* Loads data from a spreadsheet based on named ranges. Strings 'Yes' and 'No'
* are converted to booleans. One-dimensional ranges are converted to arrays
* with blank cells omitted. Assumes each named range exists.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {!Array.<string>} names A list of named ranges that should be loaded.
* @return {!Object} A dictionary with the names as keys and the values
* as the cell values from the spreadsheet.
*/
function loadDatabyName(spreadsheet, names) {
const data = {};
for (const name of names) {
const range = spreadsheet.getRangeByName(name);
if (range.getNumRows() > 1 && range.getNumColumns() > 1) {
// Name refers to a 2d range, so load it as a 2d array.
data[name] = range.getValues();
} else if (range.getNumRows() === 1 && range.getNumColumns() === 1) {
// Name refers to a single cell, so load it as a value and replace
// Yes/No with boolean true/false.
data[name] = range.getValue();
data[name] = data[name] === 'Yes' ? true : data[name];
data[name] = data[name] === 'No' ? false : data[name];
} else {
// Name refers to a 1d range, so load it as an array (regardless of
// whether the 1d range is oriented horizontally or vertically).
const isByRow = range.getNumRows() > 1;
const limit = isByRow ? range.getNumRows() : range.getNumColumns();
const cellValues = range.getValues();
data[name] = [];
for (let j = 0; j < limit; j++) {
const cellValue = isByRow ? cellValues[j][0] : cellValues[0][j];
if (cellValue) {
data[name].push(cellValue);
}
}
}
}
return data;
}
/**
* Loads options from the spreadsheet.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @return {!Object} A dictionary of options.
*/
function loadOptions(spreadsheet) {
return loadDatabyName(spreadsheet,
[NAMES.CHECK_AD_URLS, NAMES.CHECK_KEYWORD_URLS,
NAMES.CHECK_SITELINK_URLS, NAMES.CHECK_PAUSED_ADS,
NAMES.CHECK_PAUSED_KEYWORDS, NAMES.CHECK_PAUSED_SITELINKS,
NAMES.VALID_CODES, NAMES.EMAIL_EACH_RUN,
NAMES.EMAIL_NON_ERRORS, NAMES.EMAIL_ON_COMPLETION,
NAMES.SAVE_ALL_URLS, NAMES.FREQUENCY,
NAMES.FAILURE_STRINGS, NAMES.USE_SIMPLE_FAILURE_STRINGS,
NAMES.USE_CUSTOM_VALIDATION]);
}
/**
* Loads state information from the spreadsheet.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @return {!Object} A dictionary of status information.
*/
function loadStatus(spreadsheet) {
return loadDatabyName(spreadsheet,
[NAMES.DATE_STARTED, NAMES.DATE_COMPLETED,
NAMES.DATE_EMAILED, NAMES.NUM_ERRORS]);
}
/**
* Saves the start date to the spreadsheet and archives results of the last
* analysis to a separate sheet.
*
* @param {!Object} spreadsheet The spreadsheet object.
*/
function startNewAnalysis(spreadsheet) {
console.log('Starting a new analysis.');
spreadsheet.getRangeByName(NAMES.DATE_STARTED).setValue(new Date());
// Helper method to get the output area on the results or archive sheets.
function getOutputRange(rangeName) {
const headers = spreadsheet.getRangeByName(rangeName);
return headers.offset(1, 0, headers.getSheet().getDataRange().getLastRow());
}
getOutputRange(NAMES.ARCHIVE_HEADERS).clearContent();
const results = getOutputRange(NAMES.RESULT_HEADERS);
results.copyTo(getOutputRange(NAMES.ARCHIVE_HEADERS));
getOutputRange(NAMES.RESULT_HEADERS).clearContent();
}
/**
* Counts the number of errors in the results.
*
* @param {!Array.<!Object>} urlChecks A list of URL check results.
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {number} The number of errors in the results.
*/
function countErrors(urlChecks, options) {
let numErrors = 0;
for (const urlCheck of urlChecks) {
if (options.validCodes.indexOf(urlCheck.responseCode) === -1) {
numErrors++;
}
}
return numErrors;
}
/**
* Saves URLs for a particular account to the spreadsheet starting at the first
* unused row.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {!Array.<!Object>} urlChecks A list of URL check results.
* @param {!Map<string, !Object>} options options Dictionary of options.
*/
function saveUrlsToSpreadsheet(spreadsheet, urlChecks, options) {
// Build each row of output values in the order of the columns.
const outputValues = [];
for (const urlCheck of urlChecks) {
if (options.saveAllUrls ||
options.validCodes.indexOf(urlCheck.responseCode) === -1) {
outputValues.push([
urlCheck.customerId,
new Date(urlCheck.timestamp),
urlCheck.url,
urlCheck.responseCode,
urlCheck.entityType,
urlCheck.campaign,
urlCheck.adGroup,
urlCheck.ad,
urlCheck.keyword,
urlCheck.sitelink
]);
}
}
if (outputValues.length > 0) {
// Find the first open row on the Results tab below the headers and create a
// range large enough to hold all of the output, one per row.
const headers = spreadsheet.getRangeByName(NAMES.RESULT_HEADERS);
const lastRow = headers.getSheet().getDataRange().getLastRow();
const outputRange = headers.offset(lastRow - headers.getRow() + 1,
0, outputValues.length);
outputRange.setValues(outputValues);
}
for (const email of RECIPIENT_EMAILS) {
spreadsheet.addEditor(email);
}
}
/**
* Sends an email to a list of email addresses with a link to the spreadsheet
* and the results of this execution of the script.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {boolean} numErrors The number of errors found in this execution.
*/
function sendIntermediateEmail(spreadsheet, numErrors) {
spreadsheet.getRangeByName(NAMES.DATE_EMAILED).setValue(new Date());
MailApp.sendEmail(RECIPIENT_EMAILS.join(','),
'Link Checker Results',
`The Link Checker script found ${numErrors} URLs with errors in ` +
`an execution that just finished. See ${spreadsheet.getUrl()} for details.`
);
}
/**
* Sends an email to a list of email addresses with a link to the spreadsheet
* and the results across the entire account.
*
* @param {!Object} spreadsheet The spreadsheet object.
* @param {boolean} numErrors The number of errors found in the entire account.
*/
function sendFinalEmail(spreadsheet, numErrors) {
spreadsheet.getRangeByName(NAMES.DATE_EMAILED).setValue(new Date());
MailApp.sendEmail(RECIPIENT_EMAILS.join(','),
'Link Checker Results',
`The Link Checker script found ${numErrors} URLs with errors ` +
`across its entire analysis. See ${spreadsheet.getUrl()} for details.`);
}
/**
* Retrieves all final URLs and mobile final URLs in the account across ads,
* keywords, and sitelinks that were checked in a previous run, as indicated by
* them having been labeled.
*
* @param {string} labelResourceName The resource name of the label applied to
* entities that have already been checked.
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {!Object} A map of previously checked URLs with the URL as the key.
*/
function getAlreadyCheckedUrls(labelResourceName, options) {
const urls = new Set();
if (options.checkAdUrls) {
const adUrls = getCheckedAdUrls(options.checkPausedAds, labelResourceName);
for (const url of adUrls.values()) {
urls.add(url);
}
}
if (options.checkKeywordUrls) {
const keywordUrls =
getCheckedKeywordUrls(options.checkPausedKeywords, labelResourceName);
for (const url of keywordUrls.values()) {
urls.add(url);
}
}
if (options.checkSitelinkUrls) {
const sitelinkUrls =
getCheckedSitelinkUrls(options.checkPausedSitelinks, labelResourceName);
for (const url of sitelinkUrls.values()) {
urls.add(url);
}
}
return urls;
}
/**
* Returns a set of ad URLs that have already been processed. This ensures we
* only process unique URLs.
*
* @param {boolean} checkPaused Whether to include ads that are paused.
* @param {string} labelResourceName The label to identify processed/unprocessed
* ads.
* @return {!Set<string>} A set of all URLs that have already been checked.
*/
function getCheckedAdUrls(checkPaused, labelResourceName) {
const urls = new Set();
let conditions = [];
if (checkPaused) {
conditions.push(`ad_group_ad.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`ad_group_ad.status = 'ENABLED'`);
conditions.push(`campaign.status = 'ENABLED'`);
conditions.push(`ad_group.status = 'ENABLED'`);
}
conditions.push(`ad_group_ad.labels CONTAINS ALL ('${labelResourceName}')`);
const query =
`SELECT ad_group_ad.ad.final_urls, ad_group_ad.ad.final_mobile_urls from ad_group_ad where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
for (const row of rows) {
if (row.adGroupAd.ad.finalUrls) {
urls.add(row.adGroupAd.ad.finalUrls[0]);
}
if (row.adGroupAd.ad.finalMobileUrls) {
urls.add(row.adGroupAd.ad.finalMobileUrls[0]);
}
}
return urls;
}
/**
* Returns a set of keyword URLs that have already been processed. This ensures
* we only process unique URLs.
*
* @param {boolean} checkPaused Whether to include keywords that are paused.
* @param {string} labelResourceName The label to identify processed/unprocessed
* keywords.
* @return {!Set<string>} A set of all URLs that have already been checked.
*/
function getCheckedKeywordUrls(checkPaused, labelResourceName) {
const urls = new Set();
let conditions = [];
if (checkPaused) {
conditions.push(`ad_group_criterion.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`ad_group_criterion.status = 'ENABLED'`);
conditions.push(`campaign.status = 'ENABLED'`);
conditions.push(`ad_group.status = 'ENABLED'`);
}
conditions.push(
`ad_group_criterion.labels CONTAINS ALL ('${labelResourceName}')`);
conditions.push(`ad_group_criterion.type = 'KEYWORD'`);
conditions.push(`ad_group_criterion.negative = false`);
const query =
`SELECT ad_group_criterion.final_urls, ad_group_criterion.final_mobile_urls from ad_group_criterion where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
for (const row of rows) {
if (row.adGroupCriterion.finalUrls) {
urls.add(row.adGroupCriterion.finalUrls[0]);
}
if (row.adGroupCriterion.finalMobileUrls) {
urls.add(row.adGroupCriterion.finalMobileUrls[0]);
}
}
return urls;
}
/**
* Retrieves all final URLs and mobile final URLs for campaign and ad group
* sitelinks that have already been checked.
*
* @param {boolean} checkPaused Whether to include paused sitelinks or not.
* @param {string} labelResourceName The resource name of the label applied to
* campaigns and ad groups whose sitelinks have already been checked.
* @return {!Set.<string>} A set of URLs that have already been checked.
*/
function getCheckedSitelinkUrls(checkPaused, labelResourceName) {
const urls = new Set();
const campaignResourceNames =
getCheckedCampaignResourceNames(checkPaused, labelResourceName);
while (campaignResourceNames.length > 0) {
// Load campaigns 10k at a time.
let someCampaignResourceNames = campaignResourceNames.splice(0, 10000);
let query = `SELECT campaign_asset.campaign, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from campaign_asset where campaign_asset.campaign IN (${
someCampaignResourceNames.join(
', ')}) and campaign_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
let rows = AdsApp.search(query);
for (const row of rows) {
if (row.asset.finalUrls) {
urls.add(row.asset.finalUrls[0]);
}
if (row.asset.finalMobileUrls) {
urls.add(row.asset.finalMobileUrls[0]);
}
}
}
const adGroupResourceNames =
getCheckedAdGroupResourceNames(checkPaused, labelResourceName);
while (adGroupResourceNames.length > 0) {
// Load ad groups 10k at a time.
let someAdGroupResourceNames = adGroupResourceNames.splice(0, 10000);
query = `SELECT ad_group_asset.ad_group, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from ad_group_asset where ad_group_asset.ad_group IN (${
someAdGroupResourceNames.join(
', ')}) and ad_group_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
rows = AdsApp.search(query);
for (const row of rows) {
if (row.asset.finalUrls) {
urls.add(row.asset.finalUrls[0]);
}
if (row.asset.finalMobileUrls) {
urls.add(row.asset.finalMobileUrls[0]);
}
}
}
return urls;
}
/**
* A helper function that returns a list of resource names for campaigns that
* have already been processed. See getCampaignResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused campaigns.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of campaign resource names.
*/
function getCheckedCampaignResourceNames(checkPaused, labelResourceName) {
return getCampaignResourceNames(true, checkPaused, labelResourceName);
}
/**
* A helper function that returns a list of resource names for campaigns that
* have not been processed yet. See getCampaignResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused campaigns.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of campaign resource names.
*/
function getUncheckedCampaignResourceNames(checkPaused, labelResourceName) {
return getCampaignResourceNames(false, checkPaused, labelResourceName);
}
/**
* Returns a list of campaign resource names that match the specified criteria.
* In order to efficiently fetch campaign sitelinks, we need to first get a list
* of relevant campaigns. This is because we cannot filter campaign sitelinks by
* campaign attributes (only campaign resource names).
*
* NOTE: The resource names are returned quoted. This simplifies usage as
* resource names need to be quoted for use in queries.
*
* @param {boolean} alreadyChecked True if we want campaigns that have been
* processed already.
* @param {boolean} checkPaused True if we want to include paused campaigns.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of campaign resource names.
*/
function getCampaignResourceNames(
alreadyChecked, checkPaused, labelResourceName) {
const conditions = [];
// If we want campaigns we already processed, we want to see if the label is
// applied.
if (alreadyChecked) {
conditions.push(`campaign.labels CONTAINS ALL ('${labelResourceName}')`);
} else {
// If we want campaigns we haven't processed, we want to see if the label
// was not applied.
conditions.push(`campaign.labels CONTAINS NONE ('${labelResourceName}')`);
}
if (checkPaused) {
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`campaign.status = 'ENABLED'`);
}
const query = `SELECT campaign.resource_name from campaign where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
const resourceNames = [];
for (const row of rows) {
resourceNames.push(`"${row.campaign.resourceName}"`);
}
return resourceNames;
}
/**
* A helper function that returns a list of resource names for ad groups that
* have already been processed. See getAdGroupResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused ad groups.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of ad group resource names.
*/
function getCheckedAdGroupResourceNames(checkPaused, labelResourceName) {
return getAdGroupResourceNames(true, checkPaused, labelResourceName);
}
/**
* A helper function that returns a list of resource names for ad groups that
* have not been processed yet. See getAdGroupResourceNames for more details.
*
* @param {boolean} checkPaused True if we want to include paused ad groups.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of ad group resource names.
*/
function getUncheckedAdGroupResourceNames(checkPaused, labelResourceName) {
return getAdGroupResourceNames(false, checkPaused, labelResourceName);
}
/**
* Returns a list of ad group resource names that match the specified criteria.
* In order to efficiently fetch ad group sitelinks, we need to first get a list
* of relevant ad groups. This is because we cannot filter ad group sitelinks by
* ad group attributes (only ad group resource names).
*
* NOTE: The resource names are returned quoted. This simplifies usage as
* resource names need to be quoted for use in queries.
*
* @param {boolean} alreadyChecked True if we want ad groups that have been
* processed already.
* @param {boolean} checkPaused True if we want to include paused ad groups.
* @param {string} labelResourceName The resource name of the label to check.
* @return {Array.string!} An array of ad group resource names.
*/
function getAdGroupResourceNames(
alreadyChecked, checkPaused, labelResourceName) {
const conditions = [];
// If we want ad groups we already processed, we want to see if the label is
// applied.
if (alreadyChecked) {
conditions.push(`ad_group.labels CONTAINS ALL ('${labelResourceName}')`);
} else {
// If we want ad groups we haven't processed, we want to see if the label
// was not applied.
conditions.push(`ad_group.labels CONTAINS NONE ('${labelResourceName}')`);
}
if (checkPaused) {
conditions.push(`ad_group.status IN ('ENABLED', 'PAUSED')`);
conditions.push(`campaign.status IN ('ENABLED', 'PAUSED')`);
} else {
conditions.push(`ad_group.status = 'ENABLED'`);
conditions.push(`campaign.status = 'ENABLED'`);
}
const query = `SELECT ad_group.resource_name from ad_group where ${
conditions.join(' AND ')}`;
const rows = AdsApp.search(query);
const resourceNames = [];
for (const row of rows) {
resourceNames.push(`"${row.adGroup.resourceName}"`);
}
return resourceNames;
}
/**
* Retrieves all final URLs and mobile final URLs in the account across ads,
* keywords, and sitelinks, and checks their response code. Does not check
* previously checked URLs.
*
* @param {string} labelResourceName The resource name of the label to check.
* @param {Set.<string>!} checkedUrls A set of previously checked URLs.
* @param {!Array.<!Object>} urlChecks An array into which the results of each
* URL check will be inserted.
* @param {!Object} options Dictionary of options.
* @return {boolean} True if all URLs were checked.
*/
function checkUrls(labelResourceName, checkedUrls, urlChecks, options) {
let completed = true;
if (options.checkAdUrls) {
completed = completed &&
checkAdUrls(labelResourceName, checkedUrls, urlChecks, options);
if (!completed) return completed;
}
if (options.checkKeywordUrls) {
completed = completed &&
checkKeywordUrls(labelResourceName, checkedUrls, urlChecks, options);
if (!completed) return completed;
}
if (options.checkSitelinkUrls) {
completed = completed &&
checkSitelinkUrls(labelResourceName, checkedUrls, urlChecks, options);
}
return completed;
}
/**
* Retrieves all final URLs and mobile final URLs in a selector and checks them
* for a valid response code. Does not check previously checked URLs. Labels the
* entity that it was checked, if possible.
*
* @param {string} labelResourceName The resource name of the label for
* annotating
* ads when they are processed.
* @param {Set<string>!} checkedUrls A map of previously checked URLs with the
* URL as the key.
* @param {!Array.<!Object>} urlChecks An array into which the results of each
* URL check will be inserted.
* @param {!Map<string, !Object>} options options Dictionary of options.
* @return {boolean} True if all URLs were checked.
*/
function checkAdUrls(labelResourceName, checkedUrls, urlChecks, options) {
const customerId = AdsApp.currentAccount().getCustomerId();
const selector =
AdsApp.ads()
.withCondition(
`ad_group_ad.labels CONTAINS NONE ('${labelResourceName}')`);
if (options.checkPausedAds) {
selector.withCondition(`ad_group_ad.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`campaign.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
selector.withCondition(`ad_group_ad.status = 'ENABLED'`);
selector.withCondition(`campaign.status = 'ENABLED'`);
selector.withCondition(`ad_group.status = 'ENABLED'`);
}
let urlsChecked = 0;
let urlsSkipped = 0;
const ads = selector.get();
console.log(`Processing ${ads.totalNumEntities()} ads.`);
for (const ad of ads) {
const url = ad.urls().getFinalUrl();
const mobileUrl = ad.urls().getMobileFinalUrl();
const entityDetails = {
entityType: 'Ad',
campaign: ad.getCampaign().getName(),
adGroup: ad.getAdGroup().getName(),
ad: getAdAsText(ad),
keyword: '',
sitelink: ''
};
if (url && !checkedUrls.has(url)) {
urlsChecked++;
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'Ad',
campaign: ad.getCampaign().getName(),
adGroup: ad.getAdGroup().getName(),
ad: getAdAsText(ad),
keyword: '',
sitelink: ''
});
checkedUrls.add(url);
} else if (url) {
urlsSkipped++;
}
if (mobileUrl && !checkedUrls.has(mobileUrl)) {
urlsChecked++;
const responseCode = requestUrl(mobileUrl, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: mobileUrl,
responseCode: responseCode,
entityType: 'Ad',
campaign: ad.getCampaign().getName(),
adGroup: ad.getAdGroup().getName(),
ad: getAdAsText(ad),
keyword: '',
sitelink: ''
});
checkedUrls.add(mobileUrl);
} else if (mobileUrl) {
urlsSkipped++;
}
ad.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${urlsChecked} ad urls. Skipped ${
urlsSkipped} urls (duplicate).`);
return false;
}
}
console.log(`Checked ${urlsChecked} ad urls. Skipped ${
urlsSkipped} urls (duplicate).`);
return true;
}
/**
* Helper function to grab relevant text for an ad. The relevant text is
* different for each type of ad.
*
* @param {Ad!} ad The ad we need text for.
* @return {string} Relevant text to describe the ad.
*/
function getAdAsText(ad) {
// There is no AdTypeSpace method for textAd
if (ad.getType() === 'TEXT_AD') {
return ad.getHeadline();
} else if (ad.isType().expandedTextAd()) {
const eta = ad.asType().expandedTextAd();
return eta.getHeadlinePart1() + ' - ' + eta.getHeadlinePart2();
} else if (ad.isType().gmailImageAd()) {
return ad.asType().gmailImageAd().getName();
} else if (ad.isType().gmailMultiProductAd()) {
return ad.asType().gmailMultiProductAd().getHeadline();
} else if (ad.isType().gmailSinglePromotionAd()) {
return ad.asType().gmailSinglePromotionAd().getHeadline();
} else if (ad.isType().html5Ad()) {
return ad.asType().html5Ad().getName();
} else if (ad.isType().imageAd()) {
return ad.asType().imageAd().getName();
} else if (ad.isType().responsiveDisplayAd()) {
return ad.asType().responsiveDisplayAd().getLongHeadline();
} else if (ad.isType().responsiveSearchAd()) {
return ad.asType().responsiveSearchAd().getHeadlines().map(h => h.text).join(', ');
}
return 'N/A';
}
/**
* Check URLs for all keywords in the account. Skip keywords that were already
* processed. Skip URLs that were already checked. The results of each URL that
* is checked is stored in urlChecks.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* keywords.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining keywords were processed. Returns
* false otherwise.
*/
function checkKeywordUrls(labelResourceName, checkedUrls, urlChecks, options) {
const customerId = AdsApp.currentAccount().getCustomerId();
const selector =
AdsApp.keywords()
.withCondition(`ad_group_criterion.labels CONTAINS NONE ('${
labelResourceName}')`);
if (options.checkPausedKeywords) {
selector.withCondition(
`ad_group_criterion.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`campaign.status IN ('ENABLED', 'PAUSED')`);
selector.withCondition(`ad_group.status IN ('ENABLED', 'PAUSED')`);
} else {
selector.withCondition(`ad_group_criterion.status = 'ENABLED'`);
selector.withCondition(`campaign.status = 'ENABLED'`);
selector.withCondition(`ad_group.status = 'ENABLED'`);
}
let urlsSkipped = 0;
let urlsChecked = 0;
let keywordsWithNoUrls = 0;
const keywords = selector.get();
console.log(`Processing ${keywords.totalNumEntities()} keywords.`);
for (const keyword of keywords) {
const url = keyword.urls().getFinalUrl();
const mobileUrl = keyword.urls().getMobileFinalUrl();
const entityDetails = {
entityType: 'Keyword',
campaign: keyword.getCampaign().getName(),
adGroup: keyword.getAdGroup().getName(),
ad: '',
keyword: keyword.getText(),
sitelink: ''
};
if (url && !checkedUrls.has(url)) {
urlsChecked++;
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'Keyword',
campaign: keyword.getCampaign().getName(),
adGroup: keyword.getAdGroup().getName(),
ad: '',
keyword: keyword.getText(),
sitelink: ''
});
checkedUrls.add(url);
} else if (url) {
urlsSkipped++;
}
if (mobileUrl && !checkedUrls.has(mobileUrl)) {
urlsChecked++;
const responseCode = requestUrl(mobileUrl, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: mobileUrl,
responseCode: responseCode,
entityType: 'Keyword',
campaign: keyword.getCampaign().getName(),
adGroup: keyword.getAdGroup().getName(),
ad: '',
keyword: keyword.getText(),
sitelink: ''
});
checkedUrls.add(mobileUrl);
} else if (mobileUrl) {
urlsSkipped++;
}
if (!url && !mobileUrl) {
keywordsWithNoUrls++;
}
keyword.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${urlsChecked} keyword urls. Skipped ${keywordsWithNoUrls} keywords (no urls). Skipped ${urlsSkipped} urls (duplicate).`);
return false;
}
}
console.log(`Checked ${urlsChecked} keyword urls. Skipped ${keywordsWithNoUrls} keywords (no urls). Skipped ${urlsSkipped} urls (duplicate).`);
return true;
}
/**
* Helper function that checks all sitelink URLs in the account. See
* checkCampaignSitelinks and checkAdGroupSitelinks for more details.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* campaigns or ad groups.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining campaigns and ad groups were
* processed. Returns false otherwise.
*/
function checkSitelinkUrls(labelResourceName, checkedUrls, urlChecks, options) {
const uncheckedUrls = loadUncheckedSitelinkUrls(
options.checkPausedSitelinks, labelResourceName);
const checkedCampaigns = checkCampaignSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options);
if (!checkedCampaigns) {
return false;
}
const checkedAdGroups = checkAdGroupSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options);
return checkedAdGroups;
}
/**
* Check URLs for all campaign sitelinks in the account. Skip campaigns that
* were already processed. Skip URLs that were already checked. The results of
* each URL that is checked is stored in urlChecks.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* campaigns.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Map<string, !Array.<!Object>>} uncheckedUrls A map of resource name
* to URLs to check for all campaigns
* that have not been processed yet.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining campaigns were processed. Returns
* false otherwise.
*/
function checkCampaignSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options) {
const campaignResourceNames = getUncheckedCampaignResourceNames(
options.checkPausedSitelinks, labelResourceName);
let urlsSkipped = 0;
let urlsChecked = 0;
const customerId = AdsApp.currentAccount().getCustomerId();
while (campaignResourceNames.length > 0) {
// Load campaigns 10k at a time.
let someCampaignResourceNames = campaignResourceNames.splice(0, 10000);
let campaigns = AdsApp.campaigns()
.withCondition(`campaign.resource_name IN (${
someCampaignResourceNames.join(', ')})`)
.get();
for (const campaign of campaigns) {
const resourceName = campaign.getResourceName();
if (!uncheckedUrls.has(resourceName)) {
continue;
}
const urlsToCheck = uncheckedUrls.get(resourceName);
for (const urlInfo of urlsToCheck) {
const url = urlInfo.url;
const linkText = urlInfo.linkText;
if (checkedUrls.has(url)) {
// already checked this url
urlsSkipped++;
} else {
urlsChecked++;
const entityDetails = {
entityType: 'CampaignSitelink',
campaign: campaign.getName(),
adGroup: '',
ad: '',
keyword: '',
sitelink: linkText
};
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'CampaignSitelink',
campaign: campaign.getName(),
adGroup: '',
ad: '',
keyword: '',
sitelink: linkText
});
checkedUrls.add(url);
}
}
campaign.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${
urlsChecked} campaign sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return false;
}
}
}
console.log(`Checked ${urlsChecked} campaign sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return true;
}
/**
* Check URLs for all ad group sitelinks in the account. Skip ad groups that
* were already processed. Skip URLs that were already checked. The results of
* each URL that is checked is stored in urlChecks.
*
* @param {string} labelResourceName The label to identify processed/unprocessed
* ad groups.
* @param {Set<string>!} checkedUrls The set of all URLs that have already been
* checked.
* @param {!Map<string, !Array.<!Object>>} uncheckedUrls A map of resource name
* to URLs to check for all ad groups
* that have not been processed yet.
* @param {!Array.<!Object>} urlChecks An array storing the results of all URLs
* that are checked.
* @param {!Map<string, !Object>} options A mapping storing various
* configuration options.
* @return {boolean} True if all remaining ad groups were processed. Returns
* false otherwise.
*/
function checkAdGroupSitelinks(
labelResourceName, checkedUrls, uncheckedUrls, urlChecks, options) {
const adGroupResourceNames = getUncheckedAdGroupResourceNames(
options.checkPausedSitelinks, labelResourceName);
let urlsSkipped = 0;
let urlsChecked = 0;
const customerId = AdsApp.currentAccount().getCustomerId();
while (adGroupResourceNames.length > 0) {
// Load ad groups 10k at a time.
let someAdGroupResourceNames = adGroupResourceNames.splice(0, 10000);
let adGroups = AdsApp.adGroups()
.withCondition(`ad_group.resource_name IN (${
someAdGroupResourceNames.join(', ')})`)
.get();
for (const adGroup of adGroups) {
const resourceName = adGroup.getResourceName();
if (!uncheckedUrls.has(resourceName)) {
continue;
}
const urlsToCheck = uncheckedUrls.get(resourceName);
for (const urlInfo of urlsToCheck) {
const url = urlInfo.url;
const linkText = urlInfo.linkText;
if (checkedUrls.has(url)) {
// already checked this url
urlsSkipped++;
} else {
urlsChecked++;
const entityDetails = {
entityType: 'AdGroupSitelink',
campaign: adGroup.getCampaign().getName(),
adGroup: adGroup.getName(),
ad: '',
keyword: '',
sitelink: linkText
};
const responseCode = requestUrl(url, options, entityDetails);
urlChecks.push({
customerId: customerId,
timestamp: new Date(),
url: url,
responseCode: responseCode,
entityType: 'AdGroupSitelink',
campaign: adGroup.getCampaign().getName(),
adGroup: adGroup.getName(),
ad: '',
keyword: '',
sitelink: linkText
});
checkedUrls.add(url);
}
}
adGroup.applyLabel(CONFIG.label);
if (aboutToTimeout()) {
console.log(`About to timeout. Checked ${
urlsChecked} adgroup sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return false;
}
}
}
console.log(`Checked ${urlsChecked} adgroup sitelink urls. Skipped ${
urlsSkipped} (duplicate urls).`);
return true;
}
/**
* Returns a map of resource name (campaign or ad group) to a list of sitelink
* URLs to check. For each URL, we also include the sitelink link text for
* reporting reasons. These URLs may have already been tested (when checking
* other URLs). The logic to ensure URLs are only checked once is handled
* elsewhere.
*
* @param {boolean} checkPaused Whether to include sitelinks that are "paused"
* (whether the campaign
* or ad group is paused)
* @param {string} labelResourceName The label to identify processed/unprocessed
* campaigns/ad groups.
* @return {!Map<string, !Array.<!Object>>} A map of resource name to an array
* of objects ({url, linkText}).
*/
function loadUncheckedSitelinkUrls(checkPaused, labelResourceName) {
const resourceToUrlMap = new Map();
const campaignResourceNames =
getUncheckedCampaignResourceNames(checkPaused, labelResourceName);
while (campaignResourceNames.length > 0) {
// Load campaigns 10k at a time.
let someCampaignResourceNames = campaignResourceNames.splice(0, 10000);
let query = `SELECT campaign_asset.campaign, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from campaign_asset where campaign_asset.campaign IN (${
someCampaignResourceNames.join(
', ')}) and campaign_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
let rows = AdsApp.search(query);
for (const row of rows) {
const campaign = row.campaignAsset.campaign;
const finalUrl = row.asset.finalUrls[0];
const linkText = row.asset.sitelinkAsset.linkText;
if (!resourceToUrlMap.has(campaign)) {
resourceToUrlMap.set(campaign, []);
}
const urls = resourceToUrlMap.get(campaign);
// Sitelinks always have final url but might not have final mobile url
urls.push({url: finalUrl, linkText: linkText});
if (row.asset.finalMobileUrls) {
urls.push({url: row.asset.finalMobileUrls[0], linkText: linkText});
}
resourceToUrlMap.set(campaign, urls);
}
}
const adGroupResourceNames =
getUncheckedAdGroupResourceNames(checkPaused, labelResourceName);
while (adGroupResourceNames.length > 0) {
// Load ad groups 10k at a time.
let someAdGroupResourceNames = adGroupResourceNames.splice(0, 10000);
query = `SELECT ad_group_asset.ad_group, asset.final_mobile_urls, asset.final_urls, asset.sitelink_asset.link_text from ad_group_asset where ad_group_asset.ad_group IN (${
someAdGroupResourceNames.join(
', ')}) and ad_group_asset.status != 'REMOVED' and asset.type = 'SITELINK'`;
rows = AdsApp.search(query);
for (const row of rows) {
const adGroup = row.adGroupAsset.adGroup;
const finalUrl = row.asset.finalUrls[0];
const linkText = row.asset.sitelinkAsset.linkText;
if (!resourceToUrlMap.has(adGroup)) {
resourceToUrlMap.set(adGroup, []);
}
const urls = resourceToUrlMap.get(adGroup);
urls.push({url: finalUrl, linkText: linkText});
if (row.asset.finalMobileUrls) {
urls.push({url: row.asset.finalMobileUrls[0], linkText: linkText});
}
resourceToUrlMap.set(adGroup, urls);
}
}
return resourceToUrlMap;
}
/**
* Expands a URL that contains ValueTrack parameters such as {ifmobile:mobile}
* to all the combinations, and returns as an array. The following pairs of
* ValueTrack parameters are currently expanded:
* 1. {ifmobile:<...>} and {ifnotmobile:<...>} to produce URLs simulating
* clicks from either mobile or non-mobile devices.
* 2. {ifsearch:<...>} and {ifcontent:<...>} to produce URLs simulating
* clicks on either the search or display networks.
* Any other ValueTrack parameters or customer parameters are stripped out from
* the URL entirely.
*
* @param {string} url The URL which may contain ValueTrack parameters.
* @return {!Array.<string>} An array of one or more expanded URLs.
*/
function expandUrlModifiers(url) {
const ifRegex = /({(if\w+):([^}]+)})/gi;
const modifiers = {};
let matches;
let modifiedUrls;
while (matches = ifRegex.exec(url)) {
// Tags are case-insensitive, e.g. IfMobile is valid.
modifiers[matches[2].toLowerCase()] = {
substitute: matches[0],
replacement: matches[3]
};
}
if (Object.keys(modifiers).length) {
let mobileCombinations;
if (modifiers.ifmobile || modifiers.ifnotmobile) {
mobileCombinations =
pairedUrlModifierReplace(modifiers, 'ifmobile', 'ifnotmobile', url);
} else {
mobileCombinations = [url];
}
// Store in a map on the offchance that there are duplicates.
const combinations = {};
for (const url of mobileCombinations) {
if (modifiers.ifsearch || modifiers.ifcontent) {
for (const modifiedUrl of
pairedUrlModifierReplace(modifiers, 'ifsearch', 'ifcontent', url)) {
combinations[modifiedUrl] = true;
}
} else {
combinations[url] = true;
}
}
modifiedUrls = Object.keys(combinations);
} else {
modifiedUrls = [url];
}
// Remove any custom parameters
return modifiedUrls.map(url => url.replace(/{[\w_+:]+}/g, ''));
}
/**
* Return a pair of URLs, where each of the two modifiers is mutually exclusive,
* one for each combination. e.g. Evaluating ifmobile and ifnotmobile for a
* mobile and a non-mobile scenario.
*
* @param {!Object} modifiers A map of ValueTrack modifiers.
* @param {string} modifier1 The modifier to honour in the URL.
* @param {string} modifier2 The modifier to remove from the URL.
* @param {string} url The URL potentially containing ValueTrack parameters.
* @return {!Array.<string>} A pair of URLs, as a list.
*/
function pairedUrlModifierReplace(modifiers, modifier1, modifier2, url) {
return [
urlModifierReplace(modifiers, modifier1, modifier2, url),
urlModifierReplace(modifiers, modifier2, modifier1, url)
];
}
/**
* Produces a URL where the first {if...} modifier is set, and the second is
* deleted.
*
* @param {!Object} mods A map of ValueTrack modifiers.
* @param {string} mod1 The modifier to honour in the URL.
* @param {string} mod2 The modifier to remove from the URL.
* @param {string} url The URL potentially containing ValueTrack parameters.
* @return {string} The resulting URL with substitutions.
*/
function urlModifierReplace(mods, mod1, mod2, url) {
const modUrl = mods[mod1] ?
url.replace(mods[mod1].substitute, mods[mod1].replacement) :
url;
return mods[mod2] ? modUrl.replace(mods[mod2].substitute, '') : modUrl;
}
/**
* Requests a given URL. Retries if the UrlFetchApp QPS limit was reached,
* exponentially backing off on each retry. Throws an exception if it reaches
* the maximum number of retries. Throws an exception if the UrlFetchApp daily
* quota limit was reached.
*
* @param {string} url The URL to test.
* @param {!Object} options The options loaded from the configuration sheet.
* @param {!Object} entityDetails Details of the entity, e.g. type, name etc.
* @return {number|string} The response code received when requesting the URL,
* or an error message.
*/
function requestUrl(url, options, entityDetails) {
const expandedUrls = expandUrlModifiers(url);
let responseCode;
let sleepTime = QUOTA_CONFIG.INIT_SLEEP_TIME;
let numTries = 0;
for (const expandedUrl of expandedUrls) {
while (numTries < QUOTA_CONFIG.MAX_TRIES && !responseCode) {
try {
// If UrlFetchApp.fetch() throws an exception, responseCode will remain
// undefined.
const response =
UrlFetchApp.fetch(expandedUrl, {muteHttpExceptions: true});
responseCode = response.getResponseCode();
if (options.validCodes.indexOf(responseCode) !== -1) {
if (options.useSimpleFailureStrings &&
bodyContainsFailureStrings(response, options.failureStrings)) {
return 'Failure string detected';
} else if (
options.useCustomValidation &&
!isValidResponse(url, response, options, entityDetails)) {
return 'Custom validation failed';
}
}
if (THROTTLE > 0) {
Utilities.sleep(THROTTLE * MILLISECONDS_PER_SECOND);
}
} catch (e) {
if (e.message.indexOf(
'Service invoked too many times in a short time:') !== -1) {
Utilities.sleep(sleepTime);
sleepTime *= QUOTA_CONFIG.BACKOFF_FACTOR;
} else if (
e.message.indexOf('Service invoked too many times:') !== -1) {
throw EXCEPTIONS.LIMIT;
} else {
return e.message;
}
}
numTries++;
}
}
if (!responseCode) {
throw EXCEPTIONS.QPS;
} else {
return responseCode;
}
}
/**
* Searches the body of a HTTP response for any occurrence of a "failure string"
* as defined in the configuration spreadsheet. For example, "Out of stock".
*
* @param {!Object} response The response from the UrlFetchApp request.
* @param {!Array.<string>} failureStrings A list of failure strings.
* @return {boolean} Returns true if at least one failure string found.
*/
function bodyContainsFailureStrings(response, failureStrings) {
const contentText = response.getContentText() || '';
// Whilst searching for each separate failure string across the body text
// separately may not be the most efficient, it is simple, and tests suggest
// it is not overly poor performance-wise.
return failureStrings.some(
failureString => contentText.indexOf(failureString) !== -1);
}
/**
* @return {boolean} True iff there is less than TIMEOUT_BUFFER seconds left in
* the execution.
*/
function aboutToTimeout() {
return AdsApp.getExecutionInfo().getRemainingTime() < TIMEOUT_BUFFER;
}
/**
* Returns the number of days between two dates.
*
* @param {!Object} from The older Date object.
* @param {!Object} to The newer (more recent) Date object.
* @return {number} The number of days between the given dates (possibly
* fractional).
*/
function dayDifference(from, to) {
return (to.getTime() - from.getTime()) / (24 * 3600 * 1000);
}
/**
* Given a label name, ensure the label is already in the account. If the label
* is not present *and* this is a live execution, go ahead and create the label.
* If this is a preview, throw an error instructing the user to create the
* label.
*
* @param {string} labelName The name of the label to annotate processed
* entities with.
*/
function ensureLabel(labelName) {
const labels = AdsApp.labels()
.withCondition(`label.name = '${labelName}'`)
.withCondition(`label.status != 'REMOVED'`)
.get();
if (!labels.hasNext()) {
if (!AdsApp.getExecutionInfo().isPreview()) {
AdsApp.createLabel(labelName);
} else {
throw `Label ${labelName} is missing and cannot be created in ` +
`preview mode. Please run the script or create the label manually.`;
}
}
}
/**
* Removes the label from the account. Since labels cannot be removed
* in preview mode, throws an exception in preview mode.
*
* @param {string} labelName The name of the label to remove.
*/
function removeLabel(labelName) {
if (AdsApp.getExecutionInfo().isPreview()) {
throw 'Cannot remove labels in preview mode. Please run the script or ' +
'remove the label manually.';
}
const labels = AdsApp.labels()
.withCondition(`label.name = '${labelName}'`)
.withCondition(`label.status != 'REMOVED'`)
.get();
if (labels.hasNext()) {
labels.next().remove();
}
}
/**
* Validates the provided spreadsheet URL to make sure that it's set up
* properly. Throws a descriptive error message if validation fails.
*
* @param {string} spreadsheeturl The URL of the spreadsheet to open.
* @return {!Object} The spreadsheet object itself, fetched from the URL.
* @throws {Error} If the spreadsheet URL hasn't been set
*/
function validateAndGetSpreadsheet(spreadsheeturl) {
if (spreadsheeturl === 'YOUR_SPREADSHEET_URL') {
throw new Error('Please specify a valid Spreadsheet URL. You can find' +
' a link to a template in the associated guide for this script.');
}
return SpreadsheetApp.openByUrl(spreadsheeturl);
}
/**
* Validates the provided email addresses to make sure it's not the default.
* Throws a descriptive error message if validation fails.
*
* @param {!Array.<string>} recipientEmails The list of email addresses.
* @throws {Error} If the list of email addresses is still the default
*/
function validateEmailAddresses(recipientEmails) {
if (recipientEmails && recipientEmails[0] === 'YOUR_EMAIL_HERE') {
throw new Error('Please either specify a valid email address or clear' +
' the RECIPIENT_EMAILS field.');
}
}