實驗報表

您可以透過下列兩種主要方式製作實驗報表：

  • 直接查詢實驗報表：查詢 experiment 資源的指標。這個選項會在單一回應中提供控制組和實驗組的指標，以及升幅和 p 值等統計比較資料。這是製作廣告活動內實驗報表的唯一方法。
  • 廣告活動報表：使用 campaign.experiment_type 區分基準和實驗廣告活動，查詢 campaign 資源的指標。這個選項僅適用於使用個別控制組和實驗組廣告活動的實驗，例如系統管理實驗。

本指南主要著重於直接實驗報表，適用於所有支援報表的實驗類型。

直接實驗報表

您可以直接查詢 experiment 資源，擷取控制組和實驗組的成效指標和統計比較資料。

指標和統計顯著程度

對於點擊、曝光、費用、轉換和轉換價值等核心指標，experiment 資源會在同一列中提供實驗組指標 (例如 metrics.clicks) 和對照組指標 (例如 metrics.control_clicks)。

此外，這項功能還提供欄位，協助您評估實驗組之間任何差異的統計顯著程度：

  • metrics.*_p_value：如果實驗對指標沒有實際影響，觀察到的結果發生的機率。p 值越低，表示統計顯著性越高。
  • metrics.*_point_estimate：與控制組相比，實驗組在指定指標中預估的提升百分比 (正或負)。與 margin_of_error 一起使用時，這些指標會描述信賴區間，並為預估的差異指定信賴水準。預估的數量為 (實驗組 / 控制組 - 1)。點估計值是信賴區間的中心。
  • metrics.*_margin_of_error：信賴區間的半徑，以 point_estimate 為中心。系統會根據預先設定的信賴水準計算，而信賴水準取決於實驗類型。

experiment 資源支援下列核心指標欄位，包括實驗組值、控制組值，以及先前列出的統計資料欄位：

  • clicks
  • impressions
  • cost_micros
  • conversions
  • cost_per_conversion
  • conversion_value
  • conversion_value_per_cost

就轉換而言，統計欄位可透過下列 absolute_change 欄位取得，而非相對值：

如需協助建構 experiment 資源的有效查詢，請使用 Google Ads 查詢建立工具。

查詢示例

下列 GAQL 查詢會擷取實驗的重要指標：

SELECT
  experiment.experiment_id,
  experiment.name,
  experiment.type,
  metrics.clicks,
  metrics.control_clicks,
  metrics.clicks_point_estimate,
  metrics.clicks_margin_of_error,
  metrics.clicks_p_value,
  metrics.conversions,
  metrics.control_conversions,
  metrics.conversions_absolute_change_point_estimate,
  metrics.conversions_absolute_change_margin_of_error,
  metrics.conversions_absolute_change_p_value
FROM experiment
WHERE experiment.experiment_id = EXPERIMENT_ID

解讀結果

您可以根據 p 值、點估計值和誤差範圍欄位，判斷實驗是否產生具統計顯著性的結果。舉例來說，如果 conversions_absolute_change_p_value 低於您選擇的門檻 (例如 95% 信賴水準的門檻為 0.05)，且 conversions_absolute_change_point_estimate - conversions_absolute_change_margin_of_error 大於零，表示實驗組的轉換次數明顯優於控制組。

以下 Python 程式碼片段示範如何根據 p 值和升幅估算值評估結果：

Java

private void evaluateExperiment(
    GoogleAdsClient googleAdsClient, long customerId, GoogleAdsRow row) {
  Metrics metrics = row.getMetrics();
  String experimentResourceName = row.getExperiment().getResourceName();

  // 1. Evaluate conversion success as a primary success signal if available.
  // - Point Estimate: Represents the estimated average lift or difference in conversions.
  // - Margin of Error: Outlines the confidence interval bounds. Note that the margin_of_error
  //   provided by the API is calculated for a preset confidence level which is set based on the
  //   experiment type.
  // - Lower Bound: (Point Estimate - Margin of Error). If this value is above 0,
  //   we have statistical significance that performance has improved.
  double convPValue = metrics.getConversionsAbsoluteChangePValue();
  double convLift = metrics.getConversionsAbsoluteChangePointEstimate();
  double convError = metrics.getConversionsAbsoluteChangeMarginOfError();
  double convLowerBound = convLift - convError;

  if (convPValue <= P_VALUE_THRESHOLD) {
    if (convLowerBound > 0) {
      System.out.printf(
          "Significant Success: Conversions increased. Even at the lower bound, the lift is %.2f."
              + " Promoting changes.%n",
          convLowerBound);
      promoteExperiment(googleAdsClient, customerId, experimentResourceName);
      return;
    } else if ((convLift + convError) < 0) {
      System.out.printf(
          "Significant Decline: Even the upper bound (%.2f) is below zero. Ending experiment.%n",
          convLift + convError);
      endExperiment(googleAdsClient, customerId, experimentResourceName);
      return;
    }
  }

  // 2. Fall back to evaluating click metrics if conversions are inconclusive.
  double clickPValue = metrics.getClicksPValue();
  double clickLift = metrics.getClicksPointEstimate();
  double clickError = metrics.getClicksMarginOfError();
  double clickLowerBound = clickLift - clickError;

  if (clickPValue <= P_VALUE_THRESHOLD && clickLowerBound > 0) {
    System.out.printf("Click volume is significantly up (+%.1f%%).%n", clickLift * 100);

    // Graduation is only supported for separate campaign experiments, not
    // intra-campaign experiments where there is no separate treatment campaign.
    ExperimentType experimentType = row.getExperiment().getType();
    if (experimentType != ExperimentType.ADOPT_BROAD_MATCH_KEYWORDS
        && experimentType != ExperimentType.ADOPT_AI_MAX) {
      System.out.println("Graduating treatment campaign for further manual analysis.");
      graduateExperiment(googleAdsClient, customerId, experimentResourceName);
    } else {
      System.out.println(
          "Intra-campaign trial detected: graduation is not supported. Continuing to run the"
              + " experiment to gather more conversion data.");
    }
  } else {
    // 3. Print status if no action was taken.
    System.out.printf(
        "Inconclusive: No significant lift in Conversions (p=%.2f) or Clicks (p=%.2f). Current"
            + " estimated lift: %.2f +/- %.2f. Allowing the experiment to continue running.%n",
        convPValue, clickPValue, convLift, convError);
  }
}

C#

private static void EvaluateExperiment(GoogleAdsClient client, long customerId, GoogleAdsRow row)
{
    // This function evaluates performance metrics and immediately takes action
    // to update the experiment's status (promote, end, or graduate) if
    // statistical significance thresholds are met.
    var metrics = row.Metrics;
    string experimentResourceName = row.Experiment.ResourceName;

    bool hasConvMetrics = metrics.HasConversionsAbsoluteChangePValue
        && metrics.HasConversionsAbsoluteChangePointEstimate
        && metrics.HasConversionsAbsoluteChangeMarginOfError;

    bool hasClickMetrics = metrics.HasClicksPValue
        && metrics.HasClicksPointEstimate
        && metrics.HasClicksMarginOfError;

    // 1. Evaluate conversion success as a primary success signal if available.
    // - Point Estimate: Represents the estimated average lift or difference in conversions.
    // - Margin of Error: Outlines the confidence interval bounds. Note that the margin_of_error
    //   provided by the API is calculated for a preset confidence level which is set based on
    //   the experiment type.
    // - Lower Bound: (Point Estimate - Margin of Error). If this value is above 0,
    //   we have statistical significance that performance has improved.
    if (hasConvMetrics)
    {
        double convPValue = metrics.ConversionsAbsoluteChangePValue;
        double convLift = metrics.ConversionsAbsoluteChangePointEstimate;
        double convError = metrics.ConversionsAbsoluteChangeMarginOfError;
        double convLowerBound = convLift - convError;

        if (convPValue <= P_VALUE_THRESHOLD)
        {
            if (convLowerBound > 0)
            {
                Console.WriteLine(
                    $"Significant Success: Conversions increased. Even at the lower" +
                    $" bound, the lift is {convLowerBound:F2}. Promoting changes.");
                PromoteExperiment(client, customerId, experimentResourceName);
                return;
            }
            else if ((convLift + convError) < 0)
            {
                Console.WriteLine(
                    $"Significant Decline: Even the upper bound ({convLift + convError:F2}) " +
                    $"is below zero. Ending experiment.");
                EndExperiment(client, customerId, experimentResourceName);
                return;
            }
        }
    }

    // 2. Evaluate click volume as a secondary signal.
    // This is helpful as an early indicator or for lower-volume accounts.
    if (hasClickMetrics)
    {
        double clickPValue = metrics.ClicksPValue;
        double clickLift = metrics.ClicksPointEstimate;
        double clickError = metrics.ClicksMarginOfError;
        double clickLowerBound = clickLift - clickError;

        if (clickPValue <= P_VALUE_THRESHOLD && clickLowerBound > 0)
        {
            // We have a directional winner: high confidence in more traffic,
            // but not enough data to confirm conversion impact yet.
            Console.WriteLine(
                $"Click volume is significantly up (+{clickLift * 100:F1}%).");

            // Graduation is only supported for separate campaign experiments, not
            // intra-campaign experiments where there is no separate treatment campaign.
            if (row.Experiment.Type != ExperimentType.AdoptBroadMatchKeywords
                && row.Experiment.Type != ExperimentType.AdoptAiMax)
            {
                Console.WriteLine("Graduating treatment campaign for further manual analysis.");
                GraduateExperiment(client, customerId, experimentResourceName);
            }
            else
            {
                Console.WriteLine(
                    "Intra-campaign trial detected: graduation is not supported. " +
                    "Continuing to run the experiment to gather more conversion data.");
            }
            return;
        }
    }

    // 3. Print status if no action was taken.
    if (hasConvMetrics || hasClickMetrics)
    {
        string convStatus = hasConvMetrics
            ? $"Conversions (p={metrics.ConversionsAbsoluteChangePValue:F2}, " +
              $"lift={metrics.ConversionsAbsoluteChangePointEstimate:F2} +/- " +
              $"{metrics.ConversionsAbsoluteChangeMarginOfError:F2})"
            : "Conversions (not populated)";

        string clickStatus = hasClickMetrics
            ? $"Clicks (p={metrics.ClicksPValue:F2}, " +
              $"lift={metrics.ClicksPointEstimate:F2} +/- " +
              $"{metrics.ClicksMarginOfError:F2})"
            : "Clicks (not populated)";

        Console.WriteLine(
            $"Inconclusive: No significant action taken. {convStatus}, {clickStatus}. " +
            "Allowing the experiment to continue running.");
    }
    else
    {
        Console.WriteLine(
            "Conversion and click performance metrics are not yet populated. " +
            "Allowing the experiment to continue running.");
    }
}

PHP

This example is not yet available in PHP; you can take a look at the other languages.

Python

def evaluate_experiment(
    client: GoogleAdsClient, customer_id: str, row: GoogleAdsRow
) -> None:
    """Evaluates the performance of the experiment and updates it accordingly
    (for example, promotes, ends, or graduates).

    Checks conversion and click metrics against statistical significance thresholds
    to determine the appropriate action to take on the experiment.

    Args:
        client: an initialized GoogleAdsClient instance.
        customer_id: a client customer ID.
        row: a GoogleAdsRow containing the experiment and metrics.
    """
    # This function evaluates performance metrics and immediately takes action
    # to update the experiment's status (promote, end, or graduate) if
    # statistical significance thresholds are met.
    metrics = row.metrics
    experiment_resource_name = row.experiment.resource_name

    has_conv_metrics = (
        "conversions_absolute_change_p_value" in metrics
        and "conversions_absolute_change_point_estimate" in metrics
        and "conversions_absolute_change_margin_of_error" in metrics
    )
    has_click_metrics = (
        "clicks_p_value" in metrics
        and "clicks_point_estimate" in metrics
        and "clicks_margin_of_error" in metrics
    )

    # 1. Evaluate conversion success as a primary success signal if available.
    # - Point Estimate: Represents the estimated average lift or difference in conversions.
    # - Margin of Error: Outlines the confidence interval bounds. Note that the margin_of_error provided by the API is calculated for a preset confidence level which is set based on the experiment type.
    # - Lower Bound: (Point Estimate - Margin of Error). If this value is above 0,
    #   we have statistical significance that performance has improved.
    if has_conv_metrics:
        conv_p_value = metrics.conversions_absolute_change_p_value
        conv_lift = metrics.conversions_absolute_change_point_estimate
        conv_error = metrics.conversions_absolute_change_margin_of_error
        conv_lower_bound = conv_lift - conv_error

        if conv_p_value <= P_VALUE_THRESHOLD:
            if conv_lower_bound > 0:
                print(
                    "Significant Success: Conversions increased. Even at the lower"
                    f" bound, the lift is {conv_lower_bound:.2f}. Promoting"
                    " changes."
                )
                promote_experiment(
                    client, customer_id, experiment_resource_name
                )
                return
            elif (conv_lift + conv_error) < 0:
                print(
                    "Significant Decline: Even the upper bound"
                    f" ({conv_lift + conv_error:.2f}) is below zero. Ending"
                    " experiment."
                )
                end_experiment(client, customer_id, experiment_resource_name)
                return

        # 2. Evaluate click volume as a secondary signal.
        # This is helpful as an early indicator or for lower-volume accounts.
        click_p_value = metrics.clicks_p_value
        click_lift = metrics.clicks_point_estimate
        click_error = metrics.clicks_margin_of_error
        click_lower_bound = click_lift - click_error

        if click_p_value <= P_VALUE_THRESHOLD and click_lower_bound > 0:
            # We have a directional winner: high confidence in more traffic,
            # but not enough data to confirm conversion impact yet.
            print(f"Click volume is significantly up (+{click_lift*100:.1f}%).")

            # Graduation is only supported for separate campaign experiments, not
            # intra-campaign experiments where there is no separate treatment campaign.
            experiment_type_name = row.experiment.type_.name
            if (
                experiment_type_name != "ADOPT_BROAD_MATCH_KEYWORDS"
                and experiment_type_name != "ADOPT_AI_MAX"
            ):
                print(
                    "Graduating treatment campaign for further manual analysis."
                )
                graduate_experiment(
                    client, customer_id, experiment_resource_name
                )
            else:
                print(
                    "Intra-campaign trial detected: graduation is not supported. "
                    "Continuing to run the experiment to gather more conversion data."
                )
            return

    # 3. Print status if no action was taken.
    if has_conv_metrics or has_click_metrics:
        conv_status = (
            f"Conversions (p={metrics.conversions_absolute_change_p_value:.2f}, "
            f"lift={metrics.conversions_absolute_change_point_estimate:.2f} +/- "
            f"{metrics.conversions_absolute_change_margin_of_error:.2f})"
            if has_conv_metrics
            else "Conversions (not populated)"
        )
        click_status = (
            f"Clicks (p={metrics.clicks_p_value:.2f}, "
            f"lift={metrics.clicks_point_estimate:.2f} +/- "
            f"{metrics.clicks_margin_of_error:.2f})"
            if has_click_metrics
            else "Clicks (not populated)"
        )
        print(
            f"Inconclusive: No significant action taken. {conv_status}, {click_status}."
            " Allowing the experiment to continue running."
        )
    else:
        print(
            "Conversion and click performance metrics are not yet populated. "
            "Allowing the experiment to continue running."
        )

Ruby

This example is not yet available in Ruby; you can take a look at the other languages.

Perl

This example is not yet available in Perl; you can take a look at the other languages.

curl

相較於廣告活動報表的優勢

相較於分別查詢廣告活動報表，直接實驗報表具有下列優點：

  1. 集中式指標：在單一資料列中擷取控制組和實驗組的指標。
  2. 統計信賴資料：提供計算出的 p 值、點估計值和誤差範圍。
  3. 效率：不必手動合併或比較多份報表的結果。
  4. 廣告活動內支援：這是比較廣告活動內實驗的對照組與實驗組的唯一方法，流量會在單一廣告活動內分配。

廣告活動報表

對於會建立個別實驗組廣告活動的實驗 (例如 SEARCH_CUSTOM)，您可以查詢 campaign 資源，並使用 campaign.experiment_type 識別 BASE (控制組) 和 EXPERIMENT (實驗組) 廣告活動。如果您需要以更精細的層級 (例如廣告群組或關鍵字) 區隔指標，或查看 experiment 資源中沒有的廣告活動中繼資料，這個方法就很有用。不過，您需要手動進行成效比較和統計計算。

您無法使用廣告活動層級的報表，比較廣告活動內實驗的各個實驗組，因為流量分配是在單一廣告活動內部進行。查詢廣告活動內實驗的 campaign 只會傳回匯總總計。

最佳做法

  • 選取適當的信賴水準：設定較低的 p 值門檻可更快提供方向指引，尤其是在預算或轉換量較低的情況下。95% 的信賴度 (p 值 <= 0.05) 屬於學術標準，可能更適合在較長的時間範圍內取得更準確的結果。
  • 實驗執行時間要夠長：實驗應執行至少 4 週，以考量每週的成效週期、轉換延遲和學習期。
  • 給予適應期：如果廣告活動採用自動出價或測試新功能，請忽略前 1 到 2 週的資料，讓出價模型和流量重新調整至分割狀態。
  • 使用 50/50 分組：一般來說，50/50 的流量分配比例最快能得出具統計顯著性的結果。
  • 提前安排：將實驗開始日期設在 3 到 7 天後，以便廣告審查和核准程序有充足時間。
  • 每個廣告活動一次只能進行一項實驗。
上一頁
廣告活動組合實驗
下一頁
非同步錯誤