System-managed experiments

System-managed experiments are used to A/B test changes by creating one or more modifiable treatment campaigns that run alongside an original control campaign. This workflow is supported for the following ExperimentType values:

• SEARCH_CUSTOM: A custom experiment consisting of search campaigns.
• DISPLAY_CUSTOM: A custom experiment consisting of display campaigns.
• HOTEL_CUSTOM: A custom experiment consisting of hotel campaigns.
• YOUTUBE_CUSTOM: A custom experiment consisting of Video campaigns.
• PMAX_REPLACEMENT_SHOPPING: An experiment to test how your Shopping campaigns perform compared to Performance Max.

Setup

The setup for system-managed experiments follows these steps:

1. Create an Experiment.
2. Create ExperimentArm resources for control and treatment.
3. Modify the in_design_campaigns in the treatment arms.

1. Create an experiment

The first step in running an experiment using the Google Ads API is to create an Experiment. This resource defines some key information about the experiment you want to run. You don't specify any of the campaigns involved in the experiment at this step.

Here's an overview of some key fields for an Experiment:

• name: Each experiment must have a unique name.
• description: An optional field that you can use to reference later. Does not affect how the experiment runs.
• suffix: The suffix will be appended to the end of the names of the treatment campaigns so you can distinguish them from the control campaign. These concepts will be explained further in step 2.
• type: What type of experiment to run. For system-managed experiments, use SEARCH_CUSTOM, DISPLAY_CUSTOM, HOTEL_CUSTOM, YOUTUBE_CUSTOM, or PMAX_REPLACEMENT_SHOPPING.
• status: When creating an experiment, set this field to SETUP. Later on, once you begin the experiment, this field will let you check the current status.
• start_date and end_date: Specify when the experiment should start and end.
• sync_enabled: Disabled by default. If set to true, changes made to the original campaign while your experiment is running are automatically copied to the experiment campaign. Learn more.

Java

private String createExperimentResource(GoogleAdsClient googleAdsClient, long customerId) {
  ExperimentOperation operation =
      ExperimentOperation.newBuilder()
          .setCreate(
              Experiment.newBuilder()
                  // Name must be unique.
                  .setName("Example Experiment #" + getPrintableDateTime())
                  .setType(ExperimentType.SEARCH_CUSTOM)
                  .setSuffix("[experiment]")
                  .setStatus(ExperimentStatus.SETUP)
                  .build())
          .build();

  try (ExperimentServiceClient experimentServiceClient =
      googleAdsClient.getLatestVersion().createExperimentServiceClient()) {
    MutateExperimentsResponse response =
        experimentServiceClient.mutateExperiments(
            Long.toString(customerId), ImmutableList.of(operation));
    String experiment = response.getResults(0).getResourceName();
    System.out.printf("Created experiment with resource name '%s'%n", experiment);
    return experiment;
  }
}
CreateExperiment.java
      

C#

/// <summary>
/// Creates the experiment.
/// </summary>
/// <param name="client">The Google Ads client.</param>
/// <param name="customerId">The customer ID for which the call is made.</param>
/// <returns>The resource name of the newly created experiment.</returns>
private static string CreateAnExperiment(GoogleAdsClient client, long customerId)
{
    // Get the ExperimentService.
    ExperimentServiceClient experimentService = client.GetService(
        Services.V24.ExperimentService);

    // Creates the experiment.
    Experiment experiment = new Experiment()
    {
        // Name must be unique.
        Name = $"Example Experiment #{ExampleUtilities.GetRandomString()}",
        Type = ExperimentType.SearchCustom,
        Suffix = "[experiment]",
        Status = ExperimentStatus.Setup
    };

    // Creates the operation.
    ExperimentOperation operation = new ExperimentOperation()
    {
        Create = experiment
    };

    // Makes the API call.
    MutateExperimentsResponse response = experimentService.MutateExperiments(
        customerId.ToString(), new[] { operation });

    // Displays the result.
    string experimentResourceName = response.Results.First().ResourceName;

    Console.WriteLine($"Created experiment with resource name " +
        $"'{experimentResourceName}'.");
    return experimentResourceName;
}
CreateExperiment.cs
      

PHP

private static function createExperimentResource(
    ExperimentServiceClient $experimentServiceClient,
    int $customerId
): string {
    // Creates an experiment and its operation.
    $experiment = new Experiment([
        // Name must be unique.
        'name' => 'Example Experiment #' . Helper::getPrintableDatetime(),
        'type' => ExperimentType::SEARCH_CUSTOM,
        'suffix' => '[experiment]',
        'status' => ExperimentStatus::SETUP
    ]);
    $experimentOperation = new ExperimentOperation(['create' => $experiment]);

    // Issues a request to create the experiment.
    $response = $experimentServiceClient->mutateExperiments(
        MutateExperimentsRequest::build($customerId, [$experimentOperation])
    );
    $experimentResourceName = $response->getResults()[0]->getResourceName();
    print "Created experiment with resource name '$experimentResourceName'" . PHP_EOL;

    return $experimentResourceName;
}
CreateExperiment.php
      

Python

def create_experiment_resource(
    client: GoogleAdsClient, customer_id: str
) -> str:
    """Creates a new experiment resource.

    Args:
        client: an initialized GoogleAdsClient instance.
        customer_id: a client customer ID.

    Returns:
        the resource name for the new experiment.
    """
    experiment_operation: ExperimentOperation = client.get_type(
        "ExperimentOperation"
    )
    experiment: Experiment = experiment_operation.create

    experiment.name = f"Example Experiment #{uuid.uuid4()}"
    # We specify SEARCH_CUSTOM to create a standard search campaign experiment.
    # This type uses a standard draft-based workflow where the system automatically
    # creates a draft/in-design campaign for the treatment arm.
    experiment.type_ = client.enums.ExperimentTypeEnum.SEARCH_CUSTOM
    experiment.suffix = "[experiment]"
    experiment.status = client.enums.ExperimentStatusEnum.SETUP

    experiment_service: ExperimentServiceClient = client.get_service(
        "ExperimentService"
    )
    response: MutateExperimentsResponse = experiment_service.mutate_experiments(
        customer_id=customer_id, operations=[experiment_operation]
    )

    experiment_resource_name: str = response.results[0].resource_name
    print(f"Created experiment with resource name {experiment_resource_name}")

    return experiment_resource_name
      

Ruby

def create_experiment_resource(client, customer_id)
  operation = client.operation.create_resource.experiment do |e|
    # Name must be unique.
    e.name = "Example Experiment #{(Time.new.to_f * 1000).to_i}"
    e.type = :SEARCH_CUSTOM
    e.suffix = '[experiment]'
    e.status = :SETUP
  end

  response = client.service.experiment.mutate_experiments(
    customer_id: customer_id,
    operations: [operation],
  )

  experiment = response.results.first.resource_name
  puts "Created experiment with resource name #{experiment}."

  experiment
end
create_experiment.rb
      

Perl

sub create_experiment_resource {
  my ($api_client, $customer_id) = @_;

  my $experiment = Google::Ads::GoogleAds::V24::Resources::Experiment->new({
    # Name must be unique.
    name   => "Example Experiment #" . uniqid(),
    type   => SEARCH_CUSTOM,
    suffix => "[experiment]",
    status => SETUP
  });

  my $operation =
    Google::Ads::GoogleAds::V24::Services::ExperimentService::ExperimentOperation
    ->new({
      create => $experiment
    });

  my $response = $api_client->ExperimentService()->mutate({
      customerId => $customer_id,
      operations => [$operation]});

  my $resource_name = $response->{results}[0]{resourceName};
  printf "Created experiment with resource name '%s'.\n", $resource_name;
  return $resource_name;
}
create_experiment.pl
      

curl

Note: While a direct REST code sample for this step isn't provided here, you can achieve this using a manual REST request.

Refer to the Google Ads API REST interface documentation and the method-specific reference pages. You will need to construct the JSON payload based on the proto definitions.

Key Resources:

• Using REST
• REST Interface Structure
• JSON Field Mappings
• Consult the REST API Reference for the specific service and method.

2. Create experiment arms

Next, create ExperimentArm resources to define the control and treatment groups for the experiment. All arms must be created in a single request.

Each experiment must have exactly one control arm and one or more treatment arms. The control arm identifies the base campaign for comparison, and each treatment arm results in a new campaign where you can make changes to test.

You must also specify traffic_split, which is the percentage of traffic directed to each arm. The sum of traffic splits across all arms must be 100.

Java

private String createExperimentArms(
    GoogleAdsClient googleAdsClient, long customerId, long campaignId, String experiment) {
  List<ExperimentArmOperation> operations = new ArrayList<>();
  operations.add(
      ExperimentArmOperation.newBuilder()
          .setCreate(
              // The "control" arm references an already-existing campaign.
              ExperimentArm.newBuilder()
                  .setControl(true)
                  .addCampaigns(ResourceNames.campaign(customerId, campaignId))
                  .setExperiment(experiment)
                  .setName("control arm")
                  .setTrafficSplit(40)
                  .build())
          .build());
  operations.add(
      ExperimentArmOperation.newBuilder()
          .setCreate(
              // The non-"control" arm, also called a "treatment" arm, will automatically
              // generate draft campaigns that you can modify before starting the experiment.
              ExperimentArm.newBuilder()
                  .setControl(false)
                  .setExperiment(experiment)
                  .setName("experiment arm")
                  .setTrafficSplit(60)
                  .build())
          .build());

  try (ExperimentArmServiceClient experimentArmServiceClient =
      googleAdsClient.getLatestVersion().createExperimentArmServiceClient()) {
    // Constructs the mutate request.
    MutateExperimentArmsRequest mutateRequest = MutateExperimentArmsRequest.newBuilder()
        .setCustomerId(Long.toString(customerId))
        .addAllOperations(operations)
        // We want to fetch the draft campaign IDs from the treatment arm, so the easiest way to do
        // that is to have the response return the newly created entities.
        .setResponseContentType(ResponseContentType.MUTABLE_RESOURCE)
        .build();

    // Sends the mutate request.
    MutateExperimentArmsResponse response =
        experimentArmServiceClient.mutateExperimentArms(mutateRequest);

    // Results always return in the order that you specify them in the request. Since we created
    // the treatment arm last, it will be the last result.  If you don't remember which arm is the
    // treatment arm, you can always filter the query in the next section with
    // `experiment_arm.control = false`.
    MutateExperimentArmResult controlArmResult = response.getResults(0);
    MutateExperimentArmResult treatmentArmResult = response.getResults(
        response.getResultsCount() - 1);

    System.out.printf("Created control arm with resource name '%s'%n",
        controlArmResult.getResourceName());
    System.out.printf("Created treatment arm with resource name '%s'%n",
        treatmentArmResult.getResourceName());

    return treatmentArmResult.getExperimentArm().getInDesignCampaigns(0);
  }
}
CreateExperiment.java
      

C#

/// <summary>
/// Creates the experiment arms.
/// </summary>
/// <param name="client">The Google Ads client.</param>
/// <param name="customerId">The customer ID for which the call is made.</param>
/// <param name="baseCampaignId">ID of the campaign for which the control arm is
/// created.</param>
/// <param name="experimentResourceName">Resource name of the experiment.</param>
/// <returns>The control and treatment arms.</returns>
private static (MutateExperimentArmResult, MutateExperimentArmResult)
    CreateExperimentArms(GoogleAdsClient client, long customerId, long baseCampaignId,
        string experimentResourceName)
{
    // Get the ExperimentArmService.
    ExperimentArmServiceClient experimentService = client.GetService(
        Services.V24.ExperimentArmService);

    // Create the control arm. The control arm references an already-existing campaign.
    ExperimentArmOperation controlArmOperation = new ExperimentArmOperation()
    {
        Create = new ExperimentArm()
        {
            Control = true,
            Campaigns = {
                ResourceNames.Campaign(customerId, baseCampaignId)
            },
            Experiment = experimentResourceName,
            Name = "Control Arm",
            TrafficSplit = 40
        }
    };

    // Create the non-control arm. The non-"control" arm, also called a "treatment" arm,
    // will automatically generate draft campaigns that you can modify before starting the
    // experiment.
    ExperimentArmOperation treatmentArmOperation = new ExperimentArmOperation()
    {
        Create = new ExperimentArm()
        {
            Control = false,
            Experiment = experimentResourceName,
            Name = "Experiment Arm",
            TrafficSplit = 60
        }
    };

    // We want to fetch the draft campaign IDs from the treatment arm, so the
    // easiest way to do that is to have the response return the newly created
    // entities.
    MutateExperimentArmsRequest request = new MutateExperimentArmsRequest
    {
        CustomerId = customerId.ToString(),
        Operations = { controlArmOperation, treatmentArmOperation },
        ResponseContentType = ResponseContentType.MutableResource
    };


    MutateExperimentArmsResponse response = experimentService.MutateExperimentArms(
        request
    );

    // Results always return in the order that you specify them in the request.
    // Since we created the treatment arm last, it will be the last result.
    MutateExperimentArmResult controlArm = response.Results.First();
    MutateExperimentArmResult treatmentArm = response.Results.Last();

    Console.WriteLine($"Created control arm with resource name " +
        $"'{controlArm.ResourceName}.");
    Console.WriteLine($"Created treatment arm with resource name" +
      $" '{treatmentArm.ResourceName}'.");
    return (controlArm, treatmentArm);
}
CreateExperiment.cs
      

PHP

private static function createExperimentArms(
    GoogleAdsClient $googleAdsClient,
    int $customerId,
    int $campaignId,
    string $experimentResourceName
): string {
    $operations = [];
    $experimentArm1 = new ExperimentArm([
        // The "control" arm references an already-existing campaign.
        'control' => true,
        'campaigns' => [ResourceNames::forCampaign($customerId, $campaignId)],
        'experiment' => $experimentResourceName,
        'name' => 'control arm',
        'traffic_split' => 40
    ]);
    $operations[] = new ExperimentArmOperation(['create' => $experimentArm1]);
    $experimentArm2 = new ExperimentArm([
        // The non-"control" arm, also called a "treatment" arm, will automatically
        // generate draft campaigns that you can modify before starting the
        // experiment.
        'control' => false,
        'experiment' => $experimentResourceName,
        'name' => 'experiment arm',
        'traffic_split' => 60
    ]);
    $operations[] = new ExperimentArmOperation(['create' => $experimentArm2]);

    // Issues a request to create the experiment arms.
    $experimentArmServiceClient = $googleAdsClient->getExperimentArmServiceClient();
    $response = $experimentArmServiceClient->mutateExperimentArms(
        MutateExperimentArmsRequest::build($customerId, $operations)
            // We want to fetch the draft campaign IDs from the treatment arm, so the easiest
            // way to do that is to have the response return the newly created entities.
            ->setResponseContentType(ResponseContentType::MUTABLE_RESOURCE)
    );
    // Results always return in the order that you specify them in the request.
    // Since we created the treatment arm last, it will be the last result.
    $controlArmResourceName = $response->getResults()[0]->getResourceName();
    $treatmentArm = $response->getResults()[count($operations) - 1];
    print "Created control arm with resource name '$controlArmResourceName'" . PHP_EOL;
    print "Created treatment arm with resource name '{$treatmentArm->getResourceName()}'"
        . PHP_EOL;

    return $treatmentArm->getExperimentArm()->getInDesignCampaigns()[0];
}
CreateExperiment.php
      

Python

def create_experiment_arms(
    client: GoogleAdsClient,
    customer_id: str,
    base_campaign_id: str,
    experiment: str,
) -> str:
    """Creates a control and treatment experiment arms.

    Args:
        client: an initialized GoogleAdsClient instance.
        customer_id: a client customer ID.
        base_campaign_id: the campaign ID to associate with the control arm of
          the experiment.
        experiment: the resource name for an experiment.

    Returns:
        the resource name for the new treatment experiment arm.
    """
    operations: List[ExperimentArmOperation] = []

    campaign_service: CampaignServiceClient = client.get_service(
        "CampaignService"
    )

    # The "control" arm references an already-existing campaign.
    operation_1: ExperimentArmOperation = client.get_type(
        "ExperimentArmOperation"
    )
    exa_1: ExperimentArm = operation_1.create
    exa_1.control = True
    exa_1.campaigns.append(
        campaign_service.campaign_path(customer_id, base_campaign_id)
    )
    exa_1.experiment = experiment
    exa_1.name = "control arm"
    exa_1.traffic_split = 40
    operations.append(operation_1)

    # In standard campaign experiments, creating the treatment arm automatically
    # generates a draft campaign that you can modify before starting the experiment.
    operation_2: ExperimentArmOperation = client.get_type(
        "ExperimentArmOperation"
    )
    exa_2: ExperimentArm = operation_2.create
    exa_2.control = False
    exa_2.experiment = experiment
    exa_2.name = "experiment arm"
    exa_2.traffic_split = 60
    operations.append(operation_2)

    experiment_arm_service: ExperimentArmServiceClient = client.get_service(
        "ExperimentArmService"
    )
    request: MutateExperimentArmsRequest = client.get_type(
        "MutateExperimentArmsRequest"
    )
    request.customer_id = customer_id
    request.operations = operations
    # We want to fetch the draft campaign IDs from the treatment arm, so the
    # easiest way to do that is to have the response return the newly created
    # entities.
    request.response_content_type = (
        client.enums.ResponseContentTypeEnum.MUTABLE_RESOURCE
    )
    response: MutateExperimentArmsResponse = (
        experiment_arm_service.mutate_experiment_arms(request=request)
    )

    # Results always return in the order that you specify them in the request.
    # Since we created the treatment arm second, it will be the second result.
    control_arm_result: Any = response.results[0]
    treatment_arm_result: Any = response.results[1]

    print(
        f"Created control arm with resource name {control_arm_result.resource_name}"
    )
    print(
        f"Created treatment arm with resource name {treatment_arm_result.resource_name}"
    )

    return treatment_arm_result.experiment_arm.in_design_campaigns[0]
      

Ruby

def create_experiment_arms(client, customer_id, base_campaign_id, experiment)
  operations = []
  operations << client.operation.create_resource.experiment_arm do |ea|
    # The "control" arm references an already-existing campaign.
    ea.control = true
    ea.campaigns << client.path.campaign(customer_id, base_campaign_id)
    ea.experiment = experiment
    ea.name = 'control arm'
    ea.traffic_split = 40
  end
  operations << client.operation.create_resource.experiment_arm do |ea|
    # The non-"control" arm, also called a "treatment" arm, will automatically
    # generate draft campaigns that you can modify before starting the
    # experiment.
    ea.control = false
    ea.experiment = experiment
    ea.name = 'experiment arm'
    ea.traffic_split = 60
  end

  response = client.service.experiment_arm.mutate_experiment_arms(
    customer_id: customer_id,
    operations: operations,
    # We want to fetch the draft campaign IDs from the treatment arm, so the
    # easiest way to do that is to have the response return the newly created
    # entities.
    response_content_type: :MUTABLE_RESOURCE,
  )

  # Results always return in the order that you specify them in the request.
  # Since we created the treatment arm last, it will be the last result.
  control_arm_result = response.results.first
  treatment_arm_result = response.results.last

  puts "Created control arm with resource name #{control_arm_result.resource_name}."
  puts "Created treatment arm with resource name #{treatment_arm_result.resource_name}."

  treatment_arm_result.experiment_arm.in_design_campaigns.first
end
create_experiment.rb
      

Perl

sub create_experiment_arms {
  my ($api_client, $customer_id, $base_campaign_id, $experiment) = @_;

  my $operations = [];
  push @$operations,
    Google::Ads::GoogleAds::V24::Services::ExperimentArmService::ExperimentArmOperation
    ->new({
      create => Google::Ads::GoogleAds::V24::Resources::ExperimentArm->new({
          # The "control" arm references an already-existing campaign.
          control   => "true",
          campaigns => [
            Google::Ads::GoogleAds::V24::Utils::ResourceNames::campaign(
              $customer_id, $base_campaign_id
            )
          ],
          experiment   => $experiment,
          name         => "control arm",
          trafficSplit => 40
        })});

  push @$operations,
    Google::Ads::GoogleAds::V24::Services::ExperimentArmService::ExperimentArmOperation
    ->new({
      create => Google::Ads::GoogleAds::V24::Resources::ExperimentArm->new({
          # The non-"control" arm, also called a "treatment" arm, will automatically
          # generate draft campaigns that you can modify before starting the
          # experiment.
          control      => "false",
          experiment   => $experiment,
          name         => "experiment arm",
          trafficSplit => 60
        })});

  my $response = $api_client->ExperimentArmService()->mutate({
    customerId => $customer_id,
    operations => $operations,
    # We want to fetch the draft campaign IDs from the treatment arm, so the
    # easiest way to do that is to have the response return the newly created
    # entities.
    responseContentType => MUTABLE_RESOURCE
  });

  # Results always return in the order that you specify them in the request.
  # Since we created the treatment arm last, it will be the last result.
  my $control_arm_result   = $response->{results}[0];
  my $treatment_arm_result = $response->{results}[1];

  printf "Created control arm with resource name '%s'.\n",
    $control_arm_result->{resourceName};
  printf "Created treatment arm with resource name '%s'.\n",
    $treatment_arm_result->{resourceName};
  return $treatment_arm_result->{experimentArm}{inDesignCampaigns}[0];
}
create_experiment.pl
      

curl

Note: While a direct REST code sample for this step isn't provided here, you can achieve this using a manual REST request.

Refer to the Google Ads API REST interface documentation and the method-specific reference pages. You will need to construct the JSON payload based on the proto definitions.

Key Resources:

• Using REST
• REST Interface Structure
• JSON Field Mappings
• Consult the REST API Reference for the specific service and method.

A few key points:

• Exactly one arm must have control set to true.
• The traffic_split must add up to 100 across all arms.
• The control arm must specify exactly one campaign in its campaigns field.

Note: You cannot use partial failure when creating experiment arms.

3. Modify treatment campaigns

When you create a treatment arm (where control is false), the API automatically creates a draft campaign based on the control campaign and populates its resource name in the in_design_campaigns field of the treatment arm. You can treat these in-design campaigns as regular campaigns and modify them with the changes you want to test. The control campaign won't be affected. These changes are materialized into a real, servable campaign when you schedule the experiment.

At least one change must be made to an in-design campaign before you can schedule the experiment.

To retrieve the in_design_campaigns for a treatment arm, you can query GoogleAdsService:

SELECT experiment_arm.in_design_campaigns
FROM experiment_arm
WHERE experiment_arm.resource_name = "TREATMENT_ARM_RESOURCE_NAME"

Tip: Draft campaigns are only returned if you add the include_drafts=true parameter to your query.

Schedule the experiment

After creating the experiment and arms and modifying the treatment draft campaigns, you can schedule the experiment using ExperimentService.ScheduleExperiment. This is an asynchronous operation that materializes the in-design campaigns into actual campaigns, ready to serve once the experiment's start_date arrives. See Asynchronous errors for details on handling long-running operations.

Report on the experiment

Once the experiment is running, you can query for metrics to compare performance. See the Reporting guide for details.

Graduate, promote, or end the experiment

After letting the experiment run for a sufficient time, you can choose to end, promote, or graduate it using ExperimentService.

• End: If you want to stop the experiment without applying changes, use EndExperiment. The treatment campaigns stop serving, but are not removed. This is a synchronous operation.
• Promote: If you are satisfied with the treatment arm's performance and want to incorporate the changes into your original campaign, use PromoteExperiment. This copies changes from the treatment arm to the control campaign and stops the treatment arm from serving. This is an asynchronous operation; see Asynchronous errors for details. Note: Experiments of type PMAX_REPLACEMENT_SHOPPING cannot be promoted.
• Graduate: If you liked the changes, but want them to exist separately from the original campaign, use GraduateExperiment. This converts the treatment campaign into a standard, independent campaign that continues to serve outside the context of the experiment. The control campaign is not modified. This is a synchronous operation.

Type-specific requirements

PMAX_REPLACEMENT_SHOPPING:

• Requires exactly two arms (one control, one treatment).
• The control arm must specify one Shopping campaign in its campaigns field.
• For the treatment arm, you have two options for setting up the Performance Max campaign:
  • To create a new PMax campaign: Leave the treatment arm's campaigns field empty. The API automatically creates an in-design campaign based on the control Shopping campaign, as described in Step 3.
  • To use an existing PMax campaign: Specify the resource name of the PMax campaign in the treatment arm's campaigns field. If you select this option, no in-design campaign is created, and you can skip Step 3.
• It is recommended to discard the first 7 days of statistics from your evaluation to give the campaign time to finish its ramp-up and learning phase.
• Cannot be promoted.

YOUTUBE_CUSTOM:

• Supports a maximum of 10 arms.
• To test assets, set creative_asset_testing_info on the experiment arm.