In this section, we cover how you can gather historical data on various
marketing and non-marketing variables, such as advertising spend, pricing, and
revenue or performance metrics.

## Dataset specification

The following data types are required unless otherwise noted:

| Data Type | Description |
|---|---|
| Media data | Contains the exposure metric by [channel](https://developers.google.com/meridian/docs/pre-modeling/collect-data#aggregating-campaigns), geo, and time period. Possible metrics include, but are not limited to, spend, impressions, and clicks. The key is that these are intervenable units, meaning they represent media efforts that one can reasonably control. All media values must be non-negative. Must be a [summable metric](https://developers.google.com/meridian/docs/pre-modeling/collect-data#summable-metrics). |
| Media spend | Containing the media spending per channel and time span. The media data and media spend must have the same dimensions. |
| Control variables | Contains the control variables used in the model. The selection of control variables is important for estimating the causal effect from an MMM, see [Causal graph](https://developers.google.com/meridian/docs/causal-inference/causal-graph). |
| KPI | The target KPI is the model's response variable. For example, revenue amount or number of application installations. Must be a [summable metric](https://developers.google.com/meridian/docs/pre-modeling/collect-data#summable-metrics). |
| Revenue per KPI (Optional) | Contains the average revenue for a KPI unit. In the absence of accurate revenue per KPI, we strongly recommend that you approximate a rational value. If such information is unavailable, see [When the KPI is not revenue](https://developers.google.com/meridian/docs/advanced-modeling/unknown-revenue-kpi-default). Note that 'revenue per KPI' is not required if revenue is your KPI. |
| Geo population (Optional) | Contains the population for each geo. Geo population (such as Nielsen DMA TV household population) is used to scale the media metric to put all geos on a comparable scale, see [Input data](https://developers.google.com/meridian/docs/advanced-modeling/input-data) for more details about media scaling. |

Meridian offers the option to model any media channel's effect based on
reach and frequency data, see [Reach and
frequency](https://developers.google.com/meridian/docs/basics/reach-frequency).

| Data Type | Description |
|---|---|
| Reach | The reach data is the number of unique individuals exposed to the channels' ad within each time period. |
| Frequency | Frequency is the average number of times a person is exposed to an advertisement within that time period. It is equal to the total number of impressions divided by the reach for each time period. |

Meridian also offers the option to include organic media and non-media
treatments. For more information, see [Organic media and non-media
variables](https://developers.google.com/meridian/docs/advanced-modeling/organic-and-non-media-variables).

| Data Type | Description |
|---|---|
| Organic media | Organic media variables are media activities that have no direct cost. These can include, but are not limited to, impressions from newsletters, a blog post, social media activity, or email campaigns. |
| Non-media treatments | Non-media variables are marketing activities that are not directly related to media, such as running a promotion, the price of a product, and a change in a product's packaging or design. |

<br />

## Example: What an MMM dataset looks like

Meridian generally expects your marketing data to be aggregated into a single,
cohesive format, with your media data aggregated by time (for example, by week)
and ideally, by [geo](https://developers.google.com/meridian/docs/pre-modeling/collect-data#geographic-granularity). Review our
[Getting Started Notebook](https://developers.google.com/meridian/notebook/meridian-getting-started#example_dataset_preview_and_schema_overview)
for what a sample dataset for a standard [geo-level](https://developers.google.com/meridian/docs/pre-modeling/geo-selection-national-data)
model looks like.

### GitHub samples

Additional sample datasets can be found [on GitHub](https://github.com/google/meridian/tree/main/meridian/data/simulated_data/csv).

## KPI

The KPI is the \\(y\\) variable on the left-hand side of the
[Model Spec](https://developers.google.com/meridian/docs/advanced-modeling/model-spec). The KPI can be either
revenue or some other non-revenue KPI, such as conversions. Meridian requires
that the KPI is [summable](https://developers.google.com/meridian/docs/pre-modeling/collect-data#summable-metrics) across both geography and time.
Examples of summable metrics include units sold, revenue, or total conversions.
For non-summable metrics like click-through rate (CTR), you should model the
summable clicks instead.

Some modelers prefer to use a non-revenue KPI as the response variable, even
when revenue is ultimately the KPI. Meridian lets you translate KPI
units to revenue by providing revenue per KPI data for each geographic unit
and time period. For more information, see
[When the KPI is not revenue](https://developers.google.com/meridian/docs/advanced-modeling/unknown-revenue-kpi-default).

## Media, organic media, non-media treatment and control variables

Media, organic media, non-media treatment and control variables should have time
series data available.

- **Media variables:** For each paid media, the dataset must include the spend
  for each media channel, which is used as the denominator for ROI
  calculations. Meridian requires that paid media (except for R\&F channels)
  are [summable](https://developers.google.com/meridian/docs/pre-modeling/collect-data#summable-metrics) across both geography and time.

  Additionally, each paid media must include *one* of the
  following for modeling purposes:
  - A single media exposure metric, such as impressions, clicks, or spend.

    > [!NOTE]
    > **Note:** If physical impressions or click data is completely unavailable, you can use spend as a proxy for the media exposure metric. However, be aware of cost per media fluctuations. If the model assumes spend represents physical exposures, periods of high ad cost might be interpreted as high ad volume. For example, if you ramp up spend during competitive periods but the cost per media spikes, your actual physical impressions might remain flat. This might lead to inaccurate estimates for the effectiveness of a treatment.

  - Reach and frequency

- **Organic media variables:** [Organic
  media](https://developers.google.com/meridian/docs/advanced-modeling/organic-and-non-media-variables#organic_media_variables)
  has no associated spend and can
  be excluded from the media spend input. Similar to media variables,
  Meridian requires that organic media variables (except for R\&F channels)
  are [summable](https://developers.google.com/meridian/docs/pre-modeling/collect-data#summable-metrics) across both geography and time.
  Additionally, each organic media must include *one* of the following for
  modeling purposes:

  - A single media exposure metric, such as impressions or clicks.
  - Reach and frequency.
- **Non-media treatments:** Non-media variables are marketing activities that
  are not directly related to media and have no direct marketing cost
  associated with them. They differ from control variables because they are
  considered to be intervenable, and therefore are treatment variables under
  the causal model. For more information about modeling with non-media
  treatments, see [Non-media
  treatments](https://developers.google.com/meridian/docs/advanced-modeling/organic-and-non-media-variables#non-media_treatment_variables).

- **Control variables:** The purpose of control variables is to control for
  confounding. Focus on collecting variables that have a causal effect on both
  the target KPI and the media metric or media execution. Because it is
  difficult to come up with a comprehensive list of variables affecting KPI,
  it can be more practical to focus on variables that affect media budget and
  planning decisions. You can start by asking your marketing planner what
  information might have played a role, either consciously or subconsciously,
  in their decision making. For more information about modeling with control
  variables, see [Control
  variables](https://developers.google.com/meridian/docs/advanced-modeling/control-variables).

  Examples of control variables include market competition, and Google query
  volume (GQV). For more information about GQV, see [Understanding query
  volume as a confounder for search
  ads](https://developers.google.com/meridian/docs/advanced-modeling/paid-search-modeling#understanding-query-volume-confounder).
- **Seasonality-related variables:** Seasonality-related variables, such as
  holiday dummies, are typically incorporated as control variables in the
  model specification. However, Meridian is equipped with an automatic
  seasonality and trend adjustment functionality, implemented through the
  time-varying intercept [model
  specification](https://developers.google.com/meridian/docs/advanced-modeling/model-spec). Therefore, the
  inclusion of separate seasonality variables is not required.

  Alternatively, you can disable the [automatic seasonality
  adjustment](https://developers.google.com/meridian/docs/user-guide/customizing-parameters#tune-automatic-seasonality)
  and include your own seasonality variables.

## Data collection

For each of the variables, you must ascertain the type of data to be collected.
Media or marketing plans can be utilized for the purpose of determining the
appropriate variables to be collected. You can then collect media exposure for
Google channels, including metrics such as clicks and impressions, by utilizing
MMM Data Platform. Furthermore, MMM Data Platform also offers reach and
frequency data specifically for YouTube. For more information, see [Use MMM Data
Platform](https://developers.google.com/meridian/docs/pre-modeling/using-mmm-data-platform).

**Scenario:** I am an MMM developer who wants to download Google Ads data for
MMM development.

**Solution:** Use the
[MMM Data Platform](https://developers.google.com/meridian/docs/pre-modeling/using-mmm-data-platform). While
many advertisers rely on the Google Ads API for data pulls, this is not ideal
for MMM development.

Collecting [Google Query Volume (GQV)
data](https://developers.google.com/meridian/docs/pre-modeling/using-mmm-data-platform) is optional, although
omitting GQV might create bias to your model estimates. However, you can run
Meridian without GQV data.

Make sure that your data is in the proper format to run the model. For more
information about the format, see the data examples in [Supported data types and
formats](https://developers.google.com/meridian/docs/user-guide/supported-data-types-formats).

## Data imputation

When preparing your dataset, you might encounter missing or null values.
Meridian requires a complete dataset without missing values to run properly.
You must handle any gaps in your data through imputation or zero-filling before
running the model.

In general, use your best judgement when handling missing data around an
MMM dataset.

- **Media variables:** If data is missing because a specific channel was inactive during a time period or in a certain geo, fill these missing values with `0`.
- **KPI and Control variables:** If data is missing for your target metric or control variables, avoid filling them with `0` as this will skew your model's estimates. Instead, use standard data imputation techniques to estimate the missing periods. Common approaches include forward-filling, backward-filling, linear interpolation, or utilizing the historical mean for that specific geo or time period.
- **National-level media in a geo-level model:** If some media channels are only available at the national level, we recommend imputing them at the geo level. For more information and recommended imputation methods, see [National-level media in a geo-level model](https://developers.google.com/meridian/docs/pre-modeling/geo-selection-national-data#national-imputation).

### Examples of data imputation

**Example 1: Imputing missing media data (zero-filling)**

If a specific marketing channel was paused or inactive for a week, the raw data
might contain null (`NaN`) values. Because this is a media variable, you should
fill the missing data with `0` to reflect zero exposure or spend.

*Before imputation (Raw data):*

| Date (time) | Geo | TV_Spend | TV_Impressions | ... |
|---|---|---|---|---|
| 2021-03-01 | New York | $5,000 | 450,000 | ... |
| 2021-03-08 | New York | NaN | NaN | ... |
| 2021-03-15 | New York | $4,800 | 420,000 | ... |
| ... | ... | ... | ... | ... |

*After imputation (Ready for Meridian):*

| Date (time) | Geo | TV_Spend | TV_Impressions | ... |
|---|---|---|---|---|
| 2021-03-01 | New York | $5,000 | 450,000 | ... |
| 2021-03-08 | New York | **$0** | **0** | ... |
| 2021-03-15 | New York | $4,800 | 420,000 | ... |
| ... | ... | ... | ... | ... |

**Example 2: Imputing missing control variables (Interpolation)**

If a control variable like competitor sales index is missing for a specific
week, filling it with `0` would artificially skew the baseline. Instead, use an
imputation method like linear interpolation to bridge the gap between the known
values.

*Before imputation (Raw data):*

| Date (time) | Geo | Competitor_Sales | ... |
|---|---|---|---|
| 2021-03-01 | Portland | 10.5 | ... |
| 2021-03-08 | Portland | NaN | ... |
| 2021-03-15 | Portland | 11.5 | ... |
| ... | ... | ... | ... |

*After imputation (Ready for Meridian):*

| Date (time) | Geo | Competitor_Sales | ... |
|---|---|---|---|
| 2021-03-01 | Portland | 10.5 | ... |
| 2021-03-08 | Portland | **11.0** | ... |
| 2021-03-15 | Portland | 11.5 | ... |
| ... | ... | ... | ... |

## Granularity

Generally speaking, finer data granularity provides more accurate insights and
can help identify actionable results. Consider the granularity of data from the
following aspects.

### Geographic granularity

**Best Practice:** Collect data at the geo level. This level of granularity lets
you account for geo-level nuances, and use Meridian's hierarchical
Bayesian framework to yield tighter credible intervals on estimates such as ROI.
Note that certain geos can exhibit a low volume of observations. Consequently,
it is advisable to exclude those geos from the dataset before model fitting
to help ensure reliable model estimation. For more information, see
[Geo-selection and national-level
data](https://developers.google.com/meridian/docs/pre-modeling/geo-selection-national-data).

**Acceptable Alternative:** If geo level data is not available, you can use
national data. However, check that your national data has a sufficient number of
data points per effect that you are trying to measure. For more information, see
[Amount of data needed](https://developers.google.com/meridian/docs/pre-modeling/amount-data-needed).

#### Example of geo level granularity (recommended)

| Date (time) | Geo | Conversions (KPI) | TV_Spend | Social_Media_Spend | ... |
|---|---|---|---|---|---|
| 2021-01-04 | New York | 15,400 | $12,000 | $4,500 | ... |
| 2021-01-04 | Chicago | 9,850 | $8,500 | $3,200 | ... |
| 2021-01-04 | Los Angeles | 14,200 | $11,000 | $4,100 | ... |
| 2021-01-11 | New York | 16,100 | $12,500 | $4,800 | ... |
| ... | ... | ... | ... | ... | ... |

#### Example of national level granularity

| Date (time) | Conversions (KPI) | TV_Spend | Social_Media_Spend | ... |
|---|---|---|---|---|
| 2021-01-04 | 39,450 | $31,500 | $11,800 | ... |
| 2021-01-11 | 41,200 | $33,000 | $12,400 | ... |
| ... | ... | ... | ... | ... |

### Time granularity

**Best Practice:** Collect data at the weekly level. Weekly data presents an
advantageous equilibrium between the degree of variation and the extent of
noise, particularly when compared to daily or monthly data.

**Acceptable Alternative:** In the absence of weekly data, you can test daily or
monthly data as an alternative. However, when daily data is utilized, the model
can experience an extended runtime. Additionally, non-convergence or wide
credible intervals on model estimates can arise when monthly data is used.

#### Example of weekly level granularity

In this example, the `Date` column strictly advances in 7-day increments
(weekly).

| Date (time) | Geo | Conversions (KPI) | TV_Spend | Social_Media_Spend | ... |
|---|---|---|---|---|---|
| **2021-01-04** | New York | 15,400 | $12,000 | $4,500 | ... |
| **2021-01-11** | New York | 16,100 | $12,500 | $4,800 | ... |
| **2021-01-18** | New York | 14,900 | $11,800 | $4,200 | ... |
| **2021-01-25** | New York | 17,050 | $13,000 | $5,100 | ... |
| ... | ... | ... | ... | ... | ... |

### Media granularity

When determining the number of media channels to include, ensure your dataset
maintains a sufficient ratio of data points to model parameters. Evaluate your
[data-to-parameter ratio](https://developers.google.com/meridian/docs/pre-modeling/perform-eda#data-to-parameter_ratio)
to verify data sufficiency. For more specific guidance about the amount of data
needed, see
[Amount of data needed](https://developers.google.com/meridian/docs/pre-modeling/amount-data-needed).

For media channels with low media spend, it is advisable to combine them with
other channels to avoid issues with ROI estimation. For more
information, see [Channels with low
spend](https://developers.google.com/meridian/docs/post-modeling/model-debugging#channels-with-low-spend).

## Timeframe

As a general rule of thumb, historical data should be a minimum of two years'
worth of weekly data for geo-level models and three years of data for
national-level models. If only monthly data is available, then we recommend
using a minimum of three years of data. It is important for the model to have
enough data points to provide accurate calculations. However, determining the
amount of data can be more complex and ultimately depends on what your data is
like. For more specific guidance about the amount of data needed, see [Amount of
data needed](https://developers.google.com/meridian/docs/pre-modeling/amount-data-needed).

After you have collected your data, [perform an exploratory data
analysis](https://developers.google.com/meridian/docs/pre-modeling/perform-eda) to make sure that your data
is accurate and complete.

### Example of sufficient historical data

In this example, the dataset begins in January 2021 and spans continuously to
December 2022, providing the 104+ weeks per geo.

| Date (time) | Geo | Conversions (KPI) | TV_Spend | Social_Media_Spend | ... |
|---|---|---|---|---|---|
| **2021-01-04** | New York | 15,400 | $12,000 | $4,500 | ... |
| **2021-01-11** | New York | 16,100 | $12,500 | $4,800 | ... |
| ... | ... | ... | ... | ... | ... |
| **2022-12-19** | New York | 22,300 | $15,000 | $7,200 | ... |
| **2022-12-26** | New York | 24,100 | $14,500 | $6,800 | ... |

## Appendix

This appendix provides additional technical context, foundational concepts, and
supplementary guidelines to assist you in preparing your dataset for Meridian.

### Understand summable metrics

Meridian requires that your KPI and media metrics (except for R\&F channels) are
**summable** across both geography and time. This means if you add the rows of
your dataset together, the mathematical output must make logical sense.

> [!NOTE]
> **Note:** The summability requirement applies to single media exposure metrics (such as impressions, clicks, or spend). **It does not apply for media modeled
> with reach and frequency.** For more information about reach and frequency data requirements, see [Reach and frequency](https://developers.google.com/meridian/docs/advanced-modeling/reach-frequency#reach_and_frequency_data).

**Concrete examples:**

- **Summable (Do use):** Raw volumes like Total Clicks, Total Impressions, Total Spend, or Total Units Sold.
  - *Example:* If your dataset shows 100 clicks in Week 1 and 50 clicks in Week 2, adding them together correctly yields 150 total clicks.
- **Non-summable (Do not use):** Averages, rates, or percentages like Click-Through Rate (CTR), Cost-Per-Click (CPC), or Return on Ad Spend (ROAS).
  - *Example:* You cannot mathematically add rates together. For example, if Week 1 had 1,000 impressions and 20 clicks (a 2% CTR) and Week 2 had 10,000 impressions and 300 clicks (a 3% CTR), adding the rates together would give you 5%. However, your actual total is 320 clicks out of 11,000 impressions, which is an actual combined CTR of 2.9%. Because these percentages cannot be added together row-by-row, they cannot be used in the model.

If your advertising platform exports rates or averages, you must compute the raw
summable volumes before feeding the data into Meridian. For example, if your
data contains Impressions and CTR, you must calculate the total Clicks
(Impressions × CTR) and supply that summable integer to the model.

### Aggregating campaigns into media channels

Advertising platforms typically report data at a highly granular level, such as
by campaign, ad group, or creative. However, Meridian typically models media at
the channel level. To bridge this gap, you must aggregate your campaign-level
data into channel-level data.

To aggregate a platform into a single channel, sum the spend and the execution
metric (for example, impressions or clicks) across all relevant campaigns
for that specific geo and time period.

> [!NOTE]
> **Note:** Summing data across campaigns assumes that campaigns are mutually exclusive, meaning that each impression, click, or dollar spent is attributed to only one campaign without overlap or double-counting.

#### Example: Aggregating Social Media campaigns

In this scenario, a business exports raw data from a social media ads platform
containing multiple active campaigns - one geared towards prospecting, and
another towards retargeting. The raw export includes daily
campaign-level rows, which must be grouped by the start of the week and geo to
form a single weekly `Social_Media` channel for Meridian.

**Before: Platform-level export (Raw daily social media data)**

| Date | Region | Campaign ID | Campaign name | Amount spent (USD) | Impressions |
|---|---|---|---|---|---|
| 2021-01-04 | Washington | 11111111 | Q1_Prospecting | 100.00 | 10000 |
| 2021-01-04 | Washington | 22222222 | Q1_Retargeting | 50.00 | 5000 |
| 2021-01-05 | Washington | 11111111 | Q1_Prospecting | 110.00 | 11000 |
| 2021-01-05 | Washington | 22222222 | Q1_Retargeting | 60.00 | 6000 |
| 2021-01-06 | Washington | 11111111 | Q1_Prospecting | 120.00 | 12000 |
| 2021-01-06 | Washington | 22222222 | Q1_Retargeting | 45.00 | 4500 |
| 2021-01-07 | Washington | 11111111 | Q1_Prospecting | 105.00 | 10500 |
| 2021-01-07 | Washington | 22222222 | Q1_Retargeting | 55.00 | 5500 |
| 2021-01-08 | Washington | 11111111 | Q1_Prospecting | 130.00 | 13000 |
| 2021-01-08 | Washington | 22222222 | Q1_Retargeting | 65.00 | 6500 |
| 2021-01-09 | Washington | 11111111 | Q1_Prospecting | 150.00 | 15000 |
| 2021-01-09 | Washington | 22222222 | Q1_Retargeting | 80.00 | 8000 |
| 2021-01-10 | Washington | 11111111 | Q1_Prospecting | 140.00 | 14000 |
| 2021-01-10 | Washington | 22222222 | Q1_Retargeting | 70.00 | 7000 |
| 2021-01-11 | Washington | ... | ... | ... | ... |

**After: MMM dataset (Aggregated weekly into a single channel)**

To create the `Social_Media` channel, group the rows by the week
(starting on Monday, `2021-01-04`) and `Region` columns. Then, add the spend and
impression columns together for all days in that week across all campaigns.
For the week of 2021-01-04, the total spend across all 14 daily campaign records
is $1,280, and the total impressions are 128,000.

| Date (time) | Geo | Social_Media_Spend | Social_Media_Impressions |
|---|---|---|---|
| 2021-01-04 | Washington | **$1,280** | **128,000** |
| 2021-01-11 | Washington | ... | ... |