After the model is built, you must assess convergence, debug the model if
needed, and then assess the model fit.

## Assess convergence

You assess the model convergence to help ensure the integrity of your model.

The `plot_rhat_boxplot` command under `visualizer.ModelDiagnostics()` summarizes
and calculates the [Gelman \& Rubin
(1992)](https://sites.stat.columbia.edu/gelman/research/published/itsim.pdf)
potential scale reduction for chain convergence, commonly referred to as R-hat.
This convergence diagnostic measures the degree to which the variance (of the
means) between the chains exceeds what you would expect if the chains were
identically distributed.

There is a single R-hat value for each model parameter. The box plot summarizes
the distribution of R-hat values across indexes. For example, the box
corresponding to the `beta_gm` x-axis label summarizes the distribution of R-hat
values across both the geo index `g` and the channel index `m`.

Values close to 1.0 indicate convergence. R-hat \< 1.2 indicates approximate
convergence and is a reasonable threshold for many problems ([Brooks \& Gelman,
1998](https://www2.stat.duke.edu/%7Escs/Courses/Stat376/Papers/ConvergeDiagnostics/BrooksGelman.pdf)).
A lack of convergence typically has one of two culprits. Either the model is
very poorly misspecified for the data, which can be in the likelihood (model
specification) or in the prior. Or, there is not enough burnin, meaning
`n_adapt + n_burnin` is not large enough.

If you have difficulty getting convergence, see [Getting MCMC convergence](https://developers.google.com/meridian/docs/post-modeling/model-debugging#getting-mcmc-convergence).

### Generate an R-hat boxplot

Run the following commands to generate an R-hat boxplot:

    model_diagnostics = visualizer.ModelDiagnostics(mmm)
    model_diagnostics.plot_rhat_boxplot()

**Example output:**

![R-hat boxplot](https://developers.google.com/static/meridian/images/r-hat.png)

### Generate trace and density plots

You can generate trace and density plots for Markov Chain Monte Carlo (MCMC)
samples to help assess convergence and stability across chains. Each trace in
the trace plot represents the sequence of values generated by the MCMC algorithm
as it explores the parameter space. It shows how the algorithm moves through
different values of the parameters over successive iterations. In the trace
plots, try to avoid flat areas, where the chain stays in the same state for too
long or has too many consecutive steps in one direction.

The density plots on the left visualize the density distribution of sampled
values for one or more parameters obtained through the MCMC algorithm. In the
density plot, you want to see that the chains have converged to a stable density
distribution.

The following example shows how to generate trace and density plots:

    parameters_to_plot=["roi_m"]
    for params in parameters_to_plot:
      az.plot_trace(
          meridian.inference_data,
          var_names=params,
          compact=False,
          backend_kwargs={"constrained_layout": True},
      )

**Example output:**

![Trace and density plots](https://developers.google.com/static/meridian/images/trace-density-plots.png)

## Check prior and posterior distributions

When there is little information in the data, the prior and the posterior will
be similar. For more information, see [When the posterior is the same as the
prior](https://developers.google.com/meridian/docs/post-modeling/model-debugging#posterior-same-as-prior).

Channels with low spend are particularly susceptible to having an ROI posterior
similar to the ROI prior. To remediate the issue, we recommend either dropping
the channels with very low spend or combining them with other channels when
preparing the data for MMM.

Run the following commands to plot the ROI posterior distribution against the
ROI prior distribution for each media channel:

    model_diagnostics = visualizer.ModelDiagnostics(mmm)
    model_diagnostics.plot_prior_and_posterior_distribution()

**Example output:** (*Click the image to enlarge.*)

![distributions](https://developers.google.com/static/meridian/images/distributions.png)

![](https://developers.google.com/static/meridian/images/distributions.png)

By default, `plot_prior_and_posterior_distribution()` generates the posterior
and prior for ROI. However, you can pass specific model parameters to
`plot_prior_and_posterior_distribution()`, as shown in the following example:

    model_diagnostics.plot_prior_and_posterior_distribution('beta_m')

## Assess model fit

After your model convergence is optimized, assess the model fit. For more
information, see [Assess the model fit](https://developers.google.com/meridian/docs/post-modeling/model-fit)
in *Post-modeling*.

With marketing mix modeling (MMM), you must rely on indirect measures to assess
causal inference and look for results that make sense. Two good ways to do this
are by:

- Running metrics for R-squared, Mean Absolute Percentage Error (MAPE), and Weighted Mean Absolute Percentage Error (wMAPE)
- Generating plots for expected versus actual revenue or KPI, contingent upon the `kpi_type` and the availability of `revenue_per_kpi`.

> [!NOTE]
> **Note:** wMAPE is weighted by actual revenue.

### Run R-squared, MAPE, and wMAPE metrics

Goodness of fit metrics can be used as a confidence check that the model
structure is appropriate and not overparameterized. `ModelDiagnostics`
calculates `R-Squared`, `MAPE`, and `wMAPE` goodness of fit metrics. If
`holdout` is set in `ModelSpec`, then `R-squared`, `MAPE` and `wMAPE` are also
calculated for the `Train` and `Test` subsets. Be aware that goodness of fit
metrics are a measure of predictive accuracy, which is not typically the goal of
an MMM. However, these metrics still serve as a useful confidence check.

Run the following commands to generate R-squared, MAPE, and wMAPE metrics:

    model_diagnostics = visualizer.ModelDiagnostics(mmm)
    model_diagnostics.predictive_accuracy_table()

**Example output:**

![Metrics output](https://developers.google.com/static/meridian/images/metrics.png)

### Generate expected versus actual plots

Using expected versus actual plots can be helpful as an indirect method to
assess the model fit.

#### National: Expected versus actual plots

You can plot the actual revenue or KPI alongside the model's expected revenue
or KPI at the national level to help assess the model fit. *Baseline* is the
model's counterfactual estimation for revenue (or KPI) if there was no media
execution. Estimating revenue to be as close to actual revenue as possible isn't
necessarily the goal of an MMM, but it does serve as a useful confidence check.

Run the following commands to plot actual revenue (or KPI) versus expected
revenue (or KPI) for national data:

    model_fit = visualizer.ModelFit(mmm)
    model_fit.plot_model_fit()

**Example output:**

![Expected versus actual revenue
graph](https://developers.google.com/static/meridian/images/expected-vs-actual-revenue.png)

#### Geo: Expected versus actual plots

You can create the expected versus actual plots at the geo level to assess model
fit. Because there can be many geos, you might want to only show the largest
geos.

Run the following commands to plot actual revenue (or KPI) versus expected
revenue (or KPI) for geos:

    model_fit = visualizer.ModelFit(mmm)
    model_fit.plot_model_fit(n_top_largest_geos=2,
                             show_geo_level=True,
                             include_baseline=False,
                             include_ci=False)

**Example output:**

![Geo actual revenue versus estimated revenue
graphs](https://developers.google.com/static/meridian/images/geo-revenue.png)

After you are satisfied with your model fit, [analyze the model
results](https://developers.google.com/meridian/docs/user-guide/model-results-overview).