Monitor agent performance with metrics

After you launch, Business Messages tracks key performance metrics for your agent. You can access your metrics through the Business Communications Developer Console or through BigQuery.

If you need additional analytics to gauge the success of your agent, you can track custom metrics by monitoring events and other attributes of your agent's conversations.

Access your metrics

You can access partner-level and agent-level metrics in a Data Studio report through the Business Communications Developer Console. Alternatively, you can use BigQuery to browse or programmatically access your agent metrics.

To grant metrics access to other users,

  1. Add them as users to your partner account.
  2. Instruct them on how to access metrics through the Business Communications Developer Console or BigQuery.

Partner-level metrics

Partner-level metrics display data for all agents that you manage.

  1. Open the Business Communications Developer Console and sign in with your Business Messages Google account.
  2. At the bottom of the Metrics dashboard card, click View metrics.

Agent-level metrics

Agent-level metrics display data only for the selected agent.

  1. Open the Business Communications Developer Console and sign in with your Business Messages Google account.

  2. Choose your agent.

    Metrics appear in the Metrics dashboard card.

To see more detailed metrics, click Analytics in the left navigation.

Metrics through BigQuery

BigQuery is a serverless, scalable, and cost-effective data warehouse with machine learning, easy data access, and predictive analytics. You can access data in BigQuery either through the BigQuery UI or BigQuery API depending on your use case. See the BigQuery documentation.

Metrics data in BigQuery is available for 1.5 years.

To access agent metrics in BigQuery, you need to meet the following requirements:

When you have access to BigQuery, you can access your agent's data in the following project, data sets, and views:

  • Project ID: bizcomms-bm-metrics
  • Data sets: partner_metrics_user_views and partner_metrics_sa_views
  • Views: message_metrics, session_metrics, quality_metrics, and intent_metrics

For example, to see all fields from the message_metrics view, you might use the following query:

SELECT * FROM bizcomms-bm-metrics.partner_metrics_user_views.message_metrics

Available metrics

Business Messages tracks metrics across four categories: Message, Session, Quality, and Intent.

The following items are common across metrics categories:

Metric Definition Sample values
date The day (in UTC) when the session was created. 2021-12-31
See BigQuery's Date type.
agent_id The ID of the agent whose session information is being calculated. 12345678-9012-3456-789012345678
agent_name The name of the agent. Growing Tree Bank
brand_id The ID of the brand that the agent belongs to. 12345678-9012-3456-789012345678
brand_name The name of the brand that the agent belongs to. Growing Tree International
partner_id The ID of the partner that represents the brand. 12345678-9012-3456-789012345678
partner_name The name of the partner that represents the brand. XYZ Messaging

Message metrics

Message metrics are drawn from the total number of delivered messages for a given day. These metrics include the following items:

Metric Definition Sample values
message_sender Who sent the message. USER,
AGENT
message_type Type of the message. TEXT,
RICH_TEXT,
PHOTO,
RICH_CARD
representative_type Type of the representative that sent this message. Empty for messages sent by users. BOT,
HUMAN
total_messages Total number of delivered messages. 11

Session metrics

A session is a time-bound grouping of interactions, such as messages and events in a conversation. A session begins when a message is sent and a session isn't currently active. A session closes when

  • there are no messages for 24 business hours (for agents with Human-only messaging availability] (/business-communications/business-messages/guides/how-to/agents /availability). If there are < 24 business hours in next 7 days, we will use 24 hours instead.)
  • there are no messages for 30 minutes (for agents with Bot-only messaging availability)
  • you send a survey to a user
  • the user deletes the conversation
  • the user blocks the agent

Session metrics are drawn from the total number of sessions, both active and completed, for a given day. These metrics include the following items:

Metric Definition Sample values
session_initiator Who initiated the session. USER,
AGENT
session_initiating_entry_point The entry point that initiated the session. See EntryPoint.
session_containment_type The types of representative interactions in the conversation, ranging from no representative message to only bot messages to live agent messages. UNRESPONDED,
BOT_ONLY_CONVERSATION,
LIVE_AGENT_REQUEST_UNFULFILLED,
LIVE_AGENT_CONVERSATION,
CALL_INITIATED
has_agent_response The session has at least one response from the agent. TRUE,
FALSE
has_live_agent_response The session has at least one response from a live agent. TRUE,
FALSE
has_live_agent_request The session has at least one live agent request. TRUE,
FALSE
session_depth_bucket How many messages were exchanged during the sessions. 1,
2-3,
4-9,
10-24,
25-
survey_sent If a survey was sent for this session. TRUE,
FALSE
survey_responded If the user responded to the survey. TRUE,
FALSE
positive_survey_response If the survey response has a positive rating. Only meaningful when survey_responded is TRUE. TRUE,
FALSE
total_sessions Total number of sessions. 1000
average_session_depth Average number of messages exchanged during the sessions. 23.11

Quality metrics

Quality metrics are drawn from the total number of completed sessions for a given day. These metrics include the following items:

Metric Definition Sample values
date_calculated The day (in UTC) indicating when the metrics were calculated. 2021-12-31
See BigQuery's Date type.
state Current state of the agent. GOOD_STANDING,
REQUIRES_ATTENTION
survey_count Number of surveys used to calculate the survey score. If there were fewer than 10 survey responses over the last 7 days, then the most recent surveys (maximum 9) from last 35 days are used. Otherwise, all survey responses over the last 7 days are used. 84
survey_score Percentage of positive survey responses. Range from 0 to 1.
session_count Number of sessions used to calculate MRR. If there were fewer than 3 sessions over the last 7 days, then the most recent sessions (maximum 9) from the last 35 days are used. Otherwise all sessions over the last 7 days are used. 143
mrr Merchant response rate (MRR). The number of sessions that had agent responses (bot or live agent) relative to the total number of sessions. Range from 0 to 1.
hmrr Human merchant response rate (HMRR). The number of sessions that had live agent responses relative to the total number of sessions that contained one or more live agent requests or live agent responses. Range from 0 to 1.
live_agent_request_count Number of live agent requests used to calculate hmrr. If there were fewer than 10 live agent requests over the last 7 days, then the 9 most recent live agent requests from the last 35 days are used. Otherwise all live agent requests over the last 7 days are used. 42
suspension_date The day (in UTC) when the agent may be suspended due to low quality. Only set for agents with `REQUIRES_ATTENTION` state. 2021-12-31
See BigQuery's Date type.

Intent metrics

Intent metrics use machine learning to identify the intent of sessions with your agent. Intent metrics are drawn from user messages in completed sessions for a given day. English only. These metrics include the following items:

Metric Definition Sample values
intent The session intent type. SALES_LEAD,
SERVICE
count Count of sessions with the specified intent type. 11

Track custom metrics

If you have different metrics preferences or require more in-depth information than is available in the quality report, you can capture your own metrics data.

Here are some metrics that you might capture:

  • Customer satisfaction. Send surveys to capture and track user satisfaction. You can customize your survey questions to suit your data collection requirements.
  • Messages sent. Capture details each time your agent sends a message.
  • Messages received. Capture details each time your agent receives a message.
  • Agent response rates. Capture how often your agent responds to user messages. Track the differences between bot and live agent response rates.
  • Agent response times. Capture how long it takes your agent to respond to user messages. Track the differences between bot and live agent response times.
  • Delivery receipts. Capture DELIVERED events.
  • Read receipts. Capture READ events.
  • User engagement. Capture IS_TYPING events, the number of user responses, and the time difference between DELIVERED or READ events and the following user responses.

  • Interaction type. Identify how users respond to your messages. Use meaningful postback data to track suggested replies and actions, and monitor response formats to identify if users send images or plain text messages.

With your metrics captured, you can parse and organize them however best suits your business needs.

Performance monitoring and suspension thresholds

To assess agent performance, Business Messages tracks multiple metrics to ensure that agents provide a good user experience. If an agent falls below a metrics threshold, Business Messages warns you via weekly agent status emails. If an agent remains below a threshold for 28 days, Business Messages may suspend the agent. If your agent is suspended, see Fix a suspended agent.

Business Messages uses the following metrics to track agent performance:

Metric Description Volume threshold Normal volume threshold Low volume threshold
Customer satisfaction (CSAT) The customer satisfaction score indicates the overall experience the customer had after engaging with an agent. 10 surveys 80% 0%
Merchant response rate (MRR) The number of sessions that had agent responses (bot or live agent) relative to the total number of sessions. 3 sessions 95% 60%
Human merchant response rate (HMRR) The number of sessions that had live agent responses relative to the total number of sessions that contained one or more live agent requests or live agent responses 10 live agent requests 95% 0%

Each metric's volume is measured against an independent metric volume threshold, above or below which, determines if the performance of the agent can be measured against normal volume threshold or low volume threshold for that metric.

For example, if an agent has 9 surveys, 10 sessions and 9 live agent requests, then the performance of the agent is measured against a threshold of 0 for CSAT, 95 for MRR and 0 for HMRR. If the agent is below any of these thresholds, then the agent is marked in a 'Requires attention' state.

If the agent continues to be in 'Requires attention' state for 28 days or more, then the agent may be suspended.

Improve your metrics

If your agent performance is low, make sure your agent does the following.

Process and respond to messages without technical issue

  • Confirm that your webhook receives, acknowledges, processes, and routes all incoming messages, regardless of message type or content.
  • Make sure that your agent has an effective fallback strategy in case a required field is missing from a message payload. Missing or dropping user messages because a field is missing results in a bad user experience.
  • Confirm that your agent successfully sends all expected message types.

Reply intelligently and compassionately

  • Reply to every message. Automated replies can help you address common questions.
  • Be aware of users' conversation entry points and respond accordingly. For example, users coming from Google Maps are more likely interested in location-specific information like opening hours and stock checks.
  • For location-based messages, route questions to the individuals best suited to respond to them.
  • If a live agent isn't available, use automation to response to the user. Let them know your agent's chat hours and how else they might reach someone (for example, a support phone number).

Provide guidance

  • Use a welcome message, written in a friendly and welcoming tone, to set expectations about what your agent can do.
  • Use conversation starters to guide users through common or automated tasks.
  • If a user asks a question your agent can't handle, respond with guidance about the kinds of questions it can answer.

Handle data carefully

  • Whenever possible, gather location and other data from messages you receive rather than asking for it.
  • Don't request sensitive information (log in credentials, banking/credit details, personal identification numbers)
  • Only ask for data when it's required. Data overreach creates a bad user experience.
  • When you need data, be clear about why and how you'll use it.

Behave asynchronously

  • Don't mark conversations as inactive. Communications over Business Messages is ansynchronous, so users may not respond immediately.
  • Don't follow up on unresponsive users. Users aren't disconnected after navigating away from the conversation.

For additional guidance on achieving agent performance, see the requirements and guidelines.