We've created a set of resources to help you ensure your site remains available and accessible to all during the COVID-19 situation.

The Chromium Chronicle: Catching UI Regressions with Pixel Tests

Episode 10: January, 2020

by Sven Zheng in Bellevue, WA

Chrome’s testing strategy relies heavily on automated functional correctness tests and manual testing, but neither of these reliably catch minor UI regressions. Use pixel tests to automate testing your desktop browser UI.

When writing a pixel test, avoid flakiness by (1) disabling animation, (2) using mock data, and (3) testing the minimum possible surface area.

Here is a sample image used to verify pixel correctness for the omnibox:

Omnibox image used for pixel comparison.

And the code to verify the browser matches this image:


IN_PROC_BROWSER_TEST_F(SkiaGoldDemoPixelTest, TestOmnibox) {
  // Always disable animation for stability.
  ui::ScopedAnimationDurationScaleMode disable_animation(
      ui::ScopedAnimationDurationScaleMode::ZERO_DURATION);
  GURL url("chrome://bookmarks");
  AddTabAtIndex(0, url, ui::PageTransition::PAGE_TRANSITION_FIRST);
  auto* const browser_view = BrowserView::GetBrowserViewForBrowser(browser());
  // CompareScreenshot() takes a screenshot and compares it with the golden image,
  // which was previously human-approved, is stored server-side, and is managed
  // by Skia Gold. If any pixels differ, the test will fail and output a link
  // for the author to triage the new image.
  bool ret = GetPixelDiff().CompareScreenshot("omnibox",
      browser_view->GetLocationBarView());
  EXPECT_TRUE(ret);
}

This code lives at chrome/test/pixel/demo/skia_gold_demo_pixeltest.cc. The relevant headers are skia_gold_pixel_diff.h for unit tests and browser_skia_gold_pixel_diff.h for browser tests.

The pixel diff and approval process is powered by Skia Gold. Skia Gold pixel tests provide a visual approval workflow, and allow developers to accept small flakes by approving multiple gold images.

Currently the test suite is running on the Windows FYI bot. Browser tests and Views unit tests are supported.

Was this page helpful?
Yes
What was the best thing about this page?
It helped me complete my goal(s)
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It had the information I needed
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It had accurate information
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It was easy to read
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
Something else
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
No
What was the worst thing about this page?
It didn't help me complete my goal(s)
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It was missing information I needed
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It had inaccurate information
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It was hard to read
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
Something else
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.

rss_feed Subscribe to our RSS or Atom feed and get the latest updates in your favorite feed reader!