Crawl Errors now reports soft 404s
Stay organized with collections
Save and categorize content based on your preferences.
Monday, June 07, 2010
Today we're releasing a feature to help you discover if your site serves undesirable
soft or crypto 404s. A soft 404 occurs
when a webserver responds with a 200 OK
HTTP response code
for a page that doesn't exist rather than the appropriate 404 Not Found.
Soft 404s can limit a site's crawl coverage by search engines because these duplicate
URLs may be crawled instead of pages with unique content.
The web is infinite, but the time search engines spend crawling your site is limited. Properly
reporting non-existent pages with a 404 or 410 response code can improve
the crawl coverage of your site's best content. Additionally, soft 404s can
potentially be confusing for your site's visitors as described in our past blog post,
Farewell to soft 404 pages.
You can find the new soft 404 reporting feature under the Crawl errors section in
Webmaster Tools.
Here's a list of steps to correct soft 404 errors to help both Google and your users:
- Check whether you have
soft 404 errors listed in Webmaster Tools
- For the
soft 404 errors, determine whether the URL:
-
Contains the correct content and properly returns a
200 response (not actually a
soft 404)
- Should
301 redirect to a more accurate URL
- Doesn't exist and should return a
404 or 410 response
-
Confirm that you've configured the proper HTTP Response by using
Fetch as Googlebot
in Webmaster Tools
-
If you now return
404 errors, you may want to customize your 404 page to aid
your users. Our
custom 404 widget
can help.
We hope that you're now better enabled to find and correct soft 404 errors on your
site. If you have feedback or questions about the new soft 404 error reporting
feature or any other Webmaster Tools feature, please share your thoughts with us in the
Webmaster Help Forum.
Written by
Jonathan Simon,
Webmaster Trends Analyst
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
[null,null,[],[[["\u003cp\u003eSoft 404 errors occur when a web server responds with a \u003ccode\u003e200 OK\u003c/code\u003e status for a non-existent page, hindering search engine crawling and user experience.\u003c/p\u003e\n"],["\u003cp\u003eGoogle has released a new feature within Webmaster Tools to help site owners identify and address soft 404 errors.\u003c/p\u003e\n"],["\u003cp\u003eSite owners should check Webmaster Tools for soft 404 errors and either ensure content correctness, implement redirects, or return proper 404/410 responses.\u003c/p\u003e\n"],["\u003cp\u003eUtilizing "Fetch as Googlebot" and potentially customizing 404 pages can further improve site health and user experience.\u003c/p\u003e\n"]]],["The new feature in Webmaster Tools identifies `soft 404` errors, where a server incorrectly returns a `200 OK` code for non-existent pages. To correct these, users should check for listed `soft 404`s, determine if URLs should have a `200`, `301` redirect, or `404`/`410` response, and confirm the HTTP response code with Fetch as Googlebot. Customizing the `404` page is suggested. This improves crawl coverage and user experience.\n"],null,["# Crawl Errors now reports soft 404s\n\nMonday, June 07, 2010\n\n\nToday we're releasing a feature to help you discover if your site serves undesirable\n`soft` or `crypto` `404`s. A `soft 404` occurs\nwhen a webserver responds with a `200 OK`\n[HTTP response code](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)\nfor a page that doesn't exist rather than the appropriate `404 Not Found`.\n`Soft 404`s can limit a site's crawl coverage by search engines because these duplicate\nURLs may be crawled instead of pages with unique content.\n\n\nThe web is infinite, but the time search engines spend crawling your site is limited. Properly\nreporting non-existent pages with a `404` or `410` response code can improve\nthe crawl coverage of your site's best content. Additionally, `soft 404`s can\npotentially be confusing for your site's visitors as described in our past blog post,\n[Farewell to `soft 404` pages](/search/blog/2008/08/farewell-to-soft-404s).\n\n\nYou can find the new `soft 404` reporting feature under the Crawl errors section in\n[Webmaster Tools](https://search.google.com/search-console).\n\n\nHere's a list of steps to correct `soft 404` errors to help both Google and your users:\n\n1. Check whether you have `soft 404` errors listed in Webmaster Tools\n2. For the `soft 404` errors, determine whether the URL:\n 1. Contains the correct content and properly returns a `200` response (not actually a `soft 404`)\n 2. Should `301` redirect to a more accurate URL\n 3. Doesn't exist and should return a `404` or `410` response\n3. Confirm that you've configured the proper HTTP Response by using [Fetch as Googlebot](/search/blog/2009/10/fetch-as-googlebot-and-malware-details) in Webmaster Tools\n4. If you now return `404` errors, you may want to customize your `404` page to aid your users. Our [custom `404` widget](/search/docs/crawling-indexing/http-network-errors#pagegone) can help.\n\n\nWe hope that you're now better enabled to find and correct `soft 404` errors on your\nsite. If you have feedback or questions about the new `soft 404` error reporting\nfeature or any other Webmaster Tools feature, please share your thoughts with us in the\n[Webmaster Help Forum](https://support.google.com/webmasters/community).\n\n\nWritten by\n[Jonathan Simon](/search/blog/authors/jonathan-simon),\nWebmaster Trends Analyst"]]
