Find unintended pages with a site: search and the Index Coverage report, then add a noindex tag to thank-you pages, thin tag archives and internal search results, and leave them crawlable so Google sees it.

Search for your own site

Type site: followed by your domain into Google, with no space after the colon, and scroll past the pages you recognise.

Most small business sites turn up a few that nobody meant to publish. A page that says only thank you, your message has been sent. A tag archive called hops, holding one post.

A page of results for something a stranger once typed into the site's search box.

None of them was published on purpose. The site's software created each one, and nothing told Google to leave it alone.

How Google finds them

The first catches people out. A sitemap is the list of addresses a site hands to search engines, and many sitemap tools add every published page without asking.

Leaving a page out of the menu hides it from visitors, not from Google.

What a thank-you page in Google costs

A thank-you page is meant to be reached one way: by sending the form.

Once it is in search results, people land on it without sending anything, thanked for an enquiry they never made.

It also breaks the counting. If Analytics counts arrivals on that page as goals, as it should, every visitor from Google becomes an enquiry that never happened.

And where the page holds a download offered in exchange for an email address, anyone searching can take it without signing up.

Tag archives and search results

WordPress creates an archive page for every tag used on the blog. A post given six tags produces six archives, most of them holding only that post.

Those pages say nothing the post does not. They can turn up in results in place of the post itself, and whoever lands on one finds a list of one and a sidebar.

Internal search results are worse. Anyone can link to a search on your site using whatever words they like, and the results page repeats those words back, under your name.

Finding them

The site: search is the quick look. Adding a word after the domain, such as thank, tag or search, gathers each kind together.

The Index Coverage report in the new Search Console is the thorough one. Among its valid pages sits a list called indexed, not submitted in sitemap: pages Google indexed without the sitemap asking.

If your sitemap lists the pages you actually want, that list comes close to a list of pages nobody chose.

The pages tab of the Performance report then shows which of them have already appeared in results. Those come first.

A worked example

A homebrew supply shop in a light industrial unit sells malt, hops, yeast and brewing equipment online and from its counter.

Its blog carries about sixty recipes, each tagged with every hop, malt and yeast it uses, which over four years produced more than three hundred tag archives.

Searching one hop variety with the shop's name brought up a tag archive, not the product page for that hop. Further down sat the thank-you page for the free brewing guide, download link and all.

Analytics had been counting far more guide downloads than the email list had gained subscribers.

Whoever maintains the site added noindex to the thank-you pages, search results and tag archives, cut the tags to a dozen beer styles, and cleaned out the sitemap. Each recipe now links to the products it uses.

As Google recrawled, the old pages moved to the excluded list, and the guide's goal count came into line with the sign-ups.

The tag that keeps a page out

The reliable way to keep a page out of Google is a noindex instruction: a line in the page's code asking search engines not to show it.

On WordPress, most SEO plugins set it with a switch, for single pages and for whole groups such as tag archives. Many already noindex search results; check that yours does.

Take the same pages out of the sitemap. A sitemap asking Google to index a page marked noindex gives two opposite instructions, and the coverage report lists it as an error.

Noindex does not touch Analytics. The goal still counts every real enquiry.

Why robots.txt is the wrong first step

The robots.txt file tells search engines which addresses not to crawl. People often reach for it first, and it does a different job.

Blocking a page from being crawled does not keep it out of results. Google can still list an address it has seen linked, usually with a note saying no description is available.

And a page Google may not crawl is a page it cannot read, so it never sees the noindex tag. Add noindex first and let the pages drop out.

Pages that should not exist

Some of what turns up does not need hiding. It needs removing.

A landing page for a 2016 promotion, still quoting that year's price, should go, redirected to the current page for the same service. So should a tag holding one post, or a test page published by accident.

For anything sensitive, the Remove URLs tool in the old Search Console hides a page for about ninety days, time enough to delete it or mark it noindex.

The counter-case

Some sites have nothing to find: no blog, no search box, and a form that confirms on the same page. The check takes two minutes.

Some archives deserve their place. A category page with a real introduction and a dozen posts behind it is a useful page, and can stay.

The setting that hides a whole site from search engines belongs on a test copy, never a live one.

Find what Google holds that nobody chose, mark it noindex, and keep the sitemap to the pages you want found.

This week

  1. Search site: and your domain on Google.
  2. List every page nobody meant to publish.
  3. Read the indexed, not submitted list in Index Coverage.
  4. Add noindex to thank-you and search result pages.
  5. Keep, merge or noindex each tag archive.
  6. Take those pages out of the sitemap.
  7. Leave robots.txt alone until they have dropped out.

Step four fixes the counting as well as the search results: once the thank-you page is out of Google, every visit to it is a real enquiry again.

Attachment pages matter more now that image search shows the page behind each photograph, as covered in Google turns twenty and changes image search.

Frequently asked questions

Why is my thank-you page on Google?

Because nothing told Google to leave it out. Keeping it out of the menu hides it from visitors only, and many sitemap tools list every published page automatically.

What harm does an indexed thank-you page do?

People land on it without sending anything, and Analytics counts each arrival as an enquiry. Any download the page holds is free to anyone searching.

Should tag archives appear in Google?

Usually not. A tag holding one or two posts is a thin page that can turn up instead of the post itself. Keep only archives with a real introduction.

How do I find pages I never meant to publish?

Search Google for site: and your domain, then read the list of pages indexed but not submitted in your sitemap, in the Index Coverage report.

Why not block them in robots.txt?

Robots.txt stops crawling, not indexing. A blocked page can stay in results through links from elsewhere, and Google cannot read a noindex tag on a page it may not crawl.

How quickly do noindexed pages disappear?

As Google recrawls them: days for busy pages, weeks for obscure ones. For anything sensitive, the Remove URLs tool in the old Search Console hides a page sooner.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

Thank-you page showing up in Google?

Send us the address. We will list the pages Google holds that you never meant to publish, and keep them out.

Start a Conversation