Blocking Indexing

Blocking Indexing

Blocking Page Indexing Guidelines

  • Meta NoIndex is recommended because it tells the search engines they can visit a page but are not allowed to display the URL in results
    • Used to exclude individual pages from search engine indices
    • Add the noindex meta tag <meta name="robots" content="noindex">
  • Robots.txt is not recommended because it tells the search engines that the specified page can be indexed and displayed in results but that the given URL should not be crawled
  • NoFollow links are not recommended because there are other ways for the search engines to discover the page through links from other pages, analytics, etc.

Sources