Blocking Page Indexing Guidelines
- Meta NoIndex is recommended because it tells the search engines they can visit a page but are not allowed to display the URL in results
- Used to exclude individual pages from search engine indices
- Add the noindex meta tag <meta name="robots" content="noindex">
- Robots.txt is not recommended because it tells the search engines that the specified page can be indexed and displayed in results but that the given URL should not be crawled
- NoFollow links are not recommended because there are other ways for the search engines to discover the page through links from other pages, analytics, etc.