We know how important it is for you to prevent search engines from displaying your content on Google or other search engines, and we’ve come up with two solutions for you:
1. No-index/no-follow header for the Redirect to file option
Since the Download Monitor v4.5.9 update, we’ve added a no-index/no-follow header for the Redirect to file option, meaning that the URL in question has a no-index/no-follow directive in the header:
- The no-index metatag tells a search engine that even though it can crawl the page, it cannot add the page to its search index,
- The no-follow tag disallows search engines from crawling the links on that page.
So, having no-index/no-follow directives in the Redirect to file page will prevent access to the search engine’s index and, therefore, won’t show up on search engine results pages. In a nutshell, the Redirect to file page will have all its links ignored by Google and other search engines.
2. Robots.txt generation/protection setting
You can also find the Robots.txt generation function in the Settings > Advanced > Miscellaneous that helps you add the robots.txt. A robots.txt is a text file that tells the search engines where they are not allowed to go on a site.
- if there is no robots.txt, we will create it for you and add the code;
- if there is a robots.txt and it’s a real file, then click on Regenerate protection for uploads folder, and we’ll add the code;
Example:
User-agent: *
Disallow: /dlm_uploads/- if it is a virtual file, we’ll warn you to modify it; otherwise, you risk losing what you already have in robots.txt.