To prevent undesirable content while in the search indexes, website owners can instruct spiders not to crawl selected information or directories in the typical robots.txt file in the root Listing in the area. In addition, a web site could be explicitly excluded from the search engine's databases by using a https://news.rafeeg.ae