-
Rotzbua authored
Disallow crawlers to crawl and index page. Crawling causes traffic and high load. Indexation max cause unintended attention, e.g. spam, .. Reference: https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag
Rotzbua authoredDisallow crawlers to crawl and index page. Crawling causes traffic and high load. Indexation max cause unintended attention, e.g. spam, .. Reference: https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag