Files
SREBOT/web/public/robots.txt
T

8 lines
217 B
Plaintext

# Robots.txt for Toothless SQB Bot Website
# This file prevents web crawlers from scraping the site
User-agent: *
Disallow: /
# Block all crawlers from all content
# This includes search engines, scrapers, and bots