# Crawling is allowed on purpose, on every host this app answers on. # # The test deployment (buams.com) is kept out of search results with a # `X-Robots-Tag: noindex` header instead of a rule here, because the two are # not interchangeable. A URL blocked by robots.txt is never fetched, so the # noindex on it is never read, and Google goes on listing the URL it already # knows. Blocking here would keep buams.com in the results, not remove it. # # See config/seo.php and app/Http/Middleware/NoindexNonCanonicalHost.php. User-agent: * Allow: / Sitemap: https://buams.ai/sitemap.xml