# robots.txt for https://boffo.com.au/ sitemap: https://boffo.com.au/sitemaps-1-sitemap.xml # live - don't allow web crawlers to index cpresources/ or vendor/ User-agent: * Disallow: /vendor/ Disallow: /.env # Delay SemRush Site Audit to 5 second rate limit User-agent: SiteAuditBot Crawl-delay: 5