User-agent: * Allow: / Disallow: /node_modules/ # Explicitly allow AI retrieval bots (real-time AI search = traffic) User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Claude-Web Allow: / User-agent: Applebot-Extended Allow: / User-agent: GoogleExtended Allow: / User-agent: anthropic-ai Allow: / # Block junk scraper User-agent: CCBot Disallow: / Sitemap: https://superiorpetwaste.com/sitemap.xml