User-agent: * Allow: / Disallow: /dashboard Disallow: /auth/ Disallow: /billing/ Sitemap: https://scrapeio.com/sitemap.xml