# Internal GSA appliance User-agent: gsa-crawler Disallow: # Everyone else User-agent: * Disallow: /feedback/ Disallow: /readers/ Disallow: /api/ Disallow: /bugs/ Disallow: /rankit/ Disallow: /services/ Disallow: /spamscan/ Disallow: /testing/