# Disallow all crawlers access to certain pages. User-agent: * Disallow: /files/ Disallow: /themes/ Disallow: /resources/ Disallow: /hidden/