# $Id: robots.txt,v 1.6 2006/08/16 22:03:56 dries Exp $## robots.txt## This file is to prevent the crawling and indexing of certain parts# of your site by web crawlers and spiders run by sites like Yahoo!# and Google. By telling these "robots" where not to go on your site,# you save bandwidth and server resources.## This file will be ignored unless it is at the root of your host:# Used:    http://example.com/robots.txt# Ignored: http://example.com/site/robots.txt## For more information about the robots.txt standard, see:# http://www.robotstxt.org/wc/robots.html## For syntax checking, see:# http://www.sxw.org.uk/computing/robots/check.htmlUser-agent: *Crawl-delay: 10# DirectoriesDisallow: /database/Disallow: /includes/Disallow: /misc/Disallow: /modules/Disallow: /sites/Disallow: /themes/Disallow: /scripts/Disallow: /updates/Disallow: /profiles/Disallow: /taxonomy/
Disallow: /nyingmaforum/# FilesDisallow: /xmlrpc.phpDisallow: /cron.phpDisallow: /update.phpDisallow: /install.phpDisallow: /INSTALL.mysql.txtDisallow: /INSTALL.pgsql.txtDisallow: /CHANGELOG.txtDisallow: /MAINTAINERS.txtDisallow: /LICENSE.txtDisallow: /UPGRADE.txt# Paths (clean URLs)Disallow: /admin/Disallow: /aggregator/Disallow: /comment/reply/Disallow: /contact/Disallow: /logout/Disallow: /node/add/Disallow: /search/Disallow: /user/register/Disallow: /user/password/# Paths (no clean URLs)Disallow: /?q=admin/Disallow: /?q=aggregator/Disallow: /?q=comment/reply/Disallow: /?q=contact/Disallow: /?q=logout/Disallow: /?q=node/add/Disallow: /?q=search/Disallow: /?q=user/password/Disallow: /?q=user/register/