# Robots.txt file from http://steel.sci99.com # All robots will spider the domain User-agent: * Disallow: /news/channel Disallow: /curves/channel Disallow: /price/ccj/channel Disallow: /channel/*news Disallow: /channel2 Disallow: /price2 Disallow: /*PageStart Disallow: /*default.aspx Disallow: /*?l=1$ Disallow: /Default.aspx Disallow: /*Default.aspx Disallow: /*?bdclkid