Sitemap: http://techwelkin.com/sitemap_index.xml User-agent: * Disallow: /?s=* Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/themes/ Disallow: /testwork/ Disallow: /recommends/ Disallow: /tools/top-blogs/*/*.txt Disallow: /out/ Disallow: /feed/ Disallow: */feed/ Disallow: /readme.html User-agent: googlebot Allow: /wp-includes/ Allow: /wp-content/plugins/ Allow: /wp-content/themes/ User-agent: MJ12bot Disallow: / User-agent: HTTrack Disallow: / User-agent: Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com) Disallow: /