User-agent: * Disallow: /temp/ Disallow: /classifieds/mylisting/ Disallow: /xmldir/ Disallow: /rss_files/ Disallow: /rss/ Disallow: /app_feed/ Disallow: /xml_feed/ Disallow: /*spam.html$ Disallow: /*miscat.html$ Disallow: /*classifieds/dashboard_search.php* Disallow: /*/hsearch/ Allow: / User-agent : dwaarbot Disallow : / User-agent: seekbot Disallow: / User-agent: Pete-Spider Light Disallow: / User-agent: dotbot Disallow: / User-agent: twiceler Disallow: / User-agent: 008 Disallow: / User-agent: Baiduspider Disallow: / User-agent: OpenindexSpider Disallow: / User-agent: AhrefsBot Disallow: / User-agent: sistrix Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: proximic Disallow: / User-agent: MJ12bot Disallow: / User-Agent: SputnikBot Disallow: / User-agent: Riddler Disallow: / User-agent: grapeshot Disallow: / Sitemap: https://www.click.in/sitemap_index.xml Sitemap: https://www.click.in/sitemap_latest_index.xml Sitemap: https://www.click.in/sitemap_category_filter_index.xml