Home > Uncategorized > Kraken Crawler

Kraken Crawler

March 25th, 2013 Leave a comment Go to comments

Recently I have seen “kraken-crawler/0.2.0” hitting my site. This is a bot used by Kontera (advertising company) to “better understand and analyze your site’s content” (according to their support staff).

Apparently the crawler adheres to robots.txt so you can block it by adding:

User-agent: kraken-crawler/*
Disallow: /

The URL to their crawler info is broken so it’s hard to get an idea of what this is used for. If you are also seeing this bot, hopefully this helps you.

Categories: Uncategorized Tags:
  1. No comments yet.
  1. No trackbacks yet.