|
Spider |
Hits |
% of Total |
Visits |
1 |
Google Spider |
5,913,915 |
24.80% |
50,991 |
2 |
BaiDu Spider |
3,428,096 |
14.37% |
38,772 |
3 |
Wespe.de Spider |
3,102,342 |
13.01% |
690,223 |
4 |
Bing Spider |
2,435,350 |
10.21% |
15,310 |
5 |
Mozilla/5.0+(compatible;+DotBot/ 1.1;+http:/ / www.opensiteexplorer.org/ dotbot,+help@moz.com) |
1,904,064 |
7.98% |
8,552 |
6 |
magpie-crawler/1.1+(U;+Linux+amd64;+en-GB ;++http:/ / www.brandwatch.net) |
1,394,701 |
5.85% |
490 |
7 |
Yandex Spider |
1,367,563 |
5.73% |
17,061 |
8 |
Yahoo Spider |
1,147,740 |
4.81% |
2,795 |
9 |
Mozilla/5.0+(compatible;+SemrushBot/ 1.2~bl;++http:/ / www.semrush.com/ bot.html) |
556,102 |
2.33% |
13,335 |
10 |
Treato-Bot/1.0+(TREATO+LTD,+www.treato.co m,+mailto:+botadmin@treato.com) |
412,039 |
1.73% |
90 |
11 |
Vegi+bot+(we+follow+your+robots.txt+setti ngs+before+crawling,+you+can+slow+down+th e+bot+by+change+the+Crawl-Delay+parameter +in+the+settings.if+you+have+an+enquiry,+ please+email+to:+abuse-report@terrykylese oagency.com) |
405,702 |
1.70% |
11 |
12 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.4.8;+http:/ / mj12bot.com/ ) |
389,391 |
1.63% |
2,416 |
13 |
Mozilla/5.0+(compatible;+GrapeshotCrawler / 2.0;++http:/ / www.grapeshot.co.uk/ crawler.php) |
250,180 |
1.05% |
11,287 |
14 |
Mozilla/5.0+(compatible;+BLEXBot/ 1.0;++http:/ / webmeup-crawler.com/ ) |
203,923 |
0.86% |
412 |
15 |
Quick-Crawler+(+https:/ / www.scrapinghub.com/ ) |
104,479 |
0.44% |
79 |
16 |
Mozilla/5.0+(compatible;+proximic;++http: / / www.proximic.com/ info/ spider.php) |
103,712 |
0.43% |
88,514 |
17 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.4.7;+http:/ / mj12bot.com/ ) |
99,818 |
0.42% |
469 |
18 |
Mozilla/5.0+(compatible;+Cliqzbot/ 2.0;++http:/ / cliqz.com/ company/ cliqzbot) |
98,562 |
0.41% |
1,652 |
19 |
Google image crawler |
84,905 |
0.36% |
1,495 |
20 |
Yandex Images Spider |
71,177 |
0.30% |
4,323 |
21 |
Facebook external hit |
53,907 |
0.23% |
40,788 |
22 |
Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+ en-GB;+rv:1.0;+trendictionbot0.5.0;+trend iction+search;+http:/ / www.trendiction.de/ bot;+please+let+us+know+of+any+problems;+ web+at+trendiction.com)+Gecko/ 20071127+Firefox/ 3.0.0.11 |
53,188 |
0.22% |
20,130 |
23 |
Mozilla/5.0+(compatible;+Steeler/ 3.5;+http:/ / www.tkl.iis.u-tokyo.ac.jp/ ~crawler/ ) |
37,352 |
0.16% |
585 |
24 |
Mozilla/5.0+(compatible;+Pinterestbot/ 1.0;++http:/ / www.pinterest.com/ bot.html) |
29,013 |
0.12% |
3,806 |
25 |
Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10 _10_1)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 40.0.2214.111+Safari/ 537.36+moatbot |
22,886 |
0.10% |
1,963 |
26 |
robots |
21,189 |
0.09% |
4,209 |
27 |
Mozilla/5.0+(compatible;+SeznamBot/ 3.2;++http:/ / napoveda.seznam.cz/ en/ seznambot-intro/ ) |
14,000 |
0.06% |
958 |
28 |
istellabot/t.1.13 |
13,125 |
0.06% |
232 |
29 |
R6_FeedFetcher_(www.radian6.com/ crawler) |
9,574 |
0.04% |
47 |
30 |
Mozilla/5.0+(compatible;+Go-http-client/ 1.1;++centurybot9@gmail.com) |
9,524 |
0.04% |
255 |
31 |
Pu_iN+Crawler+(+http:/ / semanticjuice.com/ ) |
7,278 |
0.03% |
29 |
32 |
Mozilla/5.0+(compatible;+linkdexbot/ 2.0;++http:/ / www.linkdex.com/ bots/ ) |
6,926 |
0.03% |
3,222 |
33 |
CCBot/2.0+(http://commoncrawl.org/ faq/ ) |
6,494 |
0.03% |
143 |
34 |
Mozilla/5.0+(compatible;+SemrushBot-BA;++ http:/ / www.semrush.com/ bot.html) |
6,372 |
0.03% |
0 |
35 |
Mozilla/5.0+(compatible;+Linux+x86_64;+Ma il.RU_Bot/ 2.0;++http:/ / go.mail.ru/ help/ robots) |
6,057 |
0.03% |
1,690 |
36 |
Mozilla/5.0+(Windows+NT+6.1;+WOW64)+Apple WebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 50.0.2661.102+Safari/ 537.36;+360Spider |
5,999 |
0.03% |
3,244 |
37 |
Mozilla/5.0+(compatible;+YandexImageResiz er/ 2.0;++http:/ / yandex.com/ bots) |
4,589 |
0.02% |
3,788 |
38 |
Sogou Spider |
4,497 |
0.02% |
2,161 |
39 |
mediawords+bot+(http:/ / cyber.law.harvard.edu) |
4,378 |
0.02% |
145 |
40 |
Mozilla/5.0+(iPhone;+CPU+iPhone+OS+8_1+li ke+Mac+OS+X)+AppleWebKit/ 600.1.4+(KHTML,+like+Gecko)+Version/ 8.0+Mobile/ 12B411+Safari/ 600.1.4+(compatible;+YandexMobileBot/ 3.0;++http:/ / yandex.com/ bots) |
3,301 |
0.01% |
1,974 |
41 |
Mozilla/5.0+(Windows+NT+6.1;+WOW64)+Apple WebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 50.4.2661.102+Safari/ 537.36;+360Spider |
3,021 |
0.01% |
1,410 |
42 |
Mozilla/5.0+(compatible;+SiteExplorer/ 1.1b;++http:/ / siteexplorer.info/ Backlink-Checker-Spider/ ) |
2,981 |
0.01% |
101 |
43 |
Mozilla/5.0+(compatible;+Linux+x86_64;+Ma il.RU_Bot/ Img/ 2.0;++http:/ / go.mail.ru/ help/ robots) |
2,871 |
0.01% |
1,344 |
44 |
Mozilla/5.0+(compatible;+spbot/ 5.0.3;++http:/ / OpenLinkProfiler.org/ bot+) |
2,747 |
0.01% |
18 |
45 |
MBCrawler/1.0+(https:/ / monitorbacklinks.com) |
2,150 |
0.01% |
302 |
46 |
Mozilla/5.0+(compatible;+ExtLinksBot/ 1.5;++https:/ / extlinks.com/ Bot.html) |
2,086 |
0.01% |
34 |
47 |
Mozilla/5.0+(compatible;+TinEye-bot/ 1.31;++http:/ / www.tineye.com/ crawler.html) |
2,065 |
0.01% |
367 |
48 |
Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10 _10_1)+AppleWebKit/ 600.2.5+(KHTML,+like+Gecko)+Version/ 8.0.2+Safari/ 600.2.5+(Applebot/ 0.1;++http:/ / www.apple.com/ go/ applebot) |
1,972 |
0.01% |
665 |
49 |
Mozilla/5.0+(Windows+NT+6.1;+WOW64)+Apple WebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 50.2.2661.102+Safari/ 537.36;+360Spider |
1,911 |
0.01% |
724 |
50 |
PDF+Drive+Crawler+1.05+(+https:/ / www.pdfdrive.com/ +) |
1,828 |
0.01% |
737 |
51 |
Mozilla/5.15+(Windows;+U;+Windows+NT+5.1; +en-US;+rv:1.8.1.6)+Gecko/ 20070725+Firefox/ 2.0.0.6+-+James+BOT+-+WebCrawler+http:/ / cognitiveseo.com/ bot.html |
1,820 |
0.01% |
865 |
52 |
Twitterbot/1.0 |
1,437 |
0.01% |
621 |
53 |
GarlikCrawler/1.2+(http:/ / garlik.com/ ,+crawler@garlik.com) |
1,403 |
0.01% |
92 |
54 |
Mozilla/5.0+(compatible;+NetSeer+crawler/ 2.0;++http:/ / www.netseer.com/ crawler.html;+crawler@netseer.com) |
1,195 |
0.01% |
822 |
55 |
YisouSpider |
1,078 |
0.00% |
471 |
56 |
netEstate+NE+Crawler+(+http:/ / www.website-datenbank.de/ ) |
1,020 |
0.00% |
2 |
57 |
Mozilla/5.0+(compatible;+Yeti/ 1.1;++http:/ / naver.me/ bot) |
966 |
0.00% |
882 |
58 |
adbeat_bot |
892 |
0.00% |
453 |
59 |
Mozilla/5.0+(compatible;+linkdexbot/ 2.2;++http:/ / www.linkdex.com/ bots/ ) |
851 |
0.00% |
716 |
60 |
MaxPointCrawler/Nutch-1.10+(maxpoint.craw ler+at+maxpointinteractive+dot+com) |
847 |
0.00% |
108 |
61 |
Mozilla/5.0+(compatible;+coccocbot-web/ 1.0;++http:/ / help.coccoc.com/ searchengine) |
818 |
0.00% |
470 |
62 |
IAS+crawler+(ias_crawler;+http:/ / integralads.com/ site-indexing-policy/ ) |
726 |
0.00% |
201 |
63 |
Mozilla/5.0+(compatible;+Codewisebot/ 2.0;++http:/ / www.nosite.com/ somebot.htm) |
646 |
0.00% |
186 |
64 |
Alexa Spider |
624 |
0.00% |
111 |
65 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+DI NOSAUR+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
589 |
0.00% |
20 |
66 |
Mozilla/5.0+(compatible;+Mappy/ 1.0;++http:/ / mappydata.net/ bot/ ) |
539 |
0.00% |
20 |
67 |
Mozilla/5.0+(compatible;+Findxbot/ 1.0;++http:/ / www.findxbot.com) |
538 |
0.00% |
339 |
68 |
Mozilla/5.0+(compatible;+MojeekBot/ 0.6;++https:/ / www.mojeek.com/ bot.html) |
535 |
0.00% |
259 |
69 |
Mozilla/5.0+(iPhone;+CPU+iPhone+OS+8_0+li ke+Mac+OS+X)+AppleWebKit/ 600.1.3+(KHTML,+like+Gecko)+Version/ 8.0+Mobile/ 12A4345d+Safari/ 600.1.4+moatbot |
522 |
0.00% |
46 |
70 |
GetIntent+Crawler+(http:/ / getintent.com/ bot.html) |
508 |
0.00% |
288 |
71 |
Mozilla/5.0+(compatible;+SMTBot/ 1.0;++http:/ / www.similartech.com/ smtbot) |
414 |
0.00% |
3 |
72 |
Mozilla/5.0+(compatible;+seoscanners.net/ 1;++spider@seoscanners.net) |
405 |
0.00% |
10 |
73 |
Mozilla/5.0+(compatible;+Gluten+Free+Craw ler/ 1.0;++http:/ / glutenfreepleasure.com/ ) |
397 |
0.00% |
146 |
74 |
Mediatoolkitbot+(complaints@mediatoolkit. com) |
386 |
0.00% |
195 |
75 |
Mozilla/5.0+(compatible;contxbot/ 1.0) |
380 |
0.00% |
140 |
76 |
AdsBot-Google+(+http:/ / www.google.com/ adsbot.html) |
377 |
0.00% |
29 |
77 |
yacybot+(/global;+amd64+Linux+4.15.0-gent oo;+java+1.8.0_151;+Europe/ de)+http:/ / yacy.net/ bot.html |
376 |
0.00% |
128 |
78 |
Search4free Spider |
361 |
0.00% |
344 |
79 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+MA X+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
340 |
0.00% |
12 |
80 |
Mozilla/5.0+(compatible;+coccocbot-image/ 1.0;++http:/ / help.coccoc.com/ searchengine) |
333 |
0.00% |
156 |
81 |
Mozilla/5.0+(compatible;+SeznamBot/ 3.2-test1;++http:/ / napoveda.seznam.cz/ en/ seznambot-intro/ ) |
318 |
0.00% |
131 |
82 |
Mozilla/5.0+(Windows+NT+10.0;+WOW64;+rv:5 7.0)+Gecko/ 20100101+Firefox/ 57.0+-+James+BOT+-+WebCrawler+http:/ / cognitiveseo.com/ bot.html |
308 |
0.00% |
129 |
83 |
Clickagy+Intelligence+Bot+v2 |
291 |
0.00% |
148 |
84 |
Googlebot-Video/1.0 |
269 |
0.00% |
6 |
85 |
Slack-ImgProxy+(+https:/ / api.slack.com/ robots) |
242 |
0.00% |
219 |
86 |
Mozilla/5.0+(compatible;+SeznamBot/ 3.2-test1-1;++http:/ / napoveda.seznam.cz/ en/ seznambot-intro/ ) |
240 |
0.00% |
103 |
87 |
Mozilla/5.0+(X11;+U;+Linux+Core+i7-4980HQ ;+de;+rv:32.0;+compatible;+JobboerseBot;+ http:/ / www.jobboerse.com/ bot.htm)+Gecko/ 20100101+Firefox/ 38.0 |
228 |
0.00% |
1 |
88 |
Mozilla/5.0+(compatible;+SeznamBot/ 3.2-test4;++http:/ / napoveda.seznam.cz/ en/ seznambot-intro/ ) |
224 |
0.00% |
91 |
89 |
Mozilla/5.0+(compatible;+changedetection/ 7.0;++http:/ / www.changedetection.com/ bot.html) |
224 |
0.00% |
56 |
90 |
Mozilla/4.0+(compatible;+MSIE+8.0;+Window s+NT+6.0+;+Claritybot) |
218 |
0.00% |
186 |
91 |
Google Feedfetcher |
215 |
0.00% |
187 |
92 |
WikiDo/1.1+(http://wikido.com;+crawler@wi kido.com) |
190 |
0.00% |
28 |
93 |
ETI+SEO+bot+-+www.seo.eti.pw |
186 |
0.00% |
0 |
94 |
Googlebot/Nutch-1.7 |
180 |
0.00% |
60 |
95 |
R6_CommentReader(www.radian6.com/ crawler) |
176 |
0.00% |
16 |
96 |
ZoomBot+(Linkbot+1.0+http:/ / suite.seozoom.it/ bot.html) |
169 |
0.00% |
73 |
97 |
uipbot/1.0+(uipbot@semasio.net) |
165 |
0.00% |
9 |
98 |
Mozilla/5.0+(compatible)+SemanticScholarB ot+(+https:/ / www.semanticscholar.org/ crawler) |
153 |
0.00% |
11 |
99 |
Auto+Spider+1.0 |
150 |
0.00% |
15 |
100 |
Mozilla/5.0+(compatible;+redditbot/ 1.0;++http:/ / www.reddit.com/ feedback) |
148 |
0.00% |
15 |
101 |
yacybot+(/global;+amd64+Linux+4.15.2-gent oo;+java+1.8.0_151;+Europe/ de)+http:/ / yacy.net/ bot.html |
143 |
0.00% |
46 |
102 |
Mozilla/5.0+(compatible;+OmnizensBot/ 1.1) |
141 |
0.00% |
0 |
103 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT_NO TE_S+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
139 |
0.00% |
6 |
104 |
Mozilla/5.0+(compatible;+uCrawler/ 1.0+;++https:/ / blog.ucoz.ru/ upolicy) |
138 |
0.00% |
6 |
105 |
PDFDriveCrawler+2.36+(+http:/ / www.pdfdrive.net+) |
136 |
0.00% |
17 |
106 |
RankFlex.com+Webspider |
132 |
0.00% |
96 |
107 |
Mozilla/5.0+(compatible;+DuckDuckGo-Favic ons-Bot/ 1.0;++http:/ / duckduckgo.com) |
121 |
0.00% |
48 |
108 |
spider |
121 |
0.00% |
110 |
109 |
Mozilla/5.0+(compatible;+PaperLiBot/ 2.1;+http:/ / support.paper.li/ entries/ 20023257-what-is-paper-li) |
119 |
0.00% |
95 |
110 |
Mozilla/5.0+(Linux;+Android+5.1;+CUBOT_NO TE_S+Build/ LMY47I)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 60.0.3112.116+Mobile+Safari/ 537.36+OPR/ 44.1.2246.123029 |
119 |
0.00% |
4 |
111 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT_NO TE_S+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 63.0.3239.111+Mobile+Safari/ 537.36 |
110 |
0.00% |
4 |
112 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+MA X+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 59.0.3071.125+Mobile+Safari/ 537.36 |
109 |
0.00% |
4 |
113 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+X1 8+Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
108 |
0.00% |
4 |
114 |
Mozilla/5.0+(compatible;+PiplBot;++http:/ / www.pipl.com/ bot/ ) |
107 |
0.00% |
54 |
115 |
Mozilla/5.0+(compatible;+StorygizeBot;+ht tp:/ / www.storygize.com) |
99 |
0.00% |
67 |
116 |
Mozilla/5.0+(TweetmemeBot/ 4.0;++http:/ / datasift.com/ bot.html)+Gecko/ 20100101+Firefox/ 31.0 |
98 |
0.00% |
57 |
117 |
Mozilla/5.0+(Windows+NT+6.1;+rv:38.0)+Gec ko/ 20100101+Firefox/ 38.0+(IndeedBot+1.1) |
97 |
0.00% |
8 |
118 |
Mozilla/5.0+(compatible;+Discordbot/ 2.0;++https:/ / discordapp.com) |
97 |
0.00% |
85 |
119 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+CH EETAH+2+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
92 |
0.00% |
3 |
120 |
Mozilla/5.0+(compatible;+SurdotlyBot/ 1.0;++http:/ / sur.ly/ bot.html) |
91 |
0.00% |
31 |
121 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+EC HO+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
90 |
0.00% |
3 |
122 |
Mozilla/5.0+(compatible;+SemrushBot-BM/ 1.0;++http:/ / www.semrush.com/ bot.html) |
87 |
0.00% |
77 |
123 |
yacybot+(/global;+amd64+Linux+4.14.15-gen too;+java+1.8.0_151;+Europe/ de)+http:/ / yacy.net/ bot.html |
86 |
0.00% |
34 |
124 |
looid.com/Crawler+1.0 |
84 |
0.00% |
2 |
125 |
Googlebot+(gocrawl+v0.4) |
84 |
0.00% |
57 |
126 |
yacybot+(/global;+amd64+Linux+4.14.18-gen too;+java+1.8.0_151;+Europe/ de)+http:/ / yacy.net/ bot.html |
83 |
0.00% |
23 |
127 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+R9 +Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
81 |
0.00% |
2 |
128 |
Mozilla/5.0+(compatible;+YaK/ 1.0;+http:/ / linkfluence.com/ ;+bot@linkfluence.com) |
79 |
0.00% |
44 |
129 |
Mozilla/5.0+(compatible;+YandexAntivirus/ 2.0;++http:/ / yandex.com/ bots) |
79 |
0.00% |
5 |
130 |
Mozilla/5.0+(compatible;+BomboraBot/ 1.0;++http:/ / www.bombora.com/ bot) |
74 |
0.00% |
72 |
131 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+DI NOSAUR+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 63.0.3239.111+Mobile+Safari/ 537.36 |
74 |
0.00% |
3 |
132 |
Mozilla/5.0+(compatible;+DnyzBot/ 1.0) |
71 |
0.00% |
16 |
133 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+DI NOSAUR+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 62.0.3202.84+Mobile+Safari/ 537.36 |
71 |
0.00% |
3 |
134 |
Mozilla/5.0+(X11;+Linux+x86_64)+AppleWebK it/ 538.1+(KHTML,+like+Gecko)+JobBot/ 5.0+(compatible;++http:/ / www.jobseeker.com.au/ bot.html)+Safari/ 538.1 |
70 |
0.00% |
62 |
135 |
SundaySkyAdsTxtCrawler/ 1.0 |
67 |
0.00% |
27 |
136 |
Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10 .10;+rv:41.0)+Gecko/ 20100101+Firefox/ 55.0+BrandVeritySpider/ 1.0+(http:/ / www.brandverity.com) |
66 |
0.00% |
56 |
137 |
Mozilla/5.0+(compatible;+SemrushBot-SA/ 0.97;++http:/ / www.semrush.com/ bot.html) |
65 |
0.00% |
23 |
138 |
Screaming+Frog+SEO+Spider/ 8.3 |
65 |
0.00% |
26 |
139 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT_NO TE_S+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 63.0.3239.83+Mobile+Safari/ 537.36 |
64 |
0.00% |
2 |
140 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+NO TE+Plus+Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
62 |
0.00% |
3 |
141 |
Mozilla/5.0+(Windows+NT+6.1;+WOW64)+Apple WebKit/ 534++(KHTML,+like+Gecko)+MsnBot-Media+/ 1.0b |
62 |
0.00% |
55 |
142 |
Mozilla/5.0+(Macintosh;+U;+Intel+Mac+OS+X ;+en-US)+AppleWebKit/ 533.3+(KHTML,+like+Gecko)+cXensebot/ 1.1a;++http:/ / www.cxense.com/ bot.html/ 0.1+Safari/ 533.3 |
60 |
0.00% |
51 |
143 |
AppNexusAdsTxtCrawler/ 1.0 |
60 |
0.00% |
30 |
144 |
Mozilla/5.0+(iPhone;+CPU+iPhone+OS+9_1+li ke+Mac+OS+X)+AppleWebKit/ 601.1.46+(KHTML,+like+Gecko)+Version/ 9.0+Mobile/ 13B143+Safari/ 601.1+(compatible;+AdsBot-Google-Mobile;+ +http:/ / www.google.com/ mobile/ adsbot.html) |
59 |
0.00% |
8 |
145 |
PulsePoint-Ads.txt-Crawler/ 1.0 |
58 |
0.00% |
57 |
146 |
Synthesio+Crawler+release+MonaLisa+(conta ct+at+synthesio+dot+fr) |
58 |
0.00% |
42 |
147 |
Alexa crawler |
57 |
0.00% |
46 |
148 |
Googlebot |
57 |
0.00% |
1 |
149 |
Mozilla/5.0+(Windows+NT+6.3;compatible;+L eikibot/ 1.0;++http:/ / www.leiki.com) |
56 |
0.00% |
53 |
150 |
LinqiaCrawlerBot/1.0+(eng@linqia.com) |
55 |
0.00% |
20 |
151 |
Mozilla/5.0+(Macintosh;+U;+Intel+Mac+OS+X ;+en-US)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+cXensebot/ 2.0;++http:/ / www.cxense.com/ bot.html+Safari/ 533.3 |
52 |
0.00% |
25 |
152 |
bidswitchbot/1.0 |
51 |
0.00% |
5 |
153 |
Elmer,+the+Thinglink+ImageBot+(https:/ / support.thinglink.com/ articles/ developer-resources/ elmer-imagebot) |
50 |
0.00% |
17 |
154 |
GumGum+Bot+(http://gumgum.com;+support@gu mgum.com) |
48 |
0.00% |
24 |
155 |
Mozilla/5.0+(X11;+Linux+x86_64;+rv:33.0)+ Gecko/ 20100101+Firefox/ 33.0+fr_crawler |
48 |
0.00% |
39 |
156 |
Mozilla/5.0+(compatible;+XoviBot/ 2.0;++http:/ / www.xovibot.net/ ) |
47 |
0.00% |
46 |
157 |
Screaming+Frog+SEO+Spider/ 8.1 |
46 |
0.00% |
18 |
158 |
AdsTxtCrawler/1.0;++https:/ / github.com/ InteractiveAdvertisingBureau/ adstxtcrawler |
46 |
0.00% |
30 |
159 |
SEMrushBot |
45 |
0.00% |
18 |
160 |
DuckDuckBot/1.0;+(+http:/ / duckduckgo.com/ duckduckbot.html) |
44 |
0.00% |
34 |
161 |
Mozilla/5.0+(compatible;+PiStato/ v0.0.1;+http:/ / carsopict.com/ pages/ bot.html+) |
44 |
0.00% |
22 |
162 |
DomainStatsBot/1.0+(http:/ / domainstats.io/ our-bot) |
44 |
0.00% |
18 |
163 |
Slackbot-LinkExpanding+1.0+(+https:/ / api.slack.com/ robots) |
44 |
0.00% |
40 |
164 |
Mozilla/5.0+(compatible;+Qwant+Research+B ot/ 1.0;++https:/ / www.qwant.com/ ) |
43 |
0.00% |
6 |
165 |
Mozilla/5.0+(compatible;+FemtosearchBot/ 1.0;+http:/ / femtosearch.com) |
43 |
0.00% |
9 |
166 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT_MA NITO+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
42 |
0.00% |
3 |
167 |
Java+1.7+Apache+HttpClient+(Linux+x86_64) +/ +GnowitNewsbot+/ +Contact+information+at+http:/ / www.gnowit.com |
41 |
0.00% |
38 |
168 |
Intersearch.de Spider |
40 |
0.00% |
35 |
169 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+CH EETAH+2+Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 63.0.3239.111+Mobile+Safari/ 537.36 |
40 |
0.00% |
4 |
170 |
Mozilla/5.0+(Linux;+Android+5.1;+CUBOT_NO TE_S+Build/ LMY47I)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Version/ 4.0+Chrome/ 39.0.0.0+Mobile+Safari/ 537.36+Browser |
39 |
0.00% |
1 |
171 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+CH EETAH+2+Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 61.0.3163.98+Mobile+Safari/ 537.36 |
37 |
0.00% |
2 |
172 |
HybridBot+(hybrid.ru/ about.+If+our+bot+caused+problems+please+ contact+us.+Contact+email:+abuse@hybrid.r u) |
37 |
0.00% |
28 |
173 |
yacybot+(/global;+amd64+Linux+4.14.16-gen too;+java+1.8.0_151;+Europe/ de)+http:/ / yacy.net/ bot.html |
36 |
0.00% |
8 |
174 |
Mozilla/5.0+(Linux;+Android+5.1;+CUBOT_NO TE_S+Build/ LMY47I)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
35 |
0.00% |
2 |
175 |
SafeDNSBot+(https://www.safedns.com/ searchbot) |
33 |
0.00% |
6 |
176 |
AdsTxtCrawler/1.0 |
32 |
0.00% |
16 |
177 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT_NO TE_S+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.123+Mobile+Safari/ 537.36 |
32 |
0.00% |
1 |
178 |
Mozilla/5.0+(Linux;+Android+6.0;+CUBOT+MA X+Build/ MRA58K)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 63.0.3239.111+Mobile+Safari/ 537.36 |
31 |
0.00% |
1 |
179 |
DoCoMo Spider |
30 |
0.00% |
14 |
180 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+H3 +Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.137+Mobile+Safari/ 537.36 |
30 |
0.00% |
1 |
181 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+MA GIC+Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.123+Mobile+Safari/ 537.36 |
30 |
0.00% |
1 |
182 |
LinkedInBot/1.0+(compatible;+Mozilla/ 5.0;+Jakarta+Commons-HttpClient/ 3.1++http:/ / www.linkedin.com) |
30 |
0.00% |
29 |
183 |
Mozilla/5.0+(compatible;+archive.org_bot; +Wayback+Machine+Live+Record;++http:/ / archive.org/ details/ archive.org_bot) |
30 |
0.00% |
29 |
184 |
yacybot+(/global;+amd64+Linux+4.14.12-1-d efault;+java+1.8.0_151;+Europe/ en)+http:/ / yacy.net/ bot.html |
29 |
0.00% |
12 |
185 |
Mozilla/5.0+(compatible;+oBot/ 2.3.1;+http:/ / filterdb.iss.net/ crawler/ ) |
29 |
0.00% |
6 |
186 |
Mozilla/5.0+(compatible;+SemrushBot-SI/ 0.97;++http:/ / www.semrush.com/ bot.html) |
28 |
0.00% |
0 |
187 |
Screaming+Frog+SEO+Spider/ 6.1 |
28 |
0.00% |
8 |
188 |
NEC Spider |
28 |
0.00% |
1 |
189 |
Mozilla/5.0+(compatible;+Uptimebot/ 1.0;++http:/ / www.uptime.com/ uptimebot) |
28 |
0.00% |
26 |
190 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+H3 +Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 64.0.3282.123+Mobile+Safari/ 537.36 |
27 |
0.00% |
1 |
191 |
HubSpot+Links+Crawler+2.0+http:/ / www.hubspot.com/ |
26 |
0.00% |
23 |
192 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ en;+rv:1.9.0.13)+Gecko/ 2009073022+Firefox/ 3.5.2+(.NET+CLR+3.5.30729)+SurveyBot/ 2.3+(DomainTools) |
26 |
0.00% |
10 |
193 |
Mozilla/5.0+(compatible;+Linux+x86_64;+Ma il.RU_Bot/ Robots/ 2.0;++http:/ / go.mail.ru/ help/ robots) |
25 |
0.00% |
24 |
194 |
Mozilla/5.0+(Linux;+Android+7.0;+CUBOT+R9 +Build/ NRD90M)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 59.0.3071.125+Mobile+Safari/ 537.36 |
25 |
0.00% |
2 |
195 |
Mozilla/5.0+(iPhone;+CPU+iPhone+OS+8_3+li ke+Mac+OS+X)+AppleWebKit/ 600.1.4+(KHTML,+like+Gecko)+Version/ 8.0+Mobile/ 12F70+Safari/ 600.1.4+(compatible;+Laserlikebot/ 0.1) |
24 |
0.00% |
12 |
196 |
Mozilla/4.0+(compatible;++Vagabondo/ 4.0;+webcrawler+at+wise-guys+dot+nl;+http :/ / webagent.wise-guys.nl/ ;+http:/ / www.wise-guys.nl/ ) |
24 |
0.00% |
0 |
197 |
Mozilla/5.0+(compatible;+YandexCalendar/ 1.0;++http:/ / yandex.com/ bots) |
23 |
0.00% |
3 |
198 |
Mozilla/5.0+(compatible;+DnyzBot/ 1.0)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+HeadlessChrome / 64.0.3264.0+Safari/ 537.36 |
23 |
0.00% |
11 |
199 |
Mozilla/5.0+(compatible;+ToutiaoSpider/ 1.0;+http:/ / web.toutiao.com/ media_cooperation/ ;) |
23 |
0.00% |
23 |
200 |
Mozilla/5.0+(compatible;+YandexAccessibil ityBot/ 3.0;++http:/ / yandex.com/ bots) |
23 |
0.00% |
2 |
|
Subtotal |
23,848,819 |
99.99% |
1,055,349 |
|
Total |
23,850,039 |
100.00% |
1,055,930 |
