QUESTIONS ABOUT THIS SITE - Page 157

Undertoad · 01-08-2018, 07:20 AM

Most of the traffic is search engine bots. But the only way to know for sure is to go through the logs by hand...

Carruthers · 01-08-2018, 07:25 AM

Quote:

Originally Posted by Undertoad

Most of the traffic is search engine bots. But the only way to know for sure is to go through the logs by hand...

Thanks for that, UT.

As I type there are 262 'guests'. The number seems remarkably consistent.

glatt · 01-08-2018, 09:54 AM

It's been a while since I looked at who was online, but when I have, the majority of guests are search engine spiders.

Undertoad · 01-08-2018, 09:39 AM

About 200 of them are one single poorly-behaved French spider, with addresses like:

ip72.ip-54-36-148.eu

Who knows what they are doing, I sure don't

Undertoad · 01-08-2018, 09:48 AM

That turns out to be the "Ahrefs" search bot, which is indexing us continuously for the last several years.

https://ahrefs.com/robot

Undertoad · 01-08-2018, 09:51 AM

I can't see why they are useful so I've politely told them to go away via the file "/robots.txt". Let's see what happens in 15 minutes' time.

user-agent: AhrefsBot
disallow: /

Undertoad · 01-08-2018, 10:00 AM

Quote:

Originally Posted by Undertoad

Let's see what happens in 15 minutes' time.

OK it may take them longer to pick up the request

Clodfobble · 01-08-2018, 10:01 AM

In theory search engine spiders are good, right? I mean, everybody wants accurate search results. As long as they're not acting like that one French spider that can't stop checking for new posts every 2 seconds. (Certainly can't identify with that impulse myself. Nope. Not me.)

Undertoad · 01-08-2018, 10:13 AM

I... I can't tell any longer

People still come to us via search results, so I guess so. But this engine is Ukrainian (it was just pretending to be French) and I can't tell what it actually does.

xoxoxoBruce · 01-08-2018, 10:40 AM

Trump only knows.

Undertoad · 01-09-2018, 10:46 AM

Ahrefs has not stopped spidering so we will move to banning their IPs

Carruthers · 01-12-2018, 03:35 PM

Earlier I attempted to post a Liveleak video but, despite my best endeavours, couldn't do so.

My impression is that the string of characters/digits after the = symbol are copied, pasted in the reply box, highlighted and then have the Liveleak tags wrapped around them by means of the appropriate button.

I tried that method to no avail. Could someone kindly point me in the right direction, please?

Ta!

Undertoad · 01-12-2018, 04:05 PM

actually I think LiveLeak has stopped permitting embedded videos in general, but couldn't find proof of that

I should remove the button

Carruthers · 01-12-2018, 04:07 PM

Quote:

Originally Posted by Undertoad

actually I think LiveLeak has stopped permitting embedded videos in general, but couldn't find proof of that

I should remove the button

Thanks for that, UT!

It did occur to me that that might be the case.

sexobon · 01-12-2018, 05:56 PM

I was looking this up in the vBulletin forums and found an old '06 entry that indicates there have always been some restrictions:

Quote:

Currently there do seem to be limits as to what you can embed. I believe they have to authorise you to embed any videos tagged as "Graphic" and this may involve installing a small token exchange script on your site ...

Then I saw a '15 entry that gave an embed code:

Quote:

HTML Code:

<iframe width="640" height="360" src="http://www.liveleak.com/ll_embed?f={param}" frameborder="0" allowfullscreen></iframe>

I noticed that the LiveLeak URL in the example code above, as recent as '15, was still using "http" rather than "https" and I'm wondering if the Cellar's embed code has been changed to the latter since both the Cellar and LiveLeak are now secured with "https" URLs. Does that make a difference?

01-08-2018, 10:01 AM	#8
Clodfobble UNDER CONDITIONAL MITIGATION Join Date: Mar 2004 Location: Austin, TX Posts: 20,012	In theory search engine spiders are good, right? I mean, everybody wants accurate search results. As long as they're not acting like that one French spider that can't stop checking for new posts every 2 seconds. (Certainly can't identify with that impulse myself. Nope. Not me.) __________________ My book My other book My spirit animal

01-08-2018, 10:40 AM	#10
xoxoxoBruce The future is unwritten Join Date: Oct 2002 Posts: 71,105	Trump only knows. __________________ The descent of man ~ Nixon, Friedman, Reagan, Trump.

01-12-2018, 03:35 PM	#12
Carruthers Junior Master Dwellar Join Date: Dec 2009 Location: Buckinghamshire UK Posts: 4,059	Earlier I attempted to post a Liveleak video but, despite my best endeavours, couldn't do so. My impression is that the string of characters/digits after the = symbol are copied, pasted in the reply box, highlighted and then have the Liveleak tags wrapped around them by means of the appropriate button. I tried that method to no avail. Could someone kindly point me in the right direction, please? Ta! __________________

01-08-2018, 07:20 AM	#1
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	Most of the traffic is search engine bots. But the only way to know for sure is to go through the logs by hand...

01-08-2018, 09:54 AM	#3
glatt ™ Join Date: Jul 2003 Location: Arlington, VA Posts: 27,717	It's been a while since I looked at who was online, but when I have, the majority of guests are search engine spiders.

01-08-2018, 09:39 AM	#4
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	About 200 of them are one single poorly-behaved French spider, with addresses like: ip72.ip-54-36-148.eu Who knows what they are doing, I sure don't

01-08-2018, 09:48 AM	#5
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	That turns out to be the "Ahrefs" search bot, which is indexing us continuously for the last several years. https://ahrefs.com/robot

01-08-2018, 09:51 AM	#6
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	I can't see why they are useful so I've politely told them to go away via the file "/robots.txt". Let's see what happens in 15 minutes' time. user-agent: AhrefsBot disallow: /

01-08-2018, 10:13 AM	#9
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	I... I can't tell any longer People still come to us via search results, so I guess so. But this engine is Ukrainian (it was just pretending to be French) and I can't tell what it actually does.

01-09-2018, 10:46 AM	#11
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	Ahrefs has not stopped spidering so we will move to banning their IPs

01-12-2018, 04:05 PM	#13
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	actually I think LiveLeak has stopped permitting embedded videos in general, but couldn't find proof of that I should remove the button

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)