Saturday, 22 June 2019 17:34

Bots and your site and some more

Written by
Rate this item
(0 votes)

So running over statistics my heart nearly took a hit because I thought my site was slow and alot of online users were displayed in the Current Sessions statistics in the footer of my website. Got happy and thought "Hey, either I must upgrade or the site would suffer from being underdimensioned in connection capacity making everything slower", this was until I looked closer into who were online and saw that SEMRush were indexing my site. This is a good thing, however sending appr. 50.000 sessions simultanously isn't really pleasant to my real users and therefor I blocked the bots running on the site to get a real feeling of what is what. I both blocked via .htaccess (anothe,r link2) and PHP to be safe sure.

Here are the entries for .htaccess and your index.php file if you'd like to block SEMRush (remove the tags to regain getting indexed, SEMRush can be a valuable asset in your webmaster toolbox of source sites where you can analyze data and it can even help you get ideas to improve your website):

.htaccess entry

BrowserMatchNoCase SemrushBot bad_bot
Order Deny,Allow
Deny from env=bad_bot

PHP entry inside the index.php home file

<?php
if(preg_match('/SemrushBot/i',$_SERVER['HTTP_USER_AGENT']))
{
header('HTTP/1.0 403 Forbidden');
echo "Blocked temporary!";
exit();
}
?>

While I do agree that bots are a must to index the whole thing, there's really no need to have hundreds of bots using the site at the same time on the same site. Bots likes to surf porn? ;-D This is my own "failure" though, since I'm using another tactic instead of related feeder sites, all domains that I use for this network are being *redirected* to midnight12.com to serve what people are looking for when they enter one of the network domains. 90% of my traffic are visitor type-in traffic originating from good keyword domains. Bad thing is the address URI changes in the browser but that's really the only downside to it. It's the best setup to gain traffic level besides doing onsite SEO and keeping to the strategy of building more sites with content related to the niches that are targeted. I will optimize the code to be more friendly if someone is looking into copy parts of the website for their own sake. Copyright wise I really don't mind if ie. a footer link or sourcecode link <!-- source: domain.com --> appears somewhere if you're happy with the result otherwise just keep the source private and see how far you can bring it ;-)

Online life right now.

Pornhub.com made some cool changes I noticed, logo changed and thought I've just got myself a friend, not sure if it has something to do with me visiting their very large library of porn or if they are just happy to have friends around on the large premise of the internet. I'm trying to promote content that suits my own personal taste (this being a big mistake because the site would get more traffic if it hosted all niches), perfect content does exist but is really really rare, mostly only few streams per website if your even that lucky. If I started importing content from 3rd party feeds there should be enough for all sexualities the coming 1001 years. This would be a bad idea because my server produces timeouts at around 60.000 simultanious connections and getting indexed for that much content would produce a traffic level that would make midinght12.com too slow for users to be happy surfing it. 

The chance of hundreds of bots on your site is however slightly unreal because you probably don't have 40 related top-level .com .dk .net domains pointing to your site with A Record <Apache Server bound IP > and CNAME www linkedsite.com.
In Chrome Browser this produces a nice redirect that doesn't make your domain take a hit but instead just making a refferer on the target site logs. The theory is "The more the merrier" and site wise this suits me perfect because there's no need to have alot of sites promoting a product/niche showing shared content even though this means more content space as in more place for advertisements. Will see where it ends someday. Right now I'm testing theories gathered from years of surfing the internet.

Content must be unique to every site in your network to be home safe. This also protects you from "Duplicate content" which in reality is "Duplicate tags content". Google is a META Search Engine and looks into your content to make their algorithm produce SERP results.
Duplicate content could be <title>, <h1> and <meta> while <body> is the Search Engine´s Metadata to sort out relevancy, in between subpages on the same domain. If the same code is mentioned on another site it's something else which is called Hype. Hype is a very good thing and only seldom achieved, depends on ones network but also sometimes wallet size or product popularity, first to mill serves as father about the hyped content.

Bing is more relaxed as Google and they work slightly different, thanks for that, this is probably from being the youngstar in the group of Search Engines trying to finetune their algorithm to suit most of their users personality/personal preference/taste best, I guess this is what they call personalized content which normally runs per account like on Google. Before it was a voting/rating system that worked per-click ranking SERP results which Google Toolbar started to introduce (they were most likely the first to combine clicks/popularity with content Metadata and present them as a relavance search result, devided by keywords and present this too as a product, keyword ads/adwords). 

Storage, Streaming OS and some music.

Smartphones could learn from dualboot systems though they (producers like MotorolaNokiaApple, SamsungHTC, Huawei and the new pet Xaomei) also as ourselves like focus and else they are looking for commitment to a brand, customer care and giving an individual experience. Freedom of flavour for the ones with no concerns.

Actually I have a  project that is serving a bootable Operative System via netboot feature IPX. Just need some working clients that will be served centralized and then have a GUI giving the users the free choice of OS and flavour. Developers should be able to upload their ISO and working configuration to get listed on the boot GUI. Maybe through paying a fee. If a user needs a legal license to "play" the system then the system could use the working internet connection and paying with a creditcard option for what they need and store it per account. Flat-clients are used in businesses, they don't need to be able to play high-def games or read and store on really fast SSD discs. The thing I'm working with is however suited for both Desktop computers along with flat-clients only using local storage as a backup drive. It would bring all the available Operative Systems (all the different workspace/desktops) as networkboot option into all computers that support the IPX netboot feature, which in the end would mean a live system personalized catered to each enduser which I think is what Microsoft is aiming for building their OS entrance when installing, taking their Windows 8.1Pro installer into notice?

On fiber connections there will not be any speed issue. Fiber connections even work faster than a current-state harddrive. The only limit is the Wifi connection or cable connection to the desktop (cable being the slowest but most secure solution).

Streaming OS.

Future looks bright :-)

Music: https://www.youtube.com/watch?v=wuHmIDQQjW4

Read 1840 times Last modified on Sunday, 25 August 2019 22:07

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.

18 comments

  • Comment Link chocopie Wednesday, 26 June 2019 00:10 posted by chocopie

    jW3vbM You ave made some really good points there. I looked on the internet for more information about the issue and found most individuals will go along with your views on this site.

    #Webmaster reply


    I try make the posts as general as possible which means most people can use it for their own purpose.

    Report
  • Comment Link i love pron Wednesday, 26 June 2019 00:15 posted by i love pron

    JSR0QP This web site truly has all of the information and facts I needed concerning this subject and didn at know who to ask.

    #Webmaster reply


    You're free to ask any questions you may have. Even though I'm working in porn doesn't make the topics different, thinking of SEO and link building.

    Report
  • Comment Link eebest8 best Wednesday, 26 June 2019 17:34 posted by eebest8 best

    "Definitely, what a great blog and revealing posts, I will bookmark your website.All the Best!"

    #Webmaster reply


    Thanks for the bookmark :-)

    Report
  • Comment Link eebest8 best Wednesday, 26 June 2019 17:59 posted by eebest8 best

    "A big thank you for your article. Much obliged."

    #Webmaster reply


    Thank you :-)

    Report
  • Comment Link fiverr Friday, 05 July 2019 08:18 posted by fiverr

    Very good written article. It will be supportive to anyone who utilizes it, including me. Keep doing what you are doing – can’r wait to read more posts.

    Report
  • Comment Link oprol evorter Monday, 09 September 2019 08:24 posted by oprol evorter

    Enjoyed looking through this, very good stuff, appreciate it.

    Report
  • Comment Link nike air max 270 Friday, 11 October 2019 20:49 posted by nike air max 270

    I truly wanted to send a quick remark so as to express gratitude to you for some of the fantastic suggestions you are posting on this website. My time intensive internet lookup has now been rewarded with good ideas to exchange with my family members. I would assert that most of us site visitors actually are undoubtedly blessed to exist in a good network with very many marvellous people with insightful tactics. I feel very privileged to have seen your entire website and look forward to really more amazing minutes reading here. Thank you once again for a lot of things.

    Report
  • Comment Link lebron james shoes Saturday, 12 October 2019 19:59 posted by lebron james shoes

    I want to express appreciation to you for bailing me out of such a matter. Because of researching through the world-wide-web and obtaining ways which are not helpful, I believed my life was done. Living without the solutions to the issues you have resolved as a result of your main report is a serious case, and those that might have adversely affected my entire career if I had not come across your blog post. That expertise and kindness in touching all the pieces was very useful. I am not sure what I would have done if I had not come upon such a subject like this. I can at this point relish my future. Thanks very much for the skilled and effective help. I will not be reluctant to recommend your web blog to any person who needs guide on this area.

    Report
  • Comment Link yeezy boost 350 v2 Sunday, 13 October 2019 19:41 posted by yeezy boost 350 v2

    Thanks so much for providing individuals with such a splendid possiblity to check tips from this website. It is often very useful and as well , stuffed with a good time for me and my office fellow workers to search your site minimum 3 times in 7 days to read through the new tips you will have. And lastly, I'm just usually impressed with your eye-popping inspiring ideas you serve. Certain 4 points in this article are basically the finest we've ever had.

    Report
  • Comment Link supreme clothing Monday, 14 October 2019 15:19 posted by supreme clothing

    I actually wanted to write down a small note to be able to appreciate you for the splendid tips and hints you are writing at this website. My long internet search has now been paid with pleasant information to write about with my relatives. I 'd admit that many of us visitors are unequivocally blessed to be in a remarkable website with many marvellous professionals with good basics. I feel very privileged to have seen the website and look forward to some more exciting times reading here. Thanks a lot again for a lot of things.

    Report
  • Comment Link nike air max 270 Tuesday, 15 October 2019 11:15 posted by nike air max 270

    I and my guys ended up taking note of the nice information located on the website while suddenly I got a horrible feeling I had not expressed respect to the blog owner for those tips. My women were for that reason very interested to study them and now have in fact been using these things. Appreciate your indeed being simply kind and then for making a decision on this kind of amazing subjects most people are really needing to be aware of. Our own honest apologies for not expressing appreciation to earlier.

    Report
  • Comment Link nike basketball shoes Wednesday, 16 October 2019 07:48 posted by nike basketball shoes

    I'm writing to make you be aware of of the wonderful encounter our princess undergone visiting the blog. She came to find many things, most notably how it is like to possess a marvelous teaching mindset to make men and women with ease master a variety of grueling things. You truly surpassed our own expected results. Many thanks for supplying those invaluable, safe, explanatory and unique guidance on this topic to Ethel.

    Report
  • Comment Link supreme clothing Thursday, 17 October 2019 05:05 posted by supreme clothing

    I must show my affection for your kind-heartedness in support of those who really need help with this issue. Your real commitment to getting the message across had been astonishingly productive and have in every case permitted people much like me to realize their aims. Your important hints and tips can mean this much a person like me and extremely more to my colleagues. Thanks a ton; from each one of us.

    Report
  • Comment Link hogan outlet Friday, 18 October 2019 02:18 posted by hogan outlet

    I actually wanted to write down a word in order to thank you for the marvelous advice you are placing on this site. My incredibly long internet look up has now been rewarded with wonderful tips to write about with my family. I would mention that we readers are unquestionably blessed to be in a great website with very many special individuals with helpful tactics. I feel rather grateful to have seen your website and look forward to some more entertaining times reading here. Thanks again for all the details.

    Report
  • Comment Link cheap jordans Saturday, 19 October 2019 00:36 posted by cheap jordans

    My spouse and i ended up being quite more than happy that Ervin managed to complete his reports through the entire precious recommendations he made from your very own web page. It is now and again perplexing to simply find yourself giving out guidance which usually some other people have been selling. So we remember we have got you to appreciate because of that. The most important illustrations you have made, the easy web site menu, the friendships you will aid to engender - it's got mostly terrific, and it's assisting our son in addition to us reason why that situation is thrilling, and that's extraordinarily fundamental. Thank you for the whole lot!

    Report
  • Comment Link birkin bag Sunday, 20 October 2019 00:01 posted by birkin bag

    I and also my pals were found to be reading the nice secrets and techniques from your site and quickly I got a horrible suspicion I had not thanked the blog owner for those tips. My ladies happened to be totally passionate to read all of them and have now unquestionably been tapping into them. I appreciate you for indeed being well considerate and also for going for varieties of very good issues millions of individuals are really wanting to know about. Our own honest regret for not saying thanks to sooner.

    Report
  • Comment Link golden goose Sunday, 20 October 2019 23:07 posted by golden goose

    I enjoy you because of all of your efforts on this site. My daughter take interest in setting aside time for internet research and it's easy to see why. Most people hear all regarding the lively method you provide good guides by means of this website and as well encourage contribution from people on the subject matter and our favorite girl is always discovering a whole lot. Have fun with the rest of the year. You're performing a stunning job.

    Report
  • Comment Link moncler coats Monday, 21 October 2019 21:32 posted by moncler coats

    My wife and i were satisfied that Raymond could complete his investigation using the ideas he had from your very own weblog. It's not at all simplistic to simply find yourself freely giving things a number of people might have been making money from. Therefore we know we need the blog owner to thank because of that. All the explanations you made, the simple site menu, the friendships you will help promote - it's many extraordinary, and it's really making our son in addition to us reckon that that issue is brilliant, and that is truly indispensable. Thank you for the whole lot!

    Report