Ok, this is another techie post. Our web-designer did some things that kind of screwed us up for getting listed on Google. When Google tried to spider our homepage, they were forwarding the GoogleBot to our mobile page -- translation, Google Thinks we don't have a home page. Killed our page rank and our listing in the Organic Search.
I got that fixed last week and then noticed this week that Google kept looking for a robot.txt file. Now if you read up on the robot.txt file, you'll see it says if you don't want to control what a search-engine shouldn't see, 1 option is just to not have a robot.txt file.
However, seeing Google was looking so hard for it, I added it (it's 26 bytes big, pretty easy) and Google has since begun indexing our website like crazy. So I guess the moral is, don't be lazy, put a free-for-all robots.txt file on your website and you'll like the results.
No comments:
Post a Comment