Today I have been looking into getting my website crawled by search engine bots and went straight to Google as you do to start submitting. In the process I came across a Webmaster Help Center Article which gave some good advise so thought I’d share, here a little quote.
When your site is ready:
* Have other relevant sites link to yours.
* Submit it to Google at http://www.google.com/addurl.html.
* Submit a Sitemap as part of our Google webmaster tools. Google uses your Sitemap to learn about the structure of your site and to increase our coverage of your webpages.
* Make sure all the sites that should know about your pages are aware your site is online.
* Submit your site to relevant directories such as the Open Directory Project and Yahoo!, as well as to other industry-specific expert sites.Technical guidelines
* Use a text browser such as Lynx to examine your site, because most search engine spiders see your site much as Lynx would. If fancy features such as JavaScript, cookies, session IDs, frames, DHTML, or Flash keep you from seeing all of your site in a text browser, then search engine spiders may have trouble crawling your site.
[ad name=”front”]
I saw the mention of Lynx (Text Browser) which I have used before as I am a total fan of the Back Track Project (An Security PenTest Live CD).
As I have a few sites I wanted to install Lynx on my super sweet Mac Book (Love both PC and Mac), typed it into Google and there it is on apples site.
Once installed (all of 15 secs) I opened up Terminal typed lynx www.ezrahill.co.uk and it looked sweet. All as is so the bots should pick me up.
Quite a bit more to go over and quite a few more submissions to do to get listed in all of the big search engines. Here are a few links were you will wanna submit your site.