Crawler Pages

Some search engines, notably Yahoo! and altavista, only allow you to submit one Web page for rating. They then examine that page and index the keywords in it before following up the Web links it mentions. Ideally, this crawler page contains links to all the key pages on the site that you wish to be indexed. This should be done without any text. It only exists to help the search engines, so it doesn't have to be viewable. To prevent the crawler page from appearing in the search engine results, put a redirect on it so the searcher is passed straight to your home page.

If your site is extensive, link the master crawler page to sub-crawler pages, which list other working pages.

The maximum indexable content per page is limited to 100K. Any excess cannot be indexed.

If you are using framed pages, make sure they don't exceed the limit by getting your crawlers to test the links between the <noframes> and </noframes> sections in the master page that generates framed pages.

100 Backlinking Strategies

100 Backlinking Strategies

This ebook will give you 100 backlinking strategies. Backlinks can increase your traffic, sales and search engine ranking. You'll discover many different types of backlinks you could get and tactics you could use to acquire those those backlinks from other high PR web sites.

Get My Free Ebook


Post a comment