Search engine optimization: Difference between revisions

Revision as of 22:05, 7 July 2010

Revision as of 09:58, 8 July 2010

Line 47:

{{Main|search engine optimization methods}}

=== Getting indexed ===

The leading search engines, such as Google, Bing and Yahoo!, use [[Web crawler|crawlers]] to find pages for their algorithmic search results. Pages that are linked from other search engine indexed pages do not need to be submitted because they are found automatically. Some search engines, notably Yahoo!, operate a paid submission service that guarantee crawling for either a set fee or [[Pay per click|cost per click]].<ref>{{cite web|url=http://searchenginewatch.com/showPage.html?page=2167871|title= Submitting To Search Crawlers: Google, Yahoo, Ask & Microsoft's Live Search |date= 2007-03-12|accessdate=2007-05-15|publisher=[[Search Engine Watch]]}}</ref> Such programs usually guarantee inclusion in the database, but do not guarantee specific ranking within the search results.{{Dead link|date=April 2010}}<ref>{{Dead link|date=April 2010}}{{cite web|title=Search Submit|url=http://searchmarketing.yahoo.com/srchsb/index.php|publisher=searchmarketing.yahoo.com|accessdate=2007-05-09}}</ref> Two major directories, the Yahoo Directory and the [[Open Directorydirecth Projectengine]] bothcrawlers requiremay manuallook submissionat anda humannumber editorialof review.<ref>{{citedifferent web|url=http://searchenginewatch.com/showPage.html?page=2167881|title=factors Submittingwhen To Directories: Yahoo & The Open Directory |date= 2007-03-12|accessdate=2007-05-15|publisher=[[SearchWeb Engine Watchcrawler|crawling]]}}</ref> Googlea offerssite. [[GoogleNot Webmasterevery Tools]],page foris whichindexed anby XMLthe [[Sitemap]]search feedengines. canDistance beof createdpages andfrom submittedthe forroot freedirectory toof ensurea thatsite allmay pagesalso arebe found,a especiallyfactor pagesin thatwhether aren'tor discoverablenot bypages automaticallyget following linkscrawled.<ref name="cho">{{cite web|url=http://wwwdbpubs.googlestanford.comedu:8090/supportpub/webmasters/bin/answer.py?answer1998-51|title=40318&topicEfficient crawling through URL ordering|author=8514Cho, J., Garcia-Molina, H.|titleyear=What1998|publisher=Proceedings isof athe Sitemapseventh fileconference andon whyWorld shouldWide IWeb, haveBrisbane, one?|publisher=google.comAustralia|accessdate=2007-0305-1909}}</ref>

[[Web search engine|Search engine]] crawlers may look at a number of different factors when [[Web crawler|crawling]] a site. Not every page is indexed by the search engines. Distance of pages from the root directory of a site may also be a factor in whether or not pages get crawled.<ref name="cho">{{cite web|url=http://dbpubs.stanford.edu:8090/pub/1998-51|title=Efficient crawling through URL ordering|author=Cho, J., Garcia-Molina, H.|year=1998|publisher=Proceedings of the seventh conference on World Wide Web, Brisbane, Australia|accessdate=2007-05-09}}</ref>

=== Preventing crawling ===

{{Main|Robots Exclusion Standard}}

To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standarddirectorid [[robots.txt]] file in the root directory of the ___domain. Additionally, a page can be explicitly excluded from a search engine's database by using a [[meta tag]] specific to robots. When a search engine visits a site, the robots.txt located in the [[root directory]] is the first file crawled. The robots.txt file is then parsed, and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.<ref>{{cite web|url=http://searchengineland.com/070508-165231.php|title=Newspapers Amok! New York Times Spamming Google? LA Times Hijacking Cars.com?|publisher=[[Search Engine Land]]|date=May 8, 2007|accessdate=2007-05-09}}</ref>

=== Increasing prominence ===