Google debuted in 1998 and was the 21st search engine. The prior 20 included WebCrawler, Go.com, Lycos, Infoseek, Alta Vista, Excite, Yahoo!, Dogpile, Inktomi, HotBot, and Ask Jeeves.
While there were a number of factors which led to Google’s success including it’s minimalist home search page, it’s superior search algorithms are really what set Google apart.
The exact details of Google Search are a company secret, but how Search works in general is as follows.
Google Search has three parts:
- Crawling. Like other search engines, Google uses an automated program called Googlebot to find individual web pages and then indexes them to make them searchable. The Googlebot “crawls” billions of web pages and as it visits websites it detects links on each page and adds them to its list of pages to crawl.
- Indexing. Googlebot creates a massive index of all the words it sees on webpages it crawls as well as the words location on each page. Based on frequency and location, keywords are created in the index for each webpage.
- Page Ranking. The other 20 prior search engines, like Google, used programs to crawl and index. What really sets Google apart it how it page ranks. Effective ranking means relevant search results! When you enter a query, Google computers search the index for matching pages. The results are returned according to relevancy which is based on over 200 factors. The most important factors are (1) The frequency and location of keywords within the Web page, (2) How long the Web page has existed and (3) The number of other Web pages that link to the page in question. This third factor, looking at web page links as votes, is the key factor that allowed Google to win at search. By looking to links to a web page as a vote it is really hard to cheat the system. To get a high relevancy score your page needs to have great content so that people will link back to your page. Links from a site with a high PageRank score are valued more than links from a low page rank score.
A process called Search Engine Optimization (SEO) uses processes based on how Google and other search engines work to generate better page ranks. While there are a lot of SEO techniques, the most common relate to use and placement of keywords and creating incoming links from other sites. There are all sorts of tricks used in SEO, many of which Google’s programs know to overlook. For instance, one common trick is “overstuffing” key words by putting a lot of keywords at the bottom of the page where it does not interfere with what a page visitor sees or putting keywords in invisible text (text the color of the background). This and other “black hat” techniques such as selling or farming page links, are known by search engines and when discovered, a page can be punished by being put further down the ranking.
At Advent, I know we paid Google in order to land at the top of search results. Not sure exactly how all that works – but I’m pretty sure there’s a linkage.
Google is synonymous with web search. The company is very secretive over its methods, and critics argue that they manipulate search results.
Most people never give a second thought that the results they get from a search engine, could be “tainted” with some form of bias. It’s logical to assume that at least some manipulation can occur.
Judgement and critical thinking take a back seat to speed and convenience. Search engines give us access to quick information, that we readily assume is reliable. It is interesting when you compare results of other search engines for identical searches. The only thing I can conclude, is none are perfect.
Good points Ed!