Search Engines: A Lookback

A Book Summary on The Search, How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture

By Frances Advincula


Note: Author John Battelle starts off by detailing how he stumbled upon the year 2001’s summary on Google’s Zeitgeist, a PR tool that summarizes what the world searched for. He coins the term “The Database of Intentions,” explains the value of search to our modern world, and provides us a history of search. He also tells the story of how Google was born and their journey to success, all while exposing the inner workings of search and how it makes money. Finally, he tells about the impact and implications of search in our lives, as well as its future. However, a few details maybe outdated, as the book was published in 2005. All points of view are the original author’s; I merely summarized what he had to say.



Google’s approach to search may be the closest thing we have to “The Database of Intentions,” the aggregate result of all our searches, the history of every query we typed in the search box. Basically, it shows what we searched for and where that search led, showing insight on what we want, what we think about, what drives us. {PYP’s take: We can think of it as the world’s Facebook timeline!}


At the same time, search is not only one of the pioneers of useful web services, it is also the reason for the second wave of Internet giants (think eBay, Amazon, Yahoo, Google).  Researchers also say search is the forefront of further progress in artificial intelligence. {PYP’s take: It is interesting to note that, fast forward to today, we now have Siri, who definitely acts/talks like a human!}



Who: The younger you are and the higher your level of education, the more you use search.


What: The beauty of search is that we can query for anything under the sun; the possibilities are infinite. How we choose the words we type in the search box, however, is a mystery in itself.

Where: The most used search engines are Microsoft, Yahoo, AOL, and Google.
{PYP’s take: Now, Google is used as a verb, and Bing recently overtook Yahoo.}


When: We search the most in the morning and the evening.


Why: First, we search to find what we know already exists. We want to locate something, to find information on a topic. Second, we use search to discover what we think exists, but we have yet to find.

How Search Works: Every search engine has three main parts, the crawler, the index, and the runtime system. The crawler traverses the entire Web and sends every page it finds to a massive database called the index.  The information is then analyzed using factors such as the number and popularity of links, language, content, etc.  Afterwards, the data is sent to the runtime system, a database that is ready to serve the person who queries. The runtime system performs the ranking logic, connects the user’s query to the index, and displays the results to the user.


With this in mind, returning relevant results is no easy feat. For example, if we want to know more about Abraham Lincoln, we search for “Abraham Lincoln Biography.” However, we are not merely looking for pages with those exact keywords; a good search engine will pay attention to coherence as well. As it analyses pages, it will take into consideration if the page shows the attributes of a biography. Similarly, search must deliver results even when we misspell a word, or be flexible enough to show relevant results for subjects that are represented by different words (“soda” versus “pop”, “tennis shoes” versus “sneakers”). Search engines also worry about striking the toss-keep balance with words such as “to,” “be,” etc. Usually, tossing them out will make the engine work faster, but what if one queries “to be or not to be”? All of a sudden, those words are crucial to the query. {PYP’s take: An infographic on how Google works.}


How Search Makes Money:  Most of the revenue comes from paid search. Advertisers pay the search company a certain amount per click in exchange for their ads showing up when a user queries for something relevant to their offerings. There are also more innovative ways companies are cashing in on search; examples include targeting ads using a person’s online habits and demographic.



Archie by Alan Emtage, McGill University, 1990. The first internet-based, pre-Web search engine.


Veronica by University of Nevada students, 1993. Connected users to the document itself, versus just the machine where it is located.

WWW Wanderer by Mathey Gray of MIT, 1993. Pioneered a breadth algorithm still used today.


Web Crawler by Brian Pinkertron, University of Washington, 1994. First to index the entire contents of a webpage.


Alta Vista by Louis Monier , DEC, mid 1990s. Pioneered the use of thousands of crawlers at once.

Lycos by Dr. Michael Mauldin, Carnegie Mellon, 1994. First to use links as a way of ranking and to include a summary of the results.

Excite by six Stanford alumni, 1994. Started personalization and free email.

Yahoo by Jerry Yang and David Filo, PhD students at Stanford, 1994. Started out using a directory-type structure that organized the Web into categories. Shares stark similarities with Google (both founded by Stanford PhD students, both have the quirky culture, both have fun office complexes). by Bill Gross, founder of IdeaLab, 1997. Came up with the pay-per-click model; results were fully commercial.


Google started as a thesis topic called BackRub by Stanford PhD student Larry Page. He set out to create a system that would take the links of entire Web, analyze, and publish them in a way where one can find out who was linking to whom (unprecedented at the time), attracting the attention of Sergey Brin, another computer science PhD student at Stanford. The two came up with PageRank, an algorithm that rewarded links from important pages and penalized those that came from obscure sites (similar to the academe’s way of judging the quality of your paper through your citations and their quality).

After its debut on the Stanford site in 1996, the founders tried to license to the major players in the industry, but were turned down by companies like Yahoo for the next eighteen months. Finally deciding to start their own company, they received their first $100,000 in funding from Andy Bechtolsheim, a founder of Sun. Thus, Google was formally incorporated as Google Inc. on September 7, 1998.


The next step was to find a business model that generated money. Google turned to advertising, pioneering their text-based ads with AdWords. Led by their founders and new CEO Eric Schmidt (formerly of Sun and Novell), Google summed up its core values in their mantra – “Don’t Be Evil.”
Google continued to grow significantly from 2001 to 2004, buying DejaNews, Blogger, Picasa, and Keyhole.  Then the company released AdSense, a service that displays ads based on the contents of a page. They also started to index images and public phone-book information, partnering with companies such AT&T, Cingular, HandSpring, and AOL. After 9/11 happened, websites like weren’t able to handle the traffic, and people turned to Google to inform them. Google was finally more than a search engine, something they took advantage of by launching Google News. Later, Google launched a new version of AdWords that copied’s auction and pay-per-click approach (previously, AdWords used the cost per thousand model).  However, Google still included popularity on how they rank an ad, not merely how much the company paid. Although this decision actually makes Google more money, the public saw it as a “Don’t Be Evil” move, one that put the user’s interests before Google’s.

But as Google gained more admiration from the public and the press, not everyone was happy with it either.  Whether it was their founder’s approach, their aloofness, their unconventional hiring process, or even their cute vibe, some were not impressed by Google.

By 2004, Google realized that to be able to compete with Yahoo and Microsoft, they had to go public. That April, Google filed their formal public offering (S1) that stated how not only would they be  maintaining a high level of control, the founders would also have ten times more voting power than the rest of the shareholders, despite the fact that they would own just 30% of the shares. After an age discrimination lawsuit, an investigation due to an untimely magazine interview, a reprimand that led Google to conduct a recision offer to their employees, glitches on their auction technology, and a myriad of other PR disasters, Google finally went public on August 19, 2004. Starting at the price of $85 per share, the price quickly rose to around $100 on the first day, topping at $300 by the next summer.

Post-IPO, Google underwent a soul searching of sorts, resulting in the founders’ Tablet, a statement of what makes Google what it is. This became a guide for a reorganization that took months long. Their previous approach of giving the most resources to the top 100 projects was done away with; instead, the company segregated functionality into core groups — search, advertising, “20 percent,” and “10 percent,” with the latter two for products that are were acquisitions or unconventional.


First, search overhauled the way businesses operated, from investors looking into new prospects to real estate firms staking out new territory. Until now, Google’s routine algorithmic updates impacts the crop of small, online stores who rely on showing up in organic search results. Google continues to do these in order to control spammers, click fraud, and other unethical operations that plague the Web to this day.

Next, search made information available and permanent. We now search for everything, including potential dates, potential hires, people we just met.  Unfortunately, we might not always like what we see. For example, pre-search, unfortunate things that we may have been involved in, although public information, are somehow inconvenient to research. Now, one can merely query for your name, and voila! – the history of you, as it is published online, is available for the world to see. Of course, we cannot forget the PATRIOT  Act, which allows for our private information to be intercepted and demanded by government authorities from our ISPs, Google, etc. The million dollar question then is how do be balance between our right to know and the right of a person to his privacy?

Google also had to be very careful on the precedent they set when they were entering China. They didn’t have the luxury of being a manufacturing company; brands do not suffer by being made in China. Things are different when your business is in information. Once they budge to China, what stops another country or even a corporation from making similar demands?
{PYP’s take: A  closer look on Google and China}


Perhaps in the future, we can search for anyone in real-time, or perhaps we won’t be limited to typing in a search box.  Maybe the public will even have access to a search that understands very complex, human-like demands like IBM’s WebFountain.  For sure, the evolution of search will be influenced by its two major players and their difference. Yahoo will continue to focus on being a media business, whereas Google will keep its stance on being a technology business.  Many say Google will eventually permeate into everything we do online, including music, documents, mail, photographs, and video.  To quote directly from the book, “When it comes to search, as with the Internet itself, the most interesting stuff is yet to come.”
{PYP’s Take: To date, Google has launched Google Music, Google Docs, and Gmail, as well as acquired YouTube, and Picasa. As for the interesting stuff, it is my opinion that Google has indeed lived up to that sentiment with Google+, Search plus Your World, and Android.}


“Because of their early success, they were closed-minded and a bit arrogant. Nothing deceives like success.”- Vinod Koshla, partner at Kleiner Perkins Caufield & Byers, on advising Excite founders to buy out Lycos, and not being heeded.

“I’d rather do something interesting than something boring and get rich.” – Louis Moiner, creator of AltaVista, on leaving Compaq in 1999, having felt that AltaVista was becoming a Yahoo clone.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s