BW: The Web, According to Google

It's becoming the dominant way to search the Net, and that worries critics who see problems with its privacy practices and ranking methods

Camille Ramani helps people find jobs in the San Francisco Bay Area, no easy task in a down economy, particularly in tech-heavy California. Her job is made tougher by the fact that her clients are immigrants or political refugees, most with advanced degrees. They seek out Ramani at a nonprofit organization called Upwardly Global, which specializes in helping non-U.S. professionals find jobs. One of the first places she points them to is Google, the search engine that's so simple and fast to use that it's an ideal resource for learning quickly about the U.S.

Take Isabel, a social worker from Kenya. Ramani got her a job interview in May at a nonprofit that helps HIV/AIDS patients in San Francisco. When random Web searches left Isabel more confused than enlightened about her new field, "I sat her down in front of the computer and called up Google," recalls Ramani. "I typed in "HIV AIDS, San Francisco, organizations," and up popped all these Web sites on the topic. Isabel said: Oh my gosh! That's exactly what I need."

WORLDWIDE CLEARINGHOUSE. Ramani's experience is probably typical of that of the 81 million people who visit Google each month. The site provides answers to upwards of 200 million queries each day -- as many as 75% of all Web-search requests in North America by some estimates. It does this either through Google.com or via the Web interfaces of partner sites such as Yahoo! (YHOO ) and the America Online unit of AOL Time Warner (AOL ), which pay Google per query (but keep the revenue from advertising they sell against its searches).

Privately held Google doesn't reveal its financial results, but analysts say its revenues were about $300 million in 2002, when it delivered operating margins of 20% to 30%, and that its sales are expected to more than double over the next year. In short, Google's role as a worldwide clearinghouse for information has made it the social Web's linchpin.

Google has grown so fast that even those who use it religiously have started to worry about the tendency of its huge audience to increasingly regard the site as comprehensive when in reality its compilation of information is anything but. Here and there, moreover, Webmasters are starting to question the opaque rules that Google uses to determine what to index -- or not. Some businesses complain that because of these rules, they've been banished from the search engine without warning, explanation, or cause -- a development that can cost them dearly in terms of lost sales. All of which raises the question: Is Google getting too big for its britches?

REGULATION NEEDED? "It's the kind of question the Internet is putting to us again and again," says Jonathan Zittrain, co-director of the Berman Center for Internet & Society at Harvard Law School -- "the Net feels awfully public, but in fact almost every element of it isn't public or government-run or even subsidized. When these private parties start making individualistic decisions about who's favored and who's not, it does create serious legal questions." Zittrain argues that some regulation of Web searching might prove necessary should Google kill all its competition and become the only viable search provider.

That the issue even comes up is astonishing for a business that didn't exist until five years ago. That's when Larry Page and Sergey Brin, two Stanford University computer-science grad students, got a good idea for a search engine based on an innovation called PageRank. The system works by analyzing the linking structure of the Internet as a way to rank pages. Essentially, a Web site's influence and ranking are determined by counting how many other sites link to it.

To uncover this hierarchy, Google scours the Net using autonomous software called "bots." They scan the Web continuously, screening out blatant attempts to game Google by loading up a site with extra links -- while also giving extra weight to pages that are linked to by very prominent sites, such as CNN.com. This approach isn't really new. In academia, researchers have long rated the importance of any single paper by how many times it's cited elsewhere.

LET'S GOOGLE. What's unusual, however, is Google's ability to quickly deliver the information a Web visitor wants in a simple, uncluttered format. That's why it continues to enjoy enormous popularity, despite attempted incursions by competitors such as AllTheWeb (purchased by paid search provider Overture last March) and Teoma (which search company AskJeeves bought in September, 2001) -- both of which now rival Google in search technology.

Indeed, Google is the only site whose name has become a verb. Like Faith, most job applicants "google" their potential employers and vice versa. And googling a prospective date to learn more about the person has become de rigeur. If no more than six degrees had previously separated all the people of the world, Google has surely cut the number to four.

Take the case of Kate Ulbricht, a senior pharmacist at Massachusetts General Hospital in Boston and founder of an online journal and research tool for physicans and pharmacists that seeks to catalog discoveries in the emerging field of nontraditional medicine. While Ulbricht regularly checks a host of well-known medical sites, she says she could never accomplish her task without Google. It can automatically translate Web pages in 35 languages into English, thus widening her web of contacts to researchers whose work she previously wouldn't have been able to read. Within the next year, much of that accumulated knowledge will go into the first-ever comprehensive U.S.-published textbook summarizing research into nontraditional cures.

MISSING POLICIES. That's the shiny side of Google -- the friendly search engine that makes the world a smaller and better place. But as Google has moved toward holding a de facto monopoly on Web searches, critics have started to see another side -- one that worries them. Privacy advocates have begun to question what Google is doing with all the data it collects on the people who do the hundreds of millions of searches it processes. According to the Web site Google-Watch.org, a nonprofit group dedicated to informing the public about the search giant's inner workings, Google has no policies protecting the confidentiality of information it collects on its visitors via Web cookies. (Google execs declined to comment for this article.)

Recently, Google Watch nominated Google for the 2003 Big Brother Award, a tongue-in-cheek honor it gives each year to presumed privacy offenders. What data Google collects and keeps are of more than academic interest to privacy advocates: In an October, 2001, analysis of the Patriot Act, respected cyber-liberties advocacy group the Electronic Frontier Foundation warned: The government may now spy on innocent Americans' Web surfing, including terms entered into search engines, by merely telling a judge anywhere in the U.S. that the spying could lead to information that's "relevant" to an ongoing criminal investigation." The biggest repository of such information would be Google.

Even more disturbing to some Web-search observers are the contextual nuances that can influence Google's results. For example, type in the search term "Ruby Ridge" -- referring to a 1992 shootout in Idaho in which the FBI killed an alleged white supremacist -- and Google returns thousands of pages with tons of information. But hardly any of the links at the top of the list provide balanced information on the issue. Rather, most of the top results present extreme views of the incident.

WHAT ABOUT BOOKS? Were Google less influential that probably wouldn't matter. But as Americans, especially young ones, come to regard the leading search site as the source of all h