In 1990, a McGill student named Alan Emtage built Archie, a tool that indexed the filenames on public FTP servers. It was the first thing anyone called a search engine, and it searched almost nothing. Then the web arrived. Mosaic shipped in 1993, and within two years the number of pages had outrun anyone’s ability to keep a list.<\/p>\n
Two competing answers emerged. The first was human. In 1994 two Stanford PhD students, Jerry Yang and David Filo, started a hand-built directory of links, which became Yahoo. Editors read sites, filed them under categories, and wrote short descriptions. The results were tidy and trustworthy. They also could not scale: a team of people cannot catalogue a medium that doubles in size every few months.<\/p>\n
The second answer was mechanical. Lycos, WebCrawler, Excite, Infoseek and Inktomi sent crawlers out to fetch pages and matched your query against the words on them. AltaVista, launched in December 1995 from Digital Equipment Corporation’s research lab, was the most impressive: an index of roughly 20 million pages and a speed that made earlier tools feel like dial-up. Its weakness was the thing it measured. Ranking by what a page says about itself invites the page to lie.<\/p>\n
By 1997, anyone could game these engines. Repeat “free music” two hundred times in white text on a white background and a page about something else entirely would climb the results. Searchers learned to scroll through ten pages of junk. Coverage was poor as well. A 1998 study by Steve Lawrence and C. Lee Giles in Science<\/em> estimated the public web at about 320 million pages and found that no single engine covered more than roughly a third of them.<\/p>\n The field had two problems tangled together. Engines could not find enough of the web, and what they did find was sorted by a signal the authors of those pages controlled.<\/p>\n Larry Page arrived at Stanford in 1995 and, in 1996, began a crawler project called BackRub to map which pages linked to which. The insight came from academic publishing: a paper cited by many important papers is probably important. Sergey Brin joined him. Page and Brin were not alone in the idea. Robin Li, working at a New Jersey firm in 1996, patented a similar link-based scoring method called RankDex and later founded Baidu, and Jon Kleinberg at IBM was working on HITS at the same time. What distinguished Page and Brin was the follow-through: they built the whole stack and put it in front of users.<\/p>\n The shift was philosophical as much as technical. Link graphs turned the web’s own structure into a vote. Authors could still write whatever they liked on their pages, but they could not easily force strangers to link to them. Relevance moved from what a page claimed to what the rest of the web thought of it.<\/p>\n<\/section>\n The 1998 paper, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” presented at the Seventh International World Wide Web Conference in Brisbane, described PageRank as a random-surfer model. Imagine a person clicking links at random. Most of the time they follow a link from the current page; occasionally, with probability 0.15, they get bored and jump to a random page. The PageRank of a page is the long-run probability that this surfer is on it.<\/p>\n The calculation is circular, since a page’s score depends on the scores of the pages linking to it, which depend on their own inbound links. It is solved by iteration: start every page with equal weight, redistribute weight along links, repeat. The paper reported convergence in roughly 50 iterations on a graph of 322 million links. A page’s score is divided among its outbound links, so a link from a page with a thousand outlinks passes on far less than a link from a page with three. The damping factor, 0.85 in the paper, keeps the system from collapsing into dead ends and loops.<\/p>\n Two other ideas mattered as much as the formula. First, Google indexed anchor text, the words in the link, and associated it with the target page. A page could rank for a phrase it never used because other sites described it that way. Second, the system combined link authority with ordinary on-page signals such as term position and font prominence. PageRank was the tiebreaker that AltaVista never had.<\/p>\n The prototype fetched about 26 million pages in nine days, errors included. That figure sounds quaint, but the same constraint governs the web now: crawling costs bandwidth, compute and politeness toward the sites being fetched. Googlebot has to decide which URLs to visit, how often, and how hard to press on a given server.<\/p>\n SEOs call the result crawl budget. Google’s own documentation breaks it into two parts: a crawl capacity limit (how much a server can bear without slowing for real users) and crawl demand (how much Google wants to revisit a URL, based on popularity and staleness). For most sites the budget is irrelevant. For publishers and retailers with millions of near-duplicate URLs, such as faceted navigation and session parameters, it is the difference between being indexed and being invisible.<\/p>\n The business bet behind Google’s infrastructure was that cheap, failure-prone commodity machines, properly orchestrated, would beat expensive reliable ones. Three papers laid out the system.<\/p>\n The Google File System (SOSP 2003) spread files across thousands of ordinary servers in large chunks, replicated so that a dead disk was a routine event rather than an emergency. MapReduce (OSDI 2004), by Jeff Dean and Sanjay Ghemawat, gave engineers a way to write a computation as a map step and a reduce step and let the framework handle distribution, retries and stragglers. The paper reported that in August 2004 alone Google ran 29,423 MapReduce jobs reading about 3,288 terabytes of input. The index-building pipeline was one of its first major clients.<\/p>\n Bigtable (OSDI 2006) was the storage layer: a sparse, distributed, sorted map indexed by row key, column key and timestamp. Its flagship example was a web table keyed by reversed hostnames, so that com.cnn.www sat next to com.cnn.edition, with columns for page contents and for anchor text from linking pages. The paper counted more than sixty Google products running on it by then, among them Earth, Analytics and Personalized Search.<\/p>\n For years the index was rebuilt in large batches, and fresh content could take days or weeks to appear. In June 2010 Google announced Caffeine, built on a system called Percolator that updated the index incrementally as pages were crawled. Google said results were about 50 percent fresher. A site could now publish and rank within hours. News search, social-era content and an entire generation of publishing strategy depended on that change.<\/p>\n The scale has kept climbing. Google’s own description of Search puts the index at hundreds of billions of pages and well over 100 million gigabytes. That description is a rounded self-report that Google has not refined in years, so treat it as an order of magnitude, not a measurement.<\/p>\n<\/section>\n Google was founded in September 1998 and for its first two years it earned almost nothing. Page and Brin disliked advertising, and their Stanford-era paper included an appendix warning that ad-funded engines would be biased toward advertisers. The early revenue came from licensing the search technology, including a 2000 deal to power Yahoo’s results.<\/p>\n The pressure came from two directions: venture investors who had put in $25 million in 1999, and the dot-com crash that followed in 2000.<\/p>\n AdWords launched in October 2000 with a few hundred advertisers, priced per thousand impressions, with ads written by Google staff. The decisive redesign came in February 2002. AdWords Select moved to cost-per-click and ran an auction for each keyword.<\/p>\n The idea of paying for clicks was not Google’s. Bill Gross’s GoTo, later renamed Overture, had pioneered pay-per-click auctions in 1998, ranking purely by bid. Google changed two things. It ranked ads by bid multiplied by predicted click-through rate, which came to be folded into a Quality Score, so a relevant ad could beat a higher bid. And each advertiser paid only a small increment over the bid below it (the generalized second-price mechanism, to which economist Hal Varian gave academic backing after joining in 2002). Bad ads were penalized rather than rewarded, which kept the results useful and made clicks worth more to advertisers over time.<\/p>\n The model worked because search queries are declarations of intent. Someone typing “emergency plumber Essex” is nearer to a purchase than anyone reading a magazine. Advertisers were paying to appear at the moment a need was typed out, and could measure the outcome. Yahoo, by then the owner of Overture, sued over the patents; the case settled in 2004 with Google issuing Yahoo about 2.7 million shares.<\/p>\n In 2003 Google bought Applied Semantics for about $102 million and used its contextual technology to launch AdSense, extending Google ads onto third-party sites in exchange for a revenue share. It purchased DoubleClick for $3.1 billion in 2007 and AdMob for $750 million in 2010, taking in display and mobile advertising.<\/p>\n The revenue arc is startling in retrospect. Google reported about $86 million in 2001, around $440 million in 2002, roughly $1.5 billion in 2003 and about $3.2 billion in 2004, the year of its IPO at $85 a share, which raised roughly $1.67 billion. In 2025, Alphabet’s revenue was $402.8 billion.<\/p>\n The advertising mix has shifted, too. In the fourth quarter of 2025, Google Search and other brought in $63.1 billion and YouTube ads $11.4 billion, while Google Network, the AdSense-descended business of ads on other people’s sites, earned $7.8 billion. That last figure was slightly below the $8.0 billion of a year earlier. The thing that started the company’s money machine on the open web is now the flat part of the portfolio.<\/p>\n<\/section>\n A pattern shows up in Google’s growth: buy or build something that creates data and attention, then connect it to search and ads. Gmail launched on April 1, 2004, with a gigabyte of storage when rivals offered a few megabytes. Google acquired Keyhole in 2004, which became Earth, and launched Maps in February 2005. Google Maps now has more than two billion monthly users, according to Google. Each product made Google the default place to start something, and every starting point is a query.<\/p>\n The fear driving Google’s mobile strategy was simple: if phones were controlled by a rival that set its own search default, search revenue would be at that rival’s mercy. Google bought Android Inc. in 2005, reportedly for about $50 million. It unveiled the Open Handset Alliance in November 2007, and the first Android phone, the HTC Dream, shipped in October 2008. Android passed 2 billion active devices in 2017 and 3 billion in 2021.<\/p>\n Chrome arrived in September 2008, announced with a comic book drawn by Scott McCloud. It brought a faster JavaScript engine (V8) and a multi-process design that kept a crashed tab from taking down the window. Its real function was strategic. Web apps such as Gmail needed a fast browser, and a browser Google controlled could carry its search box and its defaults. Chrome now holds roughly two-thirds to 70 percent of global browser use, depending on the tracker and on whether you count desktop or mobile.<\/p>\n Google paid $1.65 billion in stock for YouTube in October 2006, a price that looked reckless for a site with no clear business model. In 2025, YouTube’s advertising and subscription revenue topped $60 billion, and Google reported more than 325 million paid subscriptions across its consumer services, led by Google One and YouTube Premium. YouTube is also, by most accounts, the second most used search engine on earth, the place where people go for how-to answers that text cannot give.<\/p>\n The machinery built to crawl and serve the web turned out to be sellable. App Engine launched in 2008, Compute Engine in 2012, and in 2016 Google announced its custom Tensor Processing Units for machine learning<\/a>. Cloud was the laggard for years behind Amazon Web Services and Microsoft<\/a> Azure. The picture has changed. In the fourth quarter of 2025, Google Cloud’s revenue grew 48 percent to $17.7 billion and its operating income reached $5.3 billion, up from $2.1 billion a year earlier. In the second quarter of 2026, Alphabet reported Cloud growth of 82 percent, which put the quarterly figure near $24.8 billion, roughly a fifth of company revenue.<\/p>\n<\/section>\n For a decade, Google matched strings: the characters you typed against the characters on pages, weighted by links. The problem is ambiguity. “Jaguar” can be a car, a cat or a football team. In 2010 Google bought Metaweb, whose Freebase database held structured facts about entities. In May 2012 it launched the Knowledge Graph with the slogan “things, not strings,” starting with about 500 million entities and 3.5 billion facts about them. Panels with a birthday, a population or a cast list appeared beside results, and some queries stopped needing a click at all.<\/p>\n Hummingbird, introduced in 2013, rebuilt the core algorithm to handle longer conversational queries, a response to voice search and to people typing whole questions. RankBrain, disclosed in 2015, used machine learning to interpret queries Google had never seen, a category that Google says still accounts for about 15 percent of daily searches.<\/p>\n In 2017, Google researchers published “Attention Is All You Need,” introducing the Transformer architecture. BERT, released in 2018 and folded into Search in October 2019, used it to read a query’s words in relation to the words on both sides. Google said at launch that BERT affected about one in ten English-language searches in the United States, and it spread to more than 70 languages within weeks. MUM, announced in 2021, was trained across 75 languages and could work across text and images.<\/p>\nThe mathematical shortcut<\/h3>\n
The Engineering Architecture<\/h2>\n
PageRank, in plain terms<\/h3>\n
Crawling and the budget behind it<\/h3>\n
GFS, MapReduce and Bigtable<\/h3>\n
From batches to a live index<\/h3>\n
The Monetization Engine<\/h2>\n
The company that did not know how to make money<\/h3>\n
AdWords and the auction for intent<\/h3>\n
AdSense, DoubleClick and the scale<\/h3>\n
Beyond 10 Blue Links<\/h2>\n
Maps, mail and the logic of acquisition<\/h3>\n
Android and Chrome<\/h3>\n
YouTube and the second search engine<\/h3>\n
Cloud: the infrastructure becomes a product<\/h3>\n
The Semantic and AI Era<\/h2>\n
From strings to things<\/h3>\n
Transformers: Google invents the thing that threatens it<\/h3>\n