If you can make Google bleed...

Google has a big problem. Just not the one you think.

“If you could make God bleed, people will cease to believe in Him… there will be blood in the water, and the sharks will come.” - Iron Man 2

My Dad tells this story about when he “got” the internet.  My brother was doing a school report on the Civil War and Jefferson Davis. It was hard to find books about Davis in our local library. So my Dad typed “Jefferson Davis” into our family’s AOL search and ten thousand results came up. 

It must have been 1996 or 1997.  

Google might have existed, but it was just one among many mediocre search engines, including our treasured AOL.  And then overnight, through its genius use of backlinks to curate the content of the web, Google became the only search engine that mattered.  That’s been true for close to 25 years.

But 25 years of peace breeds complacency. Paranoia is just not a part of Google culture in the way that it is at Facebook. When I joined in 2013, the onboarding manuals there included this directive: “If we don’t create the thing that kills Facebook, someone else will.”

But honestly, why should Google have worried?

The internet has consistently organized around two mediums for serving you from its infinite content-well: a-type-of-feed to push content to you and search to help you retrieve it.

Facebook once owned Feed. But that medium was innovated on by private messaging, by stories and now, by short-form-video.  Each of these offered a way to push content to you that a newsfeed could not match.

But search – search was already the seeming end state of directed content retrieval.

What could be better, after all, then typing in a few words to see all the relevant information ever generated in the world?  

The answer that always lurked in the wings – even as far back as Ask Jeeves – was that you could just ask the internet a question and get an answer. So Google invested heavily in the very AI technology that would power this with its knowledge graph and AI efforts.

And their project worked! Google invented Large Language Models…. They just didn’t bother to productize them. So like a spate of corporate labs before them - Bell inventing Unix, Xerox PARC inventing the GUI - Google is liable to be eaten by the revolution they helped birth.

At least that’s the story, right?  AI defeats Google?  Honestly, this seems wrongheaded.

The biggest threat to Google isn’t that GPT3 is going to replace search.  It’s really not - we don’t just search to generate an answer to a discrete question. The real threat to Google is that Microsoft and ChatGPT have made Google bleed. 

The God of Mountain View is mortal! The blood is in the water and the sharks are coming out. That means it’s game on for Silicon Valley to kill its greatest creation of the last 30 years. The solved problem of search is suddenly looking a little bit unsolved. That’s good news for innovators and bad news for Google. 

Because the reputation of immortality was papering over some pretty big cracks in El Goog’s armor. 

The Growing Deep Web:  Google’s Content Quality Problem

When I interned at Google, there was a mural on our office wall of a beach and a giant wave coming toward it.  This painting was called Emerald Sea.  The office I was in had been the epicenter of Google’s Google+ strategy one year before – that project was codenamed Emerald Sea.  

I was told that the team had chosen the moniker because of what it depicted – a placid beach with a giant tsunami brewing on the horizon.  The metaphorical storm was forming just down 101 at Facebook’s HQ.

I could tell you that the lesson of that moment is that Google is wrong to fear a shiny challenger. There’s plenty of room for multiple companies to thrive (as long as Apple chooses to allow them to exist). But that’s not the lesson of Google+. The useful insight is why Google was scared. 

It had nothing to do with Google wanting to be the place you share cat memes or your Uncle shares his problematic opinions. 

Google was scared because Facebook’s content and profiles were not accessible from Google search.  

Google had won Web1 because it could index and traverse all of the world’s information better than any rival, but the advent of a password-gated universe of content portended a threat that could prove existential. For the first time, there was a giant walled garden that Google could not index.  And that walled-garden is where most of the world was creating content.  It was not inconceivable that Facebook would end up with a better set of content than the rest of the World Wide Web combined.

That future didn’t happen… for Facebook. But it’s still happening for Google. Because in truth, much of the internet has always been outside of Google’s grasp.  A 2001 estimate said that, at the time, the scale of the “deep web” of unindexed content was 450-500x the public internet. 

That was a world before paywalls or subscription gated content or private APIs or social networks. So the percentage of the internet that Google cannot touch has only risen with each passing year.  That’s bad enough. But it turns out, Google is locked in a vicious cycle that makes it even worse. 

Google is an incredibly valuable traffic source for most web pages that monetize based on ads. There is a lot of money to be made in surfacing shit content that can rank highly in Google. Google knows this so it occasionally changes up the rules of the game by shifting its algorithm. But this just creates a neverending game of cat-and-mouse. So it also tries a different tactic.

It builds out a curated knowledge graph that culls through trustworthy information to surface it directly on the search results page (no clicking required!). GPT3 uses this trick, too. But this takes away traffic from the sites that do actually have good content.  

By eliminating the incentives to game search for ad money, Google has also eliminated the free-and-ad-supported business model that powered the open web. So more and more sites have to restrict their content from the prying robots of Google.

If Google will take their content without paying, then they need to keep Google out.  They need a password gate.  They need to charge for subscriptions.  

This starts a race to the bottom. 

Google fights spammers but in a way that disincentivizes the best content from being available on search. This forces good actors out of the system altogether leaving Google with more noise and ever-less signal. 

Is it any wonder that by 2019, 76% of American news outlets had joined the academic world in implementing paywalls?  Today the best content from news, to music, to streaming, to academic research, is gated behind password protected paywalls where Google can’t access it. And if current trends in culture are any indication - from OnlyFans to Patreon to NFT-gated Discords and Events - this inaccessible deep web is only going to grow.

Google’s business is directly dependent on having the best content available to index.  

But it becomes increasingly hard to index the world’s information when the world is actively hiding the best stuff from your indexers.

If the Open Web is going to fade in importance then so will its chief navigation system. 

  

Fragmentation

Imagine that you’re an entrepreneur who wants to disrupt Google.

You could try a full-frontal assault.  

You could try to compete head-on on search quality like Bing.  

You could attack Google on privacy like DuckDuckGo.  

You could aim at a higher-no-ads quality like Neeva.  

But you would have a long road to walk.  Google’s moat and reams of data ensure that it is always going to be very, very good at crawling the broad reaches of the open web.

But there’s another strategy available where Google is already getting its ass-kicked.  From commerce to finding people to searching for bars and restaurants, Google is fighting brutal battles with other search engines.  You just don’t think of their competitors as search engines.

Here’s one:  Amazon.com is the place that 74% of users start their searches for buying things online.  When you consider that eCommerce is bread-and-butter for Google’s ads business that’s uh… scary for El goog.

Or how about this: Facebook and LinkedIn are the best places to find people you know in-real-life online.  People search is basically unusable on Google except to route you to one of those services.

Then there’s Yelp for Local, Kayak/Expedia for travel, Twitter for news, Spotify for music etc. 

And we haven’t even talked about the Youths and their hatred of text content!

Instagram and TikTok are the first stop for searching for 40% of Gen-Z .

Google’s model is predicated on being a horizontal search engine.  It is designed to be the one-stop shop for searching across every intent. But as the internet has grown more and more complex, it’s not clear that a single horizontal search engine can actually dominate across intents.  And so bit by bit, use case by use case, content type by content type,  Google is losing pieces of its empire.

It’s hard to be the best at both universal and specific use cases. Consider Amazon again.  When I search for a pair of sunglasses on Amazon, I get this:

Amazon knows that I came here to buy sunglasses so it lets me start filtering that down. I can filter by brand or by rating or by delivery type or by fashion type.  It uses its knowledge of my very specific search intent to build a verticalized, focused experience.

Google is less confident that I’m searching to buy sunglasses.  I see ads for stores.  I see a link to Amazon.  I see suggested other searches I could make, or some marketing blog posts about sunglass brands. Google isn’t sure if I’m looking to shop, to research, or to learn more about sunglasses (where’s that Wikipedia link?!), it has to offer me lots of options.  

Google goes broad to satisfy any possible intent.  Amazon sells me sunglasses.

The problem for Google is that when I have a frequent intent – like shopping, reading news or looking for a person – I learn overtime that Google is inferior to a specific search intent.  

Google is still my default for less common queries.  That’s good for traffic, but it’s not nearly as defensible a position as being my most-favored-tool for a high-frequency intent.

The Platform Shifts 

The tech world has been starving for the next disruptive platform since the halcyon days of mobile.  A16z bet big on it being crypto (TBD).  OpenAI and Microsoft have bet big on it being AI.  Meta and (sometimes maybe?) Apple has bet big on it being AR/VR. 

 The uncertainty about “What comes next?” is challenging for large tech companies that have massive capabilities to explore, but lots of difficulty in turning their whole battleship toward a trend.

In 2010, Google confronted this threat with mobile.  As they saw mobile searches rise, and Apple’s iPhone taking off with its gated apps, Google saw an existential threat. If the old, decentralized web did not adopt the core innovations of mobile – location-aware, touch and small-screen friendly, sharing and app integrations – then Google was going to have to get them there.  If the web was behind, then so was Google.

The revolution…. took time and lots of capital.  

Google nudged things along by pioneering and pushing Android, then mobile wallets on Android, then giving web developers better tools to make mobile friendly sites.  Then finally they took out the big guns and said that their search algorithm would start to prioritize websites that treated mobile as a first-class-citizen in their design and feature set.

By 2017, over 60% of Google’s revenue came from mobile devices.

Google succeeded in keeping the web, and thus search at the center of the mobile internet universe.  They had limited the risk that everything would become a native app.  But can they pull off the same trick on the next platform pivot?

If Facebook or Apple really create an immersive 3d, live-in metaverse, is Google search going to be the core tool?  By owning the platform, Apple and Facebook will get the first bite at that apple.  And neither has much love loss with Google. 

If AI is truly the next platform, is keyword search and the “document-index” model of the web really valuable?  It doesn’t seem like it.

If crypto and web3 wins, with its decentralized, token-gated data-stack, can Google be sure that they will be the best tool for finding entries on an encrypted global database?  

Any of these shifts could be devastatingly disruptive to Google.  And companies aren’t great at split focus.  So Google needs to pick correctly and bet big.  The problem is – there’s also no guarantee that only one of these trends disrupts everything. 

Even if they’re right on one, the others could pose a massive threat.  Google was designed for the golden era of the web.  But if that era is ending, Google might just end with it.

What happens now…

Google’s problems are the product of an unstable digital equilibrium.  Today’s web – with open, indexable content, relatively few specialized competitors and a stable set of platforms – is in flux.

Once upon a time, Google was so dominant, its grip on attention and talent so strong, and its coffers so deep that it could fight any war.  But their empire has aged.  It has stumbled. It has shown the world that it can be beaten – even in fields that it pioneered. There’s a reason no company remains on top forever.

Time – and the creative destruction that it enables– is undefeated.

Is it going to be chat-interface sidebar that kills the Giant of Mountain View? Almost certainly not. The technology is too new, the users too unfamiliar and the design paradigm too tacked-on. 

But Google has shown that it no longer has its finger on every vein of technological progress.  That means it’s finally sensible to bet on businesses that could kill them. Heavy is the head that wears the crown. 

In a world where only the paranoid survive, Google has found its fear at a late hour.  Is it too late? We’re about to find out.